Subject: [Mtgvenue] Substantial comments on <draft-ietf-mtgvenue-iaoc-venue-selection-process-02>

From: Bob Hinden <[email-masked]>
Date: Fri, 18 Nov 2016 09:51:51 +0900

Here are my substantial comments on the draft.  I will send a second email with editorial comments.

Thanks,
Bob

> 1.2.  Requirements Language
> 
>    Requirements called out in this document are identified as either
>    "mandatory" or "desired", and considerations are tagged as
>    "Important" or "Would be nice".  For clarity, the terms are defined
>    here:
> 
>    Mandatory:  If this requirement cannot be met, a location under
>       consideration is unacceptable.  We walk away.
> 
>    Desired:  We would very much like to meet this requirement, but have
>       frequently been unable to.  The fact that we could not meet it is
>       considered in comparison to other sites.
> 
>    Important:  Can be a make-or-break consideration, but can also be
>       traded off against other considerations.
> 
>    Would be nice:  Not make-or-break, but warrants additional
>       consideration if found to be true.

As I said in todays w.g. meeting, four is two many.  My preference would be to have two or three.

> 
> 2.1.  Core Values
> 
>    The IETF has some core values that to pervade the selection process.
>    The values are not limited to the following, but at minimum include
>    them.
> 
>    Why do we meet?
>       We meet to advance development of Internet Drafts and RFCs.  We
>       also seek to facilitate attendee participation in multiple topics
>       and to enable cross-pollination of ideas and technologies.

I think this paragraph is too limited, our focus is to build the technology that the Internet is based on, just just writing IDs and RFCs.  Also, another reason we meet is to build the IETF Community.   Suggest something like:

      We meet to advance development of Internet technology and to build the IETF community.  This includes the development of Internet Drafts and RFCs, and to facilitate attendee participation in multiple topics and to enable cross-pollination of ideas and technologies.

> 
>    Internet Access:
>       As an organization, we write specifications for the Internet, and
>       we use it heavily.  Meeting attendees need unfiltered access to
>       the general Internet and our corporate networks, which are usually
>       reached using encrypted VPNs from the meeting venue and hotels,
>       including overflow hotels.  We also need open network access
>       available at high enough data rates to support our work, including
>       the support of remote participation.[MeetingNet]

I think this is missing mobile (aka cellular) access.  This is becoming more important.  I don’t think this is current a mandatory requirement that open mobile is required, but should be something that should be considered.  It may become mandatory later.

> 3.1.  Venue City Criteria
> 
>       Alissa's comment: 3.3.1 bullets 1, 5, 6, plus other bullets from
>       Section 5.1.3.1 if reformulated as criteria)
> 
>    Note that these considerations are not "make or break" but flexible
>    enough to allow for trade-offs and judgement on the part of the IAOC.
>    Ideally, a city will meet all criteria.  If all criteria cannot be
>    met, the IETF community needs to consider the trade-off acceptable.
> 
>    o  Travel to the venue is reasonably acceptable based on cost, time,
>       and burden for participants traveling from multiple regions.  It
>       is anticipated that the burden borne will be generally shared over
>       the course of the year.  [Important]

As noted in the meeting today, this is sometimes done over more than one year.

s/generally shared over the course of the year/generally shared over the course of one or two years/

> 
>       Editor query:    Resolve Important vs. Mandatory.  d/
> 
>    o  The venue is assessed as favorable for obtaining a host and
>       sponsors.  That is, the Meeting is in a location and at a price
>       that it is possible and probable to find a host and sponsors.
>       [Important]

Finding hosts is more complicated than just location and price.  It includes factors as relevant organizations in the city or country, interest by multinationals in supporting the IETF, and economic conditions.  I think it’s hard to describe this correctly.  Might be better to remove this paragraph.

>    o  It is possible to enter into a multi-event contract with the venue
>       to optimize meeting and attendee benefits, i.e., reduce
>       administrative costs and reduce direct attendee costs, will be
>       considered a positive factor.  [Would be nice]

Pleased add that this should be based on past experience with having an IETF meeting in the venue.  It would be risky to commit to multiple meetings before doing that.  Also note that this is current practice with a few venues like Berlin.

> 
>                                          October 2016
> 
> 
>    o  Available travel issue assessments (such as
>       https://travel.state.gov/content/passports/en/country.html) have
>       been pointed out the IETF community.  [Important]

This list is too short.  The US state department site is very US centric, basically any place outside of the US is scary.  Not very useful

> 
> 4.8.  Venue Selection Phases
> 
>       Alissa's comment: ( 3.5, perhaps with additional detail currently
>       in 3.3.1 bullets 2, 3, 4)
> 
>    Commencing the process four years in advance of an event results in
>    the following schedule as a guideline:

This is missing one very important step.  That is, that the IAOC sets IETF meeting dates in the future.  For example:

    https://www.ietf.org/meeting/upcoming.html

has dates through 2022.

The whole process of finding a venue is based on finding a venue that is available on our dates.  The document doesn’t mention this as far as I can tell  We don’t go to a potential venue and say what dates we could have the meeting.

I suggest adding a new Phase 1 describing this step (and renumber the others) and text to talk about the venue selection is based on venue availability or our dates.

> 
>    Phase 1:  Identification and Preliminary Investigation
>       Four years out, a process identifies cities for meetings and
>       initiates site selection.

Where is the step where the secretariat find out of a potential venue is available on our dates?  Should this be done before vetting a city with the community.  There isn’t much point to ask the community about a city if there isn’t a possible venues. I realize this is a chicken and egg problem, but it needs to be thought about.

> 
>       A.  The IAOC selects regions for meetings.
> 
>       B.  Meeting target cities per region are provided to the
>           Secretariat based upon Meetings Committee input and, if known,
>           host preferences.
> 
>       C.  Potential venues in preferred cities identified and
>           investigated, including reviews of Official Advisory Sources,
>           consultation with specialty travel services, frequent
>           travelers and local contacts to identify possible barriers to
>           holding a successful meeting in the target cities.

Might be good to mention using what we have learned by past meetings.  We know a lot more about we have had meeting before.

> 
>       D.  Investigated cities and findings are provided by the
>           Secretariat to the Meetings Committee for review.  Meetings
>           Committee makes a recommendation to the IAOC of investigated/
>           target cities to consider further as well as issues identified
>           and the results of research conducted.
> 
>    Phase 2:  Community Consultation
>       The IAOC asks the community whether there are any barriers to
>       holding a successful meeting in the target cities.  Community
>       responses are reviewed and concerns investigated.  The IAOC shall
>       maintain a public web page that lists all target cities, when
>       community input was solicited, and summarized review results.
>       Once review results are published the IAOC provides a list of
>       vetted cities to the Meetings Committee to pursue as potential
>       meeting locations.

Add something that the IAOC also publishes the results at of the community consultation on it’s web site (or an exact url).

> 
>    Phase 3:  Vetted Venues Evaluated for Site Qualification Visit
> 
>       A.  Secretariat Assesses "vetted" target cities to determine
>           availability and conformance to criteria
> 
> 
> 
> 
> Pelletier, et al.          Expires May 4, 2017                 [Page 11]
> Internet-Draft                                              October 2016
> 
> 
>       B.  Meetings Committee approves potential cities for site
>           qualification visit.

This is also missing that we visit cities where there are possible venues.  Either ones that have availability on our dates, or ones we want to find out more.  We only do a trip to a city if there are potential venues.

> 
>    There is an explicit intent to rotate meeting locations equally among
>    several places in accordance with IETF policy.  However, a consistent
>    balance is sometimes difficult to achieve.  The IAOC has an objective
>    of setting the Regions 4 years in advance, meeting in Europe, North
>    America, and Asia, with a possibility of occasionally meeting outside
>    those regions.  This policy, known as the 1-1-1* model, is set by the
>    IESG, https://iaoc.ietf.org/minutes/2010-11-10-iaoc-minutes.txt, and
>    is further discussed in [I-D.krishnan-ietf-meeting-policy].  The
>    reason for the multi-year timeframe is maximization of opportunities;
>    the smaller the time available to qualify and contract a conference
>    venue, the more stress imposed on the qualification process, and the
>    greater the risk of not finding a suitable venue or paying more for
>    it.

I think we should add that “*” meetings are infrequent.  We have done it twice in 97 meetings.  I think it should continue, but we aren’t going to do it every other year.

> 
> 5.1.5.  Experience Notes
> 
>    a.  The foregoing process works with reasonable certainty in North
>        America and Europe.
> 
>    b.  Experience to date for Asia and Latin America is that contracts
>        take longer and often will not be executed more than two years in
>        advance of the meeting.  While the IETF will have the first
>        option for the dates, for reasons not completely understood
>        contracts won't be executed.

I would describe this as “local business practices”, not “reasons not completely understood".  I think that is understood, in some regions they just don’t want to sign a contract as far in advance as we would like.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 18 Nov 2016 16:26:22 +0900

>> 2.1.  Core Values
...
>> Why do we meet? We meet to advance development of Internet Drafts
>> and RFCs.  We also seek to facilitate attendee participation in
>> multiple topics and to enable cross-pollination of ideas and
>> technologies.
>
> I think this paragraph is too limited, our focus is to build the
> technology that the Internet is based on, just just writing IDs and
> RFCs.  Also, another reason we meet is to build the IETF Community.
> Suggest something like:
>
> We meet to advance development of Internet technology and to build
> the IETF community.  This includes the development of Internet Drafts
> and RFCs, and to facilitate attendee participation in multiple topics

It is tempting to start with high-level phrasing, as you've done, but 
the danger is that it makes things too abstract and, worse, too detached.

Worst of all is that it can guide people away from our primary task:

      We have deliverables, namely the production of useful documents.

The 'community' is /not/ a primary goal, actually.  It is a really 
important attribute or mechanism for making the process of producing 
those useful document go well.  So 'community' is part of the 'how', not 
they 'why'.

>> Internet Access: As an organization, we write specifications for
>> the Internet, and we use it heavily.  Meeting attendees need
>> unfiltered access to the general Internet and our corporate
>> networks, which are usually reached using encrypted VPNs from the
>> meeting venue and hotels, including overflow hotels.  We also need
>> open network access available at high enough data rates to support
>> our work, including the support of remote
>> participation.[MeetingNet]
>
> I think this is missing mobile (aka cellular) access.  This is
> becoming more important.  I don’t think this is current a mandatory
> requirement that open mobile is required, but should be something
> that should be considered.  It may become mandatory later.

OK, but what wording needs to be changes, to what?

What does 'mobile' mean here and how is it relevant to venue selection?

>> Note that these considerations are not "make or break" but
>> flexible enough to allow for trade-offs and judgement on the part
>> of the IAOC. Ideally, a city will meet all criteria.  If all
>> criteria cannot be met, the IETF community needs to consider the
>> trade-off acceptable.
>>
>> o  Travel to the venue is reasonably acceptable based on cost,
>> time, and burden for participants traveling from multiple regions.
>> It is anticipated that the burden borne will be generally shared
>> over the course of the year.  [Important]
>

Right.  I'd already changed the wording to be:

   <c>Travel to the Venue is acceptable based on cost, time, and burden
      for participants traveling from multiple regions. It is anticipated
      hat the burden borne will be generally shared over the course
      multiple years.</c>

> Finding hosts is more complicated than just location and price.  It
> includes factors as relevant organizations in the city or country,
> interest by multinationals in supporting the IETF, and economic
> conditions.  I think it’s hard to describe this correctly.  Might be
> better to remove this paragraph.

Anyone care to comment?

>> o  It is possible to enter into a multi-event contract with the
>> venue to optimize meeting and attendee benefits, i.e., reduce
> Pleased add that this should be based on past experience with having
> an IETF meeting in the venue.  It would be risky to commit to
> multiple meetings before doing that.  Also note that this is current
> practice with a few venues like Berlin.

Right.  I'd already changed the text to be:

    <c>It is possible to enter into a multi-event contract with the Venue
       to optimize meeting and attendee benefits, i.e., reduce
       administrative costs and reduce direct attendee costs, will be
       considered a positive factor. Such a contract can be considered
       after at least one IETF meeting has been held at the Venue.</c>

> This list is too short.  The US state department site is very US
> centric, basically any place outside of the US is scary.  Not very
> useful

I certainly agree.  Do folk have other citations to consider?

>> Commencing the process four years in advance of an event results
>> in the following schedule as a guideline:
>
> This is missing one very important step.  That is, that the IAOC sets
> IETF meeting dates in the future.  For example:
>
> The whole process of finding a venue is based on finding a venue that
> is available on our dates.  The document doesn’t mention this as far
> as I can tell  We don’t go to a potential venue and say what dates we
> could have the meeting.
>
> I suggest adding a new Phase 1 describing this step (and renumber the
> others) and text to talk about the venue selection is based on venue
> availability or our dates.

I'd already modified the text to be:

    <t>The IAOC selects regions and dates for meetings.</t>

rather than creating an additional entry.

>> Phase 1:  Identification and Preliminary Investigation Four years
>> out, a process identifies cities for meetings and initiates site
>> selection.
>
> Where is the step where the secretariat find out of a potential venue
> is available on our dates?  Should this be done before vetting a city
> with the community.  There isn’t much point to ask the community
> about a city if there isn’t a possible venues. I realize this is a
> chicken and egg problem, but it needs to be thought about.

Current wording:  <section title="Identification">

  <t>Four years out, a process identifies cities that might be candidates
     for meetings: <list style="letters">

  <t>The IAOC selects regions and dates for meetings.</t>

  <t>A list of target cities per region is provided to the
     Secretariat, with host preferences, if known.</t>

  <t>Potential venues in preferred cities are identified and receive
     preliminary investigation, including reviews of Official Advisory
     Sources, consultation with specialty travel services, frequent
     travelers and local contacts to identify possible barriers to
     holding a successful meeting in the target cities.</t>

  <t>Investigated cities and findings are provided by the Secretariat
     to the Meetings Committee for further review. Meetings Committee
     makes a recommendation to the IAOC of investigated/target cities
     to consider further as well as issues identified and the results
     of research conducted.</t>
        </list></t>
       </section>

I think that covers your concern.  The "Potential" entry would worry 
about availability dates.

>> consultation with specialty travel services, frequent travelers and
>> local contacts to identify possible barriers to holding a
>> successful meeting in the target cities.
>
> Might be good to mention using what we have learned by past meetings.
> We know a lot more about we have had meeting before.

What exact wording change are you proposing?

>> target cities to consider further as well as issues identified and
>> the results of research conducted.
>>
>> Phase 2:  Community Consultation The IAOC asks the community
>> whether there are any barriers to holding a successful meeting in
>> the target cities.  Community responses are reviewed and concerns
>> investigated.  The IAOC shall maintain a public web page that lists
>> all target cities, when community input was solicited, and
>> summarized review results. Once review results are published the
>> IAOC provides a list of vetted cities to the Meetings Committee to
>> pursue as potential meeting locations.
>
> Add something that the IAOC also publishes the results at of the
> community consultation on it’s web site (or an exact url).

What's wrong with the 'shall maintain' text already there?  Again, 
please suggest specific wording changes.  (Note the current version, above.)

>> Pelletier, et al.          Expires May 4, 2017
>> [Page 11] Internet-Draft
>> October 2016
>>
>>
> This is also missing that we visit cities where there are possible
> venues.  Either ones that have availability on our dates, or ones we
> want to find out more.  We only do a trip to a city if there are
> potential venues.

Visiting a city is the 'how', not the 'what'.  The 'what' is described 
as "Secretariat Assesses..."  What's wrong this that?

>> There is an explicit intent to rotate meeting locations equally
>> among several places in accordance with IETF policy.  However, a
>> consistent balance is sometimes difficult to achieve.  The IAOC has
>> an objective of setting the Regions 4 years in advance, meeting in
>> Europe, North America, and Asia, with a possibility of occasionally
>> meeting outside those regions.  This policy, known as the 1-1-1*
>> model, is set by the IESG,
>> https://iaoc.ietf.org/minutes/2010-11-10-iaoc-minutes.txt, and is
>> further discussed in [I-D.krishnan-ietf-meeting-policy].  The
>> reason for the multi-year timeframe is maximization of
>> opportunities; the smaller the time available to qualify and
>> contract a conference venue, the more stress imposed on the
>> qualification process, and the greater the risk of not finding a
>> suitable venue or paying more for it.
>
> I think we should add that “*” meetings are infrequent.  We have done
> it twice in 97 meetings.  I think it should continue, but we aren’t
> going to do it every other year.

That's more than 2 percent...

Anyhow, current wording:

   <t>The IETF chair drives selection of "*" locations, i.e., venues
      outside the usual regions, and requires community input. These
      selections usually arise from evidence of growing interest and
      participation in the new region. Expressions of interest from
      possible hosts also factor into the meeting site selection process,
      for any meeting.</t>

So how about:

   <t>The IETF chair drives selection of "*" locations, i.e., venues
      outside the usual regions, and requires community input. These
      are exceptional selections, such as arising
      from evidence of growing interest and
      participation in the new region. Expressions of interest from
      possible hosts also factor into the meeting site selection process,
      for any meeting.</t>

In any event, I have a continuing concern about the foundational 
language for "*" venues:  It seems to imply a degree of 'marketing' the 
IETF to that region, but basically does not indicate what the purpose of 
the visit is and why that purpose constitutes justification.

>> b.  Experience to date for Asia and Latin America is that
>> contracts take longer and often will not be executed more than two
>> years in advance of the meeting.  While the IETF will have the
>> first option for the dates, for reasons not completely understood
> I would describe this as “local business practices”, not “reasons not
> completely understood".  I think that is understood, in some regions
> they just don’t want to sign a contract as far in advance as we would
> like.

You are suggesting a category of reason.  I think the language comes 
from staff and represents their feet-on-the-ground assessment.  I'll let 
you and they duke out the precise wording for this...

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Bob Hinden <[email-masked]>
Date: Fri, 18 Nov 2016 18:38:08 +0900

Dave,

Comments inline.

Thanks,
Bob

> On Nov 18, 2016, at 4:26 PM, Dave Crocker <[email-masked]> wrote:
> 
> It is tempting to start with high-level phrasing, as you've done, but the danger is that it makes things too abstract and, worse, too detached.
> 
> The 'community' is /not/ a primary goal, actually.  It is a really important attribute or mechanism for making the process of producing those useful document go well.  So 'community' is part of the 'how', not they 'why’.

I disagree, this is in a section titled “core values”.

> What does 'mobile' mean here and how is it relevant to venue selection?

We use them more and more, and mobile cellular networks be be a good solution for open network access in overflow hotels.

How about adding to the end of the paragraph:

Mobile cellular networks are becoming more important for IETF attendees at IETF meetings for the same reasons as for the venue networks.  The availability and openness of these networks should be evaluated as part of the selection process, especially in locals where open network access is restricted.

>     hat the burden borne will be generally shared over the course

Good

>      after at least one IETF meeting has been held at the Venue.</c>

Good

> rather than creating an additional entry.

Make it clear that is done earlier.  I also think the document should make it clear we are looking for venues that are available on our dates.

> I think that covers your concern.  The "Potential" entry would worry about availability dates.

I would prefer “Potential venues that are available on IETF meeting dates in preferred cities…”

> What exact wording change are you proposing?

How about adding to the list:

"experience from previous IETF meetings in the same venues"

> What's wrong with the 'shall maintain' text already there?  Again, please suggest specific wording changes.  (Note the current version, above.)

I missed that.

> Visiting a city is the 'how', not the 'what'.  The 'what' is described as "Secretariat Assesses..."  What's wrong this that?

Reading it again I now see that, but I missed it before.  Apparently wasn’t clear to me.

>     possible hosts also factor into the meeting site selection process,

I still think it should say either it doesn’t happen often, or cite that it has only happend twice in 30 years of IETF meetings.

> 
> 
> In any event, I have a continuing concern about the foundational language for "*" venues:  It seems to imply a degree of 'marketing' the IETF to that region, but basically does not indicate what the purpose of the visit is and why that purpose constitutes justification.
> 
> 
> 
> 
> 
>>> 5.1.5.  Experience Notes
>>> 
> You are suggesting a category of reason.  I think the language comes from staff and represents their feet-on-the-ground assessment.  I'll let you and they duke out the precise wording for this…

My point is that is isn’t "for reasons not completely understood”.  That reads like we don’t have any understanding.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 19 Nov 2016 08:18:13 +1300

>   <t>The IAOC selects regions and dates for meetings.</t>

I think we do need an explanatory note about dates. They are clearly out of
scope for this document, but people do occasionally wonder why we don't
pick dates to minimize costs or to match hotel availability. So perhaps we
should say something along the lines of this:

IETF dates are fixed years in advance, in conjunction with other Standards
Development Organizations, to avoid undesirable clashes. Therefore, they
cannot be varied due to venue selection issues.

Regards
   Brian Carpenter

---

From: Bob Hinden <[email-masked]>
Date: Sat, 19 Nov 2016 08:19:23 +0900

> On Nov 19, 2016, at 4:18 AM, Brian E Carpenter <[email-masked]> wrote:
> 
> On this point:
> 

+1

Bob