Subject: [Mtgvenue] Draft about high level guidance for the meeting policy of the IETF

From: Suresh Krishnan <[email-masked]>
Date: Mon, 20 Jun 2016 20:38:17 +0000

Hi all,
   I have written a draft with the intent of roughly documenting the current 
meeting policy of the IETF and eventually getting to a community consensus 
backed version that can be used to provide high level guidance to the IAOC. 
Your suggestions, comments, and criticism are greatly appreciated. The draft 
can be found here

https://tools.ietf.org/html/draft-krishnan-ietf-meeting-policy-00
https://www.ietf.org/id/draft-krishnan-ietf-meeting-policy-00.txt

Thanks
Suresh

---

From: Melinda Shore <[email-masked]>
Date: Mon, 20 Jun 2016 15:55:11 -0800

A couple of quick notes:

    The work of the IETF is primarily conducted on the working group
    mailing lists, while face-to-face WG meetings mainly provide a high
    bandwidth mechanism for working out unresolved issues.

I think we can all agree that that's the intent, but the reality
is, for better or worse, far different, and that has some impact
on how important meetings are in practice.  I don't think I've
been to a meeting in years where decisions weren't made in the
room in at least one working group.  This is, of course, ancillary
to the purpose of your document but I do think it matters quite
a bit in the broader context of how we discuss meetings (i.e. the
more central meetings are to our process in practice, the more
important attendance becomes and consequently the more important
a reasonable meeting location policy becomes).

    This meeting rotation is mainly
    aimed at distributing the travel pain for the existing IETF
    participants who physically attend meetings and for distributing the
    timezone pain for those who participate remotely

I don't think that's always been the case in the past (although I
hope it is in the future!).  There's been some hope expressed that
by meeting in new places we can attract new participants.  Again,
I'm not sure that's an argument that's been advanced by the people
planning meetings but I do think that your document is an opportunity
to resolve this question for the time being.  It may be better to
say that the meeting rotation "will be" aimed, etc.

Thanks for writing this up,

Melinda

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Jun 2016 17:54:48 -0700

On 6/20/2016 4:55 PM, Melinda Shore wrote:
> There's been some hope expressed that
> planning meetings

It has, by some.  It has been challenged, by some, absent supporting data.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Eliot Lear <[email-masked]>
Date: Mon, 20 Jun 2016 17:57:26 -0700

> It has, by some.  It has been challenged, by some, absent supporting

It is hard to have data without having performed the experiment.

Eliot

---

From: Melinda Shore <[email-masked]>
Date: Mon, 20 Jun 2016 17:04:37 -0800

> It has, by some.  It has been challenged, by some, absent supporting data.

Yes, of course, but I'm mostly thinking of this in the context
of how we've framed arguments about meeting location, not whether
or not it's actually effective.

Melinda

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Jun 2016 18:08:56 -0700

> It is hard to have data without having performed the experiment.

Oh, it's much worse than that, since we do in fact periodically claim to 
conduct 'experiments'.  There is no attempt to formulate these 
'experiments' with any explicit hypotheses, measurements, or followup 
analysess.

There wasn't for Bits n' Bites, either.  And so on...

So the supposed outreach experiments (remember we did one in Adelaide, 
too?) can't be evaluated seriously.

d/

-- 

   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 21 Jun 2016 15:22:01 +1200

Hi Suresh,

Thanks for taking up your pen.

"North America, Europe, and Asia"

I believe that we need to define those terms. They all have built-in ambiguity.

Is North America the territories covered by NAFTA, or something less?
Or possibly the territories covered by ARIN?

Is Europe the territories of the EU, or something more? Or possibly
the territories covered by RIPE-NCC?

And where is "Asia"? Does it really stand for "Asia Pacific", or possibly
the territories covered by APNIC?

Regards
   Brian

On 21/06/2016 08:38, Suresh Krishnan wrote:
> _______________________________________________
> Mtgvenue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mtgvenue
>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Jun 2016 08:17:12 -0700

> I believe that we need to define those terms. They all have built-in ambiguity.

Brian,

In abstract terms, I understand why this might be appropriate.  However 
we've gone 25+ years without precise definitions being an issue, as far 
as I recall.

So, what problem would this additional effort solve or what benefit 
would it gain?  What problems might it introduce?

d/
-- 

   Dave Crocker

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 21 Jun 2016 18:28:19 +0000

Hi Melinda,
   Thanks for your comments. Please find responses inline.

On 06/20/2016 07:55 PM, Melinda Shore wrote:
> room in at least one working group.

I too have noticed this sometime and have pointed out and said something 
along the lines of "and to be confirmed on the mailing list, right?". This 
nudge always seemed to work. I sincerely believe that this can be fixed with 
chair training.

> This is, of course, ancillary

Agree.

> by meeting in new places we can attract new participants.

Yes. I have seen claims both ways (that it is a goal and it is a non-goal) 
and I was unsure whether to put it in here. I think it would be a fine thing 
to add if the community feels that it is the right thing to do.

> Again,

Sounds great. There are two parts to this. One is to document where we are 
right now and the other is talking about the way forward. Once we have an 
initial proposal what to shoot for in the future, I will preface it with the 
"will be" text.

Thanks
Suresh

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 21 Jun 2016 20:54:56 +0200

Dave,

I agree with Brian. For example, do we want to preclude meeting south of
the Rio Grande ever again? If not, then perhaps we should say “Americas”
rather than “North America”. Do we ever want to meet in Australia or New
Zealand? The last I checked, they aren’t in Asia. And how do you count a
meeting in Hawaii?

Cheers,
Andy

On Tue, Jun 21, 2016 at 5:17 PM, Dave Crocker <[email-masked]> wrote:

> we've gone 25+ years without precise definitions being an issue, as far as
> I recall.
>
> So, what problem would this additional effort solve or what benefit would
> it gain?  What problems might it introduce?
>
> d/
> --
>
>   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 22 Jun 2016 08:05:27 +1200

Answering Dave, I think that having undefined regions in the policy
will just lead to rather pointless disputes about whether city X
is in region Y or whether it's in region *.

Personally I would tend to prefer the same definitions that the
RIRs use. As it happens, the RIRs are also experienced in choosing
venues in their regions, so there might be some help for the Meetings
Committee there.

Regards
   Brian

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 21 Jun 2016 20:25:51 +0000

Hi Dave,

> It has, by some.  It has been challenged, by some, absent supporting data.

Yes. Exactly. And may I add it is very difficult to positively prove 
causation here. There are numbers showing an uptick in attendee participation 
from a given region when a meeting is in that region. There are also numbers 
showing an increase in the number of draft authors and drafts from a given 
region (Asia) after the IETF started meeting more frequently there. On the 
other hand, the participation from the LACNIC region has already been on the 
upswing before we met in BA. i.e. we already attracted the new participants 
and then we decided to meet where they were.

Thanks
Suresh

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 21 Jun 2016 22:17:50 +0000

Hi Brian,
   Thanks for your comments.

> I believe that we need to define those terms. They all have built-in ambiguity.

Before I started writing the draft, I was also thinking about defining them 
differently. The three regions currently in the draft seem to be the best fit 
for where we have been meeting until now.

> the territories covered by APNIC?

Right. I did not go into this detail frankly because I was not sure either. 
Going forward, I think we should define the regions based on what kind of 
granularity the community wants. One potential way I thought about broadly 
splitting them up was into "Americas", EMEA, and "Asia Pacific" but your RIR 
based scheme also makes sense to me.

Thanks
Suresh

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Jun 2016 15:25:40 -0700

> Zealand? The last I checked, they aren’t in Asia. And how do you count a

Andy,

Your note seems to demonstrate a basic disconnect from the detailed 
description of the 1-1-1-* model, as it has long been described and in 
fact how it is described in the draft.  In particular, I suggest 
reviewing the difference between venues chosen under 1-1-1 versus under *.

FWIW, I interpreted Brian's concerns as being different than what you 
stated.  His are about precision, rather than being about scope, which 
is how I read yours.

d/

-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Jun 2016 15:27:30 -0700

Brian,

While I'm not inclined to ever argue that the IETF community will find a 
way to avoid debating over the details of anything, I believe the 
problem you cite hasn't shown up -- or at least not shown up in force -- 
in 25 years.

While of course we can imagine it will, why do we need to anticipate it 
with extra effort (and distraction) now?

d/

-- 

   Dave Crocker

---

From: Suresh Krishnan <[email-masked]>
Date: Tue, 21 Jun 2016 22:31:40 +0000

Hi Dave,

> problem you cite hasn't shown up -- or at least not shown up in force --

And in addition, we probably need to add some text in there anyway to 
re-evaluate whatever choices we make based on changing participation in the 
future. I am personally fine either way (leave it as is or define in a more 
granular fashion) as I see merits and problems with each approach.

Thanks
Suresh

---

From: Ole Jacobsen <[email-masked]>
Date: Tue, 21 Jun 2016 15:33:41 -0700 (PDT)

On Tue, 21 Jun 2016, Dave Crocker wrote:

> While I'm not inclined to ever argue that the IETF community will find a way
> to avoid debating over the details of anything, I believe the problem you cite
> hasn't shown up -- or at least not shown up in force -- in 25 years.
> 
> While of course we can imagine it will, why do we need to anticipate it with
> extra effort (and distraction) now?
> 
> d/
> 

Since this document is meant to document "high level guidance for the 
meeting policy" it would seem more than appropriate to clearly define
what is meant by these commonly used terms.

Ole

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Jun 2016 15:48:56 -0700

On 6/21/2016 1:25 PM, Suresh Krishnan wrote:
> es. Exactly. And may I add it is very difficult to positively prove
> upswing before we met in BA.

Suresh,

When there is no target change specified, no meaningful discussion of 
evaluation criteria, no real data measurement and no meaningful data 
analysis, it is not 'very difficult'.  It's f'ing impossible to 
legitimately assert correlation, nevermind causation.  (cf, texts on 
multi-variate data analysis or on basic research methodology.)

    1.  Participation for the meeting in the special venue is the least 
interesting datum possible, if one wants to talk about 'benefits' from 
meeting there.

    2.  Asia participation was already on the upswing prior to our going 
there more frequently.  That was, for example, an essential point that 
prompted approving meeting in Beijing.  So we can't draw /any/ clear 
conclusions.

Post-hoc citations of bits of randomly-chosen data are an example of the 
problem with how we approach 'experiments'.  They have no foundation, no 
consensus, and often no real meaning.

d/

-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Jun 2016 16:04:24 -0700

> stated.  His are about precision, rather than being about scope, which

Re-reading the above, it doesn't look obvious what I meant:  I read 
Brian as asking for a precise definition of the boundaries for each of 
the 1's.  Andy's queries seemed to be about venues far outside of any of 
the current 1's.  For example, Buenos Aires isn't ever going to count as 
North America.

Hence, precision vs. scope.

d/

-- 

   Dave Crocker

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 22 Jun 2016 03:58:56 +0200

Dave,

You’re right, I did have some questions about scope, but, I’m also
concerned about precision. Where is Istanbul classified, Europe or Asia?
Where is Hawaii classified, North America or Asia/Pacific?

Brian’s suggestion of using the RIR guidelines, and also taking advantage
of their meeting expertise, makes sense.

Cheers,
Andy

On Wed, Jun 22, 2016 at 1:04 AM, Dave Crocker <[email-masked]> wrote:

> Re-reading the above, it doesn't look obvious what I meant:  I read Brian
> as asking for a precise definition of the boundaries for each of the 1's.
> Andy's queries seemed to be about venues far outside of any of the current
> 1's.  For example, Buenos Aires isn't ever going to count as North America.
>

---

From: Olaf Kolkman <[email-masked]>
Date: Wed, 22 Jun 2016 13:42:20 +0200

> the territories covered by APNIC?

If you are looking for a source to base a regional definition on then the UN's statistical department has a [Composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings](http://unstats.un.org/unsd/methods/m49/m49regin.htm). I think that is a reasonable neutral source for this sort of work.

I think, but won't argue strongly, that the 1-1-1-* translates into: 021-150-030-*

—Olaf

- - -
Olaf Kolkman                                                         [email-masked]
Chief Internet Technology Officer
Internet Society                                                     www.internetsociety.org
                                                                     LinkedIn: OlafKolkman
                                                                     Twitter: @Kolkman
- - -

---

From: Yoav Nir <[email-masked]>
Date: Sat, 25 Jun 2016 22:30:41 +0300

Hi, Suresh

> On 21 Jun 2016, at 9:28 PM, Suresh Krishnan <[email-masked]> wrote:
> 
> Hi Melinda,

For the regular IETF week meetings, probably so. And where chair training fails, the AD might say so explicitly.

Among the suggestions made on the IETF mailing list was to emphasize interim meetings at the expense of the regular IETF week meetings. Some extolled how well this has worked for several working groups, including httpbis (working on HTTP/2) and tls (working on TLS 1.3). I followed both of these efforts, but I could not justify the expense of going to those interim meetings. The way these work is that the meeting lasts for 1.5 to 2 days. That’s a lot more time than is available in IETF week. Inevitably, decisions are made in the first few hours. Inevitably, the discussion that follows is based on those decisions. Inevitably by the end of the second day more decisions are made. Sure, everything has to be confirmed on the list, but by then there’s a huge resistance to changing what has already been discussed so thoroughly and documented in both PRs and sample code. There’s a lot more invested in that than in the output of a two-hour meeting at IETF week.

I would not like it if we fixed the tendency to make decisions at the “big” meetings only to get this back much amplified at “interim” meetings.

Yoav