Subject: Re: [BOFCHAIRS] [vmeet] Remote participation

From: Steven Bellovin <[email-masked]>
Date: Fri, 30 Mar 2012 09:20:37 +0200

On Mar 28, 2012, at 10:22 PM, Jeffrey Hutzelman wrote:
> 
> 
> While I think we (and society in general) do tend to overuse slides in
> situations where they don't really add anything,

I'll note that folks who are non-native speakers of English are often
helped significantly by having slides projected.  I've certainly
seen many people taking pictures of the screen, back before early
upload was the policy.

		--Steve Bellovin, https://www.cs.columbia.edu/~smb

---

From: SM <[email-masked]>
Date: Fri, 30 Mar 2012 11:34:22 -0700

At 08:48 30-03-2012, John C Klensin wrote:
>(1) Almost any observation that is based on the advantages of
>slides by non-native speakers also applies to real-time minutes,
>transcripts, Jabber sessions, etc.  Done well, any or all of
>them may be easier to understand than a real-time speaker,
>especially one who is speaking too quickly.

If I recall correctly, a very rough estimate of the cost for 
providing real-time transcription of one week of sessions came to 
under $10,000.

Advantages are:

   (a) The discussions are more accessible for non-English (local and
       remote) participants

   (b) Local participants in another session can follow what is going on

   (c) The transcript can be used to produce minutes for the session

   (d) The transcript can be read by people who have not attended the session

   (e) The transcript can be (easily) translated by the reader

   (f) It does not require an additional screen

   (g) A minute-taker is no longer required

   (h) Less pressure to upload slides before the session

   (i) The content of a session is searchable

Disadvantages are:

   (a) It is not always possible for the person transcribing to capture
       the name of the speaker or some technical words

   (b) It's an additional cost

   (c) There can be network connectivity issues

   (d) There may be an indirect negative impact on attendance revenue

   (e) It may be workable for regular meetings only

   (f) The content of a session is searchable

 From the IETF 82 proceedings:

   Technical plenary http://www.ietf.org/proceedings/82/minutes/plenaryt.txt
   AppArea           http://www.ietf.org/proceedings/82/minutes/appsawg.txt
   IntArea           http://www.ietf.org/proceedings/82/minutes/intarea.txt
   OpsArea           http://www.ietf.org/proceedings/82/minutes/opsawg.txt
   RtgArea           http://www.ietf.org/proceedings/82/minutes/rtgwg.txt
   SecArea           http://www.ietf.org/proceedings/82/minutes/saag.txt
   TsvArea           http://www.ietf.org/proceedings/82/minutes/tsvwg.txt

The quality of reporting varies widely.  I suggest reading the 
proceedings for an area you are involved in to get an idea of the 
remote participant experience.  For English as a foreign language, 
pick an area you are not involved in.

Regards,
-sm

---

From: SM <[email-masked]>
Date: Fri, 30 Mar 2012 16:18:57 -0700

Hi John,
At 15:14 30-03-2012, John C Klensin wrote:
>Just so we understand what we are talking about, that means all
>eight tracks, plus the plenaries?  My recollection of how much

Yes.

>skilled transcribers can do at a time combined with the length
>of our day, is that would mean at least 10 and probably 12-15,
>transcription people.  Is that roughly what the figure you give
>is based on?  And does it include airfare and hotels for those
>people (noting that they would, e.g., outnumber Secretariat
>staff for the duration of a meeting.

No, I meant doing the transcription remotely to avoid the expenses of 
airfare and hotels.  Skilled transcribers are more expensive.  BTW, I 
looked up the figures over a year ago.

>It does if people who are in the room want to see the
>transcript.  Experience at multiple IETF plenaries and several
>other organizations (including ICANN) indicates that they
>usually do.  The other advantage of transcripts is that they can
>be used by folks who don't hear well.  If they are in the room,
>that certainly requires an extra screen, at least unless you
>expect that population to all bring laptops and spend their time
>hunched over them.

The screen may not be as visible from the back of the room.  If 
people in the room want to see the transcript, they bring their 
laptop.  If you can follow the conversation, you don't need the 
transcript.  The service could be optional; it's up to the session 
chair to make the call.

>I don't think so.  For non-native speakers of whatever language
>the speakers are speaking, ability to review slides in advance
>is a big help.

Ok.

>If the speaker doesn't speak English really clearly, or speaks
>it very quickly, or both, the quality of transcription can
>easily fall below the level needed to make the above advantages
>real.  In that case, the problem is not just speaker names and
>technical words

Yes.  If the quality of transcription falls below an acceptable 
level, the question that arise is whether the people in the room have 
understood what they heard.  But let's not go there as we'll end up 
trying to solve a lot of problems.

After reading 
http://www.ietf.org/mail-archive/web/oauth/current/msg08692.html my 
opinion is that having a transcription might help in such cases.

Regards,
-sm

---

From: SM <[email-masked]>
Date: Sat, 31 Mar 2012 03:49:07 -0700

Hi John,
At 19:55 30-03-2012, John C Klensin wrote:
>Interesting idea.  Of course, if the audio fails, we would
>essentially have what used to be called a domino effect: if
>minutes and everything else depended on the transcribers, the
>transcribers were remote, and the audio went out, the WG might
>reasonably adjourn to the nearest location dispensing coffee or
>stronger beverages.

If the audio fails, you have audio recordings to work from.  It's up 
to the session chair to assess the situation and figure out what to 
do.  It is reasonable for the WG to adjourn.  It is also reasonable 
for the AD to shut down the WG if it's not delivering.

>I worry a little about expecting people to read transcripts on
>their laptops for three reasons:
>
>         (i) Trying to keep track of the laptop screen largely
>         prevents looking at the slides which, we would like to
>         hope, actually contain information.  It might interfere
>         with Jabber tracking even more.

Yes.

>
>         (ii) It might discourage getting up and standing in
>         microphone lines, etc., to comment and participate.

The transcription might provide some statistics about mic attendees, 
how much time they spend at the mic, whether the comment actually 
helps to resolve the issue being discussed.

In a session on Friday, overall, the people who stepped to the mic 
line are from North America.  There was only one non-English 
speaker.  Around 83% of the IETF output is from English 
speakers.  That may fall down to around 60% in the near future.  If 
we ignore the presentations, the numbers for standing up in 
microphone lines do not tally with the trend.

Nothing in what I suggest is going to fix that.

>         (iii) For a WG Chair (or equivalent) looking into the
>         audience and seeing who is huddled over their laptops,
>         presumably reading mail and who is paying attention.
>         Having a significant number of people huddled over their
>         laptops reading transcripts fouls up that particular bit
>         of inference because that group is mentally present in
>         the WG, while most of the mail-readers are not.

There is a significant number of tourists in a WG session.  If a 
person is not paying attention, he/she shouldn't be surprised when 
the minutes shows that he/she contributed to some decision.

BTW, I forgot to mention that some of the tourists are a bit more 
than that.  They take notes of what goes on in a session for internal 
reporting.

At 01:19 31-03-2012, Marshall Eubanks wrote:
>but  meeting consensus can have a big influence on mailing list
>consensus.)

Yes.

>The result of what we are doing here will I think inevitably at some
>point change the answer to no. And, after that, you are not having a
>WG meeting, just some sort of seminar.

If we exclude WG adoption of drafts, how many decisions are taken (to 
be confirmed on the list) in a meeting?

  OAUTH     5
  MANET     4
  TLS       3
  KITTEN    2
  DHC       2
  P2PSIP    2
  SIPCORE   2
  XRBLOCK   2
  WEBSEC    1
  ANTITRUST 1
  INSIPID   1
  KERBEROS  1
  ISIS      1
  JOSE      0
  IPSECME   0
  PKIX      0
  MILE      0
  NEA       0
  ABFAB     0
  DANE      0

The metrics used do not paint a true picture of the sessions.

Regards,
-sm

---

From: Ted Lemon <[email-masked]>
Date: Mon, 2 Apr 2012 14:32:45 +0000

> The metrics used do not paint a true picture of the sessions.

In the case of DHC at least, the metrics are pretty accurate.   There was barely time in the wg meeting to actually have any meaningful discussion about anything, so each presentation wound up being "sales pitch, a few comments, chair interrupts for time, next steps, next presentation."   It wasn't really a good use of f2f meeting time, and I'd like to do it differently at the next IETF, but I don't really know how, because there weren't a lot of presentations that the working group shouldn't have heard, and we got one of the nice like 2.5h time slots.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 02 Apr 2012 10:58:47 -0400

On Mon, 2012-04-02 at 14:32 +0000, Ted Lemon wrote:
> In the case of DHC at least, the metrics are pretty accurate.   There
>  was barely time in the wg meeting to actually have any meaningful
>  discussion about anything, so each presentation wound up being "sales
>  pitch, a few comments, chair interrupts for time, next steps, next
>  presentation."   It wasn't really a good use of f2f meeting time, and
>  I'd like to do it differently at the next IETF, but I don't really
>  know how, because there weren't a lot of presentations that the
>  working group shouldn't have heard, and we got one of the nice like
>  2.5h time slots.

Maybe you're not being conservative enough about what is a presentation
the working group should hear.  I generally try to avoid "presentations"
at all, reserving meeting time for things that actually require
face-to-face time in order to get work done and move things forward.  I
regularly turn down requests for meeting time to present some new work
or proposed work to the WG that hasn't been discussed on the meeting
list.

We still don't take a lot of decisions in any one meeting, but the ones
we do are often things for which discussion took a fair amount of time.

I don't claim to follow this principle perfectly, but it is something I
try to do, because I hate sitting in a meeting with a full agenda and
not getting work done because someone is droning on about something we
could have learned by reading a draft or having a discussion on the
mailing list.

-- Jeff

---

From: John C Klensin <[email-masked]>
Date: Mon, 02 Apr 2012 11:36:22 -0400

--On Monday, April 02, 2012 14:32 +0000 Ted Lemon
<[email-masked]> wrote:

> In the case of DHC at least, the metrics are pretty accurate.
> There was barely time in the wg meeting to actually have any
> meaningful discussion about anything, so each presentation
> wound up being "sales pitch, a few comments, chair interrupts
> for time, next steps, next presentation."   It wasn't really a
> good use of f2f meeting time, and I'd like to do it
> differently at the next IETF, but I don't really know how,
> because there weren't a lot of presentations that the working
> group shouldn't have heard, and we got one of the nice like
> 2.5h time slots.

Ted,

When the pressure for more and more meeting time is considered
in the light of my experience of the last week, I've started
wondering whether it is time to move toward a fundamental
change.  Suppose that, instead of having presentations at the
IETF meetings, we did the following:

(i) Encourage presentations to be prepared as a sequence of
slides accompanied by either written narrative or audio (or
possibly video) and made available with a cutoff not later that
the cutoff for new I-Ds and ideally earlier.   If people then
started discussing them on the relevant WG mailing lists, that
would strengthen the case for (ii) below.   

Note that recorded audio or video presentations have added
advantages: People who need a little extra time to understand
them could take that time (or back up and replay if needed),
early availability of the slides would be guaranteed, and asking
questions on mailing lists is both open-ended (no need to limit
the microphone line), more useful for anyone who has trouble
understanding English in real time, and more accessible for
anyone who can't easily translate their questions into English
in real time.  Those who hadn't really understood the relevant
I-Ds could view the presentation, read the I-D, and view the
presentation again.  And so on.

(ii) With presentations (all parts of each one, not just the
slides) available weeks before an actual IETF meeting, we would
ask the presenters to demonstrate to WG Chairs that their
subject matter (perhaps with a one or two minute review of
context) would benefit from f2f discussion time.  The strongest
demonstration would obviously be mailing list discussion after
people had looked at the presentation.

Obviously that would help the remote folks as well because
presentations could be studied at leisure and questions queued
up for f2f discussions before the relevant session even started.
Having someone who would be spending time making a presentation
under today's model instead start by responding to a series of
prepared audience questions (from both the on-site and remote
audiences) might also be a much better use of WG time.

There are lots of details associated with that sort of approach
that I haven't even started to think about, but, if it worked, I
think it could dramatically improve our overall productivity,
reverse the trend toward more presentations and work at meetings
in preference to real work on mailing lists, and reduce pressure
on meeting slots (WGs who have nothing to actually discuss cold
be denied f2f session time). WG Chairs would need discretion to
give time to things that they were confident were in the
critical path and there might need to be a fast-track appeal
procedure when presentation time was denied.   

If we could get it into our collective heads that audio or video
presentations with pre-distributed slides might actually be more
effective, it might help in a lot of ways.   Conversely, if
people needed to show that they were going to be in front of the
room to get permission to travel to IETF, it would be, IMO, far
more helpful to all concerned to have that role be "lead
discussion of presentation and I-D" rather than "give
presentation".

best,
   john

---

From: Joel jaeggli <[email-masked]>
Date: Mon, 02 Apr 2012 18:34:54 +0200

> would strengthen the case for (ii) below.   

What we've been encouraged to do, e.g. use the presentation/meeting
cycle for the problems, not the material is fundamentally at odds with
this approach. We're there to have a discussion and the slides are there
to support that. they should not be there to imho to present the
contents of the draft. We had the mailing list and the drafts themselves
to form the framework for the meeting.

> effective, it might help in a lot of ways. 

As a quite bad amateur ietf video producer, I'm going to go out on a
limb and say that a few hundred hours per meeting of cooked slides and
audio is not going to improve the productivity of the working groups
unless you're measuring that per byte.

> Conversely, if

it should be today.

---

From: John C Klensin <[email-masked]>
Date: Mon, 02 Apr 2012 14:26:46 -0400

--On Monday, April 02, 2012 18:34 +0200 Joel jaeggli
<[email-masked]> wrote:

>...
>> possibly video) and made available with a cutoff not later
>> that the cutoff for new I-Ds and ideally earlier.   If people
>> then started discussing them on the relevant WG mailing
>> lists, that would strengthen the case for (ii) below.   
 
> What we've been encouraged to do, e.g. use the
> presentation/meeting cycle for the problems, not the material
> is fundamentally at odds with this approach. We're there to
> have a discussion and the slides are there to support that.
> they should not be there to imho to present the contents of
> the draft. We had the mailing list and the drafts themselves

Joel, I apologize if I wasn't clear, but I was suggesting using
those previously-prepared presentations for online access at the
convenience of participants _before the IETF f2f meetings_ and
eliminating them from those meetings entirely.  If that is
"fundamentally at odds" with focusing the face time on problems
rather than presentations, I don't see how.

>> If we could get it into our collective heads that audio or
>> video presentations with pre-distributed slides might
>> actually be more effective, it might help in a lot of ways. 
> 
> As a quite bad amateur ietf video producer, I'm going to go
> out on a limb and say that a few hundred hours per meeting of
> cooked slides and audio is not going to improve the
> productivity of the working groups unless you're measuring
> that per byte.

And, again, I'm suggesting cutting the amount that appears
during the meeting, taking up meeting time, to nearly zero.

>> Conversely, if
>> people needed to show that they were going to be in front of
>> the room to get permission to travel to IETF, it would be,
>> IMO, far more helpful to all concerned to have that role be
>> "lead discussion of presentation and I-D" rather than "give
>> presentation".
> 
> it should be today.

Agreed.  But that isn't happening in presentation-heavy WGs.

   john

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 02 Apr 2012 14:50:48 -0400

> rather than presentations, I don't see how.

John,

I don't know about you, but I have enough trouble getting ready for a 
meeting now, reading all the latest draft versions, ramped up email 
traffic, etc.

What I *don't* need during meeting prep is to watch a bunch of videos 
about each draft on the agenda!!! Perhaps the world has decided that 
people are now too illiterate to read and must watch TV instead, but I'm 
trying to avoid that where possible. IMO reading is much more efficient 
for most things.

Certainly there *are* some things where multimodal presentation is 
preferable, and that is what the meetings (regular and interim) are for.

	Thanks,
	Paul

---

From: Melinda Shore <[email-masked]>
Date: Mon, 02 Apr 2012 10:53:30 -0800

> to form the framework for the meeting.

Hear, hear.  This strikes me as a problem that can be addressed through
better working group chair training and not through technology.

Melinda

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 2 Apr 2012 20:53:29 +0200

On Apr 2, 2012, at 8:50 PM, Paul Kyzivat wrote:
> I don't know about you, but I have enough trouble getting ready for a meeting now, reading all the latest draft versions, ramped up email traffic, etc.
> 
> What I *don't* need during meeting prep is to watch a bunch of videos about each draft on the agenda!!!

+6.02e23.

--John

---

From: Al Morton <[email-masked]>
Date: Mon, 02 Apr 2012 15:18:12 -0400

>preferable, and that is what the meetings (regular and interim) are for.
Yes, the meetings are for *interaction*. During what might start-out
as a presentation we have the opportunity to customize the content by
asking questions, and chairs can guide speakers ("you need to skip to
the issues where discussion is required now, because time is short.")

make the meetings work, not more work,
Al

---

From: John C Klensin <[email-masked]>
Date: Mon, 02 Apr 2012 15:36:59 -0400

> better working group chair training and not through technology.

I agree in principle.  But many WG chairs seem incapable of
controlling meetings and restricting the time allowed in
practice for long, sometimes incomprehensible, presentations
with little information content.   We've been saying "discussion
and interaction, not presentations" long enough and often enough
that I doubt those presentations are due to lack of training in
most cases.  

Now, I think that if WGs that couldn't get discussions focused
were shut down (if necessary over the objections of their
Chair(s)) or if ADs started denying meeting slots at IETF N+1 to
WGs who used the vast majority of time at IETF N for
presentations that did not generate discussion, learning would
set in rather quickly.  But a catalogue of how often either had
been done in, say, the last decade wouldn't be very long .

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 02 Apr 2012 15:55:56 -0400

--On Monday, April 02, 2012 14:50 -0400 Paul Kyzivat
<[email-masked]> wrote:

>...
> I don't know about you, but I have enough trouble getting
> ready for a meeting now, reading all the latest draft
> versions, ramped up email traffic, etc.
> 
> What I *don't* need during meeting prep is to watch a bunch of
> videos about each draft on the agenda!!! Perhaps the world has
> decided that people are now too illiterate to read and must
> watch TV instead, but I'm trying to avoid that where possible.
> IMO reading is much more efficient for most things.

Good answer.  I bet you even know what to do about that, if only
because you took a several year course in drinking from a fire
hose.  I'd assume you read the drafts and complain on the WG
list that you want to know if there is anything in the
presentations that isn't in the draft and, if so, what it is and
why it isn't in the draft(s).   Possibly you also do what a very
significant fraction of the community seems to do when they find
themselves sitting in a WG meeting facing a presentation that
just reprises parts of an I-D that you/they have read, namely go
read email or try to follow another WG in parallel.

I had really hoped it would be clear where I was going, but let
me be very explicit:

(i) I think there are multiple reasons why we keep getting more
and more requests for time at IETF, thereby pushing on evenings,
more of Friday, etc.  I think one of the major reasons among
that lot is that we are wasting a lot of the time that is
allocated... and I believe a presentation that is not directly
linked to, and focused on, discussion that the presenter and WG
leadership expect to be usefully occur during that session.

(ii) I think we should prefer that no new information show up in
presentations.  New organization of material, new summaries to
focus discussion or summaries of prior discussions, definitely.
But anything that could go into an I-D should go into an I-D.
That is just much better for the process as we claim it works.

(iii) If someone needs to prepare and provide a presentation to
justify their presence and existence at IETF, I think we should
arrange for them to do that without taking up meeting time (or
the time of WG participants who actually read drafts).  Videos
available well in advance might serve some of the same purpose
that "Poster Sessions" serve at many scientific meetings... and
that purpose is not to provide information that is critical to
understanding the mainstream presentations or discussions.  If
you, me, or 900 other people don't pay any attention to those
videos, the amount of critical-path information lost is likely
to be insignificant.

> Certainly there *are* some things where multimodal
> presentation is preferable, and that is what the meetings
> (regular and interim) are for.

Yep.

best,
   john

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 02 Apr 2012 21:02:21 +0100

> that I doubt those presentations are due to lack of training in

As it happens, I don't recall being at any such WG session
in Paris. Could it be that this is mostly an ex-problem? If
not, then can someone point me at some WGs that did have this
problem at IETF-83? (On-list or off-list, whatever.)

S

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 2 Apr 2012 16:07:45 -0400

> WGs who used the vast majority of time at IETF N for

If we're going to make ADs start to micromanage WGs (presumably in the
12 remaining spare minutes in the year they have), why not start with
something simpler?  For instance, "No WG gets more than 2.5 hours per
meeting, period, no exceptions" might focus the mind.  Pick some other
arbitrary time if you like.  The idea is that the scarcity of resource
will make time management more important; and that if the claim is,
"We really need this time anyway because of all the work we took
on," perhaps the response ought to be, "You should have taken less
work.  Reschedule some of it later.  People can't follow what you're
doing."

I have no idea whether this will work, and I am not really a fan of
yet more rules.  But one could make a compelling argument that the
IETF week is a shared resource, and it'd be a mistake to add another
track or stretch into Friday afternoons, since that makes everyone pay
for the management practices of some.  As a side benefit, it might
make remote participation work better too.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 02 Apr 2012 16:23:32 -0400

Frankly, it perhaps doesn't matter if we get the meetings to use the 
meeting time more efficiently. I think the highest value things that 
meetings provide are:

1) provide an artificial, but compelling, periodic deadline for 
progressing drafts, and for others to catch up on the status of those 
drafts.

2) force the primary participants to set aside their day job and 
personal concerns for a week and focus almost exclusively on the ietf tasks

3) all get on a common time zone for a week, and facilitate informal 
"hallway" conversations.

The official agenda, and the resulting proceedings, provide the formal 
justification to employers for the above. As long as they are working 
well enough to accomplish that maybe they are doing well enough.

	Thanks,
	Paul

---

From: Joel Jaeggli <[email-masked]>
Date: Mon, 02 Apr 2012 21:02:12 +0200

> Agreed.  But that isn't happening in presentation-heavy WGs.

We got 4.5 hours of meeting during 83.  If we can do it but some other
wg's can't then no technological solution is likely needed.

>    john
> 
> 
> _______________________________________________
> NOTE WELL: This list operates according to
> http://mipassoc.org/dkim/ietf-list-rules.html.
> https://www.ietf.org/mailman/listinfo/vmeet

---

From: SM <[email-masked]>
Date: Mon, 02 Apr 2012 12:24:29 -0700

Hi Ted,
At 07:32 02-04-2012, Ted Lemon wrote:
>In the case of DHC at least, the metrics are pretty 
>accurate.   There was barely time in the wg meeting to actually have 
>any meaningful discussion about anything, so each presentation wound 
>up being "sales pitch, a few comments, chair interrupts for time, 
>next steps, next presentation."   It wasn't really a good use of f2f 
>meeting time, and I'd like to do it differently at the next IETF, 
>but I don't really know how, because there weren't a lot of 
>presentations that the working group shouldn't have heard, and we 
>got one of the nice like 2.5h time slots.

There are two points in the above:

  (i)  Not enough time to have a meaningful discussion about anything

  (ii) The working group should have heard most of the presentations

One of the alternatives is to have a longer slot.  Another 
alternative is to have a 50/50 rule; the chair keeps half of the time 
for discussions and let presenters make their pitch in the other 
half.  If there are too many presentation proposals, the chair would 
have to figure out which ones to drop, or it could be first X who 
have all their material ready are on the agenda.

Regards,
-sm

---

From: Ted Lemon <[email-masked]>
Date: Tue, 3 Apr 2012 09:55:13 +0000

It's difficult to set a good balance here.   When you have a bazillion different proposals, it's not really shocking that not everybody has read every proposal.   Even if you've read every proposal, it may be difficult to remember the details of each one if you didn't read them this week.   In that context, you'd like a presentation that explains what the proposal is, as a sales pitch to get people to read the document, and as a refresher for people who read the document a month ago and read two dozen more documents since then.   Then you'd like people who haven't read the documents to read the ones they've decided are worth the time.   Then you'd like to have a discussion, because you'd have a group of people who were sufficiently up to speed to *have* a discussion.

But to do this would require more time than any working group has, and harder agenda planning than the IETF can reasonably do.   The wish to keep presentations completely out of working group meetings isn't realistic, but after this meeting I feel like we (the DHC wg) need to do something differently.

BTW, I should point out that discussion-only sessions aren't always good.   Sometimes you have people with strong opinions who aren't open to debate on those opinions, and then you get an incredibly time-wasting "discussion" at the microphone, in which no forward progress is made.   So a disproportionate amount of working group time can spent "discussing" relatively minor proposals while the serious work of the working group doesn't get discussed due to time constraints.

This is a hard problem.

For DHC, I think that in the future presentations have to be very short, like maybe 1 minute or maybe 2 minutes for a really complex draft.   Essentially, they need to be just slightly longer than an elevator pitch.   The remainder of the slot needs to be spent on discussion.   Drafts that are contentious should be discussed offline, because no actual design discussion is going to happen anyway.   The wg chairs need to use their judgment on this--there's a very fine line between a contentious discussion that can actually produce results, and one that won't.

I'm thinking the presentation on the projector needs to have a timer on it, too.

---

From: SM <[email-masked]>
Date: Tue, 03 Apr 2012 04:30:00 -0700

Hi John,
>controlling meetings and restricting the time allowed in

Most WG chairs do a decent job controlling the session.

>and interaction, not presentations" long enough and often enough

Some people find the presentations worthwhile, others don't.  Some 
people need it to justify the travel.

Sometimes there is a nine-minute presentation followed by one minute 
for discussion.  After the first few minutes, glancing at the list 
archive confirms that it's a sales pitch and that the mailing list 
has been lethargic over the last six months.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Tue, 03 Apr 2012 10:55:35 -0400

> Most WG chairs do a decent job controlling the session.

And some don't.  Let me put this personally -- while we haven't
had an product sales pitches, Joseph and I have had considerable
trouble in EAI controlling the kinds of presentations that take
nine or ten minutes that are completely redundant with what has
already occurred on the mailing list and that produce just about
zero (using your example, a minute or less) of discussion.  I
think that is a waste of meeting time, to the point that we
cancelled the EAI f2f meeting in Paris.  But, justifiably or
not, I'm got the reputation of being rather ruthless about such
things.  I feel as if a larger collection of tools, such as
agreement that it would be reasonable to force such
presentations onto the net with at most one minute, no slide,
summaries a f2f meetings, would help.  

At some level, I don't care as much whether that alternative is
required or merely an expansion of Chair options.  But, if it
were an option, then it would be also possible to push back on
requests for long (or multiple) meeting slots that might be
presentation-heavy.
 
>> with little information content.   We've been saying
>> "discussion and interaction, not presentations" long enough
>> and often enough
> 
> Some people find the presentations worthwhile, others don't.
> Some people need it to justify the travel.

> Sometimes there is a nine-minute presentation followed by one
> minute for discussion.  After the first few minutes, glancing
> at the list archive confirms that it's a sales pitch and that
> the mailing list has been lethargic over the last six months.

Yes, exactly.

I recognize that I'm pushing, possibly out of nostalgia, for an
IETF that gets most of its work done on mailing lists and for
which we should say to a WG (not always or rigidly, but often)
"if you aren't getting work done on your mailing list, then you
aren't getting work done and maybe you should be closed rather
than getting more meeting slots at IETF".  There will always be
exceptions but, when someone says something like "we had 4.5
hours at IETF and got a lot of work done", my personal reaction
isn't "hmm, I wonder how much more work that WG could do if
given more time" but "I wonder what is wrong that requires that
much meeting time to make progress" with possible answers
including (but not limited to) "not enough happening on the
mailing list between meetings" and "too much work assigned to
that WG and it is trying to do all of it in parallel".  

FWIW, I think that "if we had even more meeting time, we could
get more work done" and its frequent corollary "the amount of
work we can get done is bounded by how much meeting time we get"
tends toward a situation in which WGs or their organizational
equivalents meet for several days or a week with little
expectation of anything getting done between meetings other than
meeting preparation.  Many of us have seen organizations that
work that way, some of them are actually quite effective.
Whether it is desirable for the IETF to go in that direction or
not, I think that, if we are actually going in that direction,
we should be reviewing procedures to see if they are an
appropriate match for it --trying to push back the tide is not
usually very productive.

     john

---

From: Brian Rosen <[email-masked]>
Date: Tue, 3 Apr 2012 11:03:14 -0400

Personally, I prefer dealing with reality, rather than relying on what ideally should be.

Ideally, everyone in the room has read the document, followed the mail list discussion, and is aware of the open issues.

In reality, we are lucky to have half the room in such a state.

I usually choose to allow a presenter to spend some time summarizing where the document is before moving on to the open issues.  If I see half the presentation dealing with "level setting", then I'll take action, but that's just me.

Brian

---

From: "George, Wes" <[email-masked]>
Date: Tue, 3 Apr 2012 16:50:01 -0400

I might add one more...

Promote "WG tourism" and cross-pollination.

With this many WGs, it's simply impossible to be actively involved in every WG list for every piece of the IETF that I think might be applicable to my employer and my skill set. My hope with the meetings is that I have enough "free" timeslots (i.e. not occupied by groups that I'm actively involved in) that I can sit in on a few other groups that I'm not actively involved in and get a sense for what they're doing, maybe provide some ad hoc feedback. I've had failures, where I go to a WG meeting and can't understand enough of what's going on to contribute, but I've also had some successes, where I find something else that I can provide feedback on or that I should be more actively watching. This is probably pretty important in order to attract new folks to the list so that additional work can get done, and probably should be a consideration when we talk about format for presentations and best use of discussion during F2F meetings. I know that means that you spend some time getting those who haven't read the drafts or aren't actively involved up to speed, but I thought that it was worth mentioning as something else that distinguishes the meetings from the list.

Thanks,

Wes George

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Paul Kyzivat
> Sent: Monday, April 02, 2012 4:24 PM
> To: John C Klensin
> Cc: [email-masked]
> Subject: Re: [vmeet] [BOFCHAIRS] Remote participation
>

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: Stephen Morris <[email-masked]>
Date: Tue, 03 Apr 2012 22:59:33 +0100

On 03/04/2012 21:50, George, Wes wrote:
> With this many WGs, it's simply impossible to be actively involved 
> in every WG list for every piece of the IETF that I think might be 
> applicable to my employer and my skill set. My hope with the 
> meetings is that I have enough "free" timeslots (i.e. not occupied 
> by groups that I'm actively involved in) that I can sit in on a
> few other groups that I'm not actively involved in and get a sense
> for what they're doing, maybe provide some ad hoc feedback.

+1 to that.

But for that to happen, it does mean that presentations do have to take
account the fact that not everyone has read the draft.  Which in turn
means some "scene setting" in a presentation before getting down to the
open issues.  (In fact, in my WG I sometimes get a bit depressed how few
hands are raised to the question "who here has read the draft?".)

Stephen

---

From: Dean Willis <[email-masked]>
Date: Tue, 3 Apr 2012 19:37:44 -0500

On Apr 2, 2012, at 3:23 PM, Paul Kyzivat wrote:

> Frankly, it perhaps doesn't matter if we get the meetings to use the meeting time more efficiently. I think the highest value things that meetings provide are:
> 
> 1) provide an artificial, but compelling, periodic deadline for progressing drafts, and for others to catch up on the status of those drafts.
> 
> 2) force the primary participants to set aside their day job and personal concerns for a week and focus almost exclusively on the ietf tasks
> 
> 3) all get on a common time zone for a week, and facilitate informal "hallway" conversations.
> 
> The official agenda, and the resulting proceedings, provide the formal justification to employers for the above. As long as they are working well enough to accomplish that maybe they are doing well enough.

That's a profoundly astute and brilliant observation. I've run gong-show meetings, meetings with nothing but intense focused discussion, and meetings where the entire greybearded DNS mafia showed up to pelt me with fruit. None has been profoundly more or less effective at achieving the desired goal of furthering consensus building on the problems at hand, because most of this actually happens outside the meeting room (quite often during the meeting when I throw out people having side conversations).

And that's why virtual meetings are so hard to get right. Although we might could remote the fruit-throwing part as John suggested.

--
Dean

---

From: SM <[email-masked]>
Date: Tue, 03 Apr 2012 16:40:54 -0700

Hi John,
>zero (using your example, a minute or less) of discussion.  I

Allowing more time for discussion or a video of the presentation 
won't change much.  From the above, the presenter is either not 
following the mailing list discussions, the presenter does not 
understand the mailing list discussions, or the presenter is using 
that nine minutes for other purposes.  There may also be cultural issues.

Regards,
-sm

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 04 Apr 2012 10:05:57 -0400

On 4/3/12 4:50 PM, George, Wes wrote:
> With this many WGs, it's simply impossible to be actively involved in every WG list for every piece of the IETF that I think might be applicable to my employer and my skill set. My hope with the meetings is that I have enough "free" timeslots (i.e. not occupied by groups that I'm actively involved in) that I can sit in on a few other groups that I'm not actively involved in and get a sense for what they're doing, maybe provide some ad hoc feedback. I've had failures, where I go to a WG meeting and can't understand enough of what's going on to contribute, but I've also had some successes, where I find something else that I can provide feedback on or that I should be more actively watching. This is probably pretty important in order to attract new folks to the list so that additional work can get done, and probably should be a consideration when we talk about format for presentations and best use of discussion during F2F meetings. I know that means that you spend some time ge
 tting tho
se who haven't read the drafts or aren't actively involved up to speed, but I thought that it was worth mentioning as something else that distinguishes the meetings from the list.

I do *try* this. In fact I try it practically every meeting. But I find 
it is rarely helpful. IMO you really must have done your homework prior 
to a WG meeting in order to get any real sense of what is going on.

But perhaps it is possible to get a sense of whether a WG is functioning 
well, and hence worth investigating.

	Thanks,
	Paul