Subject: Request For IETF 77 WG and BOF Sessions Agenda, Minutes, and Presentations

From: "Wanda Lo" <[email-masked]>
Date: Wed, 12 May 2010 09:40:03 -0700

Dear WG Chairs and BOF Chairs,

We are in the process of compiling the Proceedings for IETF 77 and we are
still missing meeting minutes from various sessions. The cutoff for
submissions has past and correction submissions cutoff is today, May 12,
2010, 5 PM PT.  Please upload meeting minutes, as well as any presentations
from your sessions, as soon as possible using the Meeting Materials Manager
found here: https://datatracker.ietf.org/cgi-bin/wg/wg_proceedings.cgi.
Alternatively, you are welcome to send them to [email-masked] or
[email-masked] for manual posting.

As of today, I'm still missing minutes from:

Wednesday's Plenary

Thursday's Plenary

Application Area

-APPAREA

-CORE

-HTTPSTATE

-OAUTH

-VWRAP

Internet Area

-6LOWPAN

-SOFTWIRE

Operations Area

-DNSOP

Routing Area

-KARP

Security Area

-KRB-WG

Transport Area

-LEDBAT

IRTF Groups

-DTNRG

-RRG

Thanks,

Wanda

 

===========================

Wanda Lo

Internet Engineering Task Force (IETF)

48377 Fremont Blvd., Ste. 117 
Fremont, California 94538 / USA 
+1-510-492-4082 
+1-510-492-4001 (fax) 
 <http://www.ietf.org> www.ietf.org

-- 

Managed by Association Management Solutions (AMS)

Forum Management, Meeting and Event Planning

 <http://www.amsl.com/> www.amsl.com

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 12 May 2010 10:51:59 -0700

>As of today, I'm still missing minutes from:

...followed by about 10% of the WGs that met.

Maybe those WGs that don't turn in their minutes for IETF n should get a blue dot with a slice cut out at IETF n+1? Or the IESG member who had the highest percentage of no-minutes groups gets noted in the plenary slides? Or... ?

--Paul Hoffman, Director
--VPN Consortium

---

From: "James M. Polk" <[email-masked]>
Date: Wed, 12 May 2010 13:32:05 -0500

At 12:51 PM 5/12/2010, Paul Hoffman wrote:
>Maybe those WGs that don't turn in their minutes for IETF n should 
>get a blue dot with a slice cut out at IETF n+1? Or the IESG member 
>who had the highest percentage of no-minutes groups gets noted in 
>the plenary slides? Or... ?

perhaps an analysis should be done wrt how many WGs are repeat 
offenders of this tardiness... perhaps a pattern will emerge...

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 May 2010 15:07:38 -0400

> offenders of this tardiness... perhaps a pattern will emerge...

Yes, probably.   The pattern will be that IETF volunteers are overworked, and are inconsistent in their ability to meet IETF deadlines.   DHC minutes were late for IETF 75, early for IETF 76, and late for IETF 77.   If your theory is correct, presumably they will be early again after Maastricht.

It's unfortunate when volunteers don't have time or energy to do work to meet a deadline, but it's hardly surprising.   I'm sure it's fun to judge and moralize, but it doesn't do anything to improve the situation.

The one thing that would actually improve the situation, at least for me, would be for someone to professionally transcribe the audio from the meeting.   Volunteer minutes-takers are greatly appreciated, and often do good work, but it's hard to get good minutes in a meeting where time is so short:

Events move faster than most people type
It's easy to get distracted trying to figure out peoples' names, which they often neglect to state
A lot of participants do not speak english using the accustomed phonemes, so you need to understand what they're talking about in order to parse what they are saying in realtime
Anybody who is that familiar with the topics being discussed is probably a participant in the discussions, and thus will miss taking notes on those portions of the discussions where they are standing at the mic

The result is that even well-intentioned note-takers often do not produce notes that can be used to generate useful minutes for a meeting.   Exceptions are much appreciated, but rare.

I wound up having to sit and re-transcribe the minutes for the IETF 77 meeting, which was painful on many levels, and I will freely admit that I procrastinated for far longer than I spent actually doing the work.   Arguably a full transcript is overkill for minutes, but it's very useful for generating summaries.   I'm very happy with how the DHC minutes came out this time, but preparing them was brutal.

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 12 May 2010 12:41:00 -0700

Hi -

> From: "Ted Lemon" <[email-masked]>
> To: "James M. Polk" <[email-masked]>
> Cc: <[email-masked]>; "Paul Hoffman" <[email-masked]>; <[email-masked]>
> Sent: Wednesday, May 12, 2010 12:07 PM
> Subject: Re: Request For IETF 77 WG and BOF Sessions Agenda, Minutes,and Presentations
...
> The result is that even well-intentioned note-takers often do not produce notes 
> that can be used to generate useful minutes for a meeting.   Exceptions are
> much appreciated, but rare.
>
> I wound up having to sit and re-transcribe the minutes for the IETF 77 meeting,
> which was painful on many levels, and I will freely admit that I procrastinated for
> far longer than I spent actually doing the work.   Arguably a full transcript is
> overkill for minutes, but it's very useful for generating summaries.   I'm very
> happy with how the DHC minutes came out this time, but preparing them was brutal.

There are widely different views of what consitute "useful minutes".
>From my perspective, anything approaching a transcript is counter-productive.
What *I* like to see in the minutes is:
  - a list of the major topics discussed (no gory details, no he-said-she-said)
    (if there is some amazing insight that has everyone saying "wow", it's ok
    to write it down)
  - a list of all agreed action items, with the names of those taking responsibility
  - results of any hums, straw-polls, or other sense-of-the-room assessments
Anything more is, in my opinion, distracting and a waste of volunteer energy.

E.g.:
    Joe Schmoe reviewed the changes to the fubar i-d.  They were discussed;
    no problems were identified.  Jane Dough presented a proposal for a new work
    item on using Aether as a transport for telepathic XML.  At the request of
    AD Georg Waeschedorf, a show of hands revealed no one was willing to
    work on the proposal.  Hoa Thu Thuy accepted the action item of issuing
    a WG last call on the....

Randy

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 May 2010 15:53:37 -0400

>  - results of any hums, straw-polls, or other sense-of-the-room assessments

Sure.

> Anything more is, in my opinion, distracting and a waste of volunteer energy.

Typically you can't produce a decent summary without the transcript, unless the note-taker is sufficiently up on what is being discussed that they can produce it on the fly.   That's exciting when it happens, but it rarely happens.

The way I did the minutes for dhc this time was to present a problem statement, if applicable, a summary of the discussion, and a list of action items, for each presentation.  I also included the text of the discussion for those who might be interested, but the expectation was that most people would skip over it.   In a lot of cases there is useful information for the presenter in the discussion, and so I think it's worth including for their benefit, but I agree that for most readers it will just be noise.

Also, as a result of my detailed review of the meeting, I was able to come up with some useful discussion on the mailing list.   So I don't agree at all that the detailed review isn't worth doing.

---

From: Bernie Hoeneisen <[email-masked]>
Date: Wed, 12 May 2010 22:03:25 +0200 (CEST)

On Wed, 12 May 2010, Ted Lemon wrote:

>  *  Events move faster than most people type
>  *  It's easy to get distracted trying to figure out peoples' names, which
>     they often neglect to state
>  *  A lot of participants do not speak english using the accustomed
>     phonemes, so you need to understand what they're talking about in order
>     to parse what they are saying in realtime

Sometimes even worse applies also to certain native English speakers, of 
which some are speaking in a strange slang, others use expressions that 
are not very common, and yet others are speaking so fast that the human 
CPU of an average non-native English speaker runs out of cycles (due to 
heavy load of translating and understanding processes), even while not 
taking minutes.

>  *  Anybody who is that familiar with the topics being discussed is probably
>     a participant in the discussions, and thus will miss taking notes on
>     those portions of the discussions where they are standing at the mic

Apart from the above addition, I agree with your problem statement.

I was wondering whether a solution could be that we hire professional 
scribes to take minutes. Of course, this also leads to budget questions...

cheers,
  Bernie

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 May 2010 16:38:02 -0400

> heavy load of translating and understanding processes), even while not 

I doubt there's anybody at IETF who doesn't have this problem listening to some participants.   Much as I might wish it otherwise, I'm sure I've confounded more than a few listeners either by speaking too quickly or by unconsciously using slang.   That's the point--it's hard to keep up.

> scribes to take minutes. Of course, this also leads to budget questions...

Transcription is almost certainly cheaper, but yeah, there'd be a significant cost associated with transcribing all of the IETF streams.

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 12 May 2010 14:12:22 -0700

At 3:07 PM -0400 5/12/10, Ted Lemon wrote:
>I'm sure it's fun to judge and moralize, but it doesn't do anything to improve the situation.

Since you replied to James, not me, I can't tell if you thought my gentle-shaming suggestion was judging or moralizing. We disagree about the suggestions not doing anything to improve the situation: that is the only reason I suggested them.

>The one thing that would actually improve the situation, at least for me, would be for someone to professionally transcribe the audio from the meeting.

We disagree about that as well. The other thing that would improve the situation for everyone is better planning ahead of time on minutes-takers. Get volunteers before travelling, remind them before the meeting, and thank them profusely during and after the meeting. It is the chairs' job to be sure that the minutes are taken, not to take the minutes.

>   Volunteer minutes-takers are greatly appreciated, and often do good work, but it's hard to get good minutes in a meeting where time is so short:

This thread started because of chairs who produced *no* minutes, not because of chairs who produced minutes that were considered to be not good enough. If you will only produce minutes if they are good enough, you are failing future readers of the IETF proceedings who will only see NULL minutes.

>I wound up having to sit and re-transcribe the minutes for the IETF 77 meeting, which was painful on many levels, and I will freely admit that I procrastinated for far longer than I spent actually d…

Been there, done that, fully agree. But having half-assed minutes for the procedings is significantly better than having none.

--Paul Hoffman, Director
--VPN Consortium

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 12 May 2010 17:30:46 -0400

There is one other important property.
When the chairs realize that someone is willing to take minutes, but 
will be consistently late delivering them, then someone else needs to be 
found.  Figuring out when this applies is left as a difficult exercise 
to the chairs.

Yours,
Joel

---

From: "David Harrington" <[email-masked]>
Date: Wed, 12 May 2010 17:32:13 -0400

Has anybody tried feeding the MP3 archives into, say, Dragon?

dbh 

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Ted Lemon
> Sent: Wednesday, May 12, 2010 4:38 PM
> To: Bernie Hoeneisen
> Cc: [email-masked]; Paul Hoffman; [email-masked]
> I doubt there's anybody at IETF who doesn't have this problem 
> listening to some participants.   Much as I might wish it 
> otherwise, I'm sure I've confounded more than a few listeners 
> either by speaking too quickly or by unconsciously using 
> slang.   That's the point--it's hard to keep up.
> 
> Transcription is almost certainly cheaper, but yeah, there'd 
> be a significant cost associated with transcribing all of the 
> IETF streams.
> 
>

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 May 2010 17:38:26 -0400

> Since you replied to James, not me, I can't tell if you thought my gentle-shaming suggestion was judging or moralizing. We disagree about the suggestions not doing anything to improve the situation…

Your suggestion just seems improper.   You do not shame a volunteer for failing to deliver the goods.   You thank them profusely when they do.   I've been flamed mercilessly for insufficent promptness in producing minutes, well before the deadline, by people who were doing trip reports and felt that it was my duty to prepare text for them to use in those reports.   I don't think I've ever been thanked, other than by my co-chair, for actually producing minutes.   I have certainly never been lauded for producing minutes early.   I think it's unfortunate that we start out this discussion with a proposal to apply a stick, rather than to offer a carrot.

> We disagree about that as well. The other thing that would improve the situation for everyone is better planning ahead of time on minutes-takers. Get volunteers before travelling, remind them befor…

This presupposes the availability of a qualified volunteer.   My point earlier was that in fact a qualified volunteer is a rare and precious jewel, and that no amount of advanced planning can produce one. Either they are there, and willing to take notes for you, or they are not.   Certainly if you fail to ask them, you will not get the minutes they could have prepared for you had they been asked, but the timeliness of your request for their help will not matter if they don't exist.   Furthermore, you have no way of distinguishing between two volunteers, one of whom would take good notes, and one of whom would not, other than by trying them.   If you guess wrong, you lose.

---

From: Joseph Yee <[email-masked]>
Date: Wed, 12 May 2010 18:02:24 -0400

I tried it on old version of Dragon, by replaying the MP3 file, no digital file import, on a non IETF meeting years ago.  It did picked up a lot, but it still had issues on English and accented English (funny thing is sometimes it recognized the accented English but not the native one).  

And from past experience, make sure every speaker ID themselves really help, less effort spent to figure out where's the change of speaker.

I don't know how recent Dragon does, or if they have digital file import.  I hope they do, otherwise, you need to wait for the replay of the whole session to get the transcript.  But I'm sure *a human* still needed to review and make correction.

Joseph

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 12 May 2010 22:14:33 -0700

I did on one occasion literally transcribe the session from the
recording for the minutes. It was the longest and most boring set of
minutes I've ever written for a wg meeting.

---

From: Leonard Giuliano <[email-masked]>
Date: Thu, 13 May 2010 14:14:55 -0700

+1 on transcription.

I think everyone is correct here.  In a perfect world, a descriptive, 
concise meeting summary is the ideal solution.  And transcription would be 
long and boring.  But we don't live in a perfect world, and sometimes the 
volunteer scribe isn't very good.  Transcription is better than nothing.

How about both?  Have the audio transcribed and posted so there is a 
primary source of text along with the audio.  And then continue to have a 
scribe taking minutes and post them.  That way, if the notes are lousy (or 
missing), people can refer to the transcript.

-Lenny

On Wed, 12 May 2010, Joel Jaeggli wrote:

-) I did on one occasion literally transcribe the session from the
-) recording for the minutes. It was the longest and most boring set of
-) minutes I've ever written for a wg meeting.
-) 
-) On 05/12/2010 03:02 PM, Joseph Yee wrote:
-) > I tried it on old version of Dragon, by replaying the MP3 file, no digital file import, on a non IETF meeting years ago.  It did picked up a lot, but it still had issues on English and accented English (funny thing is sometimes it recognized the accented English but not the native one).  
-) > 
-) > And from past experience, make sure every speaker ID themselves really help, less effort spent to figure out where's the change of speaker.
-) > 
-) > I don't know how recent Dragon does, or if they have digital file import.  I hope they do, otherwise, you need to wait for the replay of the whole session to get the transcript.  But I'm sure *a human* still needed to review and make correction.
-) > 
-) > Joseph
-) > 
-) > On 2010-05-12, at 5:32 PM, David Harrington wrote:
-) > 
-) >> Has anybody tried feeding the MP3 archives into, say, Dragon?
-) >>
-) >> dbh 
-) >>
-) >>> -----Original Message-----
-) >>> From: [email-masked] 
-) >>> [mailto:[email-masked]] On Behalf Of Ted Lemon
-) >>> Sent: Wednesday, May 12, 2010 4:38 PM
-) >>> To: Bernie Hoeneisen
-) >>> Cc: [email-masked]; Paul Hoffman; [email-masked]
-) >>> Subject: Re: Request For IETF 77 WG and BOF Sessions Agenda, 
-) >>> Minutes,and Presentations
-) >>>
-) >>> On May 12, 2010, at 4:03 PM, Bernie Hoeneisen wrote:
-) >>>> Sometimes even worse applies also to certain native English 
-) >>> speakers, of 
-) >>>> which some are speaking in a strange slang, others use 
-) >>> expressions that 
-) >>>> are not very common, and yet others are speaking so fast 
-) >>> that the human 
-) >>>> CPU of an average non-native English speaker runs out of 
-) >>> cycles (due to 
-) >>>> heavy load of translating and understanding processes), 
-) >>> even while not 
-) >>>> taking minutes.
-) >>>
-) >>> I doubt there's anybody at IETF who doesn't have this problem 
-) >>> listening to some participants.   Much as I might wish it 
-) >>> otherwise, I'm sure I've confounded more than a few listeners 
-) >>> either by speaking too quickly or by unconsciously using 
-) >>> slang.   That's the point--it's hard to keep up.
-) >>>
-) >>>> I was wondering whether a solution could be that we hire 
-) >>> professional 
-) >>>> scribes to take minutes. Of course, this also leads to 
-) >>> budget questions...
-) >>>
-) >>>
-) >>> Transcription is almost certainly cheaper, but yeah, there'd 
-) >>> be a significant cost associated with transcribing all of the 
-) >>> IETF streams.
-) >>>
-) >>>
-) >>
-) > 
-)

---

From: Stephen Hanna <[email-masked]>
Date: Thu, 13 May 2010 17:32:53 -0400

If IETF wants to provide transcription, that's fine with me.

I'll just share what I do. I sit down with the minutes from
the scribes, my notes, and the MP3 file from the recording.
I play the recording while I combine the text from the
volunteer scribes. Usually, they've caught 80-90% of the
meeting so some quick editing gives pretty good minutes.
This process takes about 50% longer than the actual duration
of the meeting. As with most others here, I'm not trying to
create a transcription just the key points: decisions made,
especially insightful comments, and action items.

Thanks,

Steve

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Leonard Giuliano
> Sent: Thursday, May 13, 2010 5:15 PM
> To: Joel Jaeggli
> Cc: [email-masked]; 'Bernie Hoeneisen'; 'Paul Hoffman'; 'Ted Lemon';
> [email-masked]
> -) > I tried it on old version of Dragon, by replaying the MP3 file, no
> digital file import, on a non IETF meeting years ago.  It did picked up
> a lot, but it still had issues on English and accented English (funny
> thing is sometimes it recognized the accented English but not the
> native one).
> -) >
> -) > And from past experience, make sure every speaker ID themselves
> really help, less effort spent to figure out where's the change of
> speaker.
> -) >
> -) > I don't know how recent Dragon does, or if they have digital file
> import.  I hope they do, otherwise, you need to wait for the replay of
> the whole session to get the transcript.  But I'm sure *a human* still
> needed to review and make correction.
> -) >
> -) > Joseph
> -) >

---

From: Sandra Murphy <[email-masked]>
Date: Wed, 12 May 2010 17:14:03 -0400 (Eastern Daylight Time)

> There are widely different views of what consitute "useful minutes".

Yep.

> From my perspective, anything approaching a transcript is counter-productive.

My idea of the use of minutes is to point to them when people ask about a 
particular decision.  And to track the progress on some issue.  And find 
who spoke for what stance on the issue.  For people who did not attend 
(particularly when audio or jabber did not successfully capture what went 
on) the minutes can be very helpful.

Obviously, the audio archive could give the full experience for those who 
want it.  But it's a whole lot easier to read minutes than listen to the 
full audio (to quote Ted "painful on many levels"), the audio isn't always 
perfect and the people aren't always diligent about speaking at the mike, 
and lots of tools work better with text than with audio.  (Maybe we need 
research on a audio-grep.)

>    (if there is some amazing insight that has everyone saying "wow", it's ok

I see little benefit to this, as the agenda should provide the same.  No?

> Anything more is, in my opinion, distracting and a waste of volunteer energy.

Your experience in working groups seems to tend toward those that are 
actually coming to consensus in the meetings.  Lucky you.

--Sandy

---

From: Dean Willis <[email-masked]>
Date: Fri, 14 May 2010 03:02:47 -0500

Ted Lemon wrote:

> The one thing that would actually improve the situation, at least for 
> me, would be for someone to professionally transcribe the audio from the 
> meeting.   

Since many SIP meetings have been transcribed for a hearing-impaired 
participant, I tried that for a couple of years. It was useful as a 
reference but lousy for minutes.

> Volunteer minutes-takers are greatly appreciated, and often 
> do good work, but it's hard to get good minutes in a meeting where time 
> is so short

I found training the note-takers with a short primer to be very helpful. 
At each meeting I tell them something like:

Note the topic, the presenter, each issue, and the resolution of each 
issue or the failure to resolve if that's what happens. If any really 
impressive points are made in the discussion, jot those down if you have 
time. Do NOT record "he said, she said" material.

It seemed to work for SIP, which was a pretty darned busy working group.

--
Dean

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 14 May 2010 15:07:25 +0200

(Changing the subject line)

As an occasional notetaker, I find what Dean suggests to be very hard to do in real-time. It is often not apparent when somebody initiates a new thread at the mic whether that will evolve into something of significance. If I don't jot down what each speaker says, and then later it turns out to be important, I have to try to summarise it from memory at the end of the thread. Then I miss the next few speakers...

One frequent problem is jumping quickly between threads, without an adequate summary by the chairs, which would give notetakers an opportunity to record the important points.

Another approach is that the notetaker tries to summarise after the meeting, rather than submitting the full text, but this is additional work for (volunteer/bullied) notetakers and will result in further delays.

So I have to admit to just throwing raw notes at the chairs and leaving it to them. Of course, as a new chair, I am likely to be on the receiving end!

John

> [mailto:[email-masked]] On Behalf Of Dean Willis
> Sent: 14 May 2010 09:03
> To: Ted Lemon
> Cc: [email-masked]; [email-masked]
> Subject: Re: Request For IETF 77 WG and BOF Sessions Agenda, 
> Minutes, and Presentations
> 
> Ted Lemon wrote:
>

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 14 May 2010 09:23:34 -0400

On May 14, 2010, at 9:07 AM, Elwell, John wrote:

> As an occasional notetaker, I find what Dean suggests to be very  
> hard to do in real-time. It is often not apparent when somebody  
> initiates a new thread at the mic whether that will evolve into  
> something of significance. If I don't jot down what each speaker  
> says, and then later it turns out to be important, I have to try to  
> summarise it from memory at the end of the thread. Then I miss the  
> next few speakers...
>
> One frequent problem is jumping quickly between threads, without an  
> adequate summary by the chairs, which would give notetakers an  
> opportunity to record the important points.
>

To me, one frequent problem when taking notes is when speakers do not  
identify themselves. Even if I know who they are, I will probably be  
facing away from them and looking down, and so the speaker's id may  
not be clear. Since speaker changes typically occur at the beginning  
of new topics, this means that important parts of the discussion may  
be lost, just by taking the time to turn around and look to see who  
just said what.

And, of course, this presents an even bigger problem to those  
listening to the MP3s.

Regards
Marshall

> Another approach is that the notetaker tries to summarise after the  
> meeting, rather than submitting the full text, but this is  
> additional work for (volunteer/bullied) notetakers and will result  
> in further delays.
>
> So I have to admit to just throwing raw notes at the chairs and  
> leaving it to them. Of course, as a new chair, I am likely to be on  
> the receiving end!
>
> John
>
>

---

From: Ed Juskevicius <[email-masked]>
Date: Fri, 14 May 2010 10:07:09 -0400

+1

Regards,

Ed  Juskevicius.

On Fri, May 14, 2010 at 9:23 AM, Marshall Eubanks <[email-masked]>wrote:

>> As an occasional notetaker, I find what Dean suggests to be very hard to
>> do in real-time. It is often not apparent when somebody initiates a new
>> thread at the mic whether that will evolve into something of significance.
>> If I don't jot down what each speaker says, and then later it turns out to
>> be important, I have to try to summarise it from memory at the end of the
>> thread. Then I miss the next few speakers...
>>
>> adequate summary by the chairs, which would give notetakers an opportunity
>> to record the important points.
>>
>>
> identify themselves. Even if I know who they are, I will probably be facing
> away from them and looking down, and so the speaker's id may not be clear.
> Since speaker changes typically occur at the beginning of new topics, this
> means that important parts of the discussion may be lost, just by taking the
> time to turn around and look to see who just said what.
>
> And, of course, this presents an even bigger problem to those listening to
> the MP3s.
>
> Regards
> Marshall
>
>
>
>> meeting, rather than submitting the full text, but this is additional work
>> for (volunteer/bullied) notetakers and will result in further delays.
>>
>> So I have to admit to just throwing raw notes at the chairs and leaving it
>> to them. Of course, as a new chair, I am likely to be on the receiving end!
>>
>> John
>>
>>
>>> The one thing that would actually improve the situation, at
>>>>
>>> least for
>>>
>>>> me, would be for someone to professionally transcribe the
>>>>
>>> audio from the
>>>
>>>> meeting.
>>>>
>>>
>>>> do good work, but it's hard to get good minutes in a
>>>>
>>> meeting where time
>>>
>>>> is so short
>>>>
>>>

---

From: "David Harrington" <[email-masked]>
Date: Fri, 14 May 2010 10:19:44 -0400

Hi,

It is an unfortunate thing that sometimes what we hear is not what is
said, and this extends to what minute takers hear. 

when I prepare minutes as a chair, I go listen to the MP3 audio as
part of that process. I provide a link to the MP3 audio file, and I
try to note in the minutes what time in the MP3 a discussion starts.
This way a reader who is interested can go listen to that discussion.

Boring for me to do this? yeah. But important to make sure the minutes
provide an accurate representation of issues raised. As chair I know
most of the main players and can fill in the speaker's name when
needed, based on my recognition of their voice or recalling them
making the point.

I work for a Chinese company, and when we have a meeting in a Western
venue, the time is pretty bad for my colleagues. So they listen to the
MP3 and follow along with the presentations. I tend to follow a
transcription-style minutes because I know this could help my
colleagues understand what is being said in the MP3. (of course, lik e
most, my minutes are done closer to deadline, so they probably aren't
available when co-workers view the sldies and listen to the
presentation.)

Asia has the largest growing segment of IETF attendance, and that's
the count of people attending, not the people attending remotely. I
think transcription could be very helpful to supplement the MP3 audio.

dbh

> From: [email-masked] [mailto:[email-masked]] On 
> Cc: [email-masked]; 'Bernie Hoeneisen'; 
> [email-masked]; 'Paul Hoffman'; 'Ted Lemon'
> Minutes, andPresentations
> 
> 
> +1 on transcription.
> 

> -) > I tried it on old version of Dragon, by replaying the 
> MP3 file, no digital file import, on a non IETF meeting years 
> ago.  It did picked up a lot, but it still had issues on 
> English and accented English (funny thing is sometimes it 
> recognized the accented English but not the native one).  
> -) > 
> -) > And from past experience, make sure every speaker ID 
> themselves really help, less effort spent to figure out 
> where's the change of speaker.
> -) > 
> -) > I don't know how recent Dragon does, or if they have 
> digital file import.  I hope they do, otherwise, you need to 
> wait for the replay of the whole session to get the 
> transcript.  But I'm sure *a human* still needed to review 
> and make correction.
> -) > 
> -) > Joseph
> -) >

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 14 May 2010 09:35:16 -0500

I have largely stopped being a scribe for working group meetings (with three 
working groups at IETF 77, I was usually CHAIRING the stinking meetings 
instead), but I used to scribe for 5-7 working groups at each IETF, and my 
experience matches Marshall's.

> facing away from them and looking down, and so the speaker's id may  not 
> be clear. Since speaker changes typically occur at the beginning  of new 
> topics, this means that important parts of the discussion may  be lost, 
> just by taking the time to turn around and look to see who  just said 
> what.

I strongly suspect that Scott Bradner asked me to be a repeat scribe in 
NEWTRK as a way of keeping me away from the mike line :-)

But ... Scott did ask me to be a scribe repeatedly, and Scott had no 
co-chair, so I usually got to sit at the table in the front of the room, 
facing the group. That actually turned out to be a huge help.

When I've done that in other working groups, it's also helped me to 
associate names and faces for later conversations in the hall, which rarely 
happened when I was craning my neck around for a quick glance backwards.

If there's any operational guidance we could develop in this thread, I 
wonder if we as chairs might each consider - not talking about voting or 
consensus on this for all working groups, but each acting in the working 
group(s) we chair - asking the primary notetaker to sit at the front table 
with the chairs.

I'm remembering that DRINKS already does this (right?), and I'm remembering 
that a small number of working groups have three chairs, so we must be able 
to accommodate 3 people at the front table now. I'm hoping this wouldn't 
turn the front table into a rugby scrum.

And if we ever go back to Vienna (a lovely place), we were WAY low on power 
down the rows there, so being able to offer actual power at the front of the 
room that you didn't have to face the wall to plug into was a real 
attraction, too!

Unfortunately, the secretariat does a much better job now, so we usually 
can't offer that as a meaningful incentive... :-|

Thanks

Spencer

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 14 May 2010 11:08:05 -0400

On May 14, 2010, at 10:35 AM, Spencer Dawkins wrote:

> I have largely stopped being a scribe for working group meetings  
> (with three working groups at IETF 77, I was usually CHAIRING the  
> stinking meetings instead), but I used to scribe for 5-7 working  
> groups at each IETF, and my experience matches Marshall's.
>
>> To me, one frequent problem when taking notes is when speakers do  
>> not identify themselves. Even if I know who they are, I will  
>> probably be facing away from them and looking down, and so the  
>> speaker's id may  not be clear. Since speaker changes typically  
>> occur at the beginning  of new topics, this means that important  
>> parts of the discussion may  be lost, just by taking the time to  
>> turn around and look to see who  just said what.
>
> I strongly suspect that Scott Bradner asked me to be a repeat scribe  
> in NEWTRK as a way of keeping me away from the mike line :-)
>
> co-chair, so I usually got to sit at the table in the front of the  
> room, facing the group. That actually turned out to be a huge help.
>
> associate names and faces for later conversations in the hall, which  
> rarely happened when I was craning my neck around for a quick glance  
> backwards.
>
> If there's any operational guidance we could develop in this thread,  
> I wonder if we as chairs might each consider - not talking about  
> voting or consensus on this for all working groups, but each acting  
> in the working group(s) we chair - asking the primary notetaker to  
> sit at the front table with the chairs.
>

This actually would probably help a lot.

Marshall

> I'm remembering that DRINKS already does this (right?), and I'm  
> remembering that a small number of working groups have three chairs,  
> so we must be able to accommodate 3 people at the front table now.  
> I'm hoping this wouldn't turn the front table into a rugby scrum.
>
> And if we ever go back to Vienna (a lovely place), we were WAY low  
> on power down the rows there, so being able to offer actual power at  
> the front of the room that you didn't have to face the wall to plug  
> into was a real attraction, too!
>
> Unfortunately, the secretariat does a much better job now, so we  
> usually can't offer that as a meaningful incentive... :-|
>
> Thanks
>
> Spencer
>

---

From: "James M. Polk" <[email-masked]>
Date: Fri, 14 May 2010 13:33:39 -0500

>room, facing the group. That actually turned out to be a huge help.

While I have taken notes for my WG (TSVWG) - as chair - I can attest 
to the visibility from the front of the room being advantageous, my 
WG also has 3 chairs for nearly 4 years, which was ALWAYS cramped, 
and none of liked that situation one bit. So, the idea that all WGs 
would have (or volunteer) to undergo this I seriously wouldn't 
recommend to anyone.

But ... what about a second table on the opposite side of the 
presentation screen (i.e., the side the chairs' table isn't on) be 
for the notetakers?  The AV folks would have a little bit of work to 
do by giving that table power for laptops and setting up the table 
with 1 or 2 chairs -- but that's it. We can call it the notetaker(s) table.

Is this an acceptable compromise (that has all the advantages of 
notetakers being at the front of the room, yet not cramping the WG chairs)?

James

>rarely happened when I was craning my neck around for a quick glance backwards.
>

---

From: Ed Juskevicius <[email-masked]>
Date: Fri, 14 May 2010 14:42:51 -0400

How, how about a slightly longer table at the front, with room for one more
chair (for the notetaker)?  That way, if anyone goes to a microphone and
forgets to announce their name, the notetaker could lean over to a chair and
say "name?"

Alternatively, what about resurrecting the RFID technology that was tried
during IETF 76?  That was terrific for notetaking (when people remembered to
'swipe' their RFID-enabled name tags at a microphone) because it not only
provided info about name of the person at the mic, but it also got the
correct spelling of the name.

Regards,

Ed  "J + 10 more characters in my surname"

On Fri, May 14, 2010 at 2:33 PM, James M. Polk <[email-masked]> wrote:

>> I have largely stopped being a scribe for working group meetings (with
>> three working groups at IETF 77, I was usually CHAIRING the stinking
>> meetings instead), but I used to scribe for 5-7 working groups at each IETF,
>> and my experience matches Marshall's.
>>
>>> means that important parts of the discussion may  be lost, just by taking
>>> the time to turn around and look to see who  just said what.
>>>
>>
> While I have taken notes for my WG (TSVWG) - as chair - I can attest to the
> visibility from the front of the room being advantageous, my WG also has 3
> chairs for nearly 4 years, which was ALWAYS cramped, and none of liked that
> situation one bit. So, the idea that all WGs would have (or volunteer) to
> undergo this I seriously wouldn't recommend to anyone.
>
> But ... what about a second table on the opposite side of the presentation
> screen (i.e., the side the chairs' table isn't on) be for the notetakers?
>  The AV folks would have a little bit of work to do by giving that table
> power for laptops and setting up the table with 1 or 2 chairs -- but that's
> it. We can call it the notetaker(s) table.
>
> Is this an acceptable compromise (that has all the advantages of notetakers
> being at the front of the room, yet not cramping the WG chairs)?
>
> James
>
>
>> remembering that a small number of working groups have three chairs, so we
>> must be able to accommodate 3 people at the front table now. I'm hoping this
>> wouldn't turn the front table into a rugby scrum.
>>
>> And if we ever go back to Vienna (a lovely place), we were WAY low on
>> power down the rows there, so being able to offer actual power at the front
>> of the room that you didn't have to face the wall to plug into was a real
>> attraction, too!
>>

---

From: Brian Haberman <[email-masked]>
Date: Fri, 14 May 2010 14:47:59 -0400

> forgets to announce their name, the notetaker could lean over to a chair and

I would prefer the second table approach.  That way, the notetaker and
the Jabber scribe have the same vantage point.

Regards,
Brian

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 14 May 2010 14:12:27 -0500

If we're just dreaming on what is, for me, a lazy Friday afternoon...

> the Jabber scribe have the same vantage point.

I'd be fine with a second table.

If it was close enough for the notetaker/jabber scribe to flag down the 
chairs and say "please put me in the cue, but let me keep typing until it's 
my turn at the mike", without and distracting the working group, that would 
also help, in my experience.

Thanks,

Spencer

---

From: Ted Lemon <[email-masked]>
Date: Fri, 14 May 2010 13:29:19 -0700

> the Jabber scribe have the same vantage point.

The only problem with this is that then the note-takers are effectively excluded from the mic. And in any case this presupposes that there is a good note-taker available, which I think is an unwarranted assumption.

---

From: "James M. Polk" <[email-masked]>
Date: Fri, 14 May 2010 15:46:34 -0500

At 03:29 PM 5/14/2010, Ted Lemon wrote:
>The only problem with this is that then the note-takers are 
>effectively excluded from the mic. And in any case this presupposes 
>that there is a good note-taker available, which I think is an 
>unwarranted assumption.

... and therefore let's not do an otherwise good idea...

that's the way things stay the same -- bravo, congrats are in order!

James

---

From: Ted Lemon <[email-masked]>
Date: Fri, 14 May 2010 13:52:51 -0700

> that's the way things stay the same -- bravo, congrats are in order!

My point is that if you are going to have a note taker at a table, it makes sense for it to be the chairs' table, because there is a microphone there.  My second point is that even if this is done, it won't solve the problem that transcripts would solve. IMHO, transcripts will be of more help in the long run. I know that makes me a bad person, and will not be offended if you avoid me in the halls in Maastricht. ;')

---

From: Ted Lemon <[email-masked]>
Date: Sat, 15 May 2010 22:34:06 -0700

On May 15, 2010, at 10:38 PM, "joel jaeggli" <[email-masked]> wrote:

> There's something on order of 4 or 5 thousand hours of recorded audio If someone wants to produce transcriptions from something...

That way lies madness.  When you are running an archive that is growing, and you want to institute some new procedure with the data, start the procedure on the new data as it comes in--don't try to start at the beginning.

I think transcripts of the entire archive would be useful for historical porpoises, but transcripts of sessions as they happen would be useful for addressing the problem of late minutes, and that's the more pressing problem.

---

From: Tony Li <[email-masked]>
Date: Sun, 16 May 2010 00:38:22 -0700

Can we run it through Google Voice to see what comes out?

Tony

On 5/15/10 8:37 PM, "joel jaeggli" <[email-masked]> wrote:

> There's something on order of 4 or 5 thousand hours of recorded audio If
> someone wants to produce transcriptions from something...

Ted Lemon
> <[email-masked]> wrote:

>> My point is that if you are going to have a note taker at a table, it makes
>> sense for it to be the chairs' table, because there is a microphone there.
>> My second point is that even if this is done, it won't solve the problem that
>> transcripts would solve. IMHO, transcripts will be of more help in the long
>> run. I know that makes me a bad person, and will not be offended if you avoid
>> me in the halls in Maastricht. ;')

---

From: joel jaeggli <[email-masked]>
Date: Sun, 16 May 2010 09:58:17 -0700

On 2010-05-15 22:34, Ted Lemon wrote:
> On May 15, 2010, at 10:38 PM, "joel jaeggli"<[email-masked]>
> wrote:
>
>> There's something on order of 4 or 5 thousand hours of recorded
>> audio If someone wants to produce transcriptions from something...
>
> That way lies madness.  When you are running an archive that is
> growing, and you want to institute some new procedure with the data,
> start the procedure on the new data as it comes in--don't try to
> start at the beginning.

I agree... however it's useful for scope, there's about ~300 hours of
new recordings for each meeting, if the later can be addressed in some 
automated fashion then so can the former, or at least some of it.  if 
the later relies on human capital on a arge scale it's probably already 
insurmountable.

> I think transcripts of the entire archive would be useful for
> historical porpoises, but transcripts of sessions as they happen
> would be useful for addressing the problem of late minutes, and
> that's the more pressing problem.

---

From: "Bernard Aboba" <[email-masked]>
Date: Sun, 16 May 2010 10:47:57 -0700

Tony Li said:

"Can we run it through Google Voice to see what comes out?"

[BA] At the very least, this would be entertaining :)

---

From: joel jaeggli <[email-masked]>
Date: Sun, 16 May 2010 11:55:51 -0700

> "Can we run it through Google Voice to see what comes out?"

the sessions from hiroshima and montreal have probably the most uniform 
  and high quality SNR so far due to superior venue equipment.

---

From: Dean Willis <[email-masked]>
Date: Mon, 17 May 2010 10:19:57 -0500

Marshall Eubanks wrote:

> just by taking the time to turn around and look to see who just said what.
>

This is much less a problem if the minutes focus on the decisions made,
not on "who said what". I'm inclined to think that textually preserving
the idle banter between participants is not as important as preserving
the outcome of that banter, especially when we have recordings of the
banter itself.

If there is an especially salient point made (by somebody either known
or unknown), I might record "It was noted that....."

--
Dean

---

From: Ted Lemon <[email-masked]>
Date: Mon, 17 May 2010 10:47:47 -0500

> the later relies on human capital on a arge scale it's probably already 

300 hours at $10/hour is $3000.   Hardly insurmountable.   Granted, it'd probably cost more than that because transcription doesn't actually happen in realtime, but I doubt it'd be more than 1000 hours of work.   I'm not saying IETF has the money to pay for it, but in principle it's a very surmountable problem.  Humans who can listen and type are not expensive to hire.

---

From: Tom Taylor <[email-masked]>
Date: Mon, 17 May 2010 12:00:12 -0400

What I consider ideal is to capture not only decisions but the reasons for 
making them. That gives people something to refer back to when the topic comes 
up again, as it almost inevitably does for a major topic.

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 17 May 2010 09:11:20 -0700

This thread has diverged into two steams: note-taking for minutes, and transcription. The IETF has traditionally had minutes that were narrative and much shorter than transcription. In the past many years, the IETF (via Joel) has provided MP3s of the sessions for people who want to have an "audio transcription".

Are the people who are advocating transcription on this thread doing so to help the chairs produce narrative minutes, or as a replacement for the minutes? If the former, has anyone tried to create their minutes from a transcription and, if so, how long did it take you? If the latter, this is a significant change from the current IETF model.

--Paul Hoffman, Director
--VPN Consortium

---

From: Ted Lemon <[email-masked]>
Date: Mon, 17 May 2010 09:54:55 -0700

> Are the people who are advocating transcription on this thread doing so to help the chairs produce narrative minutes, or as a replacement for the minutes? If the former, has anyone tried to create…

I don't think that transcripts should be a substitute for minutes, because they take too long to review, and it may be difficult for someone who didn't attend to follow the discussion. I have prepared minutes from transcripts, and once I had the transcript, doing the minutes probably took three hours of my time, but I was doing a very thorough job, because I wanted the minutes to serve as a viable todo list for the wg.   I've never produced minutes I felt happy with without a transcript, whether it was one I did myself, or one that a really good note-taker did during the meeting.

---

From: Geoff Huston <[email-masked]>
Date: Tue, 18 May 2010 14:22:53 +1000

+1

Personally I'm a believer that minutes should avoid who said what - anyone wanting to find that out will wade through thte audio anyway and for everyone else the salient points is to capture decisions and "we should note this" points in a brief style - engineering notes if you prefer.

But my experience is that this preference is shared by a remarkably small minority, and the rather poor, incomplete and often tedious representation of idle microphone banter is what folk appear to want in WG minutes. It seems to me that putting more effort into this effort of the preservation of chatter is a typically misguided application of possibly productive energy where form is mistaken for substance.

  Geoff

---

From: Ted Lemon <[email-masked]>
Date: Tue, 18 May 2010 00:14:00 -0500

> But my experience is that this preference is shared by a remarkably small minority, and the rather poor, incomplete and often tedious representation of idle microphone banter is what folk appear to…

Maybe you should just go to different working groups.   While I will admit that I am often guilty of not thinking carefully before speaking at the microphone, I have the good fortune to co-chair a working group where most of the chatter at the mic is actually very useful.   Would *you* be interested?   Probably not.   That doesn't mean that what was said wasn't useful.

---

From: Geoff Huston <[email-masked]>
Date: Tue, 18 May 2010 16:33:50 +1000

> Maybe you should just go to different working groups.   While I will admit that I am often guilty of not thinking carefully before speaking at the microphone, I have the good fortune to co-chair a…

Oh, I have participated in many working groups over my last 21 years of IETF participation, and have experienced a wealth of diversity in style of working group functionality, and lack thereof. However, I hope I was sufficiently careful to note in my comment above that I was relating my experience and my impressions, and, to be clear here, I was not attempting to speak for others, nor was I attempting to tell others what they should or should not do, usefully or otherwise. I was simply relating my perceptions, for what they are worth.

regards,

  Geoff

---

From: Marshall Eubanks <[email-masked]>
Date: Tue, 18 May 2010 06:42:21 -0400

On May 18, 2010, at 2:33 AM, Geoff Huston wrote:

>>> But my experience is that this preference is shared by a  
>>> remarkably small minority, and the rather poor, incomplete and  
>>> often tedious representation of idle microphone banter is what  
>>> folk appear to want in WG minutes. It seems to me that putting  
>>> more effort into this effort of the preservation of chatter is a  
>>> typically misguided application of possibly productive energy  
>>> where form is mistaken for substance.
>>
>> Maybe you should just go to different working groups.   While I  
>> will admit that I am often guilty of not thinking carefully before  
>> speaking at the microphone, I have the good fortune to co-chair a  
>> working group where most of the chatter at the mic is actually very  
>> useful.   Would *you* be interested?   Probably not.   That doesn't  
>> mean that what was said wasn't useful.
>>
>
> Oh, I have participated in many working groups over my last 21 years  
> of IETF participation, and have experienced a wealth of diversity in  
> style of working group functionality, and lack thereof. However, I  
> hope I was sufficiently careful to note in my comment above that I  
> was relating my experience and my impressions, and, to be clear  
> here, I was not attempting to speak for others, nor was I attempting  
> to tell others what they should or should not do, usefully or  
> otherwise. I was simply relating my perceptions, for what they are  
> worth.
>

Note taking != minutes.

Notes are the raw material minutes are made of.

Regards
Marshall

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Tue, 18 May 2010 13:30:23 +0200

> I think transcription could be very helpful to supplement the MP3
audio.

+1

Mehmet

> [mailto:[email-masked]] On Behalf Of ext David Harrington
> Sent: Friday, May 14, 2010 4:20 PM
> To: 'Leonard Giuliano'; 'Joel Jaeggli'
> Cc: [email-masked]; 'Bernie Hoeneisen'; 'Ted Lemon'; 
> [email-masked]; 'Paul Hoffman'
> Subject: RE: Request For IETF 77 WG and BOF Sessions Agenda, 
> Minutes,andPresentations
> 
> Hi,
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 18 May 2010 13:45:16 +0200

> > I think transcription could be very helpful to supplement the MP3

I do not need a transcription. If something is unclear, I prefer to
listen to the original audio myself since how something was said often
carries additional information.

That said: I do not mind if people volunteers to produce written
transcriptions; but please do not make this another administrative
work item chairs have to worry about.

/js (who is optimistic to return the blue dot by July ;-)

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 18 May 2010 20:46:08 +0900

> 300 hours at $10/hour is $3000.   Hardly insurmountable.   Granted, it'd probably cost more than that because transcription doesn't actually happen in realtime, but I doubt it'd be more than 1000 h…

My only experience in transcribing is years back, with a short (maybe 
5-10 minutes) piece. It was tough, required a lot of time. There are 
people who can transcribe in real time (ICANN meetings use(d?) these, 
and they also work in courtrooms), but it's a very difficult skill 
(using a chorded keyboard) that has to be trained for a long time. Also, 
the result isn't really printable, it contains quite a lot of errors.

Once you have to give up on real-time transcription, you have to 
start/stop/rewind the recording, which means a lot of overhead. Add to 
this people not speaking very clearly, with an accent,..., bad recording 
quality, and infamiliarity of the transcriber with our technical 
terminology, and the costs will go up and up.

At $10/hour, I'd actually be glad to pay the transcription for a WG or 
two where I'm involved as chair or editor out of my own pocket, but I 
know that it's way too cheap, and even something in the $30 range won't 
be enough, by quite a substantial margin.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: "David Harrington" <[email-masked]>
Date: Tue, 18 May 2010 09:32:32 -0400

Hi,

Having detailed notes about who said what can seem useless, and just
having a bullet that says "this was decided" may seem simpler. I know
in IEEE 802.1, they use the summary style of notes, but they vote.

IETF runs on consensus. and the way to determine consensus often comes
down to understanding how many people had things to say, and it is
important to understand the technical points made, and the support
expressed (even if sometimes expressed poorly). Having that only
recorded in the audio achives is likely to be inadequate.

The IESG (or IAB if it's an appeal) may come to you and ask you to
show how you reached your assessment. If you say "it's over there in
the audio archive; go listen for yourself", you will certainly not be
very popular with I*. If you have to go chase down all the
conversations that happened during the meetings in the audio archives
yourself, you might wish more detailed minutes were written down.

I am not saying we should move toward transcription-style minutes,
because I don't think anybody will read the transcripts. I am saying
we need to make sure the minutes have an accurate reflection of what
happened during the meeting, and a short summary "we talked about this
and decided this" may be inadequate.

dbh

> [mailto:[email-masked]] On Behalf Of Geoff Huston
> Sent: Tuesday, May 18, 2010 12:23 AM
> To: Dean Willis
> Subject: Re: Notetaking
> Personally I'm a believer that minutes should avoid who said 
> what - anyone wanting to find that out will wade through thte 
> audio anyway and for everyone else the salient points is to 
> capture decisions and "we should note this" points in a brief 
> style - engineering notes if you prefer.
> 
> remarkably small minority, and the rather poor, incomplete 
> and often tedious representation of idle microphone banter is 
> what folk appear to want in WG minutes. It seems to me that 
> putting more effort into this effort of the preservation of 
> chatter is a typically misguided application of possibly 
> productive energy where form is mistaken for substance.
> 
>   Geoff 
>

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 18 May 2010 10:50:29 -0400

> and decided this" may be inadequate.

In the past, when taking minutes, I tended to create fairly verbose
ones on the grounds exactly that it was a way to indicate the degree
and roughness of consensus.  Then I read some guidelines for minute
takers, and discovered that such was actively discouraged.  But I can
put my hands on that advice just now, so I don't remember where I saw
it.  Anyone recall?  It's not in RFC 2418.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 18 May 2010 15:47:55 +0000

On Tue, May 18, 2010 at 10:50:29AM -0400, Andrew Sullivan wrote:
> and roughness of consensus.

When I've been asked to take minutes, I've done something very similar.  Like
it or not, the people speaking and the nature of their questions or points
often have significant impact on workgroup consensus.  I don't support
complete transcripts - nor do I have the ability to take notes at a speed
that would support that - but I do believe that as an organization we need
something between minutes and a full transcription.

I'll also second a comment about the scribe and lack of microphone access.
If you're going ask someone to be a scribe, please reserve some
speaking and rebuttal time for them if they're an active WG participant.
I now avoid taking that role when there is no co-scribe so that I have
access to the mic after being tail dropped on the mic queue.

-- Jeff

---

From: "David Harrington" <[email-masked]>
Date: Tue, 18 May 2010 11:48:18 -0400

I have seen the advice. I think the chairs must make a determination
of what is appropriate for their WG at the time of preparing the
minutes for the meeting. Whether you need detailed minutes or summary
minutes regarding a specific topic can vary depending on the degree of
consensus for the topic, and of course can vary by topic.

I think note-takers should keep rather detailed notes and then the
chairs can decide whether a summary is adequate for the minutes, or a
detailed report of the discussion is warranted. I don't think the note
takers should be the ones making that decision.

dbh 

> [mailto:[email-masked]] On Behalf Of Andrew Sullivan
> Sent: Tuesday, May 18, 2010 10:50 AM
> To: [email-masked]
> Subject: Re: Notetaking
>

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 18 May 2010 11:08:07 -0700

Hi -

> From: "Andrew Sullivan" <[email-masked]>
> To: <[email-masked]>
> Sent: Tuesday, May 18, 2010 7:50 AM
> Subject: Re: Notetaking
...
> it.  Anyone recall?  It's not in RFC 2418.

It used to be in http://www.ietf.org/instructions/minutes.html
It has been moved to http://www.ietf.org/old/2009/instructions/minutes.html
It's still good advice, as far as I'm concerned.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 18 May 2010 11:14:28 -0700

Hi -

> From: "David Harrington" <[email-masked]>
> To: "'Geoff Huston'" <[email-masked]>; "'Dean Willis'" <[email-masked]>
> Cc: <[email-masked]>; <[email-masked]>
> Sent: Tuesday, May 18, 2010 6:32 AM
> Subject: RE: Notetaking
...
...

But any "real" decision still needs to go back to the mailing list to determine
whether there is actually concensus support for the proposed decision.
If a decision of import is made in a WG meeting and *not* taken
to the mailing list, then IMO there may be legitimate grounds for an appeal.

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 19 May 2010 00:00:45 +0200

> yourself, you might wish more detailed minutes were written down.

Once I face this situation, I will love to go back to the audio files
because this is the most definite source out there. Why on earth
should I trust a transcript to be accurate?

Having grown up with cassette recorders, I find moving around in audio
files extremely fast - especially if the agenda and the minutes are
available.

/js

---

From: Ben Campbell <[email-masked]>
Date: Wed, 19 May 2010 16:27:14 -0500

(I'm not specifically responding to Brian, but this looks like as good a place as any to jump in on note taking mechanics)

In the last XMPP meeting, I witnessed (as a new co-chair) a radically different (to me) approach to work group note taking. It basically became a cooperative effort, with multiple note takes tag teaming for when one needed to go to the mike, or had computer difficulties, etc.

The notetakers used an online collaborative editing tool. (In this case, it was EtherPad, which unfortunately seems to have been shut down by Google--but I assume others exist or could be created)  They coordinated in real time in the jabber room. If the current primary notetaker missed a name or something, someone else would go behind to fill it in. I could see the notes appear in real time, and it was pretty amazing to watch.

Of course, this requires a good deal of trust that no one tampers with the process, but I'm not sure it would be any worse than with more conventional techniques. OTOH, having multiple sets of eyes on the notes as they are being generated may improve their accuracy.

---

From: Tony Hansen <[email-masked]>
Date: Wed, 19 May 2010 17:48:45 -0400

A number of WGs I'm involved with have been doing exactly what you 
describe, collaborative note taking, for several years. But instead of  
using EtherPad, we've been using the jabber room for their 
collaboration. It's worked very well.

     Tony Hansen
     [email-masked]

On 5/19/2010 5:27 PM, Ben Campbell wrote:
>> On May 14, 2010, at 2:48 PM, "Brian Haberman"<[email-masked]>  wrote:
>>
>>

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 19 May 2010 18:04:46 -0600

+1 

I think we need to focus on what the minutes are useful for. Some of the key things I attempt to use minutes for are:

1) updating a draft to reflect the decisions made in meeting

2) understanding what decisions were made at a recent meeting

3) looking back at old meetings to try and see why some decisions was made so I can figure out if it makes sense to re-make the decisions which often seems to be happening in WGs

4) try and figure out if there was any consensus or just silence around some topic 

I'm sure there are many other reasons but the transcript style he said she said are typically not very useful for this.

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 19 May 2010 22:01:29 -0700

On 05/17/2010 09:11 AM, Paul Hoffman wrote:
> This thread has diverged into two steams: note-taking for minutes,
> and transcription. The IETF has traditionally had minutes that were
> narrative and much shorter than transcription. In the past many
> years, the IETF (via Joel) has provided MP3s of the sessions for
> people who want to have an "audio transcription".
> 
> Are the people who are advocating transcription on this thread doing
> so to help the chairs produce narrative minutes, or as a replacement
> for the minutes? If the former, has anyone tried to create their
> minutes from a transcription and, if so, how long did it take you? If
> the latter, this is a significant change from the current IETF
> model.

It took me roughly three hours to produce a literal transcription of a
60 minute meeting and another 20 minutes to turn it into narrative
minutes (you know that material pretty well by the time you get to that
point). this was for ietf 74 I think.

> --Paul Hoffman, Director --VPN Consortium
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 20 May 2010 07:09:28 -0500

Hi, Cullen,

I'm noticing that your first two items referred to "decisions made in 
meeting(s)", and I could read the third item the same way, although that's 
less clear. Was this actually "discussion during meeting(s)"?

I'm becoming more sensitive about this based on what I think I'm seeing at 
recent IETFs. I have a sense that some groups are moving to a model where 
they really do make decisions in meetings, and then ask for rebuttals on the 
mailing list afterwards.

I have opinions about that, if it's really happening, but should ask for 
help first - since this mailing list has pretty much the entire working 
group management team for the entire IETF, I'm wondering what other people 
are seeing (am I just hanging out in the wrong places, or are others 
noticing this as well?).

Thanks,

Spencer

----- Original Message ----- 
From: "Cullen Jennings" <[email-masked]>
To: "Dean Willis" <[email-masked]>
Cc: "IETF WG Chairs" <[email-masked]>
Sent: Wednesday, May 19, 2010 7:04 PM
Subject: Re: Notetaking

3) looking back at old meetings to try and see why some decisions was made 
so I can figure out if it makes sense to re-make the decisions which often 
seems to be happening in WGs

---

From: Yaakov Stein <[email-masked]>
Date: Thu, 20 May 2010 15:20:40 +0300

Yes, recently I have been seeing WG chairs and ADs focusing on getting consensus at meetings, 
and taking the issues to the list as a matter of procedure.

I don't think that is inherently bad.
If most of the people actively involved are present at the meeting,
this method saves a lot of time.

Y(J)S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Spencer Dawkins
Sent: Thursday, May 20, 2010 15:09
To: Cullen Jennings; Dean Willis
Cc: IETF WG Chairs
Subject: Making decisions in meetings (was: Re: Notetaking)

Hi, Cullen,

Thanks,

Spencer

---

From: Dean Willis <[email-masked]>
Date: Thu, 20 May 2010 21:12:31 -0500

> Are the people who are advocating transcription on this thread doing so to help the chairs produce narrative minutes, or as a replacement for the minutes? If the former, has anyone tried to create…

I have produced minutes from professional transcriptions done for a
hearing-impaired participant in SIP, and it was rather painful -- but
given that both note-takers had catastrophic failure, it was the only
viable alternative (this was before we had recordings). I'd see if took
six to eight times as long as it does for "normal" notes.

--
Dean

---

From: Leonard Giuliano <[email-masked]>
Date: Fri, 21 May 2010 12:06:49 -0700

On Mon, 17 May 2010, Paul Hoffman wrote:

-) This thread has diverged into two steams: note-taking for minutes, and transcription. The IETF has traditionally had minutes that were narrative and much shorter than transcription. In the past many years, the IETF (via Joel) has provided MP3s of the sessions for people who want to have an "audio transcription".
-) 
-) Are the people who are advocating transcription on this thread doing so 
-) to help the chairs produce narrative minutes, or as a replacement for 
-) the minutes? If the former, has anyone tried to create their minutes 
-) from a transcription and, if so, how long did it take you? If the 
-) latter, this is a significant change from the current IETF model.

Neither.  I see transcription as a compliment to the audio recording.  If 
someone wants to go back and see what was exactly said in the meeting, 
this would be a useful resource, perhaps easier to search through than by 
navigating the audio.

Minutes/notes still have their place and transcription is no substitute.  
But if we can find a way to get good transcripts at low cost (automated), 
I think it would be a valuable complimentary resource.

-) 
-) --Paul Hoffman, Director
-) --VPN Consortium
-)

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 21 May 2010 18:47:42 -0400

On May 21, 2010, at 3:06 PM, Leonard Giuliano wrote:

> -) This thread has diverged into two steams: note-taking for  
> minutes, and transcription. The IETF has traditionally had minutes  
> that were narrative and much shorter than transcription. In the past  
> many years, the IETF (via Joel) has provided MP3s of the sessions  
> for people who want to have an "audio transcription".
> -)

A goal here IMO should be to cross link the transcript and the audio.  
I have seen this done in
video search, and it can be very effective as a means of facilitating  
media search (the presence or absence of
video is irrelevant for the search in this case). It turns out that
an audio transcription success rate of order, say, 90%, which can be  
highly annoying to read, makes it possible to robustly search for  
discussions by keywords (as they tend to be repeated).

Regards
Marshall

---

From: Dean Willis <[email-masked]>
Date: Sun, 23 May 2010 20:01:11 -0500

> discussions by keywords (as they tend to be repeated).

Check this out:

http://www.softarmor.com/mediawiki/index.php/SIP_Audio_for_IETF_74

Vijay Gurbani put it together from the IETF recordings and my minutes.

It's not cross-linked at the keyword level, just at the topic level, but
given the fallibility of voice recognition this makes sense to me.

--
Dean

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 24 May 2010 08:28:46 -0500

> given the fallibility of voice recognition this makes sense to me.

With the encouragement of Robert Sparks (and before him, Cullen
Jennings), I have promised to make the script that chops the raw
mp3 file into its constituents ready for handing it over to the
IETF tools team.

I am aiming to get this done by the Maastricht IETF.

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 26 May 2010 10:11:31 -0600

Let me separate the word "decisions" in general from topic of decisions that have WG consensus. I think that WG consensus should take into account options of all the folks involved including opinions expressed in a WG meeting at the microphone or via jabber, and opinions expressed on the email list. However, this does not mean that decisions are not made in meetings. There are non WG decisions such as an AD saying they want the work to go to WG X or Y, chair decisions such as document editors, decisions the authors of non WG drafts make about their drafts, but most importantly, the decisions that the people in the room make. 

When we take a hum in a WG meeting, every person that hums, or chooses not to hum, made some sort of decision. So when minutes say we took a hum on if the draft should say option A or option B and there was strong consensus in the room for option B, well that is usefully information about the decision that was made in the room. Now the chairs would need to merge that with input from the list to determine if options B has WG consensus.

On May 20, 2010, at 6:09 AM, Spencer Dawkins wrote:

> Hi, Cullen,
> 
> I'm noticing that your first two items referred to "decisions made in meeting(s)", and I could read the third item the same way, although that's less clear. Was this actually "discussion during meeting(s)"?
> 
> I'm becoming more sensitive about this based on what I think I'm seeing at recent IETFs. I have a sense that some groups are moving to a model where they really do make decisions in meetings, and then ask for rebuttals on the mailing list afterwards.
> 
> I have opinions about that, if it's really happening, but should ask for help first - since this mailing list has pretty much the entire working group management team for the entire IETF, I'm wondering what other people are seeing (am I just hanging out in the wrong places, or are others noticing this as well?).
> 
> Thanks,
> 
> Spencer
> 
> ----- Original Message ----- From: "Cullen Jennings" <[email-masked]>

Cullen Jennings
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 26 May 2010 14:51:50 -0400

--On Wednesday, May 26, 2010 10:11:31 AM -0600 Cullen Jennings 
<[email-masked]> wrote:

>
> Let me separate the word "decisions" in general from topic of decisions
> that have WG consensus. I think that WG consensus should take into
> account options of all the folks involved including opinions expressed in
> a WG meeting at the microphone or via jabber, and opinions expressed on
> the email list.

This is an important point.  We say that WG's make decisions on mailing 
lists, but what that really means is that to participate in a WG's 
decision-making process, it is both necessary and sufficient to do so via 
the mailing list.  A chair cannot declare consensus on an issue if there 
has been no opportunity for discussion on that issue on the mailing list, 
or without considering input from the list.  A chair _can_ declare 
consensus on an issue that has been discussed _only_ on the mailing list. 
But, as you point out, a determination of consensus needs to 
reflectopinions from all participants, regardless of the media used to 
communicate them.

> However, this does not mean that decisions are not made
> in meetings. There are non WG decisions such as an AD saying they want
> the work to go to WG X or Y, chair decisions such as document editors,
> decisions the authors of non WG drafts make about their drafts, but most
> importantly, the decisions that the people in the room make.
>
> When we take a hum in a WG meeting, every person that hums, or chooses
> not to hum, made some sort of decision. So when minutes say we took a hum
> on if the draft should say option A or option B and there was strong
> consensus in the room for option B, well that is usefully information
> about the decision that was made in the room. Now the chairs would need
> to merge that with input from the list to determine if options B has WG
> consensus.

A lot of this depends on what it means to make a decision.  While we can 
_declare_ consensus only after an issue is brought up on the mailing list, 
we often _reach_ consensus during a discussion in a face-to-face meeting. 
So in that sense, we often make decisions in meetings, and then merely 
confirm on the mailing list that there is not some previously-unknown 
objection.  I think this is a fine mode of operation, and in fact important 
if we are to make effective use of meeting time.

-- Jeff