Subject: [vmeet] draft-ietf-genarea-rps-reqs comments

From: Fred Baker <[email-masked]>
Date: Thu, 5 Jan 2012 17:54:43 -0800

A general comment throughout at least section 4 and perhaps the entire document - could we use RFC 2119 language when we're talking about requirements?

>    following the meeting online ("remote attendees").  In the past
>    decade, the IETF has tried to make it easier for remote attendees to

"decades"? We started using the MBONE tools in 1992...

>    participate in its face-to-face meetings a meaningful fashion by
>    providing supported and experimental online services.
> 
>    A functional specification is an approach to meeting one or more
>    requirement.  For example, a requirement might be "chairs must be
>    able to specify which remote attendee can speak next" and a functions
>    specification associated with that requirement might be "floor
>    control can be done through a stand-alone application or web form".
>    Functional specifications are not (currently) called out in this
>    document.

I go both ways on functional requirements such as this. My first thought is that we should specify things we know we will need; it is possible to over-specify to the point that no product (vendor or open source) meets the needs. "It would be nice to have floor control" makes sense, but I think the real requirement is that the host for the call, whatever one calls him/her, should be able to mute someone that can't or won't mute themselves. It's pretty unusual to have a floor fight over the microphone in the IETF.

>    [[[ TODO: Bar BoFs at regular IETF meetings are not listed above
>    because they mostly happen in places where remote participation can't
>    be scheduled.  However, some of them do in fact happen in regular
>    meeting rooms that might be able to use the RPS tools.  Should they
>    be included in this document? ]]]

IMHO, we should not formalize bar bofs. That has gotten Really Out Of Control, starting in Maastricht or Anaheim. If people want to sit down somewhere and discuss a topic of mutual interest, they should do so. If they want the IETF to schedule and provide for a more formal meeting, they should use the process and procedures in place for doing so. Announcing formal bar bofs is mostly painful.

> 3.2.  Locating the Meeting Information
> 
>    Finding information for the real-time audio, instant messaging, and
>    slides for an upcoming or current regular meeting is complicated by
>    that information being in many different locations on the IETF web
>    site, and the fact that the relevant URLs can change before and even
>    during the meeting.  Further, a WG chair might copy the latest
>    information and send it to the WG mailing list, but there can be
>    later changes.  Experienced remote attendees have gotten used to
>    checking just before the meeting itself, but even that does not
>    alway.

Does not alway... what?

> 3.2.1.  Audio
> 
>    A common complaint is that when a WG meeting moves to a different
>    room, remote users need to know about the move so that they can use
>    the proper URL to hear the audio stream.

That can be fixed with a change to the URL. If we have a URL that includes the working group name that redirects to the room that they're actually in, changing a room requires changing the redirect.

> 3.2.3.  Slides
> 
>    Slides are available from the meeting materials page.  Many, but
>    certainly not all, local and remote attendees know how to find the
>    meeting materials page.
> 
>    It has become fairly common for presenters to not have their
>    presentations available for distribution until just before the WG
>    meeting.  Because materials are uploaded by the WG chairs, this often
>    causes the beginning of WG meetings to be a dance involving
>    presenters giving the chairs their slides, followed by chairs
>    uploading the slides to the IETF site, followed by the chairs saying
>    "the slides are there now".

Some chairs don't allow this. I would encourage more chair to do the same.

> 3.3.  Remotely Speaking at the Mic
  ....
>    o  [[[ TODO: More here, of course. ]]]

IMHO, "more here" is the wrong answer. Instead of spending two pages on examples, we need to simply say that we need a way to facilitate remote speakers speaking for themselves.

> 3.4.  Chairs and Floor Control for Remote Attendees
> 
>    Although the previous section may seem like it is a bit harsh on WG
>    chairs, the current tools do not give them the kind of control over
>    remote attendees that they have over local attendees.  The chairs can
>    tell what is happening at the mics, but have much less view into what
>    is happening on Jabber, even if they are watching the Jabber room.
>    Without as much view, they cannot assist the flow of the conversation
> 
> 
> 
> Hoffman                   Expires July 8, 2012                 [Page 10]
> Internet-Draft          Remote Participation Reqs           January 2012
> 
> 
>    as well.
> 
>    o  Carl sees that the Jabber room has an active and useful back-
>       channel discussion during Pete's provocative presentation.  Pete
>       finishes and asks for questions.  Lee and Len and Lou and Lars
>       rush to the mic line, and it takes Robert a few seconds to get his
>       question into the Jabber room and for Sam to go to the mic.  Carl
>       tries to prioritize Sam forward in the line, but Len gets upset
>       when he does.
> 
>    o  Rachel asks a question, but Sam is not going to the mic to relay
>       it.  In fact, Sam has pretty much stopped paying attention.  Chris
>       cannot do something about the situation without making Sam look
>       bad.
> 
>    o  Pete has run over time, Robert asks what is supposed to be the
>       last question, and Pete doesn't understand what Sam said.  Carl
>       cannot tell whether to wait for Robert to rephrase the question or
>       whether Robert even heard Pete's response.
> 
>    o  [[[ TODO: More here, of course. ]]]

It seems to me that the fix for 3.3 fixes this as well. But this doesn't address floor control (Alice is eating a sandwich and slurping soda at her mike and the chair wants to mute her, Bob is talking right over someone else, ...).

Collapse this into 3.3 and then insert a section on floor control.

> 4.  Requirements for Supporting Remote Participation in Face-to-Face
>     Meetings
> 
>    This section covers the functional specification for effective remote
>    participation in meetings where some members are in a face-to-face
>    meeting, such as the regular IETF meetings an interim WG meetings
>    that are held in a meeting room.  Some of the requirements in this
>    section overlap with those in Section 5, but many are unique to
>    meetings that have a significant physical presence.
> 
>    There is an assumption in this section that the meeting chairs will
>    continue to control the flow of the discussion.  That is, if a
>    presenter is speaking and a remote attendee wants to ask a question,
>    the request to do so goes to the chair, not to the presenter.
> 
>    Recordings of the events of the meetings are valuable for remote
>    attendees who are not able to hear everything in real time.  This is
>    reflected in many requirements below.
> 
>    **Requirement 00-01**: The specifications shall rely solely upon IETF
>    and other open standards for all communications and interactions.
>    (This requirement comes from [RPS-RFP].)

So - no solution we have tried since the MBONE actually meets the requirements. I would not dismiss this as "they just want to be proprietary". The IETF has not served itself well if it doesn't take time to find out what the various solutions needed that IETF and other open standards didn't provide.

>    **Requirement 00-02**: All tools in the RPS must be able to be run on
>    the widest possible array of computers.  This means that they must be
>    able to be run as an application, from any modern web browser, or
>    from the command line of all of (at least) MacOS version 10.6 or
>    later, Windows 7 or later, and any common Linux distribution produced
>    in 2010 or later.
> 
>    [[[ TODO: Do we need to include IOS and Android platforms in that
>    list? ]]]

Well, not handheld telephones, but iPad and equivalent. 

> 4.1.  Technologies
> 
> 4.1.1.  Audio
> 
>    A few requirements come from the IETF's current use of audio in
>    meetings.  Meeting rooms have many mics: one or two for the chairs,
>    one for the presenter, and at least one for other local attendees to
>    ask questions.  Plenaries have many more mics, both at the front of
>    the room and in the audience.
> 
>    Note that the requirements here assume a very large change in the way
>    that remote participation will happen.  Instead of a remote attendee
>    typing something into the Jabber room that someone will repeat at a
>    mic in the room, remote attendees will use their own mics to speak to
>    the meeting.
> 
>    **Requirement 00-03**: Remote attendees need to be able to hear what
>    is said by local attendees and chairs at any mic in the meeting.
> 
>    **Requirement 00-04**: Remote attendees must be able to speak
>    directly to a meeting without going through a local attendee, and
>    have their speaking be heard by local attendees.  (Note that the
>    ability to speak is controlled by the chair; see Section 4.2.2.) [[[
>    TODO: is there a requirement that remote attendees who speak be
>    registered as in Requirement 00-30 and Requirement 00-31? ]]]

This has more to do with the IETF funding model and IPR models than anything. The funding issue is that the IETF is largely funded through attendance fees; presumably there would be a fee associated with registration. The IPR issue is that if someone is "in the room" (eg has signed the blue sheet) when an IPR-related topic is discussed, that has legal ramifications later. What if they're on the audio stream? I think the answer needs to be "yes", but I personally would be very happy to see it done though putting one's email address into a web page and receiving a URL in a responding email that is correct for the audio from the room.

>    A common complaint with the current RPS is that the streaming audio
>    can take more than 10 seconds (and sometimes as much as 30 seconds)
>    to reach the remote attendee.  This causes many of the problems
>    listed in Section 3.3. **Requirement 00-05**: Audio going to and from
>    remote attendees must be delivered in as close to real-time as is
>    practically possible. **Requirement 00-06**: Audible echo must be
>    damped or eliminated by the tools.
> 
>    IETF meetings happen in venues such as hotels and conference centers,
>    most of which have their own audio setups.  The IETF Secretariat
>    contracts with those venues for the use of some or all of their audio
>    system. **Requirement 00-07**: The audio system used by the RPS must
> 
> 
> 
> Hoffman                   Expires July 8, 2012                 [Page 13]
> Internet-Draft          Remote Participation Reqs           January 2012
> 
> 
>    be able to integrate with systems commonly used in the venues used
>    for IETF meetings.
> 
>    **Requirement 00-08**: Recordings of the audio for all meetings must
>    be kept for distribution after IETF meetings. **Requirement 00-09**:
>    Users must be able to easily find the audio recording of a particular
>    WG or BoF session at a particular meeting.

It would be helpful if there were some kind of indexing in the recording as well - the ability to find the discussion of a certain topic, or "Joe speaking at the mike". 

> 4.1.2.  Video
> 
>    The RFP that preceded the current document, [RPS-RFP], discusses
>    video as a requirement.  The IETF has experimented with one-way and
>    two-way video at some meetings in the past few years.  Remote
>    attendees have said that seeing people in the meetings gave them
>    better understanding of the meeting; at a recent meeting, a remote
>    presenter was able to see the people in line at the mic and was
>    better able to interact with them. [[[ TODO: determine how much of
>    this is needed for effective participation. ]]]

"need" is a fuzzy term. At Cisco, we use VoIP, webex, VoIP with video displayed on the instrument, and Telepresence. The common report is that the more a person is on video and visible to other attendees, the less the temptation to do email or other work "listening with one ear". That said, video is rarely absolutely necessary (we don't talk with each other using American Sign Language for the Deaf). It does enhance the experience.

>    **Requirement 00-10**: Remote attendees need to be able to see the
>    presenter at a meeting. **Requirement 00-11**: Remote attendees need
>    to be able to see local attendees at any mic in the meeting. [[[
>    TODO: Is there a requirement that the remote attendees see the slides
>    over video? ]]]

..."SHOULD" be able...?

>    **Requirement 00-12**: Remote attendees who are speaking over the
>    audio must be visible to the local attendees.

well, "IF the remote attendee has appropriate supporting infrastructure". If the guy's in an airport calling from his mobile phone...

>    **Requirement 00-13**: Video going to and from remote attendees must
>    be delivered in as close to real-time as is practically possible.

I blow two ways on this. I understand what you're saying, and I agree. That said, when we had a 30 second feed delay in a streaming media path I doubt that anyone thought that was a good thing or intended to make it happen - it was in that configuration "as close to real-time as [was] practically possible". I think you need to say something testable, and say how to fail the test.

>    [[[ TODO: Is there a requirement that IETF video integrate with the
>    venue video, if any? ]]]

"SHOULD"?

>    [[[ TODO: If video is provided, is there a requirement that it be
>    archived and accessible? ]]]

IMHO, while video improves the human experience, there is no information conveyed about the meeting that can't be reconstructed - slides, documents, jabber logs, and audio. After the fact, I consider it "nice to have".
 
> 4.1.3.  Instant Messaging
> 
>    As noted earlier, while the current tool's Jabber room is a good way
>    to get questions to the mic, it also becomes a second communications
>    channel that only a few people in the room are participating in.
>    This document does not address how to prevent that problem (or
>    whether it really is much of a problem).
> 
>    **Requirement 00-14**: The instant messaging system must allow anyone
>    to see all messages in the WG's or BoF's room. **Requirement 00-15**:
>    The instant messaging system must allow any registered user (even
>    those registered anonymously) to post messages in the WG's or BoF's
>    room.

IANAL... I realize that there is a disconnect between the privacy community and the legal community on this, but I suspect that the IPR issues say that we need to be able to determine after the fact who was "in the room". I can make suggestions about how to handle this, but they are out of scope here.

>    **Requirement 00-16**: Transcripts of the instant messaging for all
>    meetings must be kept for distribution after IETF meetings.
>    **Requirement 00-17**: Users must be able to easily find the instant
>    messaging transcripts of a particular WG or BoF session at a
>    particular meeting.
> 
>    **Requirement 00-18**: The instant messaging channel needs to be
>    useful for humming, which is the common IETF method of assessing
>    support.

I understand the goal, but I'm concerned about the implementation. 

Suggestion: "The IM Channel must facilitate straw polls". 

I can imagine someone saying "you can't use Jabber for IM, because it doesn't carry audio". That's not what you mean. If you mean that whatever the audio system is, remote users should be able to hum into it, what prevents them from adjusting their gain?

A better approach might be to enable some form of online poll. Yes, this gets dangerously close to voting. If the system had a way for someone to raise a hand and lower it, and reported (somehow) how many hands were raised, I could imagine a chair asking everyone (local and remote) to click hand-raise if they agree with a statement, and then tell them to take it down while s/he asked the opposite. I could imagine a model in which everyone could click one of three radio buttons: "yes", "no", and "needs further discussion to decide", and the chair sees a real time bar graph.

>    [[[ TODO: Should there be multiple rooms for a meeting?  There were
>    many requests for a separate "speak into the mic" room, but that is
>    not needed if the requirements in Section 4.1.1 are met.  Is there a
>    need for other rooms? ]]]
> 
>    [[[ TODO: Should non-registered people be allowed to read the IM
>    traffic in real time, given that anyone can register anonymously?
>    Should people registered anonymously be allowed to post in IM rooms?
>    ]]]
> 
> 4.1.4.  Slide Presentations
> 
>    **Requirement 00-19**: The input format for slide presentations must
>    be either PDF or PowerPoint. [[[ TODO: Is there a requirement to
>    support other formats? ]]]

We occasionally have people show up with Keynote. I personally have no problem with PPT/Keynote being used for the actual presentation (I personally use animation, which would otherwise have to be created as multiple slides), but there is a part of me that would like to see a ppt uploaded to the meeting materials automagically converted to pdf in the process, and I wouldn't object to being told that the upload had to be pdf. If you want to be non-proprietary, I don't have a good open source alternative to PDF.

>    **Requirement 00-20**: Presenters must be able to update their slides
>    up to just before their presentation, if such update is allowed by
>    the chairs. **Requirement 00-21**: Chairs must be able to approve or
>    disapprove of any slide submission or updates, with the default being
>    that all submissions are allowed.

Can you separate those into two paragraphs?

Speaking as a chair, anything that helps me say "I actually don't have the capability to accommodate that request" helps me in dealing with people who pull stunts like this.

>    **Requirement 00-22**: It needs to be clear to the remote attendees
>    which set of slides is being currently shown.
> 
>    [[[ TODO: Is there are requirement that remote attendees see the

"is there a"

>    slides as the people in the room?  Or, is it sufficient to keep it as
>    it is now where remote attendees must download them and speakers must
>    say which slide they are on?  If the slides will be visible to remote
>    attendees as they are presented, there will be a requirement that the
>    presenter can go back and forth in the slide deck. ]]]

If the slides are visible as presented, that will be met. My question is: is there a requirement for the speaker to select slide order in real time? If so, they have to be able to see the slides as s/he presents them. I would say "yes".

>    [[[ TODO: If the slides will be visible to remote attendees as they
>    are presented, is there a requirement that presenters be able to use
>    the equivalent of a laser pointer? ]]]

I would note that as "nice to have". Laser pointers frequently fail in the room, too.

>    [[[ TODO: Is there a requirement that animation in PowerPoint be
>    supported, or just static slides? ]]]

There are arguments on both sides there. If you force pdf, there is no animation. On webex, if you share the ppt application you get animation, but there is another mode in which you get a static version. I would list it as "nice to have".

> 4.1.5.  Shared Document Editing
> 
>    In some WG meetings, there is an attempt to edit a document with
>    input from the local attendees.  This is typically done for proposed
>    charter changes, but can also be for a WG document as well.  This is
>    usually unsuccessful, given the amount of text and the size of what
>    can be displayed on the screen.  In recent meetings, shared document
>    editing has been used for editing charters and for taking minutes of
>    meetings.

For the record, there is exactly one place where I have used this successfully. In board meetings and similar, every once in a while you have a conversation about a statement that the group would like to make. On a small number of occasions, I have found myself popping up a ppt page, writing in a strawman statement, letting the group tell me how to edit it (which invariably includes discussion of the reasons why it needs to be stated a certain way). Half an hour later we have constructed a sentence that we're willing to see in the minutes. In the end, it's really not about the statement - it's about the conversation and consensus process that created it.

In technical document discussions, I think that is far too time-consuming. I could, however, imagine the minutes to a meeting being constructed that way - we have a conversation, the secretary types "the consensus of the group was to divide by zero and drop a nuclear weapon on the moon", and anyone who disagrees then comments.

>    **Requirement 00-23**: It must be easy to start a new shared document
>    and to import existing text into a shared document.
> 
>    **Requirement 00-24**: Shared real-time editing of text-only
>    documents must be supported.  This system must allow at least three
>    people to have write access and hundreds of people to have read
>    access to any particular document.
> 
>    **Requirement 00-25**: Remote attendees can be either the writers or
>    the readers.
> 
>    **Requirement 00-26**: Those with read access need to see the edits
>    made by those with write access within less that five seconds after
>    each edit.
> 
>    **Requirement 00-27**: It must be easy to change the permissions for
>    who gets write access to a document during an editing session.
> 
>    [[[ TODO: Is this also needed for non-text documents?  If so, in what
>    formats? ]]]
> 
> 4.2.  Tools
>

---

From: John Leslie <[email-masked]>
Date: Fri, 6 Jan 2012 08:33:46 -0500

By the time I completed my first pass through the document, 
Fred Baker <[email-masked]> had written:
> 
> A general comment throughout at least section 4 and perhaps the
> entire document - could we use RFC 2119 language when we're talking
> about requirements?

   A very good point!

   It appears we have some actual MUSTs and a whole bunch of SHOULDs.

> My first thought is that we should specify things we know we will need;
> it is possible to over-specify to the point that no product (vendor
> or open source) meets the needs.

   I'm not greatly worried whether any current product meets the needs:
I'm satisfied that all the ones I've experienced need work!

> "It would be nice to have floor control" makes sense, but I think the
> real requirement is that the host for the call, whatever one calls
> him/her, should be able to mute someone that can't or won't mute
> themselves. It's pretty unusual to have a floor fight over the
> microphone in the IETF.

   Clearly, the ability to mute someone who can't or won't mute
himself is a MUST; but I would claim the MUST goes farther than that:
even soft room noise from a non-speaker can be very distracting.

> IMHO, "more here" is the wrong answer.

   I agree with Paul, not Fred, here.

   I understand Fred's frustration with two pages of examples where
the casual reader gets the roles hopelessly mixed up; but IMHO these
examples need to be documented to understand how really poorly our
current system of "channeling" works.

> Instead of spending two pages on examples, we need to simply say
> that we need a way to facilitate remote speakers speaking for
> themselves.

   The MUST, IMHO, is that remote participants MUST be able to speak
directly and be heard by all participants, whether local or remote,
subject to scheduling by the Chair.

   Some possible SHOULDs:

- a remote participant SHOULD be able to provide real-time video
  while speaking.

- the audio fed to a remote participant SHOULD exclude his own voice,
  but enable him to hear any questions raised by what he says.
  (This probably needs wordsmithing -- the extent of excluding his
  own voice varies greatly depending on how he hears it.)

- there SHOULD be a clear signal that it's time for the remote
  participant to start and/or stop speaking.

- the audio a remote participant receives SHOULD not be delayed by
  more than one second from the audio in the meeting room.

- when network issues interrupt a remote speaker, there SHOULD be
  a clear indication to local and remote participants that this
  is the reason for the silence.

- there SHOULD be a real-time display identifying remote speakers.

- remote participants SHOULD be able (if the Chair consents) to
  ask questions of the other speakers without tool-imposed delays
  exceeding two seconds.

- there SHOULD NOT be any reason to delay the start of a meeting to
  get volunteers to facilitate remote participation.

- it SHOULD be possible for two or three remote participants to
  speak at the same time, interacting without tool-imposed delays
  exceeding one second.

- there SHOULD be a mechanism for remote participants to notify
  local speakers that the audio quality is inadequate.

>>...
> It seems to me that the fix for 3.3 fixes this as well. But this
> doesn't address floor control (Alice is eating a sandwich and
> slurping soda at her mike and the chair wants to mute her,
> Bob is talking right over someone else, ...).

   I'm not sure that's "floor control" -- there's a need to recognize
that remote participants can't always tell how disruptive their
room noise would be; and "floor control" is the typical tool to
ameliorate such problems.

   "Floor control", IMHO, is really about scheduling speakers in a
manner which appears "fair enough" to all participants. One tool
not mentioned is to formalize a display of "Joe first; then Bob".

> Collapse this into 3.3 and then insert a section on floor control.

   Fred may be correct that these examples belong under 3.3...

>>... 
> So - no solution we have tried since the MBONE actually meets the
> requirements. I would not dismiss this as "they just want to be
> proprietary". The IETF has not served itself well if it doesn't take
> time to find out what the various solutions needed that IETF and
> other open standards didn't provide.

   This objection, while valid, seems out-of-scope...

> Well, not handheld telephones, but iPad and equivalent. 

   Those strike me as low-level SHOULDS, barely exceeding "would be nice"
for what we're specifying here. I'd even back off on the "any common
Linux" for the first pass.

   Eventually, of course, we'll have to do Android; but at first we'll
probably only support common Browsers on common platforms.

>>... 
>>    ability to speak is controlled by the chair; see Section 4.2.2.)
>>    [[[TODO: is there a requirement that remote attendees who speak be
> This has more to do with the IETF funding model and IPR models than
> anything.

   IMHO, anyone who "speaks" should be identified as part of the record.
I'm not sure funding issues belong in-scope; and I'd rather not confuse
folks with IPR models.

> The funding issue is that the IETF is largely funded through
> attendance fees; presumably there would be a fee associated with
> registration.

   True, but out-of-scope IMHO.

> The IPR issue is that if someone is "in the room" (eg has signed the
> blue sheet) when an IPR-related topic is discussed, that has legal
> ramifications later.

   True, but confusing.

   There is a definite "blue-sheets" issue, which probably deserves
a Requirement (though I'm not sure what it might be).

> What if they're on the audio stream? I think the answer needs to be
> "yes",

   To me, anyone who speaks should be "registered", but someone who
only listens and follows slides isn't obviously different from one
who listens to the recording later.

> but I personally would be very happy to see it done though putting
> one's email address into a web page and receiving a URL in a
> responding email that is correct for the audio from the room.

   Works for me...

>>    listed in Section 3.3.
>>    **Requirement 00-05**: Audio going to and from remote attendees
>>    must be delivered in as close to real-time as is practically
>>    possible.

   I'll break in here...

   My intuition is that there's a MUST here, of the nature of

- audio delays MUST NOT be more than 500 milliseconds greater than
  the network delays.

>>    **Requirement 00-06**: Audible echo must be damped or eliminated
>>    by the tools.

   This exceeds the possible, in a system with jitter (which we can't
avoid). OTOH, audible echo delayed one second or more is incredibly
distracting. And two or three echos of varying delays can render
a conference impossible to follow.

   I think the requirement is more like

- the tools MUST recognize audible echo, and SHOULD automatically take
  measures to reduce it to a level which won't distract listeners.

>>... 
>>    presenter at a meeting.
>>    **Requirement 00-11**: Remote attendees need to be able to see local
>>    attendees at any mic in the meeting.
>>    [[[TODO: Is there a requirement that the remote attendees see the
>>    slides over video? ]]]
> 
> ..."SHOULD" be able...?

   These three feel like SHOULDs to me, with seeing the slides exactly
as presented being the closest to a MUST. (It seems the easiest, and
getting out of sync with the slides is a _serious_ impediment.)

> well, "IF the remote attendee has appropriate supporting infrastructure".
> If the guy's in an airport calling from his mobile phone...

   That's a very real consideration (though hopefully it's not _usually_
an airport).

   To me the requirement is

- all participants MUST have a means of recognizing who is speaking
  remotely, and SHOULD be able to see video provided by the remote
  system.

> I blow two ways on this. I understand what you're saying, and I agree.

   Video generally can tolerate a several-second delay; and I wouldn't
hold it to the same standard as audio.

> That said, when we had a 30 second feed delay in a streaming media
> path I doubt that anyone thought that was a good thing or intended to
> make it happen - it was in that configuration "as close to real-time
> as [was] practically possible".

   Alas! that's true...

> I think you need to say something testable, and say how to fail the test.

   Indeed. (And that's why I think we can only specify non-network delay.)

>>    [[[ TODO: Is there a requirement that IETF video integrate with the

   I don't think there's such a need; but it would be nice for the
IETF video to be available on local laptops.

> IMHO, while video improves the human experience, there is no information
> conveyed about the meeting that can't be reconstructed - slides,
> documents, jabber logs, and audio. After the fact, I consider it "nice
> to have".

   I mostly agree; but I think we need a strong SHOULD about recording
slides as presented.

>>... 

   The problem here is that "humming" needs a definition. To me, a
"hum" button separate from a "raise hand" button is called for.

> Suggestion: "The IM Channel must facilitate straw polls". 

   Good to talk about "straw polls" in order to clarify that it must
be clear what is being hummed.

   Less good to suggest that the _tool_ should constrain how we hum.

>>    be either PDF or PowerPoint.

   PowerPoint is inadequately defined. (Actually the PDF definition is
also deficient, but only a little.)

   I'm not sure what this Requirement intends. Folks _will_ show up with
PowerPoint slide-sets; and I suppose it would be good if the tools we
use accept these about as well as a WGC's laptop would... But the real
issue is that remote participants should see the same video that local
participants do.

>>    the chairs.

   I have seen many cases of updating slides _during_ the presentation.
This doesn't feel like the right Requirement...

>>    **Requirement 00-21**: Chairs must be able to approve or
> Speaking as a chair, anything that helps me say "I actually don't have
> the capability to accommodate that request" helps me in dealing with
> people who pull stunts like this.

   These Requirements as currently worded don't seem helpful enough to
overcome Fred's reluctance.

>>...
> If the slides are visible as presented, that will be met. My question
> is: is there a requirement for the speaker to select slide order in
> real time?

   Absolutely!

> I would note that as "nice to have".

   I don't think this rises to the level of SHOULD.

>>    supported, or just static slides? ]]]

   Many presenters won't accept a rule against animation.

> There are arguments on both sides there. If you force pdf, there is no
> animation. On webex, if you share the ppt application you get animation,
> but there is another mode in which you get a static version. I would
> list it as "nice to have".

   It comes down to what we mean by "supported"... To me, the requirement
remains "visible to remote participants as presented".

>> 4.1.5.  Shared Document Editing

   Fred raises good points, but I won't comment on them.

====

   I have a bunch of scribbles which don't seem important enough to
add here.

   I will note that Requirement 00-33 probably doesn't need to specify
"the IETF Secretariat".

   In Requirement 00-43, I'd add that actually turning on "all mikes"
is often a bad choice. It kind-of works for IESG sessions, but there
are a number of instances where the flow is interrupted by "Alphonse
and Gaston" moments. It's the best-choice of those available, but it's
easy to imagine better choices.

   More particularly, "all mikes" picks up entirely too many extraneous
sounds, and (when it gets "bad enough") everything stops to diagnose
which mike(s) are responsible for the worst distractions.

   In Requirement 00-45, I think it unwise to specify a particular
authentication method.

   In Section 4.4, I think it unwise to set an expectation that remote
participants will be recognized to speak, since there may be many, and
_they_ aren't anxious to get to dinner. ;^)

--
John Leslie <[email-masked]>

---

From: Eliot Lear <[email-masked]>
Date: Fri, 06 Jan 2012 16:05:17 +0100

> "decades"? We started using the MBONE tools in 1992...

Indeed I recall someone making a comment in the Amsterdam '93 plenary
using vat <http://ee.lbl.gov/vat/>.

Eliot

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 Jan 2012 07:33:18 -0800

On 1/6/2012 7:05 AM, Eliot Lear wrote:
> Indeed I recall someone making a comment in the Amsterdam '93 plenary using vat
> <http://ee.lbl.gov/vat/>.

I don't remember which IETF it was but it was in the US, where a plenary 
included a voip demonstration by serious voice audience participation from 
Germany and Australia, as I recall.  This was my first sense of how profoundly 
things were going to be able to change.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Randy Bush <[email-masked]>
Date: Sat, 07 Jan 2012 00:44:20 +0900

> using vat <http://ee.lbl.gov/vat/>.

the old mcast tools used to be two way, it was common to have a voice
come out of the ceiling.

it was when we moved to 'modern' tools, often commercial ones oriented
to consumer one way culture, that we lost the ability for the remote
participant to be heard.

i also think that much of the discussion i have read is all waaaaaay to
specific and detailed.  let me pick on john for an example

>   own voice varies greatly depending on how he hears it.)

i suspect that what you mean is that the audio technology should be good
quality and damp feedback in both directions etc.

but why are we even at that level of detail.  i think that when we say
we want function X that it is safe to assume we want high quality X.

i think a good exercise would be to write two sets of bullets that fill
less than one page that list short term functions we need and could
realistically expect in a year and then long term wishful ideas.  and
expect that the latter will change a lot as we learn.

put this thing on a diet!

randy

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 Jan 2012 07:53:19 -0800

> expect that the latter will change a lot as we learn.

+1

d/
-- 

   Dave Crocker

---

From: Randy Bush <[email-masked]>
Date: Sat, 07 Jan 2012 01:36:35 +0900

putting my money where my mouth is ...

for personal and somewhat superstitious reasons, i do not go to meetings
in the southeastern united states.  so for atlanta, i would like
  o to be able to see and hear any of the wg/bof meetings and plenaries
  o when there are slides or movies, i want to be able to see them or
    even pull a pdf to view locally
  o i would like semi-pro video where i can see the speaker, floor
    speaker, etc.  merit used to do a quite good job of this for nanog
  o i would like to be able to make a presentation, with my face visible
    and show slides and hear and answer questions from the floor and
    other remote participants
  o i would like to be able to speak at the virtual floor mic and, if
    possible, have my face shown to the meeting room and to other remote
    folk

i do not want to constrain or over-specify how this works.  i.e. how the
speaking stone is handed around, ...  it's gonna be hard enough to do
without requiring 42 kinky sub-features and have know-it-all amateurs
micro-specifying it.

as much as possible the protocols and the software at the remote end
should be non-proprietary.  and it must run on unix, mac, linux, and
optionally ms-windows. :)

randy

ps: shoot the jabber/im as a formal channel.  it's fine for an informal
    ad hoc social s[n]ide channel

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 6 Jan 2012 09:17:25 -0800

Starting a properly-titled thread, and pulling something from Randy's short list:

>    even pull a pdf to view locally

Those are very two very different requirements. Do you have a strong preference for one way or the other? If so, why?

From the draft:

--Paul Hoffman

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 6 Jan 2012 12:27:53 -0500

>   presenter can go back and forth in the slide deck. ]]]

It has to be both IMO.

IETF meeting policy is that slides be available from the proceedings
page before the session starts, so you have to be able to pull a PDF.

However, sometimes slides change at the last minute, even during a
presentation, or there are animations or other
features that don't convey well, or the slides look different in the
original and the PDF. In addition at times I, and I assume others,
have trouble figuring out exactly what slide in the deck is being
shown, and I have even seen files be mislabeled, so I fruitlessly hunt
back and forth for the slide on the screen from a deck that doesn't
have it. So, I think you need to be able to see just exactly what is
shown at the time it's being shown.

Regards
Marshall

> --Paul Hoffman
>
> _______________________________________________
> NOTE WELL: This list operates according to
> http://mipassoc.org/dkim/ietf-list-rules.html.
> https://www.ietf.org/mailman/listinfo/vmeet

---

From: John Leslie <[email-masked]>
Date: Fri, 6 Jan 2012 13:28:11 -0500

Randy Bush <[email-masked]> wrote:
> specific and detailed.

   Were we starting from scratch, I'd agree.

   But this is a commissioned report, and I think the level of detail
is appropriate for a commissioned report, although there needs to be
an indication of lower priority for some things it lists.

> let me pick on john for an example

   No problem!

> quality and damp feedback in both directions etc.

   No -- strangely enough, I meant what I said, which had no mention of
audio quality or feedback.

   I meant that there is a need to exclude the voice of the remote
participant (which will likely be done by muting); but that this must
not prevent the remote participant from actually hearing questions
which may be raised.

   I did not mean he has to hear them in actual real-time: that may
prove impractical. But he must be aware of them in near-real-time,
and should be able to hear them in their entirety.

> but why are we even at that level of detail.

   The IAOC has determined that improved remote participation is a
priority. (I agree strongly, if you haven't guessed.)

   It makes no sense to demand improved participation and then refuse
to define what you mean by "improved participation".

> i think that when we say we want function X that it is safe to
> assume we want high quality X.

   That is not how procurement works.

   As a matter of fact, it's not clear we demand audio quality any
higher than telephone service of 40 years ago. (That question may
deserve some consideration, but it's not on the floor right now.)

> expect that the latter will change a lot as we learn.

   Were we starting from scratch, that would be an excellent start.

> put this thing on a diet!

   Much though I admire a killer-instinct for the unnecessary word,
that's not what we need right now.

   (IMHO... YMMV...)

---

From: Randy Bush <[email-masked]>
Date: Sat, 07 Jan 2012 07:37:45 +0900

> Those are very two very different requirements. Do you have a strong
> preference for one way or the other? If so, why?

they are two ways of meeting one need, to see the slides.  i am sure
there are more.  as what my list is asking is difficult, i choose not
to over-specify *how* it is accomplished.  i wanna see the slides.

my guess is that seeing them as projected, while seeing the speaker and
the floor speakers, requires more cameras and so forth, downloading the
pdf may place less burden on the implementation.  but that is just a
guess.  i can live with either.

btw, in '98, when i was laid up at home, i remotely spoke at and
participated in a caida workshop which pretty much did all this.  

i suspect we have wrapped ourselves around the axles of ephemeral tool
technologies (jabber, meewhatever, ...) and itu-like process.  we have
become cumbersome and have forgotten how to listen to the users,
experiment, prototype, ...  so what we do today is not even as useful as
what we did in the past when we were young and alive.

let's drop this massive document crap and empower some extreme media
geeks and let them play in paris and vancouver and see what actually
works and is useful.

randy