Subject: [vmeet] Meetecho tutorial at IETF83

From: Simon Pietro Romano <[email-masked]>
Date: Thu, 15 Mar 2012 13:27:39 +0100

Dear all,

I would like to remind you that we are going to offer a tutorial at the 
upcoming IETF in Paris: 
http://www.ietf.org/meeting/83/tutorials/meetecho.html.
We'll try to provide all of you (both chairs and ordinary users) with 
useful information about how to get the best out of our collaboration 
platform when joining an IETF room.
The tutorial is obviously open to remote participants. In order to let 
you join the tutorial session, a dedicated link will be made available 
here: http://ietf83.conf.meetecho.com/.

Cheers,

Simon

-- 
                             _\\|//_
                             ( O-O )
    ~~~~~~~~~~~~~~~~~~~~~~o00~~(_)~~00o~~~~~~~~~~~~~~~~~~~~~~~~
                     Simon Pietro Romano
               Universita' di Napoli Federico II
                  Computer Science Department
         Phone: +39 081 7683823 -- Fax: +39 081 7684219
                 e-mail: [email-masked]
           http://www.comics.unina.it/simonpietro.romano

     <<Molti mi dicono che lo scoraggiamento è l'alibi degli
    idioti. Ci rifletto un istante; e mi scoraggio>>. Magritte.
                          oooO
    ~~~~~~~~~~~~~~~~~~~~~~(   )~~ Oooo~~~~~~~~~~~~~~~~~~~~~~~~~
                           \ (    (   )
                            \_)    ) /
                                  (_/

---

From: John C Klensin <[email-masked]>
Date: Sun, 25 Mar 2012 10:32:55 -0400

--On Thursday, March 15, 2012 13:27 +0100 Simon Pietro Romano
<[email-masked]> wrote:

> Dear all,
> 
> I would like to remind you that we are going to offer a
> tutorial at the upcoming IETF in Paris:
> We'll try to provide all of you (both chairs and ordinary
> users) with useful information about how to get the best out
> of our collaboration platform when joining an IETF room.
> The tutorial is obviously open to remote participants. In
> order to let you join the tutorial session, a dedicated link
> will be made available here: http://ietf83.conf.meetecho.com/.

Simon,

I delayed commenting on this because the point it makes about
how the history and pattern of a key aspect of remote
participation is more important than whether I was able to be
part of the tutorial today (or get to listen to it later).  It
is a different sort of tutorial.  I started writing this note
about 10 minutes before the tutorial started.   Putting it
together, searching for where I might have reasonably looked for
information, and documenting the process took considerably
longer than the scheduled time for the tutorial itself.

Nothing is going to work well for remote users, especially
remote users who assume that they ought to be able to "just
participate", if we can't actually get all the details in place.

So, picking on you for a tutorial example, the announcement
above was sent out ten days ago.   It doesn't contain a remote
participation link.  It says that the link will be available at
will be made available here: http://ietf83.conf.meetecho.com/.
That illustrates a separate problem (see below), but ok.
However, that page shows absolutely nothing on Sunday.  It says
that one should look at the "Tutorials section", but nothing
there either.

So, nothing that your links are of the form
http://www.meetecho.com/ietf83/WGACRONYM, following the pattern
for Jabber links, etc., the moderately-interested and
moderately-experienced remote participant goes to the IETF
agenda page.  Whoops -- no acronym for the Meetecho tutorial.
No help from the Tools agenda either -- a room number, but no
links.  At this point, any sensible remote participant on the US
east coast decides that breakfast is a better idea than the
tutorial, ones on the west coast are really regretting getting
out of bed, etc.

If our would-be remote participant is _really_ determined
(whether to participate or to get frustrated), she looks at the
links on the Meetecho site and tries to synthesize.  Perhaps
http://www.meetecho.com/ietf83/meetecho, perhaps
http://www.meetecho.com/ietf83/tutorial.  Nope, both give 404
messages.  Wonder whether the normal IETF audio streams from the
room is up.. so find a WG that meets in the room later in the
week or go to the (hard to find) remote participation page, and
extract the URL
(http://ietf83streaming.dnsalias.net/ietf/ietf835.m3u), but it
yields "stream not found".    As a last resort, see if by some
chance WebEx is turned on as well as Meetecho but, at least
according to the remote participation page, no chance.

One last thing to try: was there a thread on the vmeet list that
followed up this announcement and gave a link?  So let's find
the vmeet archives assuming that one is not already subscribed
(perhaps the remote participant's main interest is getting work
done in the IETF while being remote, not designing remote
participation).  Well, it isn't a WG, so it should be listed on
the non-WG Mailing Lists page, right?   Not a trace (even after
looking for "remote participation" and "rpsreqs" too).  Not on
the active WGs page either, or course.   If our intrepid
adventurer says "aha, there is a BOF on Friday, maybe mailing
list information is there", he discovers that there is no link
to a charter and other information page (since it is a BOF, duh)
and that she agenda page shows an agenda, with no hints about
documents or mailing lists.  Being particularly determined, she
goes to the I-D tracker with the assumption that, if there is a
draft associated with a BOF, it might actually have the BOF
acronym in its name, but that fails too.  Only if someone either
knows the author's name; looks it up on the assumption that,
while it is BOF, it is treated as a General Area WG; or guesses
that, while the BOF acronym is repreqs, the draft might use
rps-reqs is one going to find that draft.  

Once found, there is a clear statement about the name and
location of the mailing list (Paul has been one of the best
people in the IETF about consistently including that information
and may have been one of the first to insist on it). but a
search of that list's archive for the Meetinfo tutorial link I
was looking for turned up nothing.

I knew enough to be able to skip several of those steps on the
way to finding out that this was hopeless.  Possibly that
knowledge led to even more frustration, as I spent time trying
things that might not have occurred to others, or perhaps it
saved time.  But the point is that we aren't getting enough of
the ducks lined up to make effective remote participation easy
and having all of the technology lined up wouldn't change that.

This experience suggests that several things are missing from
draft-ietf-genarea-rps-reqs-03 and our practices:

(1) The "Scanarios" list in Section 2 of the I-D should include
"tutorials at IETF meetings" and maybe "online tutorials on IETF
tools and similar matters" more generally.  In addition to this
Meetecho session, I can see no possible reason why the I-D
tools, RFC Editor, and even Newcomer sessions (and others)
should not be made available to remote participants.  Some of
them, if made available in recordings from prior meetings, might
even free up IETF f2f time for other things.   Probably they
aren't different in their requirements from the Scenarios that
are listed, but today's experience provides an indication that
not being explicit about these things is a way to lose or forget
them. 

(2) This experience strongly reinforces Section 3.2 of the I-D
and suggests that it doesn't go far enough.  Not only is it hard
to find these materials, but they someones don't even exist,
turning the search into an exercise in frustration or worse.
The "two different agendas" statement and bullets is not even
correct.  As far as I can tell, there are now at least two more:

	o The "Remote Participation Agenda"
	(http://www.ietf.org/meeting/83/remote-participation.htm
	l for this meeting)  and
	
	o The Meetecho Agenda.  Actually, there are two of
	these, one at the bottom of the Remote Participation
	Agenda and one at http://ietf83.conf.meetecho.com/ (with
	a link from the former).  I haven't verifies whether
	they are consistent.  It is clear that they are not
	actively maintained (e.g., there are links to Meetecho
	information for WGs that have been cancelled.

There are no links to, or hints about, the first of these on
either the datatracker or tools versions of the main agenda.  No
comments on or links from the meeting materials pages either. In
a significant improvement over some previous years, there is a
"Remote Participation" link from the main meeting page
(http://www.ietf.org/meeting/83/) that leads to the right place,
but whether someone who was simply looking for Meetecho (or
other information for a particular session) would think to
navigate back there and then guess that was the place to look is
a open question.
 
(3) In some sense, all of the above might be reasonably covered
by  **Requirement 03-61** in Section 4.13 of the current draft.
But I believe that section and requirement actually illustrate
that applicability of a general IETF problem to this particular
area: it is far more interesting to debate small, but
fascinating, technical details (e.g.,  the proposed requirements
about specific maximum millisecond delays and frame rates in the
first part of Section 4).    By contrast 03-61 essentially says
"good stuff must happen".

In particular, if we are serious, all of the remote
participation links have to be clearly available from a
comprehensive agenda page (extending the tools agenda would be
reasonable), that agenda page should be clearly identified on
the meetings page (not via a level of indirection or two), and
the default should be that any meeting or session that is
important enough to list on an IETF agenda page should have
remote access available and listed.  That agenda page should
probably show the relevant "click here for help with..." links
as well, not rely on participants (especially remote ones) to
remember where they put the email message (or last saw the web
page) that contained that information.  People should not be
expected to flip back and forth among several places to find
information and WG and BOF Chairs should not be responsible for
seeking out and organizing remote participation setups much less
remembering how and where to advertise them.  Any page that
shows remote participation information should be actively
maintained as part of the agenda process so that changes to the
meetings being help or where they are being held foul thinks up
and require (remote) chasing around to find them.  Probably that
will require additional Secretariat staff hours, but that
support is at least as important to effective remote
participation as having the right tools.  I cannot stress too
much that, if an intended remote participant cannot access the
appropriate session tools and materials without a search
adventure or reaching out in advance to a WG or session Chair,
it doesn't make any difference how good the tools are.

And, if a particular session or side-meeting does not have
remote participation support, the would be remote participant
should be able to find that out easily and immediately, rather
than having to search all possibilities (or write multiple
notes) before inferring that no support exists.

Paul, I would suggest isolating "Location of and Navigation to
remote access facilities" (or some better version of that title)
into a separate section or subsection to highlight the
importance of these issues to stress its importance.   Merely
listing a requirement that is buried in a subsection of
mostly-unrelated material is probably not good enough,
especially given the community's tendency to focus on technology
rather than the human-supported infrastructure required to make
it usable.

remotely,
   john

p.s. we once had a convention for naming Internet-Drafts.  It
was that there were only three types of names:

  draft-ietf-WGNAME-...        (for WG drafts)
  draft-OtherBodyName-...      (for a very restricted list 
                           of "Other Bodies", e.g., the IAB,
                           IANA, RFC Editor,...)
  draft-PrimaryAuthorName-...  (for everything else)

Those rules were enforced by the Secretariat and submission
process.  PrimaryAuthorName was required to be the name or
abbreviation of an Author of the document, not a third party, or
a fanciful combined name.   Other Bodies were long-lived,
well-known, and their documents were assumed to represent
consensus team efforts or important policy statements from a
body that made policy -- no purpose-invented organizations or
fanciful ones.

We expanded "WGNAME" at some stage to allow IETF Areas and
similar bodies to use their acronyms, but my recollection is
that we've seen both draft-ietf-iesg-...  and draft-iesg-...
drafts over the years.

I can't even find the rules any more.  Certainly they are not on
the I-D page.  Maybe we no longer care and, except for official
WG drafts that still follow the first convention above, the rule
is now draft-WhateverYouWantToCallIt-...

This is related to the remote participation question because
either we have to get serious about naming again, using either
tools or takedown rules to enforce whatever rules exist, or our
draft searching tools have to be able to do a deep enough search
that "rpsreqs" or "vmeet" would have been sufficient to find
draft-ietf-genarea-rps-reqs/.  Similarly, entries on the "non-WG
mailing lists" page ought to be generated automatically from
whatever process creates those lists.   If it is not, we will
continue to have one of the disconnected mentioned above.  These
things are particularly important for remote users who may not
have easy access to the time-honored search technique of knowing
who to ask to get a quick and accurate answer. If we are serious
about remote participation, we had better be serious about those
issues too.

---

From: Simon Pietro Romano <[email-masked]>
Date: Sun, 25 Mar 2012 17:34:33 +0200

Hi John,

what can I say...I'm really sorry that you had such a hard time trying 
to get the link. We actually sent the announcement on the vmeet mailing 
list as soon as we were done with the setup of the tutorial room. We had 
to first find the mixer guys, then explain to them what we needed in 
order to connect to both audio input and audio output, then help them 
with the cables and the mixer configuration, and so on and so forth. I 
can just add that the organization of the tutorial was a bit more 
complicated than with the usual IETF rooms, for the reasons that you 
illustrated in detail in your e-mail. And, yes, it is definitely true 
that the devil is in the details and that we do need to better design, 
prepare and announce events envisaging the presence of remote 
participants. As to the agenda for remote participation, it should be 
up-to-date and coherent with the actual program. The links it contains 
should be all correct and point to the right Meetecho room once they are 
activated (i.e., a few minutes before the session starts). If you find 
inconsistencies, just let us know and we'll fix them up. I hope you will 
be able to participate and have a less stressful experience next time. 
Anyhow, thank you for your precious feedback and considerations.

Cheers,

Simon

---

From: John C Klensin <[email-masked]>
Date: Sun, 25 Mar 2012 12:10:24 -0400

--On Sunday, March 25, 2012 17:34 +0200 Simon Pietro Romano
<[email-masked]> wrote:

> Hi John,
> 
> what can I say...I'm really sorry that you had such a hard
> time trying to get the link. 

Simon, thanks,

Don't be.  I think we are likely to learn far more from the
experience than we would have learned had things been more
smooth.  The underlying problem is real and it isn't your fault. 

> We actually sent the announcement
> on the vmeet mailing list as soon as we were done with the
> setup of the tutorial room. 

Good move, but far too late.  The problem for me is that I've
overwhelmed by email with a lousy S/N ratio, so I've been
following vmeet by reading through the archives every once in a
while rather than being subscribed to the mailing list.  I
finally subscribed this morning, but apparently too late to get
your message (nor had it shown up in the archives by the time
the session started).

FWIW, I'm coincidentally had connectivity problems with one of
my ISPs in the last few days, forcefully reminding me of
something we often forget: email is designed to be robust but
not necessarily real-time.

> We had to first find the mixer
> guys, then explain to them what we needed in order to connect
> to both audio input and audio output, then help them with the
> cables and the mixer configuration, and so on and so forth. I
> can just add that the organization of the tutorial was a bit
> more complicated than with the usual IETF rooms, for the
> reasons that you illustrated in detail in your e-mail.

Yes.  And it shouldn't be your problem.  Yes, things were a
little special because the tutorial was about a remote
participation tool.  But, had I decided I wanted to listen in on
one of the other tutorials, I would have had equal or greater
problems -- in your case, I at least knew where and to whom to
complain (you, your "team" list, and the vmeet list).  If I had
wanted to find out about Tools for Internet-Drafts or OAM, I
would not have had even a clue.

> And,
> yes, it is definitely true that the devil is in the details
> and that we do need to better design, prepare and announce
> events envisaging the presence of remote participants. As to
> the agenda for remote participation, it should be up-to-date
> and coherent with the actual program. The links it contains
> should be all correct and point to the right Meetecho room
> once they are activated (i.e., a few minutes before the
> session starts). If you find inconsistencies, just let us know
> and we'll fix them up. 

The one I noticed is that EAI is still listed in spite of our
canceling some days ago.  The last time I looked, the main
agenda showed it as cancelled; the tools agenda shows it as
canceled but still shows audio and Jabber links.  Whether that
is desirable or not is a question that someone should address in
a serious way.  As EAI co-chair, I certainly don't want a
permanent record of whatever is said within range of a
microphone in 212/213 on Monday afternoon registered as an EAI
audio recording -- nor do I expect to have to watch for such a
thing and ask that it be taken down.

> I hope you will be able to participate
> and have a less stressful experience next time. Anyhow, thank
> you for your precious feedback and considerations.

Based on the last time I tried to do this (Prague), it will get
a lot easier once the regular sessions start Monday morning :-).

Paul, let me make a specific suggestion in public.  If you have
not successfully drawn rpsreqs/ vmeet to a close before
Vancouver, stay home.  One learns lots of things trying to
actually do this that people will forget to tell you about and
that don't have quite the same immediacy and impact in reports
than if one experiences the frustration in person.  One of them
is that, for many people, IETF now starts with supplemental
meetings on Saturday morning.  Leaving people who are sitting in
the meeting to cope or make their own arrangements may be
reasonable but, if we believe in Internet-based remote
participation, the meeting network actually needs to be up by
breakfast time Saturday, not sometime Sunday afternoon. 

Thanks again to all for trying to make this work.

best,
   john

---

From: John Leslie <[email-masked]>
Date: Sun, 25 Mar 2012 17:54:55 -0400

> Thanks again to all for trying to make this work.

   +1

--
John Leslie <[email-masked]>