Subject: [edu-team] IETF 99 Tutorial Survey Results

From: Alexa Morris <[email-masked]>
Date: Tue, 22 Aug 2017 13:27:54 -0700

Edu Team and IETF 99 Tutorial Presenters,

Below please find links to the complete results from the IETF 99 tutorial surveys. Some respondents took the time to provide comments on how the tutorial might be improved and I’ve included those comments here so that we make sure to review them.

1. Newcomers Orientation
-   16 respondents
-    Online results: https://www.surveymonkey.com/results/SM-TGBYL9T6/
-    Ideas for improvements:
	*  Should also give a brief overview of the various subjects that IETF are currently covering.
	* Though IETF is not a "hierarchical" organization a diagram over the organization with areas, area-directors, working groups, wg-chairs, etc would be usefull to illustrate the width of the organization. Maybe also hand out some leaflet detailing the areas and briefly the working groups to give an overview of the tremendous amount of work that is being done in a wide variety of fields.
	*  I didn't know what to expect of some events, e.g. Bits and Bytes. Maybe in this tutorial you can promote hackathon activities, it could be useful even though this happens before, this could encourage someone to participate in hackathon in posterior meetings
	* Add a kind of organigram, explaining/showing the ADs and other parties of the IETF.
	* The what-to-do-if-harassed coverage didn't make me feel very reassured. It would help to have a clear pointer other than "look for someone with this color of tiny dot". Also the xmpp.net/directory.php link is broken.
 
My own thoughts on these comments: 
- Two respondents call out the desire for an organizational chart. I think the EDU team should discuss options for how to best provide this info, as clearly people are looking for it. 
- The concern about harassment is an interesting one. Perhaps we should have the Ombudsteam review the slides and provide us with some guidance. This is an important topic.

2. IRTF Overview
-  13 respondents
-   Online results: https://www.surveymonkey.com/results/SM-2JVQV9T6/
-   Ideas to improve:
	*  I liked the sci to themed pictures, but most weren't relevant to the corresponding groups. I wish you described more about what each RG does.

3. DNS Privacy
-   7 respondents
-    Survey results at https://www.surveymonkey.com/results/SM-PPCQ7T75/
-    Ideas for improvement:
	* Nope
	*  More of them and in greater detail.
	* Get more people to see this briefing. They need to know this stuff. Presenter was excellent.

4.  IEEE 802.1 Time Sensitive Networking at IETF
-   5 respondents
-   Survey results at https://www.surveymonkey.com/results/SM-GPQFQ296/
-   Ideas for improvement:
	* i did not like speakers switching in/out multiple times. one voice is better for smooth flow of concepts. Otherwise just divide all content so that same speaker comes in just once.
	* Excellent overview. Perhaps not enough time spent on the details... But the demo was great -- it drove home the industrial use case.

---

From: Fred Baker <[email-masked]>
Date: Tue, 22 Aug 2017 13:50:37 -0700

> On Aug 22, 2017, at 1:27 PM, Alexa Morris <[email-masked]> wrote:
> 

The big question in such an org chart has always been "how to draw the IAB" and "how to draw ISOC". In various org chart drawings I have seen over the years I have seen IAB and ISOC drawn as the top bubble, and quite a bit of ensuing debate terminating in "they aren't the top bubble" but nothing definitive as to where they belong. 

Excluding IAB and ISOC (and the IAOC?), it's not too hard. I think the simplest way to display ISOC and IAB without causing a riot will be to show them as adjunct organizations; "these are also involved, but it's not a hierarchy".

                          +----+
                          |IESG|
                          +----+
              Some number of areas (currently 7)
   -+--------------+--------------+-----------------
    AD             AD             AD
    |              |              |
    +- directorate +- directorate +- directorate
    |              |              |
    +- WG          +- WG          +- WG         ...
    |              |              |
    +- WG          +- WG          +- WG
    |              |              |
    +- WG          +- WG          +- WG
    |              |              |
    +- ...         +- ...         +- ...
    |              |              |

---

From: János Farkas <[email-masked]>
Date: Wed, 23 Aug 2017 10:19:26 +0200

Hi,

Thank you very much for all your help with the tutorial!
Thank you for sharing the survey results!

Regards,
Janos

---

From: Russ Housley <[email-masked]>
Date: Wed, 23 Aug 2017 12:13:27 -0400

This is from an old presentation. I think it was originally put together by Olaf.  Use any part that is helpful.

Russ

> On Aug 22, 2017, at 4:50 PM, Fred Baker <[email-masked]> wrote:
> 
> 
> 
> _______________________________________________
> edu-team mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/edu-team

---

From: Michael StJohns <[email-masked]>
Date: Wed, 23 Aug 2017 12:48:49 -0400

The more I look at this, the less I like the idea of including a wiring 
diagram for an organization that doesn't really do wiring diagrams.

Basically, this isn't information that they need to know for their first 
week and I'm loath to include it on that basis.  What they need to know 
is that working groups belong to an area and that other WGs of interest 
may be in that area so when reading the agenda concentrate on WGs of the 
same area.  I already make that point.  For the rest I refer them to the 
Tao of the IETF, the IETF website etc.

Here's the descriptive text.

      * Organized activity of the Internet Society
      * A voluntary Standards Development Organization
      * Consists of !many! Working Groups
          o Organized by Areas: Applications and Real Time, General,
            Internet, Operations and Management, Routing, Security and
            Transport
      * Most standards work is done by the Working Groups
      * Internet Architecture Board is a related organization (and
        probably is more well known publicly)
      * Lots more details – not immediately important to your meeting
        attendance.

The slide Russ provided isn't so much an organizational chart as a list 
of organizations that somehow have something to do with the nebulous 
concept of the IETF.   I took out a similarly unhelpful organizational 
slide from Scott's original deck and left it out on purpose.

Mike

---

From: Michael StJohns <[email-masked]>
Date: Wed, 23 Aug 2017 13:22:36 -0400

> 	*  Should also give a brief overview of the various subjects that IETF are currently covering.
I'm not sure how to do a brief overview of 120+ working groups. Probably 
not.   Maybe a better idea is to do another session with one AD from 
each area doing a 5 minute 2 slide brief - the IETF Newsreel of current 
topics.  It doesn't actually have to be an AD, but should be someone 
familiar with the area.  Get each AD to do a slide covering their owned 
WGs and topics.

> 	* Though IETF is not a "hierarchical" organization a diagram over the organization with areas, area-directors, working groups, wg-chairs, etc would be usefull to illustrate the width of the organi…
See my other email.
> 	*  I didn't know what to expect of some events, e.g. Bits and Bytes. Maybe in this tutorial you can promote hackathon activities, it could be useful even though this happens before, this could enc…
I'll expand a little bit on this when talking.  I did realize I glossed 
over the non-organizational organized events more in this last session.
> 	* The what-to-do-if-harassed coverage didn't make me feel very reassured. It would help to have a clear pointer other than "look for someone with this color of tiny dot". Also the xmpp.net/directo…

I've already chatted with Pete Resnick and invited him to stand up 
during the next session.  It also helps that they are now wearing a 
garish badge tag that I can add a picture of to the presentation. Still, 
given 1200 attendees and 3 Ombuds I think "Ask someone with a colored 
dot to help you" is probably the best approach.

For the latter, I'll verify the links for the next set of slides.

---

From: Alexa Morris <[email-masked]>
Date: Wed, 23 Aug 2017 13:30:30 -0700

Removing the other tutorial presenters because this is probably not all that interesting to them. 

As a general point, I don’t think that because someone points out a need for more information on a topic, we should assume that the need must be entirely fulfilled by the Newcomers Orientation. I think it’s completely acceptable — and in many cases preferable — to use the Newcomers Orientation to refer people to information available elsewhere, say at ietf.org.  But then we need to make sure that we have the relevant information online at ietf.org. More inline:

> On Aug 23, 2017, at 10:22 AM, Michael StJohns <[email-masked]> wrote:
> 
> I'm not sure how to do a brief overview of 120+ working groups. Probably not.   Maybe a better idea is to do another session with one AD from each area doing a 5 minute 2 slide brief - the IETF Newsreel of current topics.  It doesn't actually have to be an AD, but should be someone familiar with the area.  Get each AD to do a slide covering their owned WGs and topics.

We’ve talked — many times — about having brief online introduction videos for various technical topics and/or working groups. If we had something like that for each working group, you could make a simple statement pointing people to videos available in the Datracker, or some such. Online videos will reach more people than any in-person tutorial is likely to, and can be viewed anytime. 

We would need to develop a template for the video format, and strictly limit the time as well. If people like this idea I have some notions on how to do this and can pursue it further.
>> 	* Though IETF is not a "hierarchical" organization a diagram over the organization with areas, area-directors, working groups, wg-chairs, etc would be usefull to illustrate the width of the organ…

Scott has a fairly lengthly section of his online Newcomers Orientation that explains the incredibly complex structure of the IETF. While it’s true that it’s not immediately relevant to those attending their first meeting, it’s also true that some people like to understand the “big picture.” You don’t need to provide the big picture yourself, but what’s the harm in updating your final bullet to be something like:

“ Lots more details – not immediately important to your meeting attendance BUT for the insatiably curious you can view Section 5: Structure of the online IETF Newcomers Presentation on the IETF YouTube Channel (which also has lots of other interesting videos). 

IETF YouTube Channel: https://www.youtube.com/user/ietf”

-Alexa

> I'll expand a little bit on this when talking.  I did realize I glossed over the non-organizational organized events more in this last session.
> I've already chatted with Pete Resnick and invited him to stand up during the next session.  It also helps that they are now wearing a garish badge tag that I can add a picture of to the presentation. Still, given 1200 attendees and 3 Ombuds I think "Ask someone with a colored dot to help you" is probably the best approach.
>

---

From: Fred Baker <[email-masked]>
Date: Wed, 23 Aug 2017 15:11:09 -0700

> On Aug 23, 2017, at 1:30 PM, Alexa Morris <[email-masked]> wrote:

I'm OK with the idea of an introductory video. I get uncomfortable if I think it needs to be updated more frequently than the charter, and I wonder why the charter isn't adequate to the purpose. I suspect the difference between "the charter" and "a video about the charter" is data format. Speaking as a working group chair, what would you want the chair to say that isn't already in the charter?

A thought - we have a rogue's gallery of WG chair photos. We could include the video in that, or duplicate the photos and video with the charter.

I'd like you to share your ideas. I'm pretty sure I'm not tracking...

---

From: Alexa Morris <[email-masked]>
Date: Wed, 23 Aug 2017 16:07:36 -0700

Fred,

> On Aug 23, 2017, at 3:11 PM, Fred Baker <[email-masked]> wrote:
> 
> 
> 

Good questions, and this idea clearly needs to be further developed.  I wasn’t actually thinking that the video would be a repeat of the charter; I was thinking it would serve a different purpose. For example, see this video that Cullen did in 2015, while onsite at an IETF meeting, to talk about WebRTC and the RTCWEB working group: https://www.youtube.com/watch?v=1llPCXmxGak <https://www.youtube.com/watch?v=1llPCXmxGak>. Note: I had nothing to do with the production of this video, but I remember thinking it was a great idea. And it’s at almost 7,050 views, which is considerably higher than most IETF videos (which are generally just recordings of a presentation designed for and presented to a live audience).  If you look in the WebRTC video description you will see:

"What is WebRTC all about? Why are people so excited about this activity? What work is the IETF doing within the RTCWEB working group? Where is this all going? How can people get involved?”

I think that working group videos would address the “Where is this working group going? Why should I care/what does this mean? How can I get involved?" sort of questions. 

Alexa

---

From: Mirjam Kuehne <[email-masked]>
Date: Thu, 24 Aug 2017 09:36:29 +0200

Hi all,

I always liked the idea of producing short videos for each area (and
possibly each WG). I remember Kathleen did one for one of the security
WGs some years ago that was also well received.

They don't need to read out the charter, but could give a high-level
overview of the main goal and topics (pretty much what Alexa suggested
below). It would also give some "life" to the many names at the IETF
(yes, there are photos of WG chairs, but a video is still a bit more
lively).

And yes, maybe they need to be updated from time to time, but that
should be doable (if we get the ADs and WG chairs to see the usefulness
of such short intro videos).

Mirjam

On 24/08/2017 01:07, Alexa Morris wrote:
> Fred,
> 
>> On Aug 23, 2017, at 3:11 PM, Fred Baker <[email-masked]
>> <mailto:[email-masked]>> wrote:
>>
>>
>>
>>> On Aug 23, 2017, at 1:30 PM, Alexa Morris <[email-masked]
>>> <mailto:[email-masked]>> wrote:
>>>> I'm not sure how to do a brief overview of 120+ working groups.
>>>> Probably not.   Maybe a better idea is to do another session with
>>>> one AD from each area doing a 5 minute 2 slide brief - the IETF
>>>> Newsreel of current topics.  It doesn't actually have to be an AD,
>>>> but should be someone familiar with the area.  Get each AD to do a
>>>> slide covering their owned WGs and topics.
>>>
>>> We’ve talked — many times — about having brief online introduction
>>> videos for various technical topics and/or working groups. If we had
>>> something like that for each working group, you could make a simple
>>> statement pointing people to videos available in the Datracker, or
>>> some such. Online videos will reach more people than any in-person
>>> tutorial is likely to, and can be viewed anytime.
>>>
>>> We would need to develop a template for the video format, and
>>> strictly limit the time as well. If people like this idea I have some
>>> notions on how to do this and can pursue it further.
>>
>> I'm OK with the idea of an introductory video. I get uncomfortable if
>> I think it needs to be updated more frequently than the charter, and I
>> wonder why the charter isn't adequate to the purpose. I suspect the
>> difference between "the charter" and "a video about the charter" is
>> data format. Speaking as a working group chair, what would you want
>> the chair to say that isn't already in the charter?
> 
> Good questions, and this idea clearly needs to be further developed.  I
> wasn’t actually thinking that the video would be a repeat of the
> charter; I was thinking it would serve a different purpose. For example,
> see this video that Cullen did in 2015, while onsite at an IETF meeting,
> to talk about WebRTC and the RTCWEB working
> group: https://www.youtube.com/watch?v=1llPCXmxGak. Note: I had nothing
> to do with the production of this video, but I remember thinking it was
> a great idea. And it’s at almost 7,050 views, which is considerably
> higher than most IETF videos (which are generally just recordings of a
> presentation designed for and presented to a live audience).  If you
> look in the WebRTC video description you will see:
> 
> "What is WebRTC all about? Why are people so excited about this
> activity? What work is the IETF doing within the RTCWEB working group?
> Where is this all going? How can people get involved?”
> 
> I think that working group videos would address the “Where is this
> working group going? Why should I care/what does this mean? How can I
> get involved?" sort of questions. 
> 
> Alexa
> 
> 
>> A thought - we have a rogue's gallery of WG chair photos. We could
>> include the video in that, or duplicate the photos and video with the
>> charter.
>>

---

From: Michael StJohns <[email-masked]>
Date: Mon, 28 Aug 2017 15:54:00 -0400

Folks - I was serious about 5 minutes and two slides.  These *really* 
need to be bare bones and ideally updated at least every three (one?) 
years (each AD entering their third (second?) year must provide one).  
Any longer and we move from brief intro into boring bloviation.  :-)   
Also any longer and you can't actually get anyone to spend the time 
doing it.

Alternately, we hire professionals... but I don't think I'm ready for 
the IETF to have a marketing organization quite yet.

Mike

---

From: Alexa Morris <[email-masked]>
Date: Mon, 28 Aug 2017 13:45:18 -0700

I had a brief chat with someone with more experience in this area, and he said that 2 minutes is really the ideal length of time. Keeps the content focused; keeps the user interested (hopefully). 

Alexa

> On Aug 28, 2017, at 12:54 PM, Michael StJohns <[email-masked]> wrote:
> 
> Folks - I was serious about 5 minutes and two slides.  These *really* need to be bare bones and ideally updated at least every three (one?) years (each AD entering their third (second?) year must provide one).  Any longer and we move from brief intro into boring bloviation.  :-)   Also any longer and you can't actually get anyone to spend the time doing it.
> 
> Alternately, we hire professionals... but I don't think I'm ready for the IETF to have a marketing organization quite yet.
> 
> Mike
> 
> 
>

---

From: Fred Baker <[email-masked]>
Date: Mon, 28 Aug 2017 13:51:23 -0700

On Aug 24, 2017, at 12:36 AM, Mirjam Kuehne <[email-masked]> wrote:
> below).

I don't see the difference between that and the charter. Let me fill out what I mean. I'll use my working group's charter - https://datatracker.ietf.org/wg/v6ops/about/.

> The goals of the v6ops working group are:
> 
> 1. Solicit input from network operators and users to identify
> operational issues with IPv6 networks, and determine solutions or
> workarounds to those issues.
> 
> 2. Solicit input from network operators and users to identify
> operational interaction issues with the IPv4 networks, and determine
> solutions or workarounds to those issues.
> 
> 3. Solicit discussion and documentation of the issues and opportunities
> in IPv6-only operation, and of the resulting innovations.
> 
> 4. Operational solutions for identified issues should be developed in
> v6ops and documented in informational or BCP drafts.
> 
> 5. Document operational requirements for IPv6 networks.
> 
> These documents should document IPv6 operational experience, including
> interactions with IPv4, in dual stack networks, IPv6 networks with IPv4
> delivered as an overlay or translation service, or IPv6-only networks.

If I were to make a video about v6ops, I would discuss conversions among operators or operators&vendors about issues in their IPv6 networks. These would cover issues in IPv6 networks, interactions with IPv4 networks and services, operational requirements for IPv6 networks, and the current trend (on the part of some thought leaders) toward shutting down IPv4. No, I don't think I would read the charter off the screen, but - what is v6ops talking about? It's talking about the things it is chartered to talk about, and when it's not, we either change the charter or move the thread.

By the way, that current trend is not far from the minds of a lot of other people. I'm in the process of deploying IPv6 for New Tribes Mission, and asked about their various services in Rackspace and Azure. I have been very careful to characterize this as "adding IPv6 to their existing network", but the explanation my counterpart made to another guy this morning was about "moving to IPv6", e.g., shutting down IPv4. The only reason someone that deploys IPv6 *doesn't* shut down IPv4 is that they are concerned about a negative operational interaction of some kind - an application that will stop working, a host or network device that will be prevented from doing its job, an expectation (often related to security) that would no longer be met, or whatever. That rubber/road interaction has been central to v6ops for quite some time now, although the exact viewpoint has changed over time.

Cullen's video is about that. He is talking about Web RTC services and the work in the WebRTC WG supporting it. He doesn't list all of the points in https://tools.ietf.org/wg/rtcweb/charters?item=charter-rtcweb-2017-03-30.txt; it's more of a marketing pitch. But it is the list of topics in his charter, at a slightly higher level.

---

From: Fred Baker <[email-masked]>
Date: Mon, 28 Aug 2017 13:53:50 -0700

> These *really* need to be bare bones and ideally updated at least every three (one?) years (each AD entering their third (second?) year must provide one). 

And each chair each time they update their charter.

---

From: Mike StJohns <[email-masked]>
Date: Mon, 28 Aug 2017 18:14:45 -0400

Um no.  Probably not going to do 130+ videos and keep them current.   Areas yes - WGs probably not.   

Sent from my iPad

> On Aug 28, 2017, at 16:53, Fred Baker <[email-masked]> wrote:
> 
> 
>