Subject: Brilliant Plan? More hackathon, less PowerPoint-athon? (was: [irsg] IETF 99 WG Chairs Forum Minutes)

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 28 Jul 2017 12:27:21 -0500

Mirjam, thanks for the minutes.

So, just following up on two comments at the mike ...

On Thu, Jul 20, 2017 at 7:31 AM, Mirjam Kuehne <[email-masked]> wrote:

(deleted down to)

Robert Sparks said the hackathon is getting as important if not more than
> the
> work done at the WG meetings. Maybe we need to start thinking about
> having fewer WG sessions and instead focus more on hackathon work?
>

(followed a moment later by)

> Ben Campbell said that the IESG is currently discussing how to find more
> time for the hackathon.

I'm hoping that most working group chairs are aware that recent IESGs have
been experimenting with changes to the ways we meet face to face (the IETF
99 experiment was announced here:
https://mailarchive.ietf.org/arch/msg/ietf/7gtDm66kQ0BIy4LPAezxUtjBguo).

Robert's thought seemed like a bigger change than one I'd be comfortable
dropping on the IETF community without broader discussion, but I thought I
saw heads nodding as he spoke, and as Ben said, this is something the IESG
has started wondering about.

So, I'd like to discuss that, and this list seems like as good a place as
any to start.

Would your working group be willing to trade session time for hackathon
time?

Two requests ...

   - (It's possible that your answer would be "no, but what we'd like to
   have instead of session time is <InsertBrilliantPlanHere>". If that's your
   answer, please say so, but it would be helpful if you started a different
   thread with subject "Brilliant Plan? <InsertBrilliantPlanHere>", so people
   can keep different conversations straight if there's followup.
   - In any "Brilliant Plan?" thread, it's not necessary to counter with
   "that wouldn't work for my working group, because ..." Every working group
   is different, so we don't all have to agree on what some working groups
   want to do. We just need to figure out if there's enough commonality that
   we should plan to support the working groups that want to do it.

Thanks,

Spencer

Spencer, speaking as a curious AD ...

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 28 Jul 2017 18:42:22 +0100

Yes, please try it. Do hacking every morning. Start ppt at 2pm.
Or anything approaching that is a positive.

I don't have a working group to object to that at the moment:-)

S

---

From: Yaron Sheffer <[email-masked]>
Date: Sat, 29 Jul 2017 00:03:00 +0300

At the risk of taking an unpopular stand, I think this is far from 
brilliant. Or at least, does not apply to many of the working groups.

Taking my WG as an example (but I think the same is true for my previous 
WG, too): we have a bunch of large vendors that are interested in one 
technology, and each has the resources to develop it independently. In 
our case the technology is not even deeply complex, it's just that we 
need to agree on all the details. Our constituents don't care if open 
source implementations exist, and would just shrug if presented by a 
wonderful GPL'ed library. So for us, it would make sense to spend a 
little bit of time on coding to prove that the protocols are sound. But 
50% of the group's resources? No way.

Thanks,

     Yaron

---

From: Fred Baker <[email-masked]>
Date: Fri, 28 Jul 2017 18:31:57 -0700

I'm pretty much in the same boat. I have an email out to my co-chairs and they may disagree with me. But where we stand right now, v6ops is on operational working group and by charter doesn't develop code or specifications for code. I don't know what we would hack in a hackathon. 

> On Jul 28, 2017, at 2:03 PM, Yaron Sheffer <[email-masked]> wrote:
> 
> At the risk of taking an unpopular stand, I think this is far from brilliant. Or at least, does not apply to many of the working groups.
> 
> Taking my WG as an example (but I think the same is true for my previous WG, too): we have a bunch of large vendors that are interested in one technology, and each has the resources to develop it independently. In our case the technology is not even deeply complex, it's just that we need to agree on all the details. Our constituents don't care if open source implementations exist, and would just shrug if presented by a wonderful GPL'ed library. So for us, it would make sense to spend a little bit of time on coding to prove that the protocols are sound. But 50% of the group's resources? No way.
> 
> Thanks,
> 
>    Yaron
> 
>

---

From: Yoav Nir <[email-masked]>
Date: Sat, 29 Jul 2017 06:49:12 +0300

Just listen to the market. The demand for presentation slots outstrips supply, and ADs need to exhort WG chairs to reduce the count and length of time slots that we ask for.

Reducing supply by half does not seem like the way to go.

Yoav

> On 28 Jul 2017, at 20:42, Stephen Farrell <[email-masked]> wrote:
> 
>

---

From: Melinda Shore <[email-masked]>
Date: Fri, 28 Jul 2017 20:05:18 -0800

On 7/28/17 7:49 PM, Yoav Nir wrote:
> Just listen to the market. The demand for presentation slots
> outstrips supply, and ADs need to exhort WG chairs to reduce the
> count and length of time slots that we ask for.

I believe the argument is that meeting time tends to be
poorly spent (far too many presentations, not enough
actual work), and consequently a lot of that demand represents
waste.  I basically agree with that, but I also agree that
not every working group can make good use of hackathon time,
and certainly not at every meeting.  However, it might be a
good opportunity to do some whiteboarding and design team work,
which is an excellent way to move work forward.

Alternatively, simply don't hack.  None of this is compulsory,
right?

That said, my problem is this: A hackathon in the *morning*?

Melinda

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 29 Jul 2017 08:33:51 +0200

If WG chairs would start every presentation slot with this sentence, would
results improve ?

  I am sure everybody has read up on the document in question and the slides
  in before, so lets skip presenting them and lets go directly to questions
  about the item.

Aka: Its quite difficult to tune presentation materials to maximize
result oriented discussion during a given time-slot. You've got to really
know your audience upfront which can be quite difficult in many WGs.
Even chairs will have a hard time guessing their audience in many cases.

-- 
---
[email-masked]

---

From: Fred Baker <[email-masked]>
Date: Sat, 29 Jul 2017 10:27:42 -0700

I often get slides the day, or the weekend, before. I don't think there is time for the review. Yes, I can ask that they come earlier. The author still has to do the work.

Also, at least when I prepare slides, I don't plan to read them to an audience. They are prepared to support me when I am speaking. As a result, I don't subscribe to the theory that slide distribution is message distribution.

Sent using a machine that autocorrects in interesting ways...

> On Jul 28, 2017, at 11:33 PM, Toerless Eckert <[email-masked]> wrote:
>

---

From: Michael Richardson <[email-masked]>
Date: Sun, 30 Jul 2017 22:36:29 -0400

Fred Baker <[email-masked]> wrote:
    > I'm pretty much in the same boat. I have an email out to my co-chairs
    > and they may disagree with me. But where we stand right now, v6ops is
    > on operational working group and by charter doesn't develop code or
    > specifications for code. I don't know what we would hack in a
    > hackathon.

How many times are ID authors asked:
    "does this change break any existing implementations?"

How do you find this out?

BTW: Hackathons do not just hack code.  They also hack documents.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Sun, 30 Jul 2017 22:38:25 -0400

> specifications for code. I don't know what we would hack in a

To answer again in a different way: would you benefit by having
  a) less constrained schedule because other WGs went away?
  b) more operational/coding people at the meeting?

---

From: Michael Richardson <[email-masked]>
Date: Sun, 30 Jul 2017 22:39:59 -0400

> Even chairs will have a hard time guessing their audience in many

It's hard if the draft has had no discussion on the ML.
If there are a dozen threads about different issues, then I think it's pretty
easy.

Maybe that draft shouldn't have ML time then.

---

From: Warren Kumari <[email-masked]>
Date: Mon, 31 Jul 2017 13:15:43 -0400

>     > specifications for code. I don't know what we would hack in a

We also seem to be working from the assumption that everyone who
participates also writes code; this is an incorrect assumption. There
are a number of people who are not software engineers, but rather
network engineers - while their input might be useful to people
hacking code, it may not be the best use of their time...

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 31 Jul 2017 19:48:57 +0200

On Jul 29, 2017, at 08:33, Toerless Eckert <[email-masked]> wrote:
>  in before,

I wouldn’t want to lie, so I have a hard time imagining the experiment.

> so lets skip presenting them and lets go directly to questions
>  about the item.

That’s not the way it works.
If the people already had the questions, why didn’t they ask them on the mailing list?

The f2f time is good
— for educating a larger group of people (sorry, yes),
— for making complicated trade-offs and reaching decisions (which need to be validated on the mailing list, but decisions they are),
— for obtaining serendipitous input from tourists and other people who have *not* read the drafts.

(I also sense some confusion about slides: Slides are supporting a presentation, they are not meant to *be* the presentation, so having looked at them does not replace the presentation.)

> result oriented discussion during a given time-slot.

Indeed, that is hard (but so is most of the work we are doing).

> You've got to really

Right.  Is that a bug or a feature?

This discussion also lacks respect for a simple fact: IETF WGs are *wildly* different, not just in whether they write code or not.
Many of these perennial discussions repeat ingrained mantras that try to solve problems that many of our WGs simply do not have.
Experience you might be having in one corner of the IETF doesn’t always transfer to other corners.
(And in any case we need the WGs to have *chairs* so they can find solutions that work in *that* WG.)

Grüße, Carsten

---

From: Fred Baker <[email-masked]>
Date: Mon, 31 Jul 2017 10:51:35 -0700

> On Jul 31, 2017, at 10:48 AM, Carsten Bormann <[email-masked]> wrote:
> 

And in most cases, the presentation is intended to guide or inform a discussion.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 31 Jul 2017 14:00:04 -0400

> hacking code, it may not be the best use of their time...

It may not be the best of us time, but having someone to configures
equipment sitting with someone who writes code to configure equipment
can be very productive.

It can lead to clarifications about why certain parameters exist, or
cause certain options to just *go away*, because they are never useful.
(I point to a decade or more of configuring explicit broadcast addresses in
Linux distros.  No idea why it was ever there. Sure, as an override, but
you almost never need to do that, and if you do, you are already a guru)

But, again, Spencer didn't ask you why it wouldn't work for you.

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 31 Jul 2017 14:01:00 -0400

> (I also sense some confusion about slides: Slides are supporting a presentation, they are not meant to *be* the presentation, so having looked at them does not replace the presentation.)

The above supports what I tend to find to be *effective* presentations.

The thing I find to be missing from the above excellent list is *leave time
for discussion*.  Too often presentations get just enough time to speed
through the slides and not for any discussion of them.

If I wanted that, I'd watch a youtube video.

-- Jeff

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 31 Jul 2017 20:03:54 +0200

ALl in agreement, Carsten. I was bringing up the experiment to skip slides as a thought
experiment to figure out for yourself what the slides should really be used for. And
the answers can vary depending on the state of the WG.

Or as a corollary: Slides are not implicitly bad, but there are a lot of bad slides.

And everybody who was annoyed about a bad set of slides is more than welcome to approach
the presenter afterwards and offer help for the next slide deck ;-))

Cheers
   toerless

On Mon, Jul 31, 2017 at 07:48:57PM +0200, Carsten Bormann wrote:
> I wouldn???t want to lie, so I have a hard time imagining the experiment.
> 
> That???s not the way it works.
> If the people already had the questions, why didn???t they ask them on the mailing list?
> 
> The f2f time is good
> ??? for educating a larger group of people (sorry, yes),
> ??? for making complicated trade-offs and reaching decisions (which need to be validated on the mailing list, but decisions they are),
> ??? for obtaining serendipitous input from tourists and other people who have *not* read the drafts.
> 
> Experience you might be having in one corner of the IETF doesn???t always transfer to other corners.

-- 
---
[email-masked]

---

From: Melinda Shore <[email-masked]>
Date: Mon, 31 Jul 2017 10:22:43 -0800

On 7/31/17 10:01 AM, Jeffrey Haas wrote:
> The thing I find to be missing from the above excellent list is
> *leave time for discussion*.  Too often presentations get just enough
> time to speed through the slides and not for any discussion of them.

I'm not that much of a fan of strict meeting timing.  I tend
to think that if you allocate 15 minutes to a topic you're
increasing the risk of a presentation ("I have <n> minutes, which
means <m> slides"), and I think we've all seen useful discussion cut
short so that the session can move on to the next presentation.

Melinda

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 31 Jul 2017 14:34:01 -0400

For some WGs, the following seems to work.  Write the agenda being very 
clear about the split, allowing time for discussion.  Put the higher 
priority work up front, and make clear that lower priority items may get 
pushed out.  Not perfect, but it helps give the right message.

if you have an agenda with only 5 minutes per item and no discusion, 
maybe there is a different problem.

Yours,
Joel

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 31 Jul 2017 19:54:07 +0100

On 31/07/17 19:03, Toerless Eckert wrote:
> I was bringing up the experiment to skip slides 

I hope that we're not interpreting Spencer's ask to be
solely about slideware.

IMO, making better use of a week when loads of us are in
one place is entirely feasible, and experiments to see
what works well would be good. (And there's really no need
for strawman suggestions that one size fits all won't cut
it: I don't hear anyone sensible suggesting that.)

I really find it hard to believe that the formal sessions
we have now are the best option for all of the the week.
Given that we had a couple of hundred people spending the
w/e in Prague hacking, doing the tools codesprint, the ANRW
workshop and various other weekend activities, it seems
obvious to me that there is a demand, and now a significant
demand, for more than just our current set of formal WG
sessions.

If, OTOH, the people who were there at the weekend in
Prague said they find the Mon-Fri sessions *more* useful,
then that'd shut me up, but I don't think I've heard
that.

S.

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 31 Jul 2017 12:02:26 -0700

On Mon, Jul 31, 2017 at 11:54 AM, Stephen Farrell <[email-masked]
> wrote:

They're differently useful. To take a concrete example, we spent two days
hacking on TLS and QUIC stacks and then ~6 hours in QUIC-related WG
meetings. I don't believe it would have been useful to sacrifice any of
that WG meeting time for more hacking time (though given the ratios the
opposite might have been true).

FWIW, I do sometimes have the sense that some formal session or other is
not a good use of time, but I don't think that's primarily a question of
the form of the session (presentations or whatever) being useless but
rather the content being not really that relevant or important. But the
solution to that problem is to shut down WGs and documents that are
irrelevant, not to push back on people doing presentations.

-Ekr

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 31 Jul 2017 22:50:25 +0200

> we have now are the best option for all of the the week.

Definitely not.  We already start the formal sessions only at 09:30, giving us more than a third of the day before the sessions start :-)
And we managed to keep the evenings free of formal sessions, too, leaving approximately 12 of the 24 hours for non-formal meetings and sleep.

Maintaining formal meeting slots free of meetings most of your peers have to go to also helps.  Right now, the agenda planning only has conflicts between meetings — wouldn’t it be nice to be able to actively specify something like Prague’s Tuesday or Thursday morning, where, this time by accident, not much of interest to constrained node networking happened (and, of course, two CNN-related side meetings then managed to clash on the Thursday slot).

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Mon, 31 Jul 2017 20:32:43 -0400

Eric Rescorla <[email-masked]> wrote:
    > hacking on TLS and QUIC stacks and then ~6 hours in QUIC-related WG meetings.

That's awesome!

    > I don't believe it would have been useful to sacrifice any of that WG meeting
    > time for more hacking time (though given the ratios the opposite might have
    > been true).

Do you believe it would be worth to sacrifice some other WG's meeting time.
As a result, The TLS/QUIC hackers would not have (as many) conflicts, so that
could have spent more time hacking. (Alternatively, fewer days away from
home. Alternatively, company would have been willing to send n+1 people to a
meeting that was n-1/n less costly)

    > FWIW, I do sometimes have the sense that some formal session or other is not
    > a good use of time, but I don't think that's primarily a question of the form
    > of the session (presentations or whatever) being useless but rather the
    > content being not really that relevant or important. But the solution to that
    > problem is to shut down WGs and documents that are irrelevant, not to push
    > back on people doing presentations.

I agree.