Subject: Fwd: New Version Notification for draft-resnick-on-consensus-01.txt

From: Pete Resnick <[email-masked]>
Date: Wed, 13 Feb 2013 15:33:30 -0600

Folks,

I've just posted the -01 for this document. When I posted the -00, I was 
a bit quiet about it and just asked some folks on the IESG and a few 
others for their initial comments. I cleaned up a few things from those 
comments and, even though this draft still has some big gaping holes, 
I'd like to hear a bit more feedback. I figured the chairs list was a 
good place to start.

I'm happy to take comments directly to me. If you think a discussion is 
really worth having among all the chairs, I'm not opposed to that, but 
do keep in mind how many people are on this list. I've been avoiding the 
IETF general list for the moment in fear of the deep dark rathole into 
which the conversation is likely to go. Of course, the document is in 
the public archive, so if someone really wants to start discussing it 
there, they are welcome to. If I get positive feedback (and a bit more 
filled in), I'll probably take the next version to the big list and hope 
for the best.

I look forward to your comments.

pr

-------- Original Message --------
Subject: 	New Version Notification for draft-resnick-on-consensus-01.txt
Resent-From: 	<[email-masked]>
Date: 	Wed, 13 Feb 2013 13:10:32 -0800
From: 	<[email-masked]>
To: 	<[email-masked]>

A new version of I-D, draft-resnick-on-consensus-01.txt
has been successfully submitted by Pete Resnick and posted to the
IETF repository.

Filename:	 draft-resnick-on-consensus
Revision:	 01
Title:		 On Consensus and Humming in the IETF
Creation date:	 2013-02-13
Group:		 Individual Submission
Number of pages: 11
URL:             http://www.ietf.org/internet-drafts/draft-resnick-on-consensus-01.txt
Status:          http://datatracker.ietf.org/doc/draft-resnick-on-consensus
Htmlized:        http://tools.ietf.org/html/draft-resnick-on-consensus-01
Diff:            http://www.ietf.org/rfcdiff?url2=draft-resnick-on-consensus-01

Abstract:
    The IETF has had a long tradition of doing its technical work through
    a consensus process, taking into account the different views among
    IETF participants and coming to (at least rough) consensus on
    technical matters.  In particular, the IETF is supposed not to be run
    by a "majority rules" philosophy.  This is why we engage in rituals
    like "humming" instead of voting.  However, more and more of our
    actions are now indistinguishable from voting, and quite often we are
    letting the majority win the day.  This document is a collection of
    thoughts on what rough consensus is, how we have gotten away from it,
    and the things we can do in order to really achieve rough consensus.

       Note: This document contains the musings of an individual.  Right
       now, it is just some rough notes and has lots of holes that need
       to be filled in.  Even if those holes are filled, in its current
       form, it is not intended to be published as an RFC, let alone
       being a BCP for a change of IETF policy.  If it evolves into such
       a thing, great.  If it simply sparks discussion as an Internet
       Draft, that's a perfectly fine outcome.

The IETF Secretariat

---

From: Ted Lemon <[email-masked]>
Date: Thu, 14 Feb 2013 01:43:56 +0000

On Feb 13, 2013, at 4:33 PM, Pete Resnick <[email-masked]>
 wrote:
> I've just posted the -01 for this document. When I posted the -00, I was a bit quiet about it and just asked some folks on the IESG and a few others for their initial comments. I cleaned up a few things from those comments and, even though this draft still has some big gaping holes, I'd like to hear a bit more feedback. I figured the chairs list was a good place to start.

I wish I'd read this document four years ago.   Thanks for writing it.

---

From: Terry Manderson <[email-masked]>
Date: Wed, 13 Feb 2013 18:05:39 -0800

I second that, truly.

While I have existed (professionally) for many years in organisations that
like to serve as the poster child for consensus building and
multi-stakerholderism, I have not read, ever, such a well balanced
approach to the topic, especially including the nuances of the IETF's
modus operandi.

Well done.

Terry.

On 14/02/13 11:43 AM, "Ted Lemon" <[email-masked]> wrote:

>> I've just posted the -01 for this document. When I posted the -00, I
>>was a bit quiet about it and just asked some folks on the IESG and a few

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 13 Feb 2013 22:52:10 -0500

> I wish I'd read this document four years ago.   Thanks for writing it.

I wish I had read it last *week*!

	Thanks,
	Paul

---

From: Edward Crabbe <[email-masked]>
Date: Wed, 13 Feb 2013 20:11:40 -0800

Ditto;  extremely useful.  Thank you very much.

On Wed, Feb 13, 2013 at 5:43 PM, Ted Lemon <[email-masked]> wrote:

> a bit quiet about it and just asked some folks on the IESG and a few others
> for their initial comments. I cleaned up a few things from those comments
> and, even though this draft still has some big gaping holes, I'd like to
> hear a bit more feedback. I figured the chairs list was a good place to
> start.
>

---

From: Leif Johansson <[email-masked]>
Date: Thu, 14 Feb 2013 09:58:37 +0100

On 02/13/2013 10:33 PM, Pete Resnick wrote:
> Folk
<snip>
> I look forward to your comments.
liked it!

---

From: Pete Resnick <[email-masked]>
Date: Thu, 14 Feb 2013 09:56:01 -0600

Thank you for the kind words. I'm glad the document is helpful.

But I do ask that you review it to see what is missing. Surely I have 
not exhausted this topic. Please send comments about what I can add (or 
subtract) to make the document better.

(And thanks to those who sent comments privately; I'll be following up 
on those soon.)

pr

On 2/13/13 8:05 PM, Terry Manderson wrote:
> On 14/02/13 11:43 AM, "Ted Lemon"<[email-masked]>  wrote:
>
>    
>> On Feb 13, 2013, at 4:33 PM, Pete Resnick<[email-masked]>
>> wrote:
>>      

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Ted Hardie <[email-masked]>
Date: Thu, 14 Feb 2013 11:47:59 -0800

Hi Pete,

Thanks for putting together your thoughts on this.  Two things I think need
more thought:  compromise and the relationship of "all issues addressed" to
"no known technical omissions" in the RFC 2026 sense.

Your current text on compromise has this:

   This also brings up an important point about reaching consensus:
   Consensus does not really involve compromising.  "Compromising"
   implies that there remains something wrong with the outcome, but that
   the objector has simply given up.  That is not coming to consensus.

Either this isn't the same use of compromise that we had in RFC 3929, or I
disagree.   Engineering is all about trade-offs, and a lot of the
trade-offs taken up by IETF working groups shift the balance of work from
one part of the network or stack to another.  We've both seen many cases
where client/server protocol working groups have to decide whether a
particular bit of pain is taken by the client side or the server side.
Jockeying that kind of decision can take a good bit of time in a working
group, and when a group of people involved agree to a solution that shifts
work onto their own plate in order to achieve the goals of the working
group, they are compromising.  To crib from wiktionary, their action serves
to achieve consensus by mutual concession, and it's really important.
It's also not the same as "giving up"; it's making a choice that the
overall goal (achieving consensus and an interoperable standard) is more
important than a specific technical result.  Let's not discourage
compromise.

The second point is really that there are different classes of technical
objection.  A new objection (or, to be more neutral, review) that indicates
that the solution currently in hand does not meet its requirements pretty
much re-sets the clock for judging consensus if the working group
acknowledges that the review is correct.  To put that in RFC 2026's terms,
a "technical omission" at that level has to be waived by the IESG, which it
may do for reasons of timeliness or utility; a working group *must* deal
with that objection at any stage unless it gets approval to waive it. But
there are many reviews/objections which do not raise such an issue.  They
may well provide new data about the trade-offs implied by a decision or
about use cases outside the core requirement, but they are not
fundamentally "this doesn't work and here's the data".

I think a statement about working group consensus should have some more
discussion about when a previous judgement of consensus must be revisited
in light of new data.  I don't believe, personally, that it should be "must
be revisited at each new review/objection".  I think once a decision has
been made by the working group, it can be reasonable for the chairs to say
"if your objection is not at the level of 'technical omission', it is out
of the current scope of work".  The more work a working group has based on
a decision, the more I would encourage the chair to say that.  We have a
serious completion time problem in the IETF and constantly re-starting of
older discussions is a major contributor.  Even when the new
review/objection contains new data, it must be possible to say "sorry, that
question has already been called".  The impact on late arriving
participants is that they don't get to participate in all decisions, but I
think this is the right trade-off with the ability to move forward.  That
does make judging the roughness of the consent even harder, because we
generally don't checkpoint individual decisions all that well.  But I'd
suggest we put effort into fixing that, rather than allowing each review at
any stage to force a reconsideration.

regards,

Ted Hardie

On Wed, Feb 13, 2013 at 1:33 PM, Pete Resnick <[email-masked]>wrote:

> **
> Folks,
>
> I've just posted the -01 for this document. When I posted the -00, I was a
> bit quiet about it and just asked some folks on the IESG and a few others
> really worth having among all the chairs, I'm not opposed to that, but do
> keep in mind how many people are on this list. I've been avoiding the IETF
> general list for the moment in fear of the deep dark rathole into which the
> conversation is likely to go. Of course, the document is in the public
> archive, so if someone really wants to start discussing it there, they are
> welcome to. If I get positive feedback (and a bit more filled in), I'll
> probably take the next version to the big list and hope for the best.
>
> -------- Original Message --------  Subject: New Version Notification for
> draft-resnick-on-consensus-01.txt  Resent-From:
> <[email-masked]> <[email-masked]>  Date: Wed, 13
> Feb 2013 13:10:32 -0800  From: <[email-masked]><[email-masked]>  To:
> <[email-masked]> <[email-masked]>
>

---

From: Pete Resnick <[email-masked]>
Date: Thu, 14 Feb 2013 15:12:51 -0600

Hi Ted,

Excellent comments, some I've already thought about (and likely need 
more text) and some that fall into my bucket of "Yeah, I thought I might 
have to deal with that at some point". Specifics below:

On 2/14/13 1:47 PM, Ted Hardie wrote:

> Either this isn't the same use of compromise that we had in RFC 3929, 
> or I disagree.   Engineering is all about trade-offs, and a lot of the 
> trade-offs taken up by IETF working groups shift the balance of work 
> from one part of the network or stack to another.  We've both seen 
> many cases where client/server protocol working groups have to decide 
> whether a particular bit of pain is taken by the client side or the 
> server side.  Jockeying that kind of decision can take a good bit of 
> time in a working group, and when a group of people involved agree to 
> a solution that shifts work onto their own plate in order to achieve 
> the goals of the working group, they are compromising.  To crib from 
> wiktionary, their action serves to achieve consensus by mutual 
> concession, and it's really important.    It's also not the same as 
> "giving up"; it's making a choice that the overall goal (achieving 
> consensus and an interoperable standard) is more important than a 
> specific technical result.  Let's not discourage compromise.

Barry brought up a similar issue to me, and I think the short answer is 
that we are using two different meanings for the word "compromise". It 
is obviously essential that engineering tradeoffs be examined and that 
we choose one set of bad outcomes over another. However, I would 
construe this as the participants agreeing that one kind of pain is 
better than another. Even the participant taking on the pain is required 
to accept that, in some sense, the pain is OK. Otherwise, the group has 
not reached consensus. I would strongly disagree that "concession" (in 
the sense of conceding) should be involved in any way; that's equivalent 
to "giving up".

Barry's specific example to me were two unrelated technical choices and 
the idea that "horse-trading" might take place: "If you stop complaining 
about this thing that you don't like and let it in the document, I'll 
stop complaining about this other thing that you want and I'll let it in 
the document." That's completely anathema to coming to consensus.

The trick here for me is distinguishing between "weighing tradeoffs and 
collectively choosing the best set" and "deciding that the majority of 
us are willing to let some minority suffer the tradeoff, and the 
minority is unwilling to take a stand". I think the word "compromise" 
too often gets used to mean the second, and I want to discourage that 
kind of thinking.

(I strongly recommend the Sheeran book I cited in the bibliography. See 
part 2, chapter 3 for a discussion along these lines. I do suggest 
reading from the beginning to get the context; the whole book is 116 
pages, save the appendicies, so it's a short read. And you have to take 
into account that this is written about the Quakers by a Jesuit priest. 
But it's a great read.)

More text needed.

> The second point is really that there are different classes of 
> technical objection.  A new objection (or, to be more neutral, review) 
> that indicates that the solution currently in hand does not meet its 
> requirements pretty much re-sets the clock for judging consensus if 
> the working group acknowledges that the review is correct.

Assuming that what you mean by "the working group acknowledges that the 
review is correct" is that the working group agrees that the reviewer 
has correctly stated the requirement (i.e., one that the WG had 
previously agreed it would fulfill) *and* that the reviewer has 
correctly identified the failure of the current proposal to meet that 
requirement, I agree completely.

> To put that in RFC 2026's terms, a "technical omission" at that level 
> has to be waived by the IESG, which it may do for reasons of 
> timeliness or utility; a working group *must* deal with that objection 
> at any stage unless it gets approval to waive it.

Yuck. I hate the terms of "waived by the IESG". Given the above, the WG 
basically has to return to the IETF (via the IESG) and say, "We want to 
change what we said we were going to do in this charter. Everybody cool 
with that?", assuming it's a chartered requirement. The same goes if 
they are working off of a requirements document that has already gotten 
IETF consensus. But caveat who precisely is doing the waiving, I 
absolutely agree: That's a show stopper objection.

> But there are many reviews/objections which do not raise such an 
> issue.  They may well provide new data about the trade-offs implied by 
> a decision or about use cases outside the core requirement, but they 
> are not fundamentally "this doesn't work and here's the data".
>
> I think a statement about working group consensus should have some 
> more discussion about when a previous judgement of consensus must be 
> revisited in light of new data.  I don't believe, personally, that it 
> should be "must be revisited at each new review/objection".  I think 
> once a decision has been made by the working group, it can be 
> reasonable for the chairs to say "if your objection is not at the 
> level of 'technical omission', it is out of the current scope of 
> work".  The more work a working group has based on a decision, the 
> more I would encourage the chair to say that.

So I don't have a bunch of practical advice in this document yet. In 
principle, I disagree with you: A truly new objection that the WG has 
not considered before must be considered. Now, pragmatics may cause some 
amount of short-cutting: A chair might say, "That falls into the same 
category of objection that we dealt with before. We agreed that these 
sorts of things are issues, but decided that such objections are 'in the 
rough'." The objector might always come back and say, "Yeah, but I 
believe that the novelties of this issue make it different", and the 
chair will have to make a call. But my take on this is that the chair 
can't say, "That objection will simply not be heard"; the chair has to 
make the assessment that the WG really has come to (at least rough) 
consensus on the whole category of objection. Remember, there's always 
an appeal if the chair has made the wrong call.

> We have a serious completion time problem in the IETF and constantly 
> re-starting of older discussions is a major contributor.  Even when 
> the new review/objection contains new data, it must be possible to say 
> "sorry, that question has already been called".

Again, the subtlety I would insert here is the chair saying, "That 
question has already been called. Unless there are others that think 
that this new data will change their minds about how that question was 
called, I will presume that we still have consensus on this point." And 
"change their minds" is an important point: The question must have been 
decided earlier such that even previous objectors were convinced enough 
to say, "Yeah, this stinks for me, but I agree it's the tradeoff we have 
to make." Going back to the previous discussion, it's only if people 
have been "out-voted" and have "given up" that would cause them to come 
back out of the woodwork to re-argue the same point with the new data. 
Those folks have to be willing to say, "Before I was convinced that the 
tradeoff was reasonable. Has this new data *really* changed anything?"

The way to really avoid these things is to keep public issue tracker 
notes on the decisions. The chair can then say of the new data, "We 
decided exactly this with issue 23. Does this new data actually give us 
anything that you think would change that decision?" The chair can 
answer that question for themselves if they like and make the call. The 
WG can always say, "No, you made a mistake on that call; this *is* new." 
Or the objector can appeal. That's all fine.

> The impact on late arriving participants is that they don't get to 
> participate in all decisions, but I think this is the right trade-off 
> with the ability to move forward.  That does make judging the 
> roughness of the consent even harder, because we generally don't 
> checkpoint individual decisions all that well.  But I'd suggest we put 
> effort into fixing that, rather than allowing each review at any stage 
> to force a reconsideration.

Agree with the checkpointing (see above). Part of the problem with 
late-comers is that we now have WGs that last for huge amounts of time. 
If we would get our work done smartly and scoped our WG to smaller 
chunks, we'd have less of these issues.

Sounds like I have more text to write.

Thanks. Excellent comments.

pr

---

From: Warren Kumari <[email-masked]>
Date: Thu, 14 Feb 2013 16:28:19 -0500

> I second that, truly.

Thirdeded...

> approach to the topic, especially including the nuances of the IETF's

So, if this gets published (or just polished a bit) it would be very helpful to be able to take it to some other standards groups and explain what we actually *mean* by rough consensus…

W

--
I don't think the execution is relevant when it was obviously a bad idea in the first place.
This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants. 
   ---maf

Warren Kumari
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Mon, 18 Feb 2013 16:24:20 -0800

Pete and I have had some off-list exchanges; so he's already seen my 
in-line nit-picking; I'll spare the rest of you from it, and limit this 
note to larger issues about the document:

    1.  Like the rest of you, I think this is an important effort and 
much-needed.

    2.  There are some serious, strategic challenges to doing the 
document well.  That is, it has targeted an inherently difficult topic, 
IMO.  It needs to provide just the right amount of motivation, scope and 
detail.  Dangers include seeming to be too arbitrary or too abstract.  I 
think it's a good start, but as with some other comments on the thread, 
I think it needs some strategic adjustments.

    3.  The document needs to be extraordinarily careful in its use of 
terminology.  That's not an uncommon problem, but the nature of the 
paper makes it more sensitive:  For any important term that is used, its 
meaning needs to be supplied explicitly, rather than making the 
assumption that all readers share the same understanding.  The exchange 
about "compromise" was a good example.  (And I, too, feel Pete's meaning 
was idiosyncratic in ways that hurt the document.)

My own view is that the scope of the document needs to be a bit larger 
and like some others I think it needs to include text that provide 
exemplars of practical application.

Significantly, many readers of the document will have little IETF 
experience, but quite a bit of experience with other groups.  They will 
(and already do) attempt to re-apply their familiar models, even though 
the IETF actually employs very different models.  So the IETF's models 
and their motivations need to be explained.

With all that in mind, here's some suggested text.  I'll stress that I 
view the following as merely elaborating and extending what Pete already 
has, rather than suggesting something "different".  The text is meant to 
establish a somewhat stronger base for the detail that Pete is pursuing[*]:

      It will help to understand the meaning of the IETF's term 'rough 
consensus' by first understanding the nature of the IETF's goal for all 
its work:  It seeks to have its work deployed and used at very large 
scale.  This means that the work must be readily understood and adopted 
by a very wide range of independent readers, developers, managers and 
operators.  It also means that the technology must be readily operable 
in a very wide range of conditions.  No matter how skilled and 
well-intentioned, a small, isolated group of writers rarely can produce 
work that satisfies these requirements. The world is too complex; too 
many important issues are likely to be missed by a small, isolated team. 
Satisfying both the politics of wide-scale adoption and the mechanics of 
independent operation typically requires a diverse and substantial set 
of participants who contribute to and support the work.

     Rough Consensus is the label the IETF uses for the underlying 
process that produces viable work.  At its most efficient, for the 
easiest cases, rough consensus can look like a simple moment of 
decision-making, with an apparent super-majority, but that perception is 
misleading.

     Rather, the actual model is of a collaborative discussion process 
that raises and resolves issues, eventually developing a stable and 
broad base of support, for work that has a solid base of analysis to 
justify the choices made in its development.  That is, the "community" 
develops active support for the result, because that community created 
it and understand it, through the process of participating in setting 
its goals, selecting among alternative solutions, and resolving concerns.

    All participants are collaborators, including those with 
responsibility for group management (Authors, Design Team, Chairs, ADs). 
  That is, their primary role is to facilitate collaboration, rather 
than to assert management decisions of their own preference.  Managers 
often do assert their authority, but this ultimately is, again, for 
efficiency.  If the group disagrees with an assertion, dispute 
resolution is between equals, where possible.

     Resolution of an issue does not require that the person raising it 
prevail or even agree with the resolution; but it does require that the 
group pay serious heed to the issue and consider its merits against 
alternatives.

     By the time a decision has been made, there is a broadly-shared 
understanding of its formation and its likely utility.  We call that 
understanding 'Rough Consensus'.

d/

[*]  Pete is legitimately concerned that my text will move the 
document's scope to be distractingly broad.  I share the concern, which 
is why the sequence of the text stays focused on the practical basis and 
meaning of the core term he discusses in his document.
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net