Subject: Issues around sponsoring individual documents

From: Lisa Dusseault <[email-masked]>
Date: Thu, 6 Sep 2007 17:44:32 -0700

The Applications area does not have a lot of attendees or WGs (I  
oversee five, and of those five, three are within a document or two  
of closing).  Much of the current work is being done as individual  
submissions (abbreviated IS in this email) [0].  I'd like to get some  
input on how IS's should be handled.  I have many opinions on IS  
tradeoffs, having written several and sponsored more, but I'm trying  
to phrase these questions without entirely presupposing my own  
answers, and to reflect conflicted opinions and the criticisms I've  
heard.

(BTW, I'm sure I can follow the advice of "Use your judgement" if  
anybody decides to say that, but it doesn't really inform that  
judgement does it?  )

Is an IS that defines a new protocol for the Standards Track fine in  
general?
Is an IS that extends a standard protocol developed in a WG fine in  
general?
Is an IS that obsoletes a standard protocol developed in a WG fine in  
general?

Do IS's suffer from less review?  Is that a problem?
Whose responsibility is it to get sufficient review?

Is it mostly well-connected individuals that can use the IS track,  
knowing an AD to do the sponsoring?  Or mostly individuals with a lot  
of time on their hands?
An IS can have a non-AD document shepherd.  Does encouraging that  
improve the situation or make IS publishing even more dependent on  
the author's connections?

Should I limit time spent sponsoring IS's? [1]  IESG work plus IS  
work could consume 30 hours a week, or 40, or 50...

Assuming I limit the potentially endless amount of work devoted to  
IS's, do I limit it algorithmically (e.g. first come first served),   
as a matter of pure taste, or other?
How should I prioritize IS sponsoring work? Which documents get my  
attention first?  [2]

When there's a question of consensus for an IS document, should I  
just drop the document?
Assuming I try to determine consensus how do I do so without an  
official owning WG -- consensus calls to the IETF discuss list?   
Choose a related list and hope people are paying attention?
Do I need to ensure there's a quorum?

How would appeals against IS documents affect answers to these  
issues? (Usually individual ADs take the first stab at handling  
appeals, formal or informal, against WG chair decisions.  When  
there's no WG chair involved with a doc, I guess the appeal would go  
straight to the whole IESG...)
Does sponsoring many IS documents give an AD, and the IESG as a  
whole, too much power?

Are we discouraging legitimate WGs by encouraging IS's?
What other purposes do WG's serve besides simply publishing  
documents, that are not met by IS's? [3]
If we turned down ISs in Apps, would the proposed work die, go  
elsewhere or ... ?  Would that be bad?

Lisa

[0] Note "Independent Submission" is a different thing, an I-D that  
goes through the RFC Editor to RFC and never gets an IETF last call,  
so it's marked as not an IETF document

[1] So far I have been sponsoring nearly every document I'm asked to,  
not really limiting.  However I can see time limiting becoming required.

[2] My current practice is to do WG-related work first, then IS- 
sponsoring work in rotation.  Note that with WG-related work, the WG  
chair does some work which I have to do with IS documents. IS  
documents are usually more work for me than WG products.

[3] I'll take a first stab at this one:
  - creating buy-in among potential implementors,
  - developing relationships that can lead to more interop work than  
otherwise,
  - choosing document editors that can reflect or build consensus,  
strongarm them if necessary

---

From: John Leslie <[email-masked]>
Date: Thu, 6 Sep 2007 21:34:34 -0400

> Is an IS that defines a new protocol for the Standards Track fine in  

   I'd describe them as "necessary, rather than "fine".

> Is an IS that extends a standard protocol developed in a WG fine in  

   This falls a bit short of "fine". Such extensions deserve formal
review...

> Is an IS that obsoletes a standard protocol developed in a WG fine in  

   I doubt there's any other practical way. (These do seem to attract
enough review...)

> Do IS's suffer from less review?

   Yes.

> Is that a problem?

   Not necessarily.

> Whose responsibility is it to get sufficient review?

   Yours, I suppose... (That's why we pay you the big bucks, right?)

> knowing an AD to do the sponsoring?  Or mostly individuals with a lot  

   A bit of both... The latter particularly need significant review.

> the author's connections?

   It certainly improves life for the AD...

> work could consume 30 hours a week, or 40, or 50...

   Yes.

   (Alas, we have rather a backlog of apparea work which seems to
have no way of proceeding except via ISs...)

> as a matter of pure taste, or other?

   No algorithm can yield reasonable results. You have to "use your
judgment". ;^)

> How should I prioritize IS sponsoring work? Which documents get my  

   Clearly the updates to mainstream, high-usage RFCs are squeaking
petty loudly...

> When there's a question of consensus for an IS document, should I  

   Bad idea...

   This situation is exactly where we need WG-style process.

> Do I need to ensure there's a quorum?

   You can always _try_ a consensus call an the apparea list...

   (TANSTAA quorum.)

> issues?

   At all costs, please avoid considering suicide!

   (Obviously, appeals are a sign of some serious malfunction.)

> (Usually individual ADs take the first stab at handling  appeals,
> formal or informal, against WG chair decisions.  When  there's no
> WG chair involved with a doc, I guess the appeal would go  straight
> to the whole IESG...)

   I don't read our process documents that way. Even when your own
action is being appealed, an AD should make the first attempt at
resolution.

> Does sponsoring many IS documents give an AD, and the IESG as a  

   Perhaps, but I think of it as rather too many headaches.

> Are we discouraging legitimate WGs by encouraging IS's?

   No. I believe WGs are being discouraged for other reasons.

> documents, that are not met by IS's? [3]

- review
- polishing details
- discussion on a better targeted mailing list
- ability to schedule physical meetings if necessary
- etc...

> elsewhere or ... ?  Would that be bad?

   Mostly go elsewhere, with "die" a close second. IMHO, it is bad
because solutions get delayed -- often until some half-assed idea
takes hold by IETF inaction.

--
John Leslie <[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Thu, 06 Sep 2007 22:03:57 -0400

Lisa Dusseault wrote:
> oversee five, and of those five, three are within a document or two of
> closing).  Much of the current work is being done as individual
None of the above.  All of them are acceptable according to process, but
that doesn't mean that they're fine in general.

I see two cases where ISes are appropriate vehicles for standards actions:

1. The proposal is simple and noncontroversial, and there is already a
significant and identifiable community in the IETF (or close to it) that
can competently evaluate it.   There's really no need for a WG to refine
the proposal because the proposal is simple enough that its effects are 
easily understood.  (examples include some - not all - proposals for new
email headers, new MIME types, that sort of thing)

2. Careful analysis reveals that the proposal only affects a narrow and
well-identified group of people, and those people have already reached
consensus on the proposal. So there's no need to spin up a WG that
consists only of that set of people.

> Whose responsibility is it to get sufficient review?
Yes they do, in general.  It's not a problem as long as the process is
used only for cases like the ones I mention above.  generally, having
the IS process seems like a good thing, but it does require AD/IESG
discretion to prevent it from being used as a end-run around the full WG
process.
> knowing an AD to do the sponsoring?  Or mostly individuals with a lot
It's not an either-or.  Both sets of individuals enjoy an advantage.  A
well-connected individual with a lot of time on his hands enjoys an even
greater advantage.  :)

IMHO, there's nothing wrong with the standards process having a lower
barrier for simple, uncontroversial, easy to analyze proposals, than it
has for proposals that require more deliberation  by a WG and/or by
IESG.  However if a well-connected individual can somehow use this lower
barrier to get his IS adopted as a standard before a less well-connected
individual or industry group can get his/their IS on a similar topic
submitted to IESG (or before they can get a WG formed to work on that
problem), that might be a problem.
> improve the situation or make IS publishing even more dependent on the
> author's connections?
Offhand I think the more people available to shepherd a document, the
wider the set of authors who might obtain sponsors. 
> Should I limit time spent sponsoring IS's? [1]  IESG work plus IS work
> could consume 30 hours a week, or 40, or 50...
Yes.  WG proposals should take precedence over ISs, and you should
probably set limits on the time you spend doing IESG work.  Otherwise it
will eat your life (it did mine).    I don't think that we should expect
ADs or IESG to evaluate IS proposals within strict time limits like we
do for WG proposals.
> How should I prioritize IS sponsoring work? Which documents get my
I think if I were doing it, I'd basically use a rough FIFO order so that
old proposals would definitely get looked at.  but within a set of
documents submitted within a particular time-slot, I'd use a greedy
algorithm - the documents that involve the least work for the AD would
get handled first.  after that, the documents that are harder to review
or provide feedback for, but which seemed to have the most benefit for
the community would get processed. 
> When there's a question of consensus for an IS document, should I just
> drop the document?
I don't think such documents should be dropped immediately.  But neither
do I think the AD needs to spend a lot of effort sorting them out. 
Basically I'd try to make it up to the IS author and those who commented
on it to try to reach a compromise by themselves.  Give them, say, a 4
week timeout. After that timeout, one of two things happens: (1) the
parties are in agreement that this (perhaps updated) document is the way
forward.  (2) the parties couldn't fully agree but the author argues
that the (perhaps updated) document has rough community consensus (the
objections notwithstanding) and meets the quality requirements for
standards-track outlined in RFC2026 or wherever.   If the document was
updated, a new Last Call is needed.  After that Last Call (if any) the
AD re-evaluates the situation and makes a recommendation to the full
IESG to either drop the document (requiring the author to re-apply for
IS action after some time) or approve it as-is (despite objections if any).
> How would appeals against IS documents affect answers to these issues?
> formal or informal, against WG chair decisions.  When there's no WG
> chair involved with a doc, I guess the appeal would go straight to the
> whole IESG...)
> Does sponsoring many IS documents give an AD, and the IESG as a whole,
> too much power?
AD and IESG discretion is needed to keep this from happening.  But
should IESG decide that the way to make progress in IETF is to do as
much as possible with ISes and avoid forming working groups, it's hard
to know what the best remedy is.

Basically I think that it might be wise to have more formal criteria for
what makes a valid IS, and to make approval of an IS subject to appeal
if those criteria are violated.
> Are we discouraging legitimate WGs by encouraging IS's?
I think that there are cases where ISs are used to process things that
really need more discussion and review, because many in industry see our
WG process is seen as burdensome...and sometimes, subject to input from
the "wrong" people.  but that might be more indicative of problems with
our WG process than of problems with our IS process.
> What other purposes do WG's serve besides simply publishing documents,
> that are not met by IS's? [3]
Assuming that IETF can artract a wide enough set of interested parties,
WGs are much better than non-WGs at determining requirements,  avoiding
output that harms valid interests, and review and refinement of
nontrivial specifications.  Also, there are processes required of WGs to
ensure fairness that can't be applied to individuals.
> elsewhere or ... ?  Would that be bad?
What would happen, and whether that would be bad, depends on the
specific work. 

Keith

---

From: Martin Duerst <[email-masked]>
Date: Fri, 07 Sep 2007 19:36:49 +0900

>answers, and to reflect conflicted opinions and the criticisms I've  

I think the main case for IS is that there is less overhead.
There are cases where something started as an IS, but there was
controversy, so a WG was founded (LTRU, which I'm co-chairing).
Without controversy, it wouldn't have needed a WG, because two
predecessor documents (RFC 1766 and RFC 3066) were done as ISes.

>Is an IS that defines a new protocol for the Standards Track fine in  

Protocol is a very wide term. Something major like mail, HTTP, Atom, Jabber,...
most probably no, but there are minor protocols that could easily
be done by IS.

>Is an IS that extends a standard protocol developed in a WG fine in  

In most cases, yes. One of the goals of good protocol design is to
make this possible. Things are often done on the mailing list of the
former WG, or are done even while the WG is still around, but
preferring to work on the main pieces, and not taking on a draft
because they think it's uncontroversial and can be done as an IS.

>Is an IS that obsoletes a standard protocol developed in a WG fine in  

If that protocol is dead anyway, it will be difficult to find a WG
chair and WG members,...

I think there is another question that you may want to pose here:
Is an IS that moves a Proposed/Draft Standard developed in a WG
fine in general?

And I would say yes to that question. The URI spec is definitely
a good example.

>Do IS's suffer from less review?  Is that a problem?

Some probably do. But there are very thin WGs, and there are WGs
with lots of members but rarely anybody reading a document.
More review is always better, but never easy to get.

>Whose responsibility is it to get sufficient review?

Mostly the author's.

>knowing an AD to do the sponsoring?  Or mostly individuals with a lot  

Time is needed both for WG drafts and for individual drafts.
Connections are good, but they serve more to reduce the inhibition
threshold. The first and foremost thing you need as an IS sumbmitter
is to know that this can actually be done, and that happens mostly
through connections.

>the author's connections?

I think it's a good idea. It can definitely take off some load
from the AD.
But it may not always work out, or not always the same way.
For draft-duerst-archived-at, the main thing that my shepheard
did was to submit it as an IS when I didn't have the time to do so.
After that, he wasn't reachable by email for a while.

>work could consume 30 hours a week, or 40, or 50...

You definitely have to be careful with your load! I'm not sure
why you say "IESG work plus IS work", after all, ISes also go
for IESG approval. I'd written "WG work and IS work".

The Application Area had lots more WGs a few years ago. You may
scare off a few people by telling them that you won't accept
a draft unless it comes through a WG, but you may then just
get more WGs. Other Areas have some semi-standing WGs for
extensions and updates and other ongoing work. We could have
a "standing Email WG" and a "standing HTTP WG" and so on,
but I don't think that would decrease your workload or
would significantly improve the quality of what's now
published via ISes.

>How should I prioritize IS sponsoring work? Which documents get my  

I think a major point is weeding out stuff that's not yet ready
or doesn't really fit in your area, and so on.

>When there's a question of consensus for an IS document, should I  

That very much depends on the nature of the stuff, and the
nature of the non-consensus.

>official owning WG -- consensus calls to the IETF discuss list?

There is a Last Call anyway, so I would only use the IETF list
in very special situations.

>Choose a related list and hope people are paying attention?

Related lists are definitely a good thing. If there is no
related list, that may be a sign that it doesn't belong in
the IETF. But you shouldn't have to do that. The submitter
(or shepherd) should come to you and give you pointers to
discussions on such lists that already took place.

>Do I need to ensure there's a quorum?

There are no quora for WGs, so why would there be for ISes?
I think one good check I'd use is whether there was discussion.
In many ways, discussion with some degree of disagreement is
better than no discussion, because it shows you that people
have read the document, and have thought things through and
hashed them out.

>straight to the whole IESG...)

If it smells like appeals, it definitely smells like WG.

>Does sponsoring many IS documents give an AD, and the IESG as a  

I don't think so. People can always complain during IETF Last Call,
and if something is controversial, it will need a WG.
Having ISes as a ligthweight process is a great advantage
of the IETF, even if for you personally, ISes are more work
than WG drafts.

>Are we discouraging legitimate WGs by encouraging IS's?

There is a wide spectrum where you can have it both ways.
Which way you want it is mostly your call (and Chris',
unless you are ready to have 10 WGs and don't mind if
he only has 5 :-).

>documents, that are not met by IS's? [3]

Besides what you say below, I think WGs are crucial for document
suites. I don't think something like e.g. the series of documents
currently being worked on by EAI could be handled as ISes.

>elsewhere or ... ?  Would that be bad?

If you think it belongs elsewhere, you should turn it down.
Although I think you as an AD can sponsor just about any IS,
I'd use "does it fall into the IETF work area" and "does it
fall into the Application Area" as main criteria.

Regards,    Martin.s

>[2] My current practice is to do WG-related work first, then IS- sponsoring work in rotation.  Note that with WG-related work, the WG  

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Eliot Lear <[email-masked]>
Date: Fri, 07 Sep 2007 13:29:18 +0200

Hello Lisa,

First let me say that I appreciate your openness in raising this issue.  
Obviously you're in the role because we do trust your judgment (nyeh 

> Is an IS that defines a new protocol for the Standards Track fine in 

It Depends. 

Could you imagine DKIM going forward as an individual submission?  I 
couldn't.  We could never get the community to agree to it.    Let's 
return to the purpose of WGs in the first place.  They exist, in my 
opinion, as a convenience for area directors.  You use them as a tool to 
drive consensus.  It's a very heavy weight process which should be 
avoided when it can be.  In spite of what I write below, however, I 
largely agree with Keith in terms of applicability of ISes.

And so the standard (ahem) I would apply were I in your shoes (ouch) 
would be the following six questions:

   1. Is this work appropriate to standardize?  Is it important enough
      and is it in an area in which we have competence?  If no, stop
      here and kill.
   2. Will there be more than one or two implementors?  If no, stop here
      and kill.  Without sufficient intent from implementors, you
      probably don't have a viable specification, with or without a
      working group.  Otherwise you're now to the point where you or a
      stuckee probably have to read the whole thing.  So...
   3. Are you confident that the community would agree with the approach
      taken?  If not, perhaps a BoF?
   4. Would the work benefit from working group review through forging
      of consensus through compromise and dialog?  If yes, create a
      working group.  Otherwise..
   5. Is the work ready for advancement?  You have the right and
      responsibility to say, sometimes, "yes, this is improtant, but no
      this is not of sufficient quality that I believe it should be
      advanced, because...", followed by "take the following actions to
      improve the document and come back to me..."  Some people say a WG
      can help here, but I'm not convinced.
   6. Otherwise, sure, go ahead.  But again, use your judgment.  I can't
      envision all the exigencies.

I take the tact that individual submissions should have the same 
priority as working group submissions once you've answered yes to both 
(1) and (2), with the understanding that you will then prioritize your 
work so as not to consume more than a reasonable amount of time on the 
AD job.  The last thing we want is for ADs to burn out.  If this means 
that documents are not put out fast enough, let's deal with that problem 
separately.  To answer (3) and (4) there's no reason why you couldn't 
really on a group of advisors you trust.  Call them document shepherds 
or whatever.

If after last call you judge there not to be consensus there is no 
reason you couldn't send the thing to a working group to go get fixed.  
This will of course depend on the authors' wishes to a great extent.  If 
they don't want to go through the pain of a working group, it's very 
hard to push something through in those circumstances.

> Is an IS that extends a standard protocol developed in a WG fine in 

Same answer as above.

> Is an IS that obsoletes a standard protocol developed in a WG fine in 

Same answer as above.
> Whose responsibility is it to get sufficient review?

Joint between you and the authors.
> knowing an AD to do the sponsoring?  Or mostly individuals with a lot 

How does one answer this?

> improve the situation or make IS publishing even more dependent on the 

In an informal sense, yes.  No reason for you not to have help.

> could consume 30 hours a week, or 40, or 50...

You should limit the time you put into the IESG, but prioritize what is 
important, be it IS or WG.

> as a matter of pure taste, or other?

I'm going to presume from this line of questioning that you receive a 
lot of garbage.  If the abstract looks stupid, unimportant, or otherwise 
indicates that the doc is not appropriate for the IETF, trust that the 
rest of the document is as well and say so and move on.

> How should I prioritize IS sponsoring work? Which documents get my 
See above.

> When there's a question of consensus for an IS document, should I just 
No.  If it's important figure out what to do.  Perhaps that's a working 
group.  Perhaps that's a set of clearly defined changes.  Perhaps you 
have to drop it, but if it really is important, others will be willing 
to offer alternatives, given the chance.

> Choose a related list and hope people are paying attention?

Yes.  Consensus calls to the IETF discuss list, as well as anywhere else 
the info comes from.  But you might want to draw people's attention to 
something you think is important.

> Do I need to ensure there's a quorum?

You don't need a quorum, but you'd better be very sure of yourself 
without one.  I think it's a good idea for people to step and say, "this 
is a good idea" or "this is a bad idea" or "here is what would make this 
a good idea".  Absent that, you should be conservative.

> chair involved with a doc, I guess the appeal would go straight to the 

If someone questions your consensus call that's a pretty easy appeal to 
handle.  Also, your peers should give you advice about consensus before 
you make the call.

> Does sponsoring many IS documents give an AD, and the IESG as a whole, 

No.  You still should heavily weight community consensus.  And the IAB 
needs to pay attention.

> Are we discouraging legitimate WGs by encouraging IS's?
Since a WG is a heavy weight tool for you, who cares?
 
> elsewhere or ... ?  Would that be bad?

It Depends.  If it's bad work, no.  If it's work that's not appropriate 
for the IETF, no.  If it's unimportant, no.  Otherwise, yes.

Eliot

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Sep 2007 08:31:36 -0400

Lisa,

We've discussed aspects of this topic fairly extensively
off-list, starting many months ago.  I will try to avoid
repeating that discussion here, but the summary is that I
believe the growth of work being done in nominal ISs is a
healthy adaptation to some evolution in the Applications Area.
Some of our subject matter has gotten fairly specialized and we
are seeing input from fewer generalists who have really put in
the work to understand subject matter and context.   At least
some of the IS work has developed because of a perception that
the WG process can actually produce less useful review because
it introduces a lot of noise and opinions from people who don't
know much about the subject matter and who are unwilling to
actually read and study drafts (rather than, e.g., responding
loudly to what they perceive as happening on mailing list
threads).  

The review effort for 2821bis and 2822bis is fairly symptomatic
in this regard, with a number of comments early in the process
from people who had either not bothered to study the drafts, had
not understood or accepted the constraints on an effort to move
existing documents to Draft, or wanted to use the revision
effort as a platform for completely revising the Internet email
model.  While the level of attention being paid to the work has
waxed and waned (something that can also happen in a WG), I
believe that we have actually gotten better input and review
than might have been the case with a WG, especially had that WG
drifted off into needing to educate a larger population or
dispose of irrelevant concerns.

I think that, if we have a problem, it is not having more
variations than either "WG" or single-model Individual
Submission process.   But, even today, we don't have a single
model for the latter: we have IS efforts that are really
individual work; efforts that are the result of a self-selected
but closed design team; efforts that are widely publicized to
the IETF community and discussed on public, IETF-based, lists
but not organized as WGs, and so on.  I also believe that the
true measure of the size of the Applications Area is better
reflected in the amount of work we are getting out than in a
count of WGs.

Some comments on your specific questions below, but I'm largely
in agreement with John Leslie and Martin Duerst and won't repeat
their remarks.

--On Thursday, 06 September, 2007 17:44 -0700 Lisa Dusseault
<[email-masked]> wrote:

>... 
> (BTW, I'm sure I can follow the advice of "Use your judgement"
> if anybody decides to say that, but it doesn't really inform
> that judgement does it?  )

"Use your judgment" is, in fact, probably the right answer.  I
hope this discussion provides you the information and
calibration you need.

> Is an IS that defines a new protocol for the Standards Track
> fine in general?
> Is an IS that extends a standard protocol developed in a WG
> fine in general?
> Is an IS that obsoletes a standard protocol developed in a WG
> fine in general?

In each case, I don't think there is an "in general".  I think
you need to monitor the situations and the amount and quality of
review.   I think it is also important that we not automatically
assume that having a WG implies extensive and competent review
from diverse parties.  Often that is that case, and perhaps it
is a reasonable presumption in the absence of evidence to the
contrary.  But there have certainly been WGs --and particular
documents coming out of otherwise well-behaved WGs-- in which
there has been evidence that no careful, competent, and
disinterested review occurs.

> Whose responsibility is it to get sufficient review?

The responsibility ought to lie with the authors and proposers.
I believe that it is their responsibility to convince you, the
IESG, and whatever portion of the community is interested and
relevant that there is sufficient interest and review to justify
standardization.  Regardless of how it is developed, one of the
requirements for an IETF standards-track document must be that
people --other than the developers and, where relevant, the
firms for which they work-- actually care whether there is a
standard in the area and what it contains.  One key difference
between WG-produced and individual submission documents is that,
because WGs go through a formal qualification and chartering
process, there is a presumption that their work is relevant,
appropriate for standardization, and has been reviewed.  None of
those presumptions automatically apply to an IS document; the
authors of such documents need to convince the community that
those criteria are met.   On the other hand, none of them are
automatically true of a WG product, but it is up to those who
are convinced that one or more criteria have not been met to
demonstrate that.

> Is it mostly well-connected individuals that can use the IS
> track, knowing an AD to do the sponsoring?  Or mostly
> individuals with a lot of time on their hands?

I hope not.  John's and Martin's comments are relevant here, but
I would also point out that people who are well-connected also
often have a much easier time getting WGs approved and through
the system.  Unfortunately, there is little or no substitute for
experience in the IETF (not the same thing as being
well-connected, but highly correlated often the more important
of the two) as an aid to getting things through the system.

>...
> Should I limit time spent sponsoring IS's? [1]  IESG work plus
> IS work could consume 30 hours a week, or 40, or 50...
> 
> Assuming I limit the potentially endless amount of work
> devoted to IS's, do I limit it algorithmically (e.g. first
> come first served),  as a matter of pure taste, or other?
> How should I prioritize IS sponsoring work? Which documents
> get my attention first?  [2]

I think you have to look at subject matter and depth of support.
My observation above about a variety of mechanisms being
subsumed under "individual submissions" is important here.  In
most cases, if some individual shows up and says "I have this
pet idea and I think it should be an IETF standard", you should
send him or her away until and unless they can demonstrate
support in the community for both the idea and for standardizing
it.   Put differently, use your own judgment, but remember that
no idea has a right to be standardized, whether as an individual
submission or via a WG.  For this, and for several of your other
questions, I think the burden is on the proposer to convince you
that the idea is appropriate for standardization, that there is
sufficient interest, that adequate review has occurred, and that
an appropriate degree of consensus has been achieved.  

> issues? (Usually individual ADs take the first stab at
> handling appeals, formal or informal, against WG chair
> decisions.  When there's no WG chair involved with a doc, I
> guess the appeal would go straight to the whole IESG...)

No, the individual AD still takes the first stab.  Agreeing to
sponsor an IS doesn't automatically make you an advocate of any
issue involving it.  And the second step, according to the
procedures, is a discussion with the IETF Chair.

> Does sponsoring many IS documents give an AD, and the IESG as
> a whole, too much power?

Not in and of itself.  Consistent bad judgment might, but this
is not the only topic around which bad judgment can result in a
concentration of power in the IESG.  And, if the community
doesn't have a way to deal with bad judgment by ADs, the problem
to be solved isn't one of too many IS documents.

> Are we discouraging legitimate WGs by encouraging IS's?

Occasionally, probably so.  We are also discouraging
illegitimate WGs, would-be WGs that waste a lot of time and
never get organized, WGs with such a high noise:signal ratio
that it is impossible to get work done or evaluate consensus,
and so on.  The bottom line is that either the individual
submission process or the WG one can be abused.

>...

    john

---

From: "tom.petch" <[email-masked]>
Date: Fri, 7 Sep 2007 15:42:33 +0200

Lisa

After perusing your e-mail, I am suffering from question overload.

I think that IS suffer from too little review and, as such, damage the IETF.

Sometimes, the first I am aware of them is when they are approved by the IESG;
sometimes, they come to my attention when a Last Call discussion on the main
IETF list highlights a deficiency which could have been fixed long before Last
Call; sometimes such a Last Call discussion refers back to an approved RFC which
now turns out to have a deficiency.  Of course, any of this can happen with any
I-D from any source but IS appear to contribute more than their share.

I have put time and effort into trying to find if and where these I-Ds get
discussed prior to Last Call and, apart from URI, have largely failed.  I do
receive and mostly look at each and every I-D announcement but do not always
realise from the title and description whether or not I whould be interested.

I often see posts which are concerned that the IESG have too much power, and in
general, I disagree, but this is one area where is seems that they do, severally
and collectively.

This is a general, cross-area, comment and should not be taken as specific to
the Apps area, or to you personally.

Tom Petch

---

From: Jari Arkko <[email-masked]>
Date: Fri, 07 Sep 2007 15:31:59 +0300

Lisa,

You already got excellent feedback, but a couple of additional notes
below:

First, you know but the readers of this list might not know that there's
an ION on this topic:

  http://www.ietf.org/IESG/content/ions/ion-ad-sponsoring.html

The ION talks about, among other things, what factors the AD should
consider when thinking about sponsoring a document (Section 5)
and in what situations requiring all work to come from WGs is
problematic (Section 6).

> knowing an AD to do the sponsoring?

The AD should not accept proposals only from his or her friends;
they can come from anybody and the same criteria should be
used to judge all proposals. However, the ability to make such
submissions is not perhaps widely enough known.

> Does encouraging that improve the situation or make IS publishing even
> more dependent on the author's connections?

In my experience, having a shepherd does help. You get one more
additional pair of eyes looking at the document, plus you can delegate
some of the discussion during Last Call or with other reviewers to the
shepherd.

However, I don't always find a shepherd for individual submissions
even if I want to. That's fine, too.

> Should I limit time spent sponsoring IS's?
> How should I prioritize IS sponsoring work?

There's really no other answer than use your judgment. Work
important for the Internet should get priority. Documents with
many problems can consume a lot of time; in practice I at least
also give priority for good and/or small documents that do
not take a lot of my time. (My bar for sponsoring, say, a bug
fix document to an existing RFC where the WG no longer
exists is very low.)

> Choose a related list and hope people are paying attention?

The ION says that all individual documents (even Experimental ones)
should go through IETF Last Call, because no last call was performed
in a WG. So the minimum bar is review on the IETF discuss list.
However, acceptance through silence is not a good option for this
type of documents, as there is no guarantee that no one outside
the authors and the AD has read them. As a result, it is often useful
to solicit review from specific experts or lists, and make sure you
get that review.

> How would appeals against IS documents affect answers to these issues?

The same rules apply. In this case there is no WG chair action to appeal.
However, just as your actions in AD review can be appealed for regular
documents, they can also be appealed for these documents.

> elsewhere or ... ?  Would that be bad?

Both tools should be used for their appropriate purpose. There's
a middleground where you could do something either in a WG
or as an individual submission. If you find that your area has
a significant number of individual submissions that could or
perhaps even should be WG documents instead, maybe you
need more working groups :-)

For instance, some areas have area working groups that
also deliver documents (bug fixes, architectural
discussion documents, small extensions, etc).

A protocol set that has a lot of maintenance issues could
get a maintenance working group to deal with it instead
of sending documents directly to you.

Jari

---

From: James M Snell <[email-masked]>
Date: Fri, 07 Sep 2007 08:32:07 -0700

Lisa,

Having written several IS drafts, one of which reached Proposed Standard
status and another Experimental RFC, and another that I believe might
have helped to motivate this particular note from you, I'd have to say
that I think individual submissions are an important part of the process
if handled properly.

IS's can suffer from less review.  The only way they can succeed is if
there is an existing community that is interested enough to provide
adequate review and if the author(s) of the spec are willing to active
engage and respond to that community.

In the case of the Atom-related specifications that I have produced, I
have been fortunate in that the existing WG has been very willing to
provide adequate feedback and the chairs have been very willing to put
up with discussion about non-chartered topics.  Controversies that have
come up have been dealt with openly and honestly and the result has been
very positive overall.

I think the burden to demonstrate that standards track IS's have had
adequate review falls on the author(s) and the AD who chooses to sponsor
it.  If the AD does not want to sponsor the draft, then so be it.  The
author(s) have the option of pursuing the creation of a formal WG if
they think the work is important enough.  Alternatively, the author(s)
can choose to take another path such as publishing as an Informational
or Experimental RFC, or publishing outside the IETF entirely.

I would suggest that IS's that have a demonstrated record of being
discussed and hashed out within an existing community such as a WG
should be given priority in the queue.  Doing so should encourage spec
author(s) to take their IS's to the community and helps to address the
concerns you are raising.  If the community shows an interest, and if
the author(s) demonstrate a willingness to listen and respond to the
community, then the IS should be allowed to progress.

- James

Lisa Dusseault wrote:
> 
> The Applications area does not have a lot of attendees or WGs (I oversee
> five, and of those five, three are within a document or two of
> tradeoffs, having written several and sponsored more, but I'm trying to
> phrase these questions without entirely presupposing my own answers, and
> to reflect conflicted opinions and the criticisms I've heard.
> 
> anybody decides to say that, but it doesn't really inform that judgement
> does it?  )
> 
> knowing an AD to do the sponsoring?  Or mostly individuals with a lot of
> time on their hands?
> Assuming I limit the potentially endless amount of work devoted to IS's,
> do I limit it algorithmically (e.g. first come first served),  as a
> matter of pure taste, or other?
> Assuming I try to determine consensus how do I do so without an official
> owning WG -- consensus calls to the IETF discuss list?  Choose a related
> list and hope people are paying attention?
> (Usually individual ADs take the first stab at handling appeals, formal
> or informal, against WG chair decisions.  When there's no WG chair
> involved with a doc, I guess the appeal would go straight to the whole
> IESG...)
> If we turned down ISs in Apps, would the proposed work die, go elsewhere
> or ... ?  Would that be bad?
> 
> Lisa
> 
> [0] Note "Independent Submission" is a different thing, an I-D that goes
> through the RFC Editor to RFC and never gets an IETF last call, so it's
> marked as not an IETF document
> 
> [2] My current practice is to do WG-related work first, then
> IS-sponsoring work in rotation.  Note that with WG-related work, the WG
> chair does some work which I have to do with IS documents. IS documents
> are usually more work for me than WG products.
>

---

From: Dave Crocker <[email-masked]>
Date: Fri, 07 Sep 2007 08:58:13 -0700

John C Klensin wrote:
>     but the summary is that I

+1 to all of John's summary comments.  They do such a nice job about the 
question that they probably should be published on a page somewhere.

> "Use your judgment" is, in fact, probably the right answer. 

Obtaining the support of an AD is an important part of the filtering process 
for IETF work, whether as a WG or an IS. That should not change.

Although I think that John did not quite say it this way, I would suggest 
using 3 questions:

1. Import:  Is the work worthy of sponsoring and spending time on?

2. Process:  Is there a strong reason for pressing to have the work incur the 
administrative effort and protracted schedule of an IETF working group? What 
is it?

3. Support:  Is the work already being done diligently without a formal 
working group or is there a good indication it will be?  That is, are there 
adequate indications of current (or likely) community need and participation 
to make it likely that the document a) will be (or has been) developed with 
consensus and b) will be used by the community?

Good work can happen outside an IETF working group.  Bad work can happen 
inside one.  So the most useful way to decide whether to sponsor some work is 
to consider import (quality and impact) and support questions rather than 
process questions.

> assume that having a WG implies extensive and competent review

+1. Perhaps a heuristic -- along with my #3, above, is to consider the 
complexity and risk of the work.  That said, one can argue that working groups 
actually increase risk. (Yes, I'm serious. Note John's comments about 
uninformed participation.  I'll also note that working groups have a habit of 
expanding scope and functionality of the work and that that is usually 
detrimental to its timeline and utility, IMO.)

 > IESG, and whatever portion of the community is interested and

+1.

Perhaps more ought to lie with authors and proposers for working groups, but 
it's clear that an IS places (or should place) more burden on them.

>     One key difference
> appropriate for standardization, and has been reviewed. 

It also has the considerable benefit of being required to state its goals and 
deliverables explicitly (and clearly?)

  None of
> those presumptions automatically apply to an IS document;

Not automatically but I would claim that *all* good work should contain 
statements about this in the document.

> subsumed under "individual submissions" is important here. 

+2

Focus on the nature, quality and importance of the work, not on whether it is 
formally a wg.  Productivity and utility, rather than formality.

  In
> it. 

+1.  But then, this is exactly the type of shepherding role that ADs provide 
for wg formation, too, I think.

> to be solved isn't one of too many IS documents.

Interesting topic.

> never get organized, WGs with such a high noise:signal ratio

+1.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 7 Sep 2007 13:00:51 -0700

On Sep 7, 2007, at 8:32 AM, James M Snell wrote:

> have helped to motivate this particular note from you,...

Yes, but this wasn't the only one!  Efforts by Dave Crocker and John  
Klensin are also prompting this line of questioning.  But Paul's  
request added to yours to publish the Atom feature advertisement spec  
did prompt finishing the email to send it.

I asked Paul if he'd be document shepherd for that spec, by the way,  
to do the work of determining what consensus was for which features  
to include, to write the doc shepherd's overview & help the process  
along.  I'm not sure he thought that was inappropriate or just not  
something he had the time to do, but he declined.

thanks,
Lisa

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 7 Sep 2007 13:20:53 -0700

Didn't mean to name names publicly!  But as long as I am, I might as  
well characterize the ISs.

Some are being run practically as if they are WG documents, with  
commentary on active WG or post-WG lists.  James' and John's are in  
that category, as well as the IMAP i18n/collation docs.
One is likely to become the standard extensibility discovery  
mechanism for a WG standard -- James'.
Some are revising very widely used standards.  John's and Dave's are  
in that category.
Nearly half are URI scheme registrations, which must be IETF  
Consensus documents in order for the IANA registration to be  
allowed.  Others register a single mail header, VCard property  
(Cullen's) or MIME type.
A few are effectively from outside organizations including Odette,  
ISO, GSM, 3GPP and XMPP.
One is attempting to describe requirements for future work, rather  
than define an implementable standard -- Sam's.

Three ID authors requesting sponsoring have been fellow ADs (Sam,  
Cullen, Chris).
Nine primary authors were people I've never met.
Eight primary authors were people I have met at IETF meetings or  
elsewhere.

IMO, most are good causes.  Most are good quality.  YMMV.

Lisa

On Sep 7, 2007, at 1:00 PM, Lisa Dusseault wrote:

> Yes, but this wasn't the only one!  Efforts by Dave Crocker and  
> John Klensin are also prompting this line of questioning.  But  
> Paul's request added to yours to publish the Atom feature  
> advertisement spec did prompt finishing the email to send it.
>
> I asked Paul if he'd be document shepherd for that spec, by the  
> way, to do the work of determining what consensus was for which  
> features to include, to write the doc shepherd's overview & help  
> the process along.  I'm not sure he thought that was inappropriate  
> or just not something he had the time to do, but he declined.
>
> thanks,
> Lisa

---

From: Martin Duerst <[email-masked]>
Date: Sat, 08 Sep 2007 16:42:34 +0900

Two additional comments, one on something Tom says below and
one on James' situation.

>realise from the title and description whether or not I whould be interested.

I think this is a valid point. The IETF discourages leaving any potentially
instable pointers in documents, and this includes pointers to mailing
lists, issues lists, and so on. For RFCs, this makes quite a bit of sense
(although it may sometimes go too far), and it tends to affect Internet
Drafts strongly. After all, everybody wants to publish Internet Drafts as
RFCs, so they better conform to the rules for RFCs.

For people with some basic knowlegde about IETF WGs, that's no problem
for WG drafts. Take the group of letters appearing in the draft name
after draft-ietf-, and work from there. However, for individual drafts,
it is a problem. What I have been doing, at least for some drafts, is to
add a special paragraph, ideally in the Abstract (because that goes
out with the announcement), that may read something like this
(taken from an actual draft of mine):
    [RFC Editor: Please remove this paragraph before publication.] This is
    a draft to update RFC 3987 and move towards IETF Draft Standard.
    For an issues list/change log and additional information, please
    see http://www.w3.org/International/iri-edit.
I think we should incourage things like this more.

As for the Atom-related drafts that James was working on,
I think that's one situation where I could understand if the AD
just told the author and the WG: Please make this a WG draft.
I don't want to say that it has to be done that way, but if
there is a WG anyway, and the draft is discussed anyway on the
WG list, and is closely related to the actual WG work
(e.g. an extension, or even a way to discover extensions),
if I were an AD, I'd definitely think about trying to reduce
my workload by having a WG chair do the shephearding ex officio,
rather than politely asking him/her and being declined.
Creating a WG is a huge overhead, but having a WG being
formally responsible for a draft that they discuss anyway
isn't too much overhead. Just an idea.

Regards,    Martin.

---

From: Dave Crocker <[email-masked]>
Date: Sun, 09 Sep 2007 08:56:47 -0700

> instable pointers in documents, and this includes pointers to mailing

Well, not really.  Internet Drafts frequently state where to go for online 
discussion.  Certainly all the drafts I've been involved in in recent years do 
this.  And there is an established method of marking text that is to be 
removed prior to RFC publication.

There was a suggestion to mandate having all Internet Drafts contain an email 
address for discussion, but it was not pursued. As I recall, the fact that the 
information can be ferreted out, for working group drafts, was the reason for 
not requiring it to be listed in every I-D.

My own view is that the entire purpose of an I-D is for discussion, so the 
address ought to be mandated, to make sure that the problem you cite does not 
occur.  It is easy information to provide and it saves potential contributors 
quite a bit of frustration.

> just told the author and the WG: Please make this a WG draft.

It is common for "related" drafts to be outside the scope of the wg charter. 
The AD cannot simply instruct the wg to go beyond that scope.  So it is not 
uncommon for an informal wg task to be pursued formally as an individual 
submission.

> rather than politely asking him/her and being declined.

Yup.

> isn't too much overhead. Just an idea.

Given the need to modify the charter, it actually is very high overhead.

d/

-- 

   Dave Crocker

---

From: Lars Eggert <[email-masked]>
Date: Mon, 10 Sep 2007 14:04:48 +0300

On 2007-9-9, at 18:56, ext Dave Crocker wrote:
> It is common for "related" drafts to be outside the scope of the wg  
> charter. The AD cannot simply instruct the wg to go beyond that  
> scope.  So it is not uncommon for an informal wg task to be pursued  
> formally as an individual submission.

The AD actually can, if the WG is interested, because the charter is  
negotiated between the AD and the WG, followed by an IESG approval.

(If the WG - i.e., the expert community around a protocol - is not  
interested in a document, that should indicate that further questions  
may need to be asked about it before it should become an IS.)

> Given the need to modify the charter, it actually is very high  

Not really. Minor recharterings to add well-defined work items aren't  
complicated.

Lars

---

From: Lars Eggert <[email-masked]>
Date: Mon, 10 Sep 2007 14:17:28 +0300

Hi,

you've gotten a lot of good feedback already. My viewpoint differs a  
bit, in that I'm more hesitant about the widespread use of the IS  
path. An IS should be a rare exception, with the much more well- 
defined and -tested WG process the rule.

Because people see the IS path as a shortcut - and even worse, an  
"insider" shortcut -  the more you let them take it, the more  
requests you will get in the future. It starts to erode the process.  
Just say no. Or, actually, ask yourself if the gain from pushing  
something through as an IS is worth the pain in terms of extra  
management and maybe future work. If the gain is worth it *to you*  
(don't forget the IS path is at your discretion, and your judgement  
decides) then sure, go ahead.

But in general, I'd encourage you to tell people to publish their  
documents through the appropriate working group, and if there isn't  
one, tell them to prepare a BOF. WGs have existed to only work on one  
or two documents (PMTUD, for example). WGs come with chairs and they  
come with an established management process, both of which reduce  
your workload (even more so with PROTO).

Lars

PS: Another alternative to IS is something like TSVWG (and now  
OPSAWG) to work in a more open fashion on documents and topics that  
maybe don't warrant their own WG.

---

From: John C Klensin <[email-masked]>
Date: Mon, 10 Sep 2007 10:57:58 -0400

--On Monday, September 10, 2007 2:17 PM +0300 Lars Eggert 
<[email-masked]> wrote:

> Hi,
>
> you've gotten a lot of good feedback already. My viewpoint
> differs a bit, in that I'm more hesitant about the widespread
> use of the IS path. An IS should be a rare exception, with the
> much more well-defined and -tested WG process the rule.

Lars, I have to disagree, unless the WG process can be made both 
a lot more efficient and a lot lower noise.   See my earlier 
note about this and please understand that the Apps area may be 
more subject to the noise factor than other areas... for several 
reasons, some good and some bad.

> Because people see the IS path as a shortcut - and even worse,
> an "insider" shortcut -  the more you let them take it, the
> more requests you will get in the future.

Well, some of us don't see it primarily as a shortcut (although 
sometimes it might be) but as a more efficient way to get some 
kinds of work done.

> It starts to erode
> the process. Just say no. Or, actually, ask yourself if the
> gain from pushing something through as an IS is worth the pain
> in terms of extra management and maybe future work. If the
> gain is worth it *to you* (don't forget the IS path is at your
> discretion, and your judgement decides) then sure, go ahead.

> But in general, I'd encourage you to tell people to publish
> their documents through the appropriate working group, and if
> there isn't one, tell them to prepare a BOF.

Which, under most circumstances, immediately adds at least four 
to six months to the process without, in most cases, getting any 
incremental work done.  It also tends to open up discussions 
about what problem should be solved.  That is extremely useful 
in some cases and a total waste of time in others, but the 
length of time it takes seems to be independent of value.

As costs rise and industry pressure for timely results 
increases, "go spend an extra six (or more) months organizing a 
WG and then take twice the time a WG should take, not due to 
tying down details but with trying to deal fairly with 
irrelevant noise" is going to be increasingly equivalent to 
"take that work outside the IETF".   I don't think an IETF 
effort should take one day less time than it takes to get things 
right (separating me from some folks who favor "accept the 
tolerably good as long as it is timely") but I don't think it 
should take one week longer, either.  I don't think that 
convincing people to go elsewhere is a desirable outcome.  That 
is especially true when the issue at hand is revision/ 
advancement of existing IETF standards, where we know who most 
of the experts are, have mailing lists on which to "advertise" 
for participants, have constraints about what changes can be 
made, and really don't need to spend WG time reopening old 
issues.

> WGs have existed
> to only work on one or two documents (PMTUD, for example). WGs
> come with chairs and they come with an established management
> process, both of which reduce your workload (even more so with
> PROTO).

Having been an AD, although long ago, and watched the process 
for years, I question that generality of this assertion too. 
Sometimes they do, sometimes they don't.

> OPSAWG) to work in a more open fashion on documents and topics
> that maybe don't warrant their own WG.

Sure.  But it is not clear to me that those groups really force 
the kind of topic-specific expert review that we assume that we 
get from "ordinary" WGs.  If they do, that is fine.  But, if 
they don't, we may delude ourselves by then using a shortcut 
community review process that is inappropriate to those groups. 
With the diversity of Apps such groups tend to be particularly 
inappropriate (been there, tried that).  We have tried, or at 
least discussed, a number of other alternatives to "pure" 
individual submission handling in Apps.  I've shared some of 
these with Lisa.  They include special open directorate review 
meetings (perhaps more like SAAG than like TSVWG, but I'm not 
sure any more), extending the Apps Area review process (which 
seems to work well), open review sessions at IETF meetings 
(tried once with IDNAbis and something I would recommend based 
on that experience) and generally holding independent submission 
authors and advocates responsible for demonstrating that 
adequate review has occurred.

One size does not fit all.

regards,
    john

---

From: Lars Eggert <[email-masked]>
Date: Mon, 10 Sep 2007 18:29:18 +0300

Hi, John,

On 2007-9-10, at 17:57, ext John C Klensin wrote:
> Lars, I have to disagree, unless the WG process can be made both a  
> lot more efficient and a lot lower noise.   See my earlier note  
> about this and please understand that the Apps area may be more  
> subject to the noise factor than other areas... for several  

I do agree that because APP produces standards that tend to get wider  
exposure outside the IETF, it's a tricky area, and progressing work  
can take a lot of effort. (TSV is comfortable wedged between INT and  
APP, so we are are a bit more sheltered.)

> sometimes it might be) but as a more efficient way to get some  

Being flippant for a second, let's all use this more efficient way to  
get work done then! There is a tragedy-of-the-commons effect here -  
the IS procedure is only powerful if used sparingly, because  
otherwise the benefits go away. I completely agree with you that for  
important work that needs to happen on an aggressive timeline and is  
not controversial, it is a valid option that makes a lot of sense.  
But it comes at a price, namely, taking up a LOT more AD cycles than  
WG documents do.

For special work items, IS is the right thing to do. But not everyone  
can be special.  One rule of thumb that I've used in the past is "is  
this document as important as 3-5 WG documents?" Because that's my  
totally subjective measure of how much additional effort it takes. If  
the proponents asking for IS have a convincing answer, well, good.

> Which, under most circumstances, immediately adds at least four to  
> six months to the process without, in most cases, getting any  
> incremental work done.  It also tends to open up discussions about  
> what problem should be solved.  That is extremely useful in some  
> cases and a total waste of time in others, but the length of time  
> it takes seems to be independent of value.
>
> As costs rise and industry pressure for timely results increases,  
> "go spend an extra six (or more) months organizing a WG and then  
> take twice the time a WG should take, not due to tying down details  
> but with trying to deal fairly with irrelevant noise" is going to  
> be increasingly equivalent to "take that work outside the IETF".    
> I don't think an IETF effort should take one day less time than it  
> takes to get things right (separating me from some folks who favor  
> "accept the tolerably good as long as it is timely") but I don't  
> think it should take one week longer, either.  I don't think that  
> convincing people to go elsewhere is a desirable outcome.  That is  
> especially true when the issue at hand is revision/ advancement of  
> existing IETF standards, where we know who most of the experts are,  
> have mailing lists on which to "advertise" for participants, have  
> constraints about what changes can be made, and really don't need  
> to spend WG time reopening old issues.

I'm all for streamlining the WG process, but that's a different  
discussion. IS lets you bypass some of the delays associated with  
having a WG, which makes it attractive, but the AD pays that cost in  
terms of cycles. That model doesn't scale.

> Having been an AD, although long ago, and watched the process for  
> years, I question that generality of this assertion too. Sometimes  
> they do, sometimes they don't.

Yes, I shouldn't have generalized this. But at least for me  
personally, the benefits of having shepherds and a well-defined  
process reduce the workload.

> Sure.  But it is not clear to me that those groups really force the  
> kind of topic-specific expert review that we assume that we get  
> from "ordinary" WGs.  If they do, that is fine.  But, if they  
> don't, we may delude ourselves by then using a shortcut community  
> review process that is inappropriate to those groups.

Oh, yes, having these catch-all WGs doesn't much help with getting  
documents adequately reviewed. It does help with the management  
aspect though.

> inappropriate (been there, tried that).  We have tried, or at least  
> discussed, a number of other alternatives to "pure" individual  
> submission handling in Apps.  I've shared some of these with Lisa.   
> They include special open directorate review meetings (perhaps more  
> like SAAG than like TSVWG, but I'm not sure any more), extending  
> the Apps Area review process (which seems to work well), open  
> review sessions at IETF meetings (tried once with IDNAbis and  
> something I would recommend based on that experience) and generally  
> holding independent submission authors and advocates responsible  
> for demonstrating that adequate review has occurred.

It may well be that a catch-all WG is the wrong model for APP. It's  
not a spectacular success in TSV, either, but it's better than any  
other alternative we though of. I only brought it up to bring it to  
Lisa's attention as an option to consider.

Lars

---

From: Keith Moore <[email-masked]>
Date: Mon, 10 Sep 2007 12:42:54 -0400

> Lars, I have to disagree, unless the WG process can be made both a lot
> more efficient and a lot lower noise.   See my earlier note about this
> and please understand that the Apps area may be more subject to the
> noise factor than other areas... for several reasons, some good and
> some bad.
Certainly the WG process needs to be made more efficient, and the noise
reduced.  I just don't see the IS process as a way to solve this
problem.  If not used judiciously the IS process will just further lower
the quality of IETF's work. 

> Which, under most circumstances, immediately adds at least four to six
> months to the process without, in most cases, getting any incremental
> work done.  It also tends to open up discussions about what problem
> should be solved.  That is extremely useful in some cases and a total
> waste of time in others, but the length of time it takes seems to be
> independent of value.
True enough.  That discussion does take at least four to six months -
independent of how difficult the problem actually is.  You want to
reserve WGs for the tasks where such discussions are useful and
appropriate and will contribute to the quality of the overall effort. 
And really this four to six months is what it takes for a group of
people with diverse backgrounds to begin to understand each other and
start speaking in somewhat the same language.  For new standards that
need input from people with diverse backgrounds, this process is
necessary - even though product development people will hate it.  For
established standards that just need revision, I agree that the process
is usually not necessary and just wasteful.  One exception would be when
an established standard has seen widespread use outside its intended
scope, and needs to be revised in light of the scope as newly understood.

(Actually my impression is that if a WG is chartered to only develop
"requirements" before needing re-chartering, then the WG takes about
12-18 months just to develop requirements.  If the WG is chartered to do
additional work, there's more incentive to get the requirements part
done faster.  And IMHO WGs should never be chartered to do
"requirements", because this causes too many decisions to be carved in
stone before their effects are understood.  "design goals" would be much
better.   But somehow the "requirements" meme has stuck in IETF and
won't go away.)

Keith

---

From: "tom.petch" <[email-masked]>
Date: Mon, 10 Sep 2007 20:02:10 +0200

I said before on this Apps list that I saw some IS as potentially damaging: the
sort of thing I had in mind is the I-D draft-hartman-webauth-phishing-05.txt,
currently  being discussed on the IETF main list.  This has failed to achieve
consensus and the document shepherd now proposes to seek consensus on a non-IETF
list.  OK, the list is public and publicised, but this topic is a really big one
for Internet users and if the IETF cannot muster a Working Group to hammer the
ideas into shape, then we should not be working on it - IMHO.  And, by
implication, no AD should be sponsoring it.

This is about the tenth such discussion on the main IETF list that has grabbed
my attention in 2007 - details vary, but all seem to say to me that 'this I-D
should not
have got this far in this state' - and almost all are IS.  Ironically, one such
I-D is draft-housley-tls-authz-extns which Sam Hartman sponsored and then didn't
and then ... well, read the archives of the main list.

Tom Petch

---

From: Keith Moore <[email-masked]>
Date: Mon, 10 Sep 2007 15:54:52 -0400

> implication, no AD should be sponsoring it.

I don't entirely agree with that.  If the list is well-known, accepts
input from anyone, and tries to reach consensus (rather than, say,
trying to promote some organization's agenda), I don't see why the
results of a discussion on said list shouldn't be given a fair amount of
weight by an IETF AD when trying to estimate soundness and community
support.  Unfortunately I've seen a fair number of intentionally-biased
lists outside of IETF (and a few inside of IETF).

That said, list consensus (whether an IETF list or not) is not the same
thing as IETF consensus.  Even a chartered IETF working group should not
assume that group consensus is a sufficient condition for adoption. 

Keith

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 10 Sep 2007 13:56:37 -0700

> assume that group consensus is a sufficient condition for adoption.

Thanks, this pretty much states my intention and I believe Alexey's  
on discussing the hartman phishing draft :)  The list is public and  
archived and listed on the IETF non-WG lists page.  The fact that  
discussion will take place there is known now to the main IETF list.   
The result will be a bunch of input for Sam to revise his document,  
*not* an IETF consensus we can bash people over the head with :)

Lisa