From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 13 Nov 2003 17:41:52 -0600

Please send corrections to these notes (and make sure your names are
spelled correctly!) to me and [email-masked].

Thanks,

Spencer

------=_NextPart_000_00A8_01C3AA0D.6C2C86A0
Content-Type: text/plain;
	name="newtrk.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="newtrk.txt"

New IETF Standards Track Discussion BOF (newtrk)

Thursday, November 13 at 1530-1730

CHAIRS: Scott Bradner <[email-masked]>

DESCRIPTION:
The consensus in the problem working group is that the current IETF 3-
stage standards track described in RFC 2026 [RFC2026] is not working as
originally intended.  The draft problem statement document says:

     "The current hierarchy of Proposed, Draft and Full Standard
     maturity levels for specifications is no longer being used in the
     way that was envisioned when the stratification was originally
     proposed.  In practice, the IETF currently has a one-step
     standards process that subverts the IETF's preference for
     demonstrating effectiveness through running code in multiple
     interoperable implementations and compresses the process that
     previously allowed specifications to mature as experience was
     gained with actual implementations:"

The draft document then goes on to list 4 observations:

  1/ few documents actually progress after being published as PS
  2/ there is a perception that the IESG raised the quality requirement
  3/ in spite of the raised quality requirement, running code is not
      required
  4/ there seems to be a reinforcing feedback loop involved: vendors
     implement and deploy PS documents so the IESG tries to make the PS
     documents better

The draft problem document concludes that the 3-stage process is
excessive.

This BOF will discuss the observation that the current IETF 3-stage
standards track is not working and discuss options for different
standards
track processes.

Expected output of BOF:
        1/ criteria for declaring "success" for a new IETF standards
track
        2/ next steps

AGENDA:

  1/ description of current IETF Standards Track (see presentation)

Scott described the archeology, dating from RFC 1083 by Jon Postel
  included three-stage process, with "proposed protocols"
  no one knows why it became "proposed standard"
  RFC 1310 had first standalone process description,
  has been three-stage ever since

As many as 22 versions of BGP draft - MANY stages...

Draft Standard was drafted to prove IPR interoperability as well

Full Standard = Wide Use

  2/ observations from problem working group - Elwyn Davies

2.4 section of problem statement draft, rev 5

"three-stage process not used properly"
  subversion and compression
  higher quality bar (at least a perception)
  quality checked by thought exercises, not running code
  proposed standards deployed as if they were mature
  this raises the quality bar and slows things down,
    so people sometimes deploy I-Ds...

No aftercare for standards
  no bug reports, periodic reviews

Scott - purpose of Draft Standard is a quality/clarity check of
implementation

Bob Hinden - can implement from the spec - we've lost our running code
heritage

Dol strom? - vendors don't wait for proposed standard

  3/ what other SDOs do - Scott Bradner

W3C uses three-plus stages, pretty much what we use
  working draft is internal-only

Joel Halperin - ISO was a committee draft with number assigned at the
beginning of the process
  draft international standard, international standard with full
approval

Global Grid Forum - modeled after IETF with minor differences
  four types of documents
  no document has reached our second stage
  have to show interoperability and running code
  80 percent researchers, so specs lag review pretty badly
  working groups should shut down as soon as possible (sometimes too
soon)

Steve Woodlock - Open Group (will have slides)
  10-week timed process after draft iteration
  company review process and change requests followed by voting
  board approval

Tom Taylor - ITU-T
  has Study Groups, made up of working parties and questions
  recommendations are outputs of process
  all decisions made at meetings
  editor just edits
  driven by written contributions
  last call when "finished", started by working parties
  if consensus, study group just approves the document
  if not consensus, try again, but governments are voters
  maintenance - recommendations issued as new versions, identified by
date
    assigned to a question for maintenance

Stephen Hayes - 3GPP
  either 1-stage or n-stage, conceptually
  one type of document - technical specification
  increase in functionality over time - version number increments
    major updates every 18 months
  three-level version numbers
  strong rules about backwards compatibility, but things do change over
time

Alex ???? - some of these organizations are based on corporate
membership
  this is a radical difference
  internal review
  IEEE has similar process
    starts with contributions, goes through drafts, then to ballots in
WG and final committee approval
    membership requirements are very strict (based on individuals)
  document changes include patches of earlier specs

  4/ proposals for alternate standards track processes

        draft-ietf-problem-issue-statement-04.txt
*       draft-dawkins-pstmt-twostage-01.txt - Dave Crocker

Barriers, unused stages, unused process, false advertising,
uncoordinated use of drafts, cruft in the archives
Working Group Snapshot, Proposed Standard, Internet Standard
PS requires running code, but just one implementation
PS is technical vetting, Internet Standard is a popularity contest
No requirement for multiple implementations - one widely deployed
implementation can be a standard
PS goes away after 36 months
Working Group Snapshot is same as Scott's, working group
synchronization, but IESG comments, doesn't approve
  totally under control of working group

Pete Resnick - for IS, does "community adoption" mean one
implementation?
  yes, although it sounds crazy...

Scott - "it goes away"?
  goes to Historic (RFCs don't go away)

John Leslie - can you get to PS in less than 6 months?
  it's an ID, it can be handled like an ID

*       draft-bradner-ietf-stds-trk-00.txt - Scott

IESG has changed the bar - doesn't let nits go with Proposed Standards
Vendors don't wait for PS, and implement from random internet drafts
no difference between draft standard and full standard
"running code" shifted one step to the left
Suggested "stable snap shot" (SSS) - level of IESG review is very low
  immature prestandard specifications
  note omissions of requirements
proposed standard is largely unchanged, Internet Standard is DS+S

Keith Moore - "no known technical ommissions" in 2026 means a lot more
than it used to (security)
  now there's a nits list, etc.

Chris Bemondi - I-D nits focuses on normative references, so lengthens
publications queue, too

Bob Hinden - we have I-D and working group draft

Christian - we have seen this creeping insertion of a new step in the
process
  working group draft is now the old PS, modulo some things

Aaron Falk - are SSSs ephemeral?
  depends on the proposal, but yes, in this proposal

Craig White - what's the "big I Internet"?
  not an isolated IP network

*       draft-iesg-hardie-outline-00.txt - Leslie Daigle
        draft-iesg-hardie-category-descriptions-00.txt

Uses a cooking analogy (ingredients, baking, baked, eaten, boiled)
Boiled is "cooked by another process" - Experimental, etc.
"Baking" could be archival
could loosen lockstep reference requirements - difference in archival
series
new archival series, "Candidate Specification"
new designation, "Full Standard" for documents that prove
interoperability
Aside - might want to talk about sharing RFC series (includes non-IETF
documents)

Bob Hinden - doing a lot of cooking, but we're starving to death :-}
  one of the problems we have is WG thinks it's finished, IESG doesn't
  Ted also has document on document approval

Kurt Zelinga? - are individual submissions different?
  Ted also proposes a variety of working group types

Aaron Falk - what about "spoiled" category?
  working group work items are an archival series

*        draft-loughney-what-standards-00.txt - John Loughney

IETF has produced lots of RFCs, not a lot of standards
Mobile Phone TCP team that just used STD 7/RFC 793, and ...
Bugs in published specs - errata? new working group? what do you
implement?
No way to figure out what the best version of a protocol is...
We just produce more specifications.
Better web interface? Maintenance teams? Early STD teams? Enhanced STD
numbers?
id/loc split with RFC numbers!

next steps - get running code for this problem
- 61-119 RFCs for TCP, so ...?

* Maintaining Standards - Brian Carpenter

I'm trying to deprecate a feature in a draft standard - SHOULD? MUST? no
version numbers
FORTRAN knew how to deprecate features
TCP is the extreme example?
Would love to have a one-step standards process with version numbers
  and clear mechanism for identifying interoperability

Scott - remember IPR "feature" of Draft Standard when we evaluate
proposals

Spencer - what about conditional IPR statements based on standards
status/standards track?
  new IPR procedure is better, but we still need to nail this down

Dave Perkins - versioning - what about references to specific versions?
  Brian - ASN had that problem - we need to understand this aspect of
normative references
    need to be able to do both kinds of normative references

  5/ what would define success in a revised IETF Standards track - Scott

more advancement?
fewer products based on random I-Ds?
better understanding within IETF/WG?
trade press understanding?

Aaron Falk - documents are in state that corresponds to their status now
- less of that
  43 implementations of a PS is a failure? If it happens all the time,
yes
  effort required for a change proportional to size of change

Keith Moore - names of standard types need to coorrespond to reality
  standards must be of high quality
  be clear on the kind of endorsement
  be clear on currency - "relevancy"

Alan DeCock - define success in terms of failure - failed documents
archive?
  and these are the reasons why...

Harald Alverstrand - the success of the IETF is in producing standards
that make the Intenet work
  we don't do this for fun
  guessing what to implement, and getting it wrong, is a failure
  our metrics should correspond to this - clarity x 4

Mark Handley - success is that the Internet works better
  but it works fairly well on Internet Drafts and Proposed Standards
  clarity and relevance

Bill Strawn - standards track isn't too difficult to understand
  don't care about the press
  we have a lot of problems - how many are the result of slow
publication process?

Elliot Lear - success is when you take a TCPdump, identify the protocols
that are running, and show they are documented

John Loughney - no objective measure of success
  there will be problems - success is in the eye of the beholder
  there is some problem - we may not agree on what it is, but ...
  relevance of the work we do matters

Dave Crocker - this environment is too solution-rich!
  too many possibilities to tweak - what needs change badly?
  media silliness doesn't do any damage, and we can't fix it anyway
  cruft in archives is more of a problem
  minimize number of changes to minimize risk

Christian - really like Scott's proposal
  most realistic
  other proposals aren't that different
  we aren't here to protect the Internet and prevent harm - it's a
negative statement that requires censorship
  we need to swing back the other way
  the IETF spends a lot of time doing nothing
  if we don't make it possible for people to agree without two years of
censorship, we will fail

Bob Hinden - we've stopped building the Internet - but stuff is still
being added to the Internet
  control didn't work

Harald - making the Internet work better means experiments, things we
can do, fun
  doesn't mean blocking stuff, it means doing stuff

  6/ open discussion

Is there consensus we need a change - hummed "yes"

Brian Zorn - "success of the IETF" in my own terms - "less trouble"
  two protocols I'm working on are I-Ds, multiple implementations,
shipping products, nobody implemented the RFC
  if it's not on time, don't even bother
  can't change protocols between I-D versions, and it's already shipping
  Individual submissions?
  need version numbers, with interoperability statements
  protocol that has taken years to come out - the people who want to use
it have surrendered and left the building
  too fast and too slow... I-Ds vs PS

Melinda Shore - retrieval - standardization status isn't archiving,
everything should be archieved and retrievable
  need protocol specification version numbers
  choosing working group documents is full of whimsy today, and should
stay that way
  what you need to read to understand the working group - charter is
insufficient
  building a standards track around what's a working group document is
the wrong thing to do

Keith Moore - is something wrong with our labeling? probably, but
something is wrong with our process
  iterating toward perfection - FIN wait
  groups need to shut down so they don't start doing strange things
  document process needs to reflect this
  don't leave out Individual Submissions - have to be accommodated
  need some kind of waterfall model for all this stuff - don't make IESG
evaluate clean-sheet proposals with no experience
  sketches before snapshots? WGs shoot down incomplete specs and don't
read complete ones, so we read powerpoint

Louis - customers of these documents don't care about labels
  IDs used to be "ideas"
  the market picks what they want to deploy
  Internet Drafts is fine, except for non-archiving
  IETF doesn't make Internet work, it's where good work gets done
  make some tweaks and don't change much - adding process just won't
happen

Elwyn - Backwards compatibility - would my 1995 Sun/3 work today?
  should we require backwards compatibility for standards at some level?
  pre-standard Ethernet and modem vendors offered guaranteed upgrades to
standard versions

Scott - how do we make changes to standards-track protocols?

Elwyn - we need to be careful about what we standardize

Alex - two elements need to be addressed - number of stages and movers
between stages
  everybody has stages, no matter where you go
  need interoperable implementations
  moving between stages doesn't work well at IETF
  movers - individuals, chairs, next level starts to break down
  labeling could be radically improved - no indication of protocol,
standard level, version number...

Jonathan Sadler - stages of normative references?

Kurt - Individual submissions are very important - what the IETF is
built on

Brian Zorn - protocols that were individual submissions, but shouldn't
have been
  - WG yes, charter no, companies shipping

Aaron - multiple stages with interoperable implementations - we don't do
conformance testing

Alex - at most two stages, we need the IPR hook, need synchronization
between IETF formality and Internet reality
  there's a huge disconnect here - need a lower bar with
cross-functional review

Mark Handley - make the Internet work better? discourage implementation,
encourage experimentation, discourage deployment
  can't tell stages of a draft - no idea what attributes an I-D has
under current process (reviewed by congestion police)

Alex - documents advanced in meetings, three per year?
  Scott - work done on mailing list

SUMMARY - strong consensus that change is needed, snapshot needed,
multiple implementations needed, one stage not enough

WG Fodder - Harald
  do we have a rough idea? yes, need more discussion for WG
  Melinda - we really do know what the question is, just not the answer
    need more openness with other changes under consideration

  Bob Hinden - go for a WG now, there is a problem to solve
  Craig White - go back to interoperability testing? What is running
  code? We have no clarity about this
  Alex - know what the problem is and have suggestions on how to fix it
  ??? - we need a Solutions WG
  Mark Handley - have strong consensus change is needed, need to address
  it somewhere

  can we write a WG charter now? yes in the room

  volunteers for WG charter team? Send volunteers to [email-masked]
  we'll start a mailing list

Designated ID stage?

---------------- end -------------------------
http://eikenes.alvestrand.no/pipermail/solutions/2003-November/000345.html

Lucy E. Lynch 				Academic User Services
Computing Center			University of Oregon
llynch  @darkwing.uoregon.edu		(541) 346-1774/Cell: 912-7998