Subject: Re: Time to say "NO!!" to AUTH4200 (Re: AUTH48 checking the different formats (Re: [rfc-i] Public archival of AUTH48 communications))

From: David Noveck <[email-masked]>
Date: Mon, 28 Feb 2022 04:55:19 -0500

While I am in favor of greater transparency in this area and I feel that
the discussion has been helpful, I think the fundamental issue is that the
AUTH48 process has gotten out of control, which it clearly has, given that
there are documents that have languished in this state for over 25 weeks
rather than 48 *hours*, which was the original intention.   I can see no
justification for this expansion to 25 x 7 x 24 =  4200.

Perhaps with the formatting issues that now exist there is a need to be
more realistic and move to AUTH168 or even AUTH192 but a line clearly has
to be drawn and AUTH4200 is on the wrong side of it.

It gives time for issues on which consensus has been declared to be
relitigated, perhaps endlessly, and that is clearly bad.  Another source is
ADs showing an inappropriate degree of tolerance for lazy authors and that
has to stop somehow.9oj

The IESG has tried to address this issue in a statement on 1/5/2006 but,
here we are 16 years later and the problem still exists.

I'd like to ask all ADs with documents beyond the AUTH1000 point to
investigate and suggest ways to address the issue for real.

On Mon, Feb 28, 2022, 3:37 AM Carsten Bormann <[email-masked]> wrote:

> >> With the move to XML and renderings of a new RFC in different
> >> formats, who is responsible for reviewing the different
> >> renderings for unintended changes in meaning? If it's the
> >> authors, I'd hope that a change in AUTH48 might offer some way
> >> to spread the burden.
> >>
> >> FWIW
> >
> > Larry, based on the last document with which I went through
> > AUTH48 (in the first half of 2020), it wasn't clear who, if
> > anyone, was responsible for checking the different formats and
> > renderings.
>
> “Responsible”: The authors of course (in theory).
>
> > Of course, the theory is that, if the XML is
> > correct, everything else should take care of itself.
>
> Well, that isn’t even a theory.
> Xml2rfc has its mysterious ways.
>
> More importantly, implementers will look at a rendering (TXT, HTML/PDF).
> If that is confusing, you will have interoperability problems (or low
> take-up).
> So authors will typically check one of the renderings in some detail.
> (Only with a lot of luck, they will seriously both TXT and HTML or PDF,
> which unfortunately is really needed in particular since 3.10.0.)
>
> Where aspects of the XML are not visible in the renderings, the RPC has
> started to ask specific questions.
> E.g., for the type= attributes of <sourcecode elements.
> I’m not sure there is a checklist of invisible aspects.
> It would help to have an debug-the-XML rendering where all these invisible
> aspects become visible.
>
> In summary, I think that a theory where authors are responsible for the
> XML will stay fiction.
> The RPC is much better off doing this, being used to operate in that noisy
> environment, and (I hope) more aware what went wrong in previous RFCs.
>
> Grüße, Carsten
>
>

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Feb 2022 10:45:16 -0500

Perhaps the story of AUTH4200 could be more clearly communicated so that we
can understand if it's an anomaly.

My experience, for instance, with RFC8415 was that AUTH48 stretched into 2-3 months, because
the length of the document meant that a few authors just couldn't find the
time to do the full review required.  A significant issue was that it had
spent a long time in the Q, and authors had swapped too much context out.

Basically, the best way to make it all work faster would be to just do it
faster  :-)

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 28 Feb 2022 15:50:35 -0000

Y'all might find Christian's work in RFC 8963 useful in this regard.
Although only a small and random sample of RFCs, it does show some of the variation in Auth48 and the (possibly anecdotal) causes thereof.
Adrian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 28 Feb 2022 18:48:50 +0100

On 2022-02-28, at 10:55, David Noveck <[email-masked]> wrote:
> 
> I'd like to ask all ADs with documents beyond the AUTH1000 point to investigate and suggest ways to address the issue for real.  

Apart from the technical mismatches I just brought up, there also is a scheduling problem.  Doing AUTH48 even for a medium-sized document such as RFC-to-be 9176 (60 PDF pages; 74 in TXT form) requires setting aside some time.  Today, the RPC is essentially a black box that suddenly spews a significant work order at you.  Not totally surprising (one can look at the queue, look at average sojourn times, and make guesses), but not plannable either.

So in this case I decided to leave all the early AUTH48 work with the main author/editor, and I will only come in when the document is nearly stable, which has given me some time to weave this work into the scheduling.  But there just is no way to do this in less than 4 weeks (which is still AUTH672, phew).

If we plan to move towards a more author-friendly form of document production, some interaction about scheduling the author work will need to be added.
And yes, “kein Plan überlebt den ersten Feindkontakt”… (*).

Grüße, Carsten

(*): Actually, Moltke said “Kein Operationsplan reicht mit einiger Sicherheit über das erste Zusammentreffen mit der feindlichen Hauptmacht hinaus”.  Apparently, he didn’t have an RPC helping him :-)

---

From: David Noveck <[email-masked]>
Date: Mon, 28 Feb 2022 16:58:25 -0500

On Mon, Feb 28, 2022, 10:46 AM Michael Richardson <[email-masked]>
wrote:

> time to do the full review required.

A full review should not be necessary.  The only review should be of the
changes made by the RFC editor.   Perhaps the problem was that these were
some excessive.

I had the advantage for RFC of only having two authors.  I wound up having
to say "no we don't want to do that" a bunch of times but it didn't add too
much time since my co-author was of the same opinion.

A significant issue was that it had
> spent a long time in the Q, and authors had swapped too much context out.

Ouch!

> Basically, the best way to make it all work faster would be to just do it

Good luck with that.

Part of the problem is that there is no clear definition of what the RFC
editor is to do.  I've found them helpful on commas and split infinitives
and formatting issues.   Was the problem that the RFC editor was trying to
do too much?

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 1 Mar 2022 00:17:24 +0100

> A full review should not be necessary. 

I wish that were true...

> The only review should be of the changes made by the RFC editor.   Perhaps the problem was that these were some excessive.   

The RFC editor often makes suggestions when text is unclear.
In many cases these are great, in some cases these are wrong, but easily corrected (or rejected).
The interesting cases are where the authors then need to find out what was actually meant.

>> spent a long time in the Q, and authors had swapped too much context out.

Not just in the RFC editor queue (even though a MISSREF contributed a year in my example), also in the IESG processing, which can be very focused on narrow issues (and cause a lot of Brownian motion).
Have a look at the graphics and history in

https://datatracker.ietf.org/doc/draft-ietf-core-resource-directory/

WGLC in 1H2019, IESG processing until early 2021, 1 year MISSREF…
So it’s been three years since we did the final heavy lifting.
Certainly swapped out by now.

(And that was a very successful development, pretty cutting-edge when we started it in 2011, with other SDOs then picking up on the way much of the innovation in their own subset derivatives.  It just took a lot of time to get the nooks and crannies right, with several author changes and job changes with the existing authors in that time.)

Grüße, Carsten

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 28 Feb 2022 15:24:36 -0800

> > A full review should not be necessary. 

IMO a full review is necessary, not least to ensure that the document
remains self-consistent and that any changes applied during editing are
uniformly applied to all relevant parts of the document, not just in the
subset of places that the editor noticed.

This is analogous to a situation that arises when doing code review.
If you change an API's semantics, you had better go around and check
all the callers of that API to see if they need adjustments, not just the
callers of the API that your test suite happens to exercise.  With
specifications we do not have the rigid rules of function names to identify
the places to check, and the safe thing to do is read the whole document
with any changes that were made fresh in mind.

-Ben

---

From: John C Klensin <[email-masked]>
Date: Mon, 28 Feb 2022 19:11:01 -0500

--On Monday, February 28, 2022 15:24 -0800 Benjamin Kaduk
<[email-masked]> wrote:

> IMO a full review is necessary, not least to ensure that the
> document remains self-consistent and that any changes applied
> during editing are uniformly applied to all relevant parts of
> the document, not just in the subset of places that the editor
> noticed.
> 
> This is analogous to a situation that arises when doing code
> review. If you change an API's semantics, you had better go
> around and check all the callers of that API to see if they
> need adjustments, not just the callers of the API that your
> test suite happens to exercise.  With specifications we do not
> have the rigid rules of function names to identify the places
> to check, and the safe thing to do is read the whole document

Carsten,

(1) Full agreement with Ben.  I'd add that, long ago, we went
directly from IESG approval to the RFC Editor to publication
without an intervening check like AUTH48.   The check was
instituted after the IESG and community were very unhappy with
how a few documents came out of that process.

(2) The alternative to something very much like AUTH48 (as Ben
describes it) is to hand the document over to the RPC when we
(the IESG or a WG) would otherwise decide it was ready for Last
Call, get it edited into final form (plus or minus RFC rather
than I-D boilerplate), and then do the Last Call and IESG review
on  the edited and finished document.  Of course that means
that, if _any_ substantive changes show up during or after Last
Call, the document would need to be edited again and go through
Last Call again.   

After noting that some other SDOs do things quite similar to
that, the community has looked at similar options several times
and concluded that the additional time and costs it would
introduce are just not worth it.  Certainly the odds that it
would save time relative to the current model are very low.  At
least most times when people have argued for going down that
path in the past, the argument as been that it would improve
quality.  Whether that is true or not, and whether the quality
improvements would be worth the costs, would probably depend on
how closely IETF participants look at those supposedly final
documents.

    john