Subject: [rfc-i] can we just drop the disclaimer that seems to have appeared?

From: Joseph Lorenzo Hall <[email-masked]>
Date: Wed, 2 Oct 2019 14:14:46 -0400

(Apologies, I sent this a week ago and apparently it's been in the
moderation queue as I'm not a subscriber to this list. Someone might
want to check and see if there are others in limbo.)

Hi, first, thank you for all your hard work.

I was excited to see that the datatracker now has verified errata
versions of RFCs up. As someone that spends time with archivists and
librarians (my PhD is from a School of Information), this is a very
welcome development!

However, I noticed this new feature while looking at the HTTP/2 ORIGIN
frame RFC (RFC 8336):

https://datatracker.ietf.org/doc/rfc8336/

However, when I click on that link, it takes me to

https://www.rfc-editor.org/rfc/beta/errata/RFC8336.html

Which has a ginormous and ugly and scary notice that says that despite
it including verified errata, that the "rendering" (whatever that means
in this context) should not be used for reference. This is on top of the
red coloring used for the errata'd text and the "expand" button that
shows the verified errata contents.

That big warning seems a bit much from a usability perspective (people
are less likely to read documents with big scary warnings at the top)
I'd love to just get rid of that thing or have a much more innocuous
thing inline next to the errata-changed text that said something short
and sweet and then linked to why using a document that actually
incorporates the errata should be discouraged as a reference (which
seems silly, but I realize you have good reasons for this kind of thing
that are beyond me).

best, Joe

-- 
Joseph Lorenzo Hall
Chief Technologist, Center for Democracy & Technology [https://www.cdt.org]
1401 K ST NW STE 200, Washington DC 20005-3497
e: [email-masked], p: 202.407.8825, pgp: https://josephhall.org/gpg-key
Fingerprint: 3CA2 8D7B 9F6D DBD3 4B10  1607 5F86 6987 40A9 A871

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 2 Oct 2019 11:46:15 -0700

+1 to this.

In this vein, I am aware of two soon-to-be-verified errata which are just
clear defects in the specification (example:
https://www.rfc-editor.org/errata_search.php?rfc=8452)
What best serves the interest of the standard is for the reader to be sent
to the corrected version and to treat that as the appropriate reference for
standardization.

-Ekr

---

From: Heather Flanagan <[email-masked]>
Date: Wed, 2 Oct 2019 12:46:27 -0700

HI all,

I understand the warning is big, somewhat obnoxious, and possibly redundant. And that’s exactly as I think is appropriate and necessary. 

Having the documents available with verified errata is helpful, but these cannot, at any point, be considered the canonical version fo the spec. RFCs do not change once published; this is a core tenet of the RFC Series. Pointing people to unofficial, informal material as “the” source is inappropriate. To add to that, and I know this has been discussed, the errata have not gone through a consensus process. Considering the RFC to be complete only with this added information adds an escape clause to the review and publication process I think is unacceptable. 

I am supporting having these annotated RFCs available as an aid to the reader, but I want consumers to really think about what they’re looking at. And given how individuals tend to ignore header and boilerplate style material, having the current header as prominent as it is now is the only way I can see to get that information across.

-Heather

> On Oct 2, 2019, at 11:46 AM, Eric Rescorla <[email-masked]> wrote:
> 
> +1 to this.
> 
> In this vein, I am aware of two soon-to-be-verified errata which are just clear defects in the specification (example: https://www.rfc-editor.org/errata_search.php?rfc=8452 <https://www.rfc-editor.org/errata_search.php?rfc=8452>)
> What best serves the interest of the standard is for the reader to be sent to the corrected version and to treat that as the appropriate reference for standardization.
> 
> -Ekr
> 
> 
> 
> 
> On Wed, Oct 2, 2019 at 11:15 AM Joseph Lorenzo Hall <[email-masked] <mailto:[email-masked]>> wrote:
> https://datatracker.ietf.org/doc/rfc8336/ <https://datatracker.ietf.org/doc/rfc8336/>
> 
> https://www.rfc-editor.org/rfc/beta/errata/RFC8336.html <https://www.rfc-editor.org/rfc/beta/errata/RFC8336.html>
> 
> Chief Technologist, Center for Democracy & Technology [https://www.cdt.org <https://www.cdt.org/>]
> e: [email-masked] <mailto:[email-masked]>, p: 202.407.8825, pgp: https://josephhall.org/gpg-key <https://josephhall.org/gpg-key>
> [email-masked] <mailto:[email-masked]>
> https://www.rfc-editor.org/mailman/listinfo/rfc-interest <https://www.rfc-editor.org/mailman/listinfo/rfc-interest>

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 2 Oct 2019 19:49:25 +0000

Can we add <blink> ?

I think Heather’s rationale make a lot of sense and prefer to leave it.

---

From: Joseph Lorenzo Hall <[email-masked]>
Date: Wed, 2 Oct 2019 15:50:30 -0400

On Wed, Oct 2, 2019 at 3:46 PM Heather Flanagan <[email-masked]> wrote:

> HI all,
>
> I understand the warning is big, somewhat obnoxious, and possibly
> redundant. And that’s exactly as I think is appropriate and necessary.
>
> Having the documents available with verified errata is helpful, but these
> cannot, at any point, be considered the canonical version fo the spec. RFCs
> do not change once published; this is a core tenet of the RFC Series.
> Pointing people to unofficial, informal material as “the” source is
> inappropriate. To add to that, and I know this has been discussed, the
> errata have not gone through a consensus process. Considering the RFC to be
> complete only with this added information adds an escape clause to the
> review and publication process I think is unacceptable.
>
>
I think this is short-sighted and overly formulaic — e.g., if we did a
rough calculation as to what number of errata would even need to be
considered for a consensus process I suspect it would be a vanishing
fraction of all errata.

That being said, I've said my piece and you are the decider. best, Joe

> I am supporting having these annotated RFCs available as an aid to the
> reader, but I want consumers to really think about what they’re looking at.
> And given how individuals tend to ignore header and boilerplate style
> material, having the current header as prominent as it is now is the only
> way I can see to get that information across.
>
> -Heather
>

---

From: "Scott O. Bradner" <[email-masked]>
Date: Wed, 2 Oct 2019 16:09:22 -0400

+1

> On Oct 2, 2019, at 3:46 PM, Heather Flanagan <[email-masked]> wrote:
> 
> HI all,
> 
>> In this vein, I am aware of two soon-to-be-verified errata which are just clear defects in the specification (example: https://www.rfc-editor.org/errata_search.php?rfc=8452)

---

From: Ted Hardie <[email-masked]>
Date: Wed, 2 Oct 2019 13:17:27 -0700

Greetings,

>> review and publication process I think is unacceptable.
Good news!  You're both right.  But you have the equivalent of pronoun
trouble.

The point Heather is making is that the RFC is archival and does not
change, and that is true and a very fundamental.  But the point Eric (and I
think Joe) are making is that the *specification for $FOO* is not
necessarily similarly unchanging.  The specification of what to do to
achieve interoperable operations (or correctness in cases like the
AES-GCM-SIV case Eric pointed out) is the RFC plus whatever corrections
have been accepted and implemented by the community.  If there are no
corrections, then it is just the RFC.  In the good cases of corrections,
there are formal errata which have been accepted by the relevant group; in
the bad cases we have lore, where knowing the internal state of some WG
tells you which update drafts MUST be implemented to get it to work.

The state-of-play proposal made a while back tried to give a name and
reference for the * current specification of $FOO* without touching the
nature of RFCs.  It was trying to get rid of the prevalence of lore by
creating a common way of summarizing what got added or taken away to the
currently published RFCs to get to common interoperability.  That didn't
succeed as a proposal, but I think its useful context here, because it
tells you why people want the verified errata version to be easily used by
a implementer--because its the most up-to-date thing we have to help people
achieve interoperability.

I have no UX credentials and I easily ignore large red things, so I'm not
giving anyone any advice on look and feel.  But please do keep in mind that
desirable goal:  limiting the amount of external knowledge/lore an
implementer has to have to get the "specification for $FOO" right.  That's
what helps us get interoperability (the end goal), and we should keep the
friction on getting there down as low as we reasonably can.

Just my two cents,

regards,

Ted

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 2 Oct 2019 13:29:59 -0700

On Wed, Oct 2, 2019 at 1:18 PM Ted Hardie <[email-masked]> wrote:

>>> Having the documents available with verified errata is helpful, but
>>> these cannot, at any point, be considered the canonical version fo the
>>> spec. RFCs do not change once published; this is a core tenet of the RFC
>>> Series. Pointing people to unofficial, informal material as “the” source is

Precisely. At the end of the day, the output of the IETF is technical
specifications. The initial version of those technical specifications is
published as RFCs, but occasionally those RFCs themselves contain errors
which do not match what the technical specification clearly needs to say.
The errata process allows us to correct those errors, but we then need a
mechanism to get those in front of people who are looking to implement the
specification. Having that mechanism have a large warning of this kind
partly defeats that mechanism.

I am aware of the argument Heather raises about consensus, but I believe it
actually pushes us in the other direction. We routinely make minor changes
to documents in AUTH-48 and as long as those changes are clearly in line
with consensus, those documents do not go back to the WG or to IETF LC, and
are merely subject to approval by the ADs or WG chairs. The kinds of
changes we are talking about for errate (e.g., the typo for 8452) are
generally of that nature -- or even more minor. Given that those same
people are generally the errata approvers, this seems like a natural
extension of that process.

-Ekr

---

From: Adam Roach <[email-masked]>
Date: Wed, 2 Oct 2019 15:37:05 -0500

On 10/2/19 3:29 PM, Eric Rescorla wrote:
> I am aware of the argument Heather raises about consensus, but I 
> believe it actually pushes us in the other direction. We routinely 
> make minor changes to documents in AUTH-48 and as long as those 
> changes are clearly in line with consensus, those documents do not go 
> back to the WG or to IETF LC, and are merely subject to approval by 
> the ADs or WG chairs. The kinds of changes we are talking about for 
> errate (e.g., the typo for 8452) are generally of that nature -- or 
> even more minor. Given that those same people are generally the errata 
> approvers, this seems like a natural extension of that process.

I was just about to make this point; so take this message as someone 
else who has thought about this issue a lot reaching the same conclusion.

/a

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 2 Oct 2019 16:51:06 -0400

We bend the need for explicit confirmation of rough consensus in order 
to get through Auth48.  (It takes way longer than we want anyway.)  That 
doe snot mean that the practice there reflects an agreement that the 
rough consensus is not important.  In fact, ADs are expected to send 
things back to the list if the changes get significant.

If we as a community want to adopt some form of recognized definiton as 
to what the spec for $foo is, I would be happy to see that happen.  All 
efforts to do so have failed.  Errata do not meet that goal.
I am willing to see the documents displayed with errata with 
sufficiently clear warnings (which is what Heather and the team are 
doing.)  But displaying them in a way which could be easily confused 
with the actual RFC content would be VERY bad.

So again, if folks want a "this is the current specification for $foo" 
then lets put together a way to get community consensus on such a 
process, including how that process gets community consensus on the 
answers for each $foo.

Yours,
Joel

On 10/2/2019 4:29 PM, Eric Rescorla wrote:
> 
> 
> On Wed, Oct 2, 2019 at 1:18 PM Ted Hardie <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Greetings,
> 
>     On Wed, Oct 2, 2019 at 12:51 PM Joseph Lorenzo Hall <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
> 
> 
>         On Wed, Oct 2, 2019 at 3:46 PM Heather Flanagan
>         <[email-masked] <mailto:[email-masked]>> wrote:
> 
>             HI all,
> 
>             I understand the warning is big, somewhat obnoxious, and
>             possibly redundant. And that’s exactly as I think is
>             appropriate and necessary.
> 
>             Having the documents available with verified errata is
>             helpful, but these cannot, at any point, be considered the
>             canonical version fo the spec. RFCs do not change once
>             published; this is a core tenet of the RFC Series. Pointing
>             people to unofficial, informal material as “the” source is
>             inappropriate. To add to that, and I know this has been
>             discussed, the errata have not gone through a consensus
>             process. Considering the RFC to be complete only with this
>             added information adds an escape clause to the review and
>             publication process I think is unacceptable.
> 
> 
> 
>     Good news!  You're both right.  But you have the equivalent of
>     pronoun trouble.
> 
>     change, and that is true and a very fundamental.  But the point Eric
>     (and I think Joe) are making is that the *specification for $FOO* is
>     not necessarily similarly unchanging.  The specification of what to
>     do to achieve interoperable operations (or correctness in cases like
>     the AES-GCM-SIV case Eric pointed out) is the RFC plus whatever
>     corrections have been accepted and implemented by the community.  If
>     there are no corrections, then it is just the RFC.  In the good
>     cases of corrections, there are formal errata which have been
>     accepted by the relevant group; in the bad cases we have lore, where
>     knowing the internal state of some WG tells you which update drafts
>     MUST be implemented to get it to work. 
> 
> 
> which do not match what the technical specification clearly needs to 
> say. The errata process allows us to correct those errors, but we then 
> need a mechanism to get those in front of people who are looking to 
> implement the specification. Having that mechanism have a large warning 
> of this kind partly defeats that mechanism.
> 
> I am aware of the argument Heather raises about consensus, but I believe 
> it actually pushes us in the other direction. We routinely make minor 
> changes to documents in AUTH-48 and as long as those changes are clearly 
> in line with consensus, those documents do not go back to the WG or to 
> IETF LC, and are merely subject to approval by the ADs or WG chairs. The 
> kinds of changes we are talking about for errate (e.g., the typo for 
> 8452) are generally of that nature -- or even more minor. Given that 
> those same people are generally the errata approvers, this seems like a 
> natural extension of that process.
> 
> -Ekr
> 
>

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 2 Oct 2019 14:01:54 -0700

> things back to the list if the changes get significant.

I want to clarify that the changes I am talking about are material, but not
significant.
For instance, the erratum for RFC 8452 is a typo in a hex string in a text
vector.
Under no usual circumstances would that require a fresh declaration of
consensus.

> with the actual RFC content would be VERY bad.

As Ted says, this is perhaps a pronoun issue. What I care about is that
people
know what the specification is, and presently the RFC memorializes one point
in time for that. I don't agree that people having the correct view of the
specification
and not being aware that it's not precisely the RFC is "VERY bad",
especially
when compared to having people have the incorrect view of the specification.

-

> process, including how that process gets community consensus on the

I had understood the errata process to be that process. Again, to take the
example
of 8452, the spec is just flat-out wrong and the correct spec is the one
with the
verified erratum, no matter how we choose to display it.

-Ekr

---

From: Randy Bush <[email-masked]>
Date: Wed, 02 Oct 2019 14:20:58 -0700

> RFCs do not change once published; this is a core tenet of the RFC
> Series.

yep

a good errata mechanism is a good companion to this

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 3 Oct 2019 11:21:24 +1300

Eric,

On 03-Oct-19 09:29, Eric Rescorla wrote:
...
 
> Precisely. At the end of the day, the output of the IETF is technical specifications. The initial version of those technical specifications is published as RFCs, but occasionally those RFCs themselves contain errors which do not match what the technical specification clearly needs to say. The errata process allows us to correct those errors, but we then need a mechanism to get those in front of people who are looking to implement the specification. Having that mechanism have a large warning of this kind partly defeats that mechanism.

Exactly the opposite, IMHO. It's *because* the annotated version flags the updates that it's useful. The reader now knows where to look in an implementation to find whether it is correct or whether the implementor was misled by the errors in the original RFC.
 
> I am aware of the argument Heather raises about consensus, but I believe it actually pushes us in the other direction. We routinely make minor changes to documents in AUTH-48 and as long as those changes are clearly in line with consensus, those documents do not go back to the WG or to IETF LC, and are merely subject to approval by the ADs or WG chairs. The kinds of changes we are talking about for errate (e.g., the typo for 8452) are generally of that nature -- or even more minor. Given that those same people are generally the errata approvers, this seems like a natural extension of that process.

It is, but it's after the checkpoint provided by RFC publication (at which point the boilerplate about "this is a draft" is removed). To me, that is a strong argument for including very clear markers of the changes.

If you really want to fix this problem: https://tools.ietf.org/html/draft-klensin-isdbis

Regards
    Brian

---

From: Adam Roach <[email-masked]>
Date: Wed, 2 Oct 2019 17:40:45 -0500

> Exactly the opposite, IMHO. It's *because* the annotated version flags the updates that it's useful. The reader now knows where to look in an implementation to find whether it is correct or whether…

Oh, yeah, flagging the updates is extremely useful, and I don't think 
anyone is arguing against doing so. (I'm the one who wrote the part of 
the SOW that calls for such changes to be clearly styled as different 
than the rest of the document, and I think it's pretty important 
myself.) The pushback I'm seeing in this thread pertains to the rather 
stern black-box warning at the top of the errata-patched documents. I 
think it's probably fine to inform readers that the document 
incorporates errata that have been verified since the corresponding RFC 
was published; but the current language goes much further than that.

/a

---

From: Jim Schaad <[email-masked]>
Date: Wed, 2 Oct 2019 21:07:44 -0700

-----Original Message-----
From: rfc-interest <[email-masked]> On Behalf Of Adam
Roach
Sent: Wednesday, October 2, 2019 3:41 PM
To: Brian E Carpenter <[email-masked]>; Eric Rescorla
<[email-masked]>
Cc: RFC Interest <[email-masked]>; Heather Flanagan
<[email-masked]>; Joseph Lorenzo Hall <[email-masked]>
Subject: Re: [rfc-i] can we just drop the disclaimer that seems to have
appeared?

> Exactly the opposite, IMHO. It's *because* the annotated version flags the
updates that it's useful. The reader now knows where to look in an
implementation to find whether it is correct or whether the implementor was
misled by the errors in the original RFC.

Oh, yeah, flagging the updates is extremely useful, and I don't think anyone
is arguing against doing so. (I'm the one who wrote the part of the SOW that
calls for such changes to be clearly styled as different than the rest of
the document, and I think it's pretty important
myself.) The pushback I'm seeing in this thread pertains to the rather stern
black-box warning at the top of the errata-patched documents. I think it's
probably fine to inform readers that the document incorporates errata that
have been verified since the corresponding RFC was published; but the
current language goes much further than that.

[JLS] One of the other issues that needs to be taken into consideration is
the quality of the Errata database.  There are a number of errata where
applying the errata leads to something that is not very usable.  If you look
at errata 9 in https://www.rfc-editor.org/rfc/beta/errata/RFC4753.html then
you will see that text was removed rather than being replaced with correct
text.  This makes it more difficult to say that this is a corrected version
of the document.  At some point a pass over that database is going to be
needed both to deal with errata that are not well formed,  such as errata
1800 in the same document and those which have multiple replacements in a
single errata rather than being multiple errata.  This may be done by a
standardized way but it is not currently done.

Jim

/a

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 3 Oct 2019 00:19:15 -0400

Dear colleagues,

It so happens that I work for the Internet Society, but I am not
speaking for it.

On Wed, Oct 02, 2019 at 12:46:27PM -0700, Heather Flanagan wrote:

> these cannot, at any point, be considered the canonical version fo
> the spec. RFCs do not change once published; this is a core tenet of
> the RFC Series.

It seems to me that this is a quite narrow meaning of "RFCs do not
change", and that taken to its natural conclusion the correct action
would be to suppress completely the errata system.

It may well be that I have overlooked something, but everything I have
ever understood about publication (and about the RFC errata system)
says that an erratum is never published to create a substantive
change, nor even to correct a mere typographical error (Cf. _The
Chicago Manual of Style_, but MLA and other manuals I can consult
without a visit to the Big House of Books all seem to agree about
this.).  It's to correct an error in the _text_, not in the
_concepts_, that causes the text somehow to be wrong such that a
reader would be misled.

There is a distinction to be made between "an error in the text" and
"an error in the concepts described in the text".  It is the former
that errata are supposed to repair.  The idea, then, is that the
mistake is "merely in the printed text" and not strictly speaking in
the document itself.  In other words, the idea is that if everyone had
done everything perfectly, the erratum would not exist.  (I observe
that some publishers have distinguised between errata and corrigenda,
in order to distinguish who is at fault.  I'm pleased the IETF doesn't
do that, though perhaps erring on the side of making the authors/WGs
responsible would be better justified.)  In this sense, then, the
erratum exists to make the printed text more accurately cleave to what
it ought to have been, rather than the corrupt thing that came out of
the all too human process.

Now, one might argue that the RFC series is fully electronic for
practical purposes, so that it is stupid and silly to use some
"correcting this thing to what it should _really_ say" mechanism.  Why
not just patch the "original" directly?  Well, that won't work, of
course, because every now and then it might be important to know
whether someone was referring to the pre-erratum or post-erratum
version, and you couldn't know that by reference to the same RFC
number without any other stuff on it.  Interoperation could depend on
this, so keeping the difference visible is clearly valuable.

Yet it would be perverse to insist that the version without errata
ought to be the one to refer to.  If you get a physical book
containing (say) mathematical proofs in it, and there's an erratum on
p 719, you don't tell people to ignore the erratum pasted in the back
(unless youre trying to make some obscure point about the difficulty
of typesetting equations correctly).  You refer to the book, and if
there are errata that are relevant you draw the reader's attention to
them.  It therefore seems perverse to have an ***RFC Police Force
Warning*** on the top to say that the rendering may not (MAY NOT?)  be
used as a reference.

Of course, the most natural thing to do, given that integers are free
and all that, is to generate a new, obsoleting RFC automatically to
replace the RFC with the error as soon as an erratum is verified.  I
suppose there are good reasons that people decided not to do this.
Perhaps it has to do with the exalted and holy way the network
community has come to treat the term "RFC".  But in response, perhaps
the Editor could create a new series -- say, cRFC (for corrigendum
RFC) -- that contains automatically-generated patched versions of the
RFC proper, containing only and all verified and non-held errata.
That would allow canonical reference to the verified-corrected text
while yet preserving the official, errors-and-all, unchanged RFCs.  It
probably requires extending the range to numerics, though:
000000.0000001 or something.  Heck, you could even make them
completely isomorphic with the "real" RFCs by having each "real" RFC
correspond to cRFC NNNNNNNN.0

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 11 Oct 2019 16:42:51 +1100

On Thu, Oct 3, 2019, at 05:46, Heather Flanagan wrote:
> reader, but I want consumers to really think about what they’re looking 
> at. And given how individuals tend to ignore header and boilerplate 
> style material, having the current header as prominent as it is now is 
> the only way I can see to get that information across.

Thread necromancer here.

Are you saying "warnings are ignored, so we added a bigger warning"?

In contrast, tools.ietf.org is used as a primary source for RFCs Internet-wide.  The HTML conversion there makes non-canonical changes, of which we routinely hear about the way it can create broken references (though that is no real fault of the tool, it's an impossible problem).  There, all we have is a neat banner at the top pointing to a copy of the canonical version using the link text "txt".

A similar effect could be gained by adding a single line saying "This copy of RFC XXXX has 3 errata applied. View the _canonical version_."  Even aligning the banner with the text block would make it less glaring.

Of course, this is rfc-editor.org and the RFC Series Editor has discretion there.  I hope that the tool can be modified to show these views in a less glaring format for use by other sites.

p.s., The RFC8336 erratum Joe points to seems to have been applied with extra whitespace not present in the erratum itself.  The ^^^ is a redundancy that would need to be fixed, of course.