Subject: [Edm] Some thoughts on Sep 1 meeting, Issue 1

From: Vijay Gurbani <[email-masked]>
Date: Sun, 6 Sep 2020 18:51:43 -0500

Dear Tommy, Mirja, all: Thank you for organizing the virtual meeting on
Tue, Sep 1.

As the instigator of the IETF WG Chairs list thread on archiving
implementations [1], I have some thoughts, but unfortunately due to a
conflict with meetings on my calendar, I had to leave early before the
second half of the meeting on last Tue when this issue was discussed.
Thank you for considering this thread of enough interest to put it on the
EDM agenda.

My main question in the IETF Working Group Chair thread was simple: How do
the larger set of R&D folks who use the protocols we create find relevant
code and other accoutrements that are of interest to them without wading
into the IETF arcana?  (As a definition, "IETF arcana" is the collective
knowledge we, on this list, have gathered from participating in the
organization for a sustained period.  We know what a datatracker is, what
the difference between an individual submission versus a WG submission is,
we know where Wikis related to WG pages are, we know what WG deliverables
are, etc.  This collected knowledge is not readily available to the many
people who want to use our protocols, and it is not clear to me that this
knowledge would be of use to them unless they want to participate in the
IETF to the extent that we do.  They need to implement a standardized
protocol, for the most part, that is their primary concern.)

What is the most visible output of the IETF?  Perhaps unarguably, it is an
RFC.  Even someone who has at best, a foggy idea of what the IETF is, knows
what a RFC does.  When I talk to developers at companies and students at
universities, if they have heard of IETF at all, it is mostly through
knowing that some organization called IETF produces these things called
RFCs.  That's it. Perhaps for them that is enough.  And if you buy that
argument, then the corollary is that we should do everything that we can to
make sure that the implementers have all of the information they need to
implement the protocol from the RFC itself.

The primary artifact of the protocol is the code that implements it.  Not
all code is equal.  Some, like a SIP server, a QUIC implementation, a HTTP
Server, a TLS implementation are complex and it is a foregone conclusion
that we can simply park some code in an archive that implements such
complex protocols.  However, we also produce routing protocols, hashing
functions, best practices on how to handle dual IP stacks, ALTO entities
that use HTTP and JSON to converse with each other, and so on.  Such code
is small, perhaps modular, stand-alone and often can easily be made
available in a repository much more readily than complex systems like QUIC,
SIP, TLS, or HTTP.

For the more complex protocols and their implementations, wikis and GitHub
repositories that point the new implementer to the community of interest is
sufficient.  For the less complex protocols that do not need a large
community of interest, a simple code archival mechanism appears to be
reasonable.  To the extent that the archive should be IETF owned, it makes
sense to home the archive under ietf.org, as some of the IETF wikis are.

So, the next question is how to let the implementer know of these resources
when they read a RFC or an I-D?  RFC 7942 is an excellent start.  It
suggests having an Implementation Section in an I-D, but requests the
removal of such a section from an RFC-to-be.  This implementation section
can be used to document the datatrackers, wikis, and GitHub repositories
for the more complex code and a link to the archival area for the less
complex code.  (I realize that IETF does not have authority over links like
GitHub that are not under the ietf.org administrative domain, but perhaps
for large projects, it is adequate that they stay in repositories like
GitHub.) Perhaps RFC 7942 can be updated so that the Implementation Section
is not removed when the I-D becomes an RFC, and perhaps verbiage can be
added to it during the update that essentially says "caveat emptor", but in
more words and with appropriate constraints.  But the basic concern of
making an RFC self contained such that it has links to implementations is
adequately accomplished by an updated RFC 7492.

Thank you for reading so far, and thank you again for considering this of
enough interest to put it on the EDM agenda.  I look forward to your
thoughts on this.

[1] A quick recap and restatement of the need for a stable archival area
controlled by the IETF is in
https://mailarchive.ietf.org/arch/msg/wgchairs/He68KsKijRg2FAIpxRFi5MyJGis/
 That link contains other links to the original thread for those of you who
want to catch up with the discussion.

Cheers,

- vijay

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 8 Sep 2020 19:44:12 -0500

Hi, Vijay,

On Sun, Sep 6, 2020 at 6:52 PM Vijay Gurbani <[email-masked]>
wrote:

> So, the next question is how to let the implementer know of these
> resources when they read a RFC or an I-D?  RFC 7942 is an excellent start.
> It suggests having an Implementation Section in an I-D, but requests the

ISTM that it's worth considering what's possible without revising RFC 7942.

Everyone may already know this, but the RFC Editor now has a way of
inlining errata - for example, check
https://www.rfc-editor.org/rfc/inline-errata/rfc3261.html. *

If we can figure out how to track "what's worth pointing to", perhaps the
RFC editor could also inline pointers, rather than dorking with the RFC
text itself.

Best,

Spencer

* I believe Adam Roach did the heavy lifting on proving this was feasible
when he was on the IESG. I'm not sure where the actual code used today came
from, but I do want to give credit when I can.

> --
> Edm mailing list
> [email-masked]
> https://www.iab.org/mailman/listinfo/edm
>

---

From: Vijay Gurbani <[email-masked]>
Date: Wed, 9 Sep 2020 10:06:37 -0500

On Tue, Sep 8, 2020 at 7:44 PM Spencer Dawkins at IETF <
[email-masked]> wrote:

> [...]

Dear Spencer: Nice to hear from you, and thank you for your comment.

I am all for considering what's possible without revising RFC 7942, no
problem, as long as we can redirect the interested parties to the
appropriate code archives.

The work you cited that Adam has done on RFC 3261 for errata looks very
neat.  This could be a potential way to proceed if IAB/IETF decided not to
go the route of updating RFC 7942 or inserting some other text in the
RFC-to-be.  Some issues to consider if we go the inlining route would be
(a) how to aesthetically present the information on the RFC masthead, (b)
what is the expectation when someone prints the RFC with such inling?  Will
the inline text show up?  (In the case of RFC 3261 with inlined errata, I
note that all expanded errata are printed, non expanded errata appear as
green text in the generated PDF.  By "expanded" I mean that if I click on
one of the errata, it expands into a rectangle at the appropriate place in
the RFC, and that entire rectangle is then printed in the generated PDF.)

Thanks,

- vijay

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 10 Sep 2020 18:40:31 +0200

Hi Vijay,

Thanks for moving this discussion forward.

To focus on the formal bits for a second: RFC7942 doesn’t actually require that the section is removed. This says "Authors are requested to” but I guess it’s for the authors/wg to decide. Also this is an IETF-consensus document which I think was AD-sponsored. So the way to update it is “simply” to write a bis-draft and bring it to gendisptach these days.

More to the actually point: I understand your motivation about people only knowing RFCs but none of the IETF process but I’m not sure putting this information in an RFC is the right tool. I bed it’s actually not very hard to find some suitable code on GitHub these days, especially as you said if this is about larger projects (which in the best case are actively maintained). For smaller code snippets we often put pseudo code in the document of appendix. So that’s another option.

Mirja

> On 9. Sep 2020, at 17:06, Vijay Gurbani <[email-masked]> wrote:
> 
> On Tue, Sep 8, 2020 at 7:44 PM Spencer Dawkins at IETF <[email-masked]> wrote:
> Hi, Vijay,
> 
> [...]
> Everyone may already know this, but the RFC Editor now has a way of inlining errata - for example, check https://www.rfc-editor.org/rfc/inline-errata/rfc3261.html. *
> 
> If we can figure out how to track "what's worth pointing to", perhaps the RFC editor could also inline pointers, rather than dorking with the RFC text itself. 
> 
> I am all for considering what's possible without revising RFC 7942, no problem, as long as we can redirect the interested parties to the appropriate code archives.
> 
> The work you cited that Adam has done on RFC 3261 for errata looks very neat.  This could be a potential way to proceed if IAB/IETF decided not to go the route of updating RFC 7942 or inserting some other text in the RFC-to-be.  Some issues to consider if we go the inlining route would be (a) how to aesthetically present the information on the RFC masthead, (b) what is the expectation when someone prints the RFC with such inling?  Will the inline text show up?  (In the case of RFC 3261 with inlined errata, I note that all expanded errata are printed, non expanded errata appear as green text in the generated PDF.  By "expanded" I mean that if I click on one of the errata, it expands into a rectangle at the appropriate place in the RFC, and that entire rectangle is then printed in the generated PDF.)
> 
> Thanks,
> 
> - vijay
> -- 
> Edm mailing list
> [email-masked]

---

From: Vijay Gurbani <[email-masked]>
Date: Thu, 10 Sep 2020 12:49:28 -0500

> Thanks for moving this discussion forward.

Dear Mirja: Thank you for your time and your comments.  More inline.

> To focus on the formal bits for a second: RFC7942 doesn’t actually require
> that the section is removed. This says "Authors are requested to” but I
> guess it’s for the authors/wg to decide. Also this is an IETF-consensus
> document which I think was AD-sponsored. So the way to update it is
> “simply” to write a bis-draft and bring it to gendisptach these days.
>

I agree that RFC 7942 only "requests" not "mandates" that the section be
removed.  However, the practical effect of this request is that most
authors will simply ask for the section to be removed.  That was the case
in a particular draft that I was reviewing for Gen-Art (which actually
resulted in the thread I opened up on the WG Chairs list).  Furthermore, to
be frank, before I reviwed the draft in question for Gen-ART, I was not
even aware of RFC 7942.  Certainly, it is fairly new (having been minted in
2016), but none of the I-Ds or RFCs I have read post-2016 even mention an
Implementation Status section.

It certainly does not portend well that we are not eating our own dog
food.  I looked at two active WGs --- quic and httpbis --- to see if an RFC
7942 section is present in any of the active I-Ds.  Of the 24 I-Ds in these
groups, I could not find such a section [1].  Apologies, I am not picking
on these groups, just trying to get some objective data points by looking
at a couple of active WGs.  The irony is that the base protocol document
for QUIC [2] has a "Note to Readers" section that provides the very
information --- where to find the GitHub repository --- that we are seeking
to institutionalise.  I am not sure what will happen to this note when QUIC
becomes an RFC.

> More to the actually point: I understand your motivation about people only
> knowing RFCs but none of the IETF process but I’m not sure putting this
> information in an RFC is the right tool. I bed it’s actually not very hard
> to find some suitable code on GitHub these days, especially as you said if
> this is about larger projects (which in the best case are actively
> maintained). For smaller code snippets we often put pseudo code in the
> document of appendix. So that’s another option.
>

I understand and respect your view that putting this information in an RFC
is not the right thing to do.  I am trying to convince you that it is, and
I hope that I succeed in some measure.

The proposal we are considering --- i.e., putting some text in RFCs to
guide legions of implementers not familiar with "IETF arcana" --- is
already being used loosely as I pointed out above for QUIC.  It seems to me
that we can do better by institutionalising it in some manner that RFC 7942
did not. Perhaps as you suggest, the answer is to simply write a RFC
7942-bis draft.  If you think this is an avenue we should explore, I am
happy to spin up a rfc7942-bis  so to provide more structure to the
conversation.

[1] I will like to point out that of the 24 I-Ds, not all deal with
specific protocol/algorithmic issues, so please account for that reality.
[2] https://www.ietf.org/id/draft-ietf-quic-transport-30.txt

Thanks,

- vijay

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 10 Sep 2020 20:33:07 +0200

Just very quickly some points. Having read all RFCs that we have published in the four years of my AD time I can tell you that RFC7942 is used form time to time. However, it also addresses a different problem then what you ask for, namely how do people in the wg know about existing implementations. QUIC decides to use another tool for this purpose; I guess for several reason, one being that many implementors in QUIC who have been new to the IETF are actually more familiar with GitHub than any IETF procedures. My expectation is that note you mentioned for QUIC will be removed before publication as RFC.

> On 10. Sep 2020, at 19:49, Vijay Gurbani <[email-masked]> wrote:
> 
> On Thu, Sep 10, 2020 at 11:40 AM Mirja Kuehlewind <[email-masked]> wrote:
> Hi Vijay,
> 
> I agree that RFC 7942 only "requests" not "mandates" that the section be removed.  However, the practical effect of this request is that most authors will simply ask for the section to be removed.  That was the case in a particular draft that I was reviewing for Gen-Art (which actually resulted in the thread I opened up on the WG Chairs list).  Furthermore, to be frank, before I reviwed the draft in question for Gen-ART, I was not even aware of RFC 7942.  Certainly, it is fairly new (having been minted in 2016), but none of the I-Ds or RFCs I have read post-2016 even mention an Implementation Status section.  
> 
> It certainly does not portend well that we are not eating our own dog food.  I looked at two active WGs --- quic and httpbis --- to see if an RFC 7942 section is present in any of the active I-Ds.  Of the 24 I-Ds in these groups, I could not find such a section [1].  Apologies, I am not picking on these groups, just trying to get some objective data points by looking at a couple of active WGs.  The irony is that the base protocol document for QUIC [2] has a "Note to Readers" section that provides the very information --- where to find the GitHub repository --- that we are seeking to institutionalise.  I am not sure what will happen to this note when QUIC becomes an RFC.
>  
> I understand and respect your view that putting this information in an RFC is not the right thing to do.  I am trying to convince you that it is, and I hope that I succeed in some measure.  
> 
> The proposal we are considering --- i.e., putting some text in RFCs to guide legions of implementers not familiar with "IETF arcana" --- is already being used loosely as I pointed out above for QUIC.  It seems to me that we can do better by institutionalising it in some manner that RFC 7942 did not. Perhaps as you suggest, the answer is to simply write a RFC 7942-bis draft.  If you think this is an avenue we should explore, I am happy to spin up a rfc7942-bis  so to provide more structure to the conversation.
> 
> [1] I will like to point out that of the 24 I-Ds, not all deal with specific protocol/algorithmic issues, so please account for that reality.

---

From: Vijay Gurbani <[email-masked]>
Date: Thu, 10 Sep 2020 13:49:30 -0500

Dear Mirja: Thanks for your time.  More inline.

On Thu, Sep 10, 2020 at 1:33 PM Mirja Kuehlewind <[email-masked]>
wrote:

> Just very quickly some points. Having read all RFCs that we have published
> in the four years of my AD time I can tell you that RFC7942 is used form
> time to time. However, it also addresses a different problem then what you
> ask for, namely how do people in the wg know about existing
> implementations.

OK, it is good to know that we are eating our own dog food, albeit in small
portions.  If RFC7942 is in the ballpark, but addresses a slightly
different problem, then perhaps we can see what can be salvaged by an
rfc7942-bis.  The problem of people in the WG knowing about existing
implementations is (I believe) easily solved by anyone participating in a
WG meeting (or list), most should quickly pick up where the most promising
implementations are.  The bigger problem is what about others that do not
participate in the WG deliberations, and more importantly, how to highlight
implementations that tracked the I-D to when it became an RFC, but all of a
sudden, in the post RFC world, there is no clear path for newcomers on how
to find the code.

QUIC decides to use another tool for this purpose; I guess for several
> reason, one being that many implementors in QUIC who have been new to the
> IETF are actually more familiar with GitHub than any IETF procedures.

Right, and the fact that QUIC put this GitHub URL in the I-D itself
underscores my point on why it is important to maintain such links to help
non-IETF implementers by providing them information in the RFC itself.
Especially after the I-D becomes an RFC.

> My expectation is that note you mentioned for QUIC will be removed before
> publication as RFC.
>

That would be a tremendous loss of information for the non-IETF development
community.  I am sure some will stumble on the QUIC GitHub, and I am
equally sure that many will not.  I am arguing to make such an informal
tie-in more explicit and institutionalized.

Thanks,

- vijay

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 11 Sep 2020 08:44:55 +1200

Hi,

I too think that RFC7942 is sadly under-used; it's very rare that I see it during
Gen-ART reviews and I would like to see more. I have used it myself in a few drafts.

However, I think it would be a serious mistake to include such information
in RFCs. RFCs are archival documents with an infinite lifetime. Do we really
want them to contain URLs for GitHub repositories of prototype implementations?
Or what amounts to advertising for specific products? Should RFC 2460 include
a pointer to the first commercial IPv6 implementation (IBM AIX 4.3)?

A dogfood wiki on the IETF site would be the best place for an ongoing (but
surely not permanent) record of implementations, IMHO.

Regards
   Brian Carpenter

On 11-Sep-20 06:49, Vijay Gurbani wrote:
> On Thu, Sep 10, 2020 at 1:33 PM Mirja Kuehlewind <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Just very quickly some points. Having read all RFCs that we have published in the four years of my AD time I can tell you that RFC7942 is used form time to time. However, it also addresses a different problem then what you ask for, namely how do people in the wg know about existing implementations. 
> 
> 
> OK, it is good to know that we are eating our own dog food, albeit in small portions.  If RFC7942 is in the ballpark, but addresses a slightly different problem, then perhaps we can see what can be salvaged by an rfc7942-bis.  The problem of people in the WG knowing about existing implementations is (I believe) easily solved by anyone participating in a WG meeting (or list), most should quickly pick up where the most promising implementations are.  The bigger problem is what about others that do not participate in the WG deliberations, and more importantly, how to highlight implementations that tracked the I-D to when it became an RFC, but all of a sudden, in the post RFC world, there is no clear path for newcomers on how to find the code.
> 
>     QUIC decides to use another tool for this purpose; I guess for several reason, one being that many implementors in QUIC who have been new to the IETF are actually more familiar with GitHub than any IETF procedures. 
> 
> 
> Right, and the fact that QUIC put this GitHub URL in the I-D itself underscores my point on why it is important to maintain such links to help non-IETF implementers by providing them information in the RFC itself.  Especially after the I-D becomes an RFC.
>  
> 
>     My expectation is that note you mentioned for QUIC will be removed before publication as RFC.
> 
> 
> That would be a tremendous loss of information for the non-IETF development community.  I am sure some will stumble on the QUIC GitHub, and I am equally sure that many will not.  I am arguing to make such an informal tie-in more explicit and institutionalized.
> 
> Thanks,
> 
> - vijay
>

---

From: Vijay Gurbani <[email-masked]>
Date: Thu, 10 Sep 2020 16:24:14 -0500

Dear Brian: Thank you for your comments!  Having co-authored one RFC with
you, I value your experience and judgment.

Please see inline.

On Thu, Sep 10, 2020 at 3:45 PM Brian E Carpenter <
[email-masked]> wrote:

> Hi,
>
> I too think that RFC7942 is sadly under-used; it's very rare that I see it
> during Gen-ART reviews and I would like to see more. I have used it myself
> in a few drafts.
>
> in RFCs. RFCs are archival documents with an infinite lifetime. Do we
> really want them to contain URLs for GitHub repositories of prototype
> implementations?
>

Indeed, RFCs are archival documents with an infinite lifetime, but RFCs are
also scientific documents, and as any scientific document, sometimes their
content is absolute (Shannon's "Mathematical Theory of Communications"
paper, for instance), and sometimes their content can be updated (the
hundreds of documents the scientific community is producing now on
Covid-19).  IEEE and ACM are encouraging scientific papers to contain links
to datasets and experiments that are hosted for archival by these
institutions.  Why can't we?

In the extreme case, if you read a peer-reviewed scientific publication in
a top quality journal and click on an expired link in that journal, the act
of not finding the contents at the end of the link likely should not
invalidate the entire research. I think that we are doing a disservice to
the future readers (and implementers) of RFCs by assuming that they will
not pay attention to the "caveat emptor" text associated with any GitHub
repositories, especially if these warnings are in a section that screams
that the contents of the GitHub repository were current as of the
publication date of this RFC and that no endorsement of the GitHub
repository is made by the IETF beyond the publication date of the RFC.  I
really see no harm, I see a lot of benefit.

Or what amounts to advertising for specific products? Should RFC 2460
> include a pointer to the first commercial IPv6 implementation (IBM AIX 4.3)?
>

These are good questions; they have answers that would require us to come
to consensus on.  I will certainly give you my answer to this and hope that
it changes your mind, or if not, at least incentivize you to explore the
issue from a different angle.  In my association with the IETF, I have come
to appreciate the communal way in which we develop our protocols.  This is
inspite of the fact that each one of us --- at least a majority --- work
for corporations that fund us to attend the meetings and conduct the work.
If because of this prevelant communal zeitgeist in the IETF, we don't have
a "Company-X version of Procol P", or a "Company-Y version of Protocol P",
why do we assume the community will tacitly endorse implementations that
amount to advertising for specific products?  Surely, we have come beyond
that, no?  We have guardrails in place (WG chairs, AD, IESG, perhaps ISOC
and IAB) to make sure that this does not happen.  So what are we reticient
about?

Regarding whether RFC 2460 should have included a pointer to the first
commercial implementation of IPv6 from IBM, well sure, with a constraint:
as long as IBM was willing to make the code available openly (as happens in
GitHubs now), and allows others to experiment with it and make it better, I
don't see the harm.  Of course, if IBM was not willing to make the code
available publicly, then this line of thinking is moot.

Thank you much!

Cheers,

- vijay