Subject: [rfc-i] Accessing RFCs in XML

From: Richard Barnes <[email-masked]>
Date: Thu, 10 Oct 2019 17:10:29 -0400

Hi all,

Over the past several years, it seems like many (most?) RFCs have been
submitted and AUTH48'ed in XML.  Certainly all of the ones I've authored
have been.  Yet, I can't seem to find the XML on rfc-editor.org.  Am I
missing something?

Thanks,
--Richard
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 14:18:43 -0700

It would be good to have the ones that were submitted by the author with
XML but even better would be the last XML from before the final nroff
stage. This would provide the best basis for doing a bis

-Ekr

---

From: Richard Barnes <[email-masked]>
Date: Thu, 10 Oct 2019 17:21:13 -0400

Yes, the XML I'm after here is the final one that was agreed between the
authors and the RPC.
--Richard

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Thu, 10 Oct 2019 21:52:54 +0000

From: rfc-interest <[email-masked]> on behalf of Richard Barnes <[email-masked]>
Date: Thursday, October 10, 2019 at 5:21 PM
To: Eric Rescorla <[email-masked]>
Cc: RFC Interest <[email-masked]>
Subject: Re: [rfc-i] Accessing RFCs in XML

Yes, the XML I'm after here is the final one that was agreed between the authors and the RPC.
--Richard

On Thu, Oct 10, 2019 at 5:19 PM Eric Rescorla <[email-masked]<mailto:[email-masked]>> wrote:
It would be good to have the ones that were submitted by the author with XML but even better would be the last XML from before the final nroff stage. This would provide the best basis for doing a bis

-Ekr

Over the past several years, it seems like many (most?) RFCs have been submitted and AUTH48'ed in XML.  Certainly all of the ones I've authored have been.  Yet, I can't seem to find the XML on rfc-editor.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__rfc-2Deditor.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=SZp9QRNzdbA-8qxjWXf4EDvn14CuShqiO2voCivGftg&s=AfJpyVCa1caq8IHA4a0YOetrlgP_90rFAL7PtunfxoM&e=>.  Am I missing something?

Thanks,
--Richard

There are various reasons that the RFC Editor doesn’t just put those out on the web server. But they DO have them saved and are quite happy to send you a copy if you ask.

                Tony

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 14:55:27 -0700

On Thu, Oct 10, 2019 at 2:53 PM HANSEN, TONY L <[email-masked]> wrote:

> *From: *rfc-interest <[email-masked]> on behalf of
> Richard Barnes <[email-masked]>
> *Date: *Thursday, October 10, 2019 at 5:21 PM
> *To: *Eric Rescorla <[email-masked]>
> *Cc: *RFC Interest <[email-masked]>
> *Subject: *Re: [rfc-i] Accessing RFCs in XML
>
>
>
> have been.  Yet, I can't seem to find the XML on rfc-editor.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__rfc-2Deditor.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=SZp9QRNzdbA-8qxjWXf4EDvn14CuShqiO2voCivGftg&s=AfJpyVCa1caq8IHA4a0YOetrlgP_90rFAL7PtunfxoM&e=>.
> Am I missing something?
>
>
>
> Thanks,
>
> --Richard
>
>
>
>
>
>
>
> There are various reasons that the RFC Editor doesn’t just put those out
> on the web server. But they DO have them saved and are quite happy to send
> you a copy if you ask.
>
>
What are those reasons?

-Ekr

---

From: Heather Flanagan <[email-masked]>
Date: Thu, 10 Oct 2019 15:39:02 -0700

> On Oct 10, 2019, at 2:55 PM, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
> On Thu, Oct 10, 2019 at 2:53 PM HANSEN, TONY L <[email-masked] <mailto:[email-masked]>> wrote:
> From: rfc-interest <[email-masked] <mailto:[email-masked]>> on behalf of Richard Barnes <[email-masked]>
> To: Eric Rescorla <[email-masked] <mailto:[email-masked]>>
> Cc: RFC Interest <[email-masked] <mailto:[email-masked]>>
> On Thu, Oct 10, 2019 at 5:19 PM Eric Rescorla <[email-masked] <mailto:[email-masked]>> wrote:
> 
> Over the past several years, it seems like many (most?) RFCs have been submitted and AUTH48'ed in XML.  Certainly all of the ones I've authored have been.  Yet, I can't seem to find the XML on rfc-editor.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__rfc-2Deditor.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=Kz8VdgPVctDNSNPJ6PsBaw&m=SZp9QRNzdbA-8qxjWXf4EDvn14CuShqiO2voCivGftg&s=AfJpyVCa1caq8IHA4a0YOetrlgP_90rFAL7PtunfxoM&e=>.  Am I missing something?
> 
>  
> 
> Thanks,
> 
> --Richard
> 
>  
> 
>  
> 
>  
> 

This is something that has been discussed periodically; the answer remains the same. The reasons are that in the pre-v3 universe, changes happen between the final XML and the publication of the RFC. Given the understanding that people do not read warning labels (or boilerplate, or most anything in the header), we make absolutely sure they understand that there may be discrepancies between the XML we give them and the final output by making this an explicit ask/response every time.

In the v3 universe, that is not necessary for the final XML will not contain any substantive content changes from what’s published.

-Heather

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 16:23:24 -0700

On Thu, Oct 10, 2019 at 3:39 PM Heather Flanagan <[email-masked]> wrote:

> This is something that has been discussed periodically; the answer remains
> the same. The reasons are that in the pre-v3 universe, changes happen
> between the final XML and the publication of the RFC. Given the
> understanding that people do not read warning labels (or boilerplate, or
> most anything in the header), we make absolutely sure they understand that
> there may be discrepancies between the XML we give them and the final
> output by making this an explicit ask/response every time.
>

This seems oddly paternalistic. In the v2 world, there's no sense in which
the XML is the same as the final RFC, and textual changes aside, you can't
even produce the final RFC from it because the pagination is potentially
different. Is there any evidence that this has ever produced significant
confusion?

-Ekr

---

From: "Paul Hoffman" <[email-masked]>
Date: Thu, 10 Oct 2019 16:46:11 -0700

On 10 Oct 2019, at 16:23, Eric Rescorla wrote:

> Is there any evidence that this has ever produced significant
> confusion?

I have first-hand experience with authors of a -bis document who were 
even warned. Not only did they forget that, but they complained bitterly 
when there was a late surprise due to them not having started off doing 
a diff between the XML and the text in the RFC. It was fixable, of 
course, but I understand the RPC's hesitance to just put them up 
willy-nilly.

--Paul Hoffman

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 16:59:57 -0700

> course, but I understand the RPC's hesitance to just put them up

Of course, one might argue that the source of this problem is not going and
retrospectively correcting the XML.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 17:05:50 -0700

On Thu, Oct 10, 2019 at 4:59 PM Eric Rescorla <[email-masked]> wrote:

> Of course, one might argue that the source of this problem is not going
> and retrospectively correcting the XML.
>

And, responding to myself...

The fact that you say this confusion happened under the current system and
after someone was warned seems to undercut the argument that somehow
handing these out retail rather than wholesale is a fix, other than that it
generally makes it harder to get the XML at all.

-Ekr

---

From: heather flanagan <[email-masked]>
Date: Thu, 10 Oct 2019 17:39:45 -0700

On Oct 10, 2019, at 5:05 PM, Eric Rescorla <[email-masked]> wrote:

Of course, one might argue that the source of this problem is not going and retrospectively correcting the XML.

The fact that you say this confusion happened under the current system and after someone was warned seems to undercut the argument that somehow handing these out retail rather than wholesale is a fix, other than that it generally makes it harder to get the XML at all.

And at this point, we’re just dueling anecdotes. I do not have stats on how many times confusion has been avoided by our attempts to make sure the bis authors are aware of the potential differences between the pre-v3 XML and the final RFC. Stats on a negative are notoriously difficult to capture.

I can, however, point to research on whether or not people read warning labels, fine print, etc. (No, mostly they don’t.) I can also point to people noting on IETF lists that they never read things like the boilerplate or other front matter.

So, I’m willing to go out on a limb and say “people aren’t likely to read any warnings I put up there, and it is quite possible they will make potentially serious mistakes in their docs if they don’t pay attention, so let’s leave this rather small but important hoop in place to help everyone avoid issues.” Will it work 100% of the time? Demonstrably not. But I think it will help enough to make it worth while.

-Heather

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 11 Oct 2019 11:46:01 +1100

> So, I’m willing to go out on a limb and say “people aren’t likely to read any warnings I put up there, and it is quite possible they will make potentially serious mistakes in their docs if they don…

To add a bit more anecdata...

As a doc author who has done more than one bis, it's significantly easier to start with an existing document than to hand-port the .txt back to XML (or, more likely, Markdown). So, I usually start with the last known source file I have, hand-porting changes from the RPC where I can (big shout-out to Julian here, who often provides this as a service).

Do I get it 100% right? Of course not. However, it's pragmatic, and hopefully, if the RPC catches an error the first time around, they'll catch it the second time too.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 10 Oct 2019 17:49:36 -0700

On Thu, Oct 10, 2019 at 5:39 PM heather flanagan <[email-masked]>
wrote:

> And at this point, we’re just dueling anecdotes.I do not have stats on how
> many times confusion has been avoided by our attempts to make sure the bis
> authors are aware of the potential differences between the pre-v3 XML and
> the final RFC. Stats on a negative are notoriously difficult to capture.
>

> I can, however, point to research on whether or not people read warning
> labels, fine print, etc. (No, mostly they don’t.) I can also point to
> people noting on IETF lists that they never read things like the
> boilerplate or other front matter.
>
> So, I’m willing to go out on a limb and say “people aren’t likely to read
> any warnings I put up there, and it is quite possible they will make
> potentially serious mistakes in their docs if they don’t pay attention, so
> let’s leave this rather small but important hoop in place to help everyone
> avoid issues.”
>

As I said, this seems oddly paternalistic.

We're an open organization and absent some strong data to the contrary, we
should make things available, not make people jump through hoops to get
them.

-Ekr

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 10 Oct 2019 21:35:53 -0400

Why are we berating this?  We are in the process of switching to XMLv3, 
where the XML will be the definitive form and will be avaialble.  So 
there is a plan being executed that will make the XML available going 
forward.

Yours,
Joel

On 10/10/2019 8:49 PM, Eric Rescorla wrote:
> 
> 
> On Thu, Oct 10, 2019 at 5:39 PM heather flanagan <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
> 
> 
>>     On Oct 10, 2019, at 5:05 PM, Eric Rescorla <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>
>>
>>     On Thu, Oct 10, 2019 at 4:59 PM Eric Rescorla <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>
>>
>>
>>         On Thu, Oct 10, 2019 at 4:45 PM Paul Hoffman
>>         <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>         Of course, one might argue that the source of this problem is
>>         not going and retrospectively correcting the XML.
>>
>>
>>     The fact that you say this confusion happened under the current
>>     system and after someone was warned seems to undercut the argument
>>     that somehow handing these out retail rather than wholesale is a
>>     fix, other than that it generally makes it harder to get the XML
>>     at all.
>>
> 
>     And at this point, we’re just dueling anecdotes.I do not have stats
>     on how many times confusion has been avoided by our attempts to make
>     sure the bis authors are aware of the potential differences between
>     the pre-v3 XML and the final RFC. Stats on a negative are
>     notoriously difficult to capture.
> 
> 
>     I can, however, point to research on whether or not people read
>     warning labels, fine print, etc. (No, mostly they don’t.) I can also
>     point to people noting on IETF lists that they never read things
>     like the boilerplate or other front matter.
> 
>     So, I’m willing to go out on a limb and say “people aren’t likely to
>     read any warnings I put up there, and it is quite possible they will
>     make potentially serious mistakes in their docs if they don’t pay
>     attention, so let’s leave this rather small but important hoop in
>     place to help everyone avoid issues.” 
> 
> 
> We're an open organization and absent some strong data to the contrary, 
> we should make things available, not make people jump through hoops to 
> get them.
> 
> -Ekr
>

---

From: Richard Barnes <[email-masked]>
Date: Thu, 10 Oct 2019 22:03:20 -0400

> output by making this an explicit ask/response every time.

Well, now I'm confused and concerned.  As an author, I approve the final
XML.  If there are substantive differences between what I approved and what
is published, that seems like a serious defect in the publication process.
And if there aren't, then publishing the XML shouldn't lead to any
confusion.

--Richard

>
> In the v3 universe, that is not necessary for the final XML will not
> contain any substantive content changes from what’s published.
>
> -Heather
>

---

From: Richard Barnes <[email-masked]>
Date: Thu, 10 Oct 2019 22:04:21 -0400

I know the future is bigger than the past and all, but as Mark pointed out,
some of us still care about old RFCs 🤷‍♂️
--RLB

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Oct 2019 06:22:39 +0200

On 11.10.2019 04:03, Richard Barnes wrote:
> ...
> XML.  If there are substantive differences between what I approved and
> what is published, that seems like a serious defect in the publication
> process.  And if there aren't, then publishing the XML shouldn't lead to
> any confusion.
> ...

FWIW, if there are more than minimal changes, I would absolutely prefer
that this was actually published as a new Internet Draft before going to
RFC.

Yes, edits in AUTH48 are well-intended, but sometimes they *are* wrong,
and more eyes on the changes could help avoiding that.

Best regards, Julian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 11 Oct 2019 00:41:00 -0400

Julian, are you really asking that Auth48, after it completes, gets an 
additional WG-2-week so WG members can decide if the changes were 
significant and correct?
People are complaining about publication taking too long as it is.
We do have a process that copies the AD and the shepherd on Auth48.
And folks do look for significant changes and mostly try to prevent 
them.  (I have seen authors ask for significant changes during Auth48, 
and have them told "no, that is not our remit".)

Sure, mistakes can get made.  Which is wy there are extra eyes on the 
discussion.

Yours,
Joel

PS: And again, once we move to v3, the XML will be the definitive 
document.  So reviewing the final XML will give full assurance.  (I 
believe it is expected that many folks will choose to review view one of 
the output formats.  But that is the Authors' choices.)

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Oct 2019 06:48:36 +0200

> additional WG-2-week so WG members can decide if the changes were

If the changes were significant? Yes. Not necessarily a new LC, but at
least a chance to review what changed.

And *optimally*, these changes would also be applied in whatever source
control system the WG uses.

Best regards, Julian

---

From: Michael Richardson <[email-masked]>
Date: Fri, 11 Oct 2019 11:52:54 +0200

> least a chance to review what changed.

Area Directors have to approve major changes, and sometimes they *DO* send
the changes back to the WG.  But, it's an exception path, not a regular one.

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 11 Oct 2019 06:20:02 -0700

Indeed, and I expect that the chance that people will have to do -bises of
existing RFCs is quite high.

On Thu, Oct 10, 2019 at 7:04 PM Richard Barnes <[email-masked]> wrote:

> I know the future is bigger than the past and all, but as Mark pointed
> out, some of us still care about old RFCs 🤷‍♂️
> --RLB
>

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 13:27:29 +0000

> Indeed, and I expect that the chance that people will have to do -bises of existing RFCs is quite high.

Yes, IASA2 had to do much hand-editing to turn published RFC's into markdown.

> On Thu, Oct 10, 2019 at 9:36 PM Joel M. Halpern <mailto:[email-masked]> wrote:

Berated not debated. Ha!  Great auto-correct  Berated.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 11 Oct 2019 09:59:39 -0400

1) This is a policy that has been in effect for MANY years
2) since we are moving to a new XML format, and new XML will be 
available, it follows that this problem is going to decrease by itself.

Hence changing the policy NOW to create mechanisms to post all the old 
XML seems counterproductive when we need the RPC working to get the new 
format running smoothly.  And when the problem you are raising will 
solve itself over time.

Yours,
Joel

On 10/11/2019 9:20 AM, Eric Rescorla wrote:
>     I know the future is bigger than the past and all, but as Mark
>     pointed out, some of us still care about old RFCs 🤷‍♂️
>     --RLB
> 
>     On Thu, Oct 10, 2019 at 9:36 PM Joel M. Halpern <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>          > On Thu, Oct 10, 2019 at 5:39 PM heather flanagan
>         <[email-masked] <mailto:[email-masked]>
>          > <mailto:[email-masked] <mailto:[email-masked]>>>
>         wrote:
>          >
>          >
>          >
>         <mailto:[email-masked]>
>          >>     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>          >>
>          >>
>          >>
>          >>     On Thu, Oct 10, 2019 at 4:59 PM Eric Rescorla
>         <[email-masked] <mailto:[email-masked]>
>          >>     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>          >>
>          >>
>          >>
>          >>         <[email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>> wrote:
>          >>
>          >>             > Is there any evidence that this has ever produced
>          >>             significant
>          >>             > confusion?
>          >>
>          >>             when there was a late surprise due to them not
>         having
>          >>             started off doing
>          >>             a diff between the XML and the text in the RFC.
>         It was
>          >>             fixable, of
>          >>             course, but I understand the RPC's hesitance to
>         just put
>          >>             them up
>          >>             willy-nilly.
>          >>
>          >>
>          > [email-masked] <mailto:[email-masked]>
>         [email-masked] <mailto:[email-masked]>

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 11 Oct 2019 07:04:19 -0700

> 1) This is a policy that has been in effect for MANY years

Yes, and?

> available, it follows that this problem is going to decrease by itself.

At some incredibly slow rate; we're going to be creating -bis documents for
documents created over the past 10 years for quite some time.

> format running smoothly.

Mechanism? I'm talking about taking the current directory of those XML
files and putting it on a Web site. As you observe, no more of these
documents are being created, so this hardly seems onerous.

-Ekr

> And when the problem you are raising will
> solve itself over time.
>
> Yours,
> Joel
>
> >          >>             > Is there any evidence that this has ever
> produced
> >          >>             significant
> >          >>             > confusion?
> >          >>
> >          >>             I have first-hand experience with authors of a
> -bis
> >          >>             document who were
> >          >>             even warned. Not only did they forget that, but
> they
> >          >>             complained bitterly

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 07:10:16 -0700

> On Oct 10, 2019, at 7:03 PM, Richard Barnes <[email-masked]> wrote:
> ...
> 
> Well, now I'm confused and concerned.  As an author, I approve the final XML.  If there are substantive differences between what I approved and what is published, that seems like a serious defect in the publication process.  And if there aren't, then publishing the XML shouldn't lead to any confusion.
> 

As an author pre-v3, you approved the final content, not the final XML. In v3, you are approving both the final content and the final XML. 

-Heather

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 11 Oct 2019 10:13:17 -0400

If you mean "when the AD, authors, or shepherd determine that the 
changes are too large, it should go to the WG" that is already the 
practice (as Michael said separately.)
If you mean "The WG should get to check all of the changes to see if 
they are too large" that seems to create significantly more delay for 
very little benefit.

Yours,
Joel

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 10:14:45 -0400

On Fri, Oct 11, 2019 at 10:10 Heather Flanagan <[email-masked]> wrote:

> As an author pre-v3, you approved the final content, not the final XML. In
> v3, you are approving both the final content and the final XML.
>

Ok, so it sounds like we’re in the latter case.  So can we just have
someone put the final XML files in the directory with the TXT and PDF?

—Richard

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 10:18:50 -0400

On Fri, Oct 11, 2019 at 10:14 Richard Barnes <[email-masked]> wrote:

>> As an author pre-v3, you approved the final content, not the final XML.
>> In v3, you are approving both the final content and the final XML.
>>
>

Specifically, what I’m asking for is a one time upload of the v2 XML for
existing RFCs.  Since as Joe notes, new stuff is already going to be
published in XML.   This should be like a 5min operation.

—Richard

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 07:23:44 -0700

> On Oct 11, 2019, at 7:18 AM, Richard Barnes <[email-masked]> wrote:
> 
> 
> 
> On Fri, Oct 11, 2019 at 10:14 Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
> On Fri, Oct 11, 2019 at 10:10 Heather Flanagan <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Oct 10, 2019, at 7:03 PM, Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
>> ...
>> 
> Ok, so it sounds like we’re in the latter case.  So can we just have someone put the final XML files in the directory with the TXT and PDF?
> 
> Specifically, what I’m asking for is a one time upload of the v2 XML for existing RFCs.  Since as Joe notes, new stuff is already going to be published in XML.   This should be like a 5min operation. 

The original reason we haven’t posted pre-v3 XML still stands. It’s not a question of how much time it will take to do it (though now is a really bad time to ask the RPC to be doing *anything* other than getting through the implementation of v3) but whether it’s a good idea at all. 

-Heather

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Oct 2019 18:10:10 +0200

On 11.10.2019 00:39, Heather Flanagan wrote:
> ...
>> What are those reasons?
>
> This is something that has been discussed periodically; the answer
> remains the same. The reasons are that in the pre-v3 universe, changes
> happen between the final XML and the publication of the RFC. Given the
> most anything in the header), we make absolutely sure they understand
> that there may be discrepancies between the XML we give them and the
> final output by making this an explicit ask/response every time.
> ...

FWIW, there are other good reasons to make the XML public. For instance,
to help analyzing the use of the vocabulary, helping to make decisions
about the V3 format.

Best regards, Julian

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 12:27:22 -0400

On Fri, Oct 11, 2019 at 10:23 AM Heather Flanagan <[email-masked]>
wrote:

> The original reason we haven’t posted pre-v3 XML still stands. It’s not a
> question of how much time it will take to do it (though now is a really bad
> time to ask the RPC to be doing *anything* other than getting through the
> implementation of v3) but whether it’s a good idea at all.
>

You just told me that there are no substantive changes between the
author-approved XML and final XML, and if there's a substantive difference
between the final XML and final TXT, that again seems like a serious defect
in the publication process.  So to the degree the earlier policy was based
on risk of confusion, I'm not understanding how that confusion arises.

--Richard

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 09:33:41 -0700

> On Oct 11, 2019, at 9:27 AM, Richard Barnes <[email-masked]> wrote:
> 
> 
> 
> On Fri, Oct 11, 2019 at 10:23 AM Heather Flanagan <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Oct 11, 2019, at 7:18 AM, Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> 
>> 
> You just told me that there are no substantive changes between the author-approved XML and final XML, and if there's a substantive difference between the final XML and final TXT, that again seems like a serious defect in the publication process.  So to the degree the earlier policy was based on risk of confusion, I'm not understanding how that confusion arises.

When I’m talking about no substantive changes between the author-approved XML and the final XML, I’m talking about v3 and only v3. And the v3 XML files are being made available.

Where we get into differences between the XML and the author-approved TXT, we’re talking about v2, where changes may have been made during AUTH48 that ended up in the nroff and not the XML. V2 XML files are available upon request, if we have them. Not all authors submitted XML.

-Heather

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 12:44:23 -0400

On Fri, Oct 11, 2019 at 12:33 PM Heather Flanagan <[email-masked]>
wrote:

>>>> Well, now I'm confused and concerned.  As an author, I approve the
>>>> final XML.  If there are substantive differences between what I approved
>>>> and what is published, that seems like a serious defect in the publication
> When I’m talking about no substantive changes between the author-approved
> XML and the final XML, I’m talking about v3 and only v3. And the v3 XML
> files are being made available.
>

You said, "As an author pre-v3, you approved the final content, not the
final XML".  But concretely, the artifact that the RPC and I agreed on was
an XML file.  I presume this was similar with other authors doing AUTH48 in
XML.

> Where we get into differences between the XML and the author-approved TXT,
> we’re talking about v2, where changes may have been made during AUTH48 that
> ended up in the nroff and not the XML. V2 XML files are available upon
> request, if we have them. Not all authors submitted XML.
>

This is a distinction without a difference.  If they're available on
request, they should be available on HTTP request, i.e., on a web page.
There is nothing gained by making me send email.

--Richard

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 12:47:15 -0400

On Fri, Oct 11, 2019 at 12:44 PM Richard Barnes <[email-masked]> wrote:

>>>>> As an author pre-v3, you approved the final content, not the final
>>>>> XML. In v3, you are approving both the final content and the final XML.
>>>>>
>>>>
>>> The original reason we haven’t posted pre-v3 XML still stands. It’s not
>>> a question of how much time it will take to do it (though now is a really
>>> bad time to ask the RPC to be doing *anything* other than getting through
>>> the implementation of v3) but whether it’s a good idea at all.
>>>
>>
>> Where we get into differences between the XML and the author-approved
>> TXT, we’re talking about v2, where changes may have been made during AUTH48
>> that ended up in the nroff and not the XML. V2 XML files are available upon

Alternatively, if this is more effective, please consider this email a
request, submitted via email, for all final XMLv2 files that the RFC editor
has.  Feel free to respond 1:1.

--Richard

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 09:51:12 -0700

> On Oct 11, 2019, at 9:44 AM, Richard Barnes <[email-masked]> wrote:
> 
> 
> 
> On Fri, Oct 11, 2019 at 12:33 PM Heather Flanagan <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Oct 11, 2019, at 9:27 AM, Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> 
>> 
> You said, "As an author pre-v3, you approved the final content, not the final XML".  But concretely, the artifact that the RPC and I agreed on was an XML file.  I presume this was similar with other authors doing AUTH48 in XML.

Not necessarily, no. It varies from one set of authors to the next. 

> This is a distinction without a difference.  If they're available on request, they should be available on HTTP request, i.e., on a web page.  There is nothing gained by making me send email.

Except one final, explicit attempt to make sure that authors know that what they’re getting might not reflect the final content of the RFC. 

I can see a world where this is slightly more automated, in that the person doing the request needs to click on some kind of acknowledgement, but I suspect that will be equally annoying to those who just want the files. I also think it’s easier to ignore a click through than it is to ignore a human sending you email. Still, worth discussing after the RPC is through the hurdle of v3 implementation.

-Heather

---

From: Ted Lemon <[email-masked]>
Date: Fri, 11 Oct 2019 14:12:58 -0400

I think it is unfortunate that we can’t get the last xml file. I think it would be a mistake to label that xml file the “xml file for rfcXXXX.”  However, it is the xml file for some previous version. Making it available as that seems to solve EKR’s problem. There is a risk of error; when filing a bis document, the first version should have no diffs with the RFC. 

As you say, asking the RFC editor to solve this problem right now seems like a really bad idea.   So while in the abstract I think there is a low cost path to something useful, I am not suggesting that it can be done today. 

> On Oct 11, 2019, at 12:51, Heather Flanagan <[email-masked]> wrote:
> 
> 
> 
>>> On Fri, Oct 11, 2019 at 12:33 PM Heather Flanagan <[email-masked]> wrote:
>>> 
>>> 
>>>> On Fri, Oct 11, 2019 at 10:23 AM Heather Flanagan <[email-masked]> wrote:
>>>>> 
>>>>>

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 14:40:49 -0400

On Fri, Oct 11, 2019 at 12:47 PM Richard Barnes <[email-masked]> wrote:

>>>> Specifically, what I’m asking for is a one time upload of the v2 XML
>>>> for existing RFCs.  Since as Joe notes, new stuff is already going to be
>>> When I’m talking about no substantive changes between the
>>> author-approved XML and the final XML, I’m talking about v3 and only v3.
>>> And the v3 XML files are being made available.
>>>
>>

Heather: Could you please provide an ETA for a response to this request?

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Fri, 11 Oct 2019 18:41:40 +0000

From: rfc-interest <[email-masked]> on behalf of Ted Lemon <[email-masked]>
Date: Friday, October 11, 2019 at 2:13 PM
Subject: Re: [rfc-i] Accessing RFCs in XML

Nothing is preventing you from getting the XML. It’s just that the RFC Editor has chosen to require you to get it through an email request rather than putting it up on a web site.

I’ve never had a problem getting one from them, when they existed.

                Tony

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 12:11:16 -0700

> On Oct 11, 2019, at 11:40 AM, Richard Barnes <[email-masked]> wrote:
> 

An exact one? No, I can’t. I have asked the RPC to prioritize v3 work over everything else at this point. I will be raising your request with the RSOC (since it has implications) when I meet with them later this month.

-Heather

---

From: Richard Barnes <[email-masked]>
Date: Fri, 11 Oct 2019 15:13:50 -0400

On Fri, Oct 11, 2019 at 3:11 PM Heather Flanagan <[email-masked]> wrote:

> An exact one? No, I can’t. I have asked the RPC to prioritize v3 work over
> everything else at this point. I will be raising your request with the RSOC
> (since it has implications) when I meet with them later this month.
>

The E in ETA is "Estimated" :)  And I thought you said these files were
available by email.

--Richard

---

From: Heather Flanagan <[email-masked]>
Date: Fri, 11 Oct 2019 12:17:42 -0700

> On Oct 11, 2019, at 12:13 PM, Richard Barnes <[email-masked]> wrote:
> 
> 
> 
> On Fri, Oct 11, 2019 at 3:11 PM Heather Flanagan <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Oct 11, 2019, at 11:40 AM, Richard Barnes <[email-masked] <mailto:[email-masked]>> wrote:
>> 
> The E in ETA is "Estimated" :)  And I thought you said these files were available by email.

They are. One by one. Not in a full tar ball of everything, which is I believe what you asked for.

If you need a specific one, drop a note to the [email-masked], and we’ll get it to you. If you want all of them ever turned in, you will have to wait until we have time to sort it out. And no, I don’t have an estimated time for that. 

-Heather

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 11 Oct 2019 12:50:54 -0700

On Fri, Oct 11, 2019 at 12:17 PM Heather Flanagan <[email-masked]>
wrote:

>> An exact one? No, I can’t. I have asked the RPC to prioritize v3 work
>> over everything else at this point. I will be raising your request with the
>> RSOC (since it has implications) when I meet with them later this month.
>>
>
> If you need a specific one, drop a note to the [email-masked],
> and we’ll get it to you. If you want all of them ever turned in, you will
> have to wait until we have time to sort it out. And no, I don’t have an
> estimated time for that.
>

Can you explain what it is that needs to be sorted out? It seems like basic
data retention practices would have them all on a disk somewhere. Is there
some technical difficulty with quickly assembling them into a tarball?

-Ekr

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 11 Oct 2019 19:52:56 +0000

*   The E in ETA is "Estimated" :)  And I thought you said these files were available by email.

Available as in, requests for copies will be processed individually and emailed.

I suggest you back down a bit?

---

From: Jeffrey Yasskin <[email-masked]>
Date: Fri, 11 Oct 2019 12:55:12 -0700

On Fri, Oct 11, 2019 at 12:51 PM Eric Rescorla <[email-masked]> wrote:

> Can you explain what it is that needs to be sorted out? It seems like
> basic data retention practices would have them all on a disk somewhere. Is
> there some technical difficulty with quickly assembling them into a tarball?
>

What is urgent about this? It seems like a good thing to do eventually
(although the fact that the expert on the thread thinks it's a bad idea
gives me pause), but it doesn't seem worth distracting any of the folks
making the v3 transition happen.

Jeffrey

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 11 Oct 2019 13:01:26 -0700

> making the v3 transition happen.

Well, the fact that it's said to be distracting is itself concerning. This
seems like a straightforward request that, assuming reasonable data
retention, should be straightforward to quickly fulfill. I'm trying to
determine why that's not the case. If the answer is that there's some
technical obstacle, then perhaps we can get past that, and I'd be certainly
happy to be told that that couldn't be debugged for some reasonable time.
However, if the answer is that the RSE just doesn't think that members of
the community should be allowed to have copies of all the XML files, than
that's something else entirely.

-Ekr

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 11 Oct 2019 22:07:28 +0200

> On 11 Oct 2019, at 21:52, Salz, Rich <[email-masked]> wrote:
> 

+1

    Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 11 Oct 2019 22:12:36 +0200

> On 11 Oct 2019, at 22:01, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
>> On Fri, Oct 11, 2019 at 12:55 PM Jeffrey Yasskin <[email-masked]> wrote:
>>>> On Fri, Oct 11, 2019 at 12:17 PM Heather Flanagan <[email-masked]> wrote:
>>>> 
>>>> 
>>> Can you explain what it is that needs to be sorted out? It seems like basic data retention practices would have them all on a disk somewhere. Is there some technical difficulty with quickly assembling them into a tarball?
>>  
>> What is urgent about this? It seems like a good thing to do eventually (although the fact that the expert on the thread thinks it's a bad idea gives me pause), but it doesn't seem worth distracting any of the folks making the v3 transition happen.
> 
> Well, the fact that it's said to be distracting is itself concerning. This seems like a straightforward request that, assuming reasonable data retention, should be straightforward to quickly fulfill. I'm trying to determine why that's not the case. If the answer is that there's some technical obstacle, then perhaps we can get past that, and I'd be certainly happy to be told that that couldn't be debugged for some reasonable time. However, if the answer is that the RSE just doesn't think that members of the community should be allowed to have copies of all the XML files, than that's something else entirely.

I think the RFC Editor's responses on this has been perfectly clear, and ignoring them and throwing out fictitious reasons to refute is unhelpful.

    Henrik

---

From: "Bron Gondwana" <[email-masked]>
Date: Sat, 12 Oct 2019 07:16:44 +1100

On Fri, Oct 11, 2019, at 14:41, HANSEN, TONY L wrote:
> *From: *rfc-interest <[email-masked]> on behalf of Ted Lemon <[email-masked]>
> *Date: *Friday, October 11, 2019 at 2:13 PM

> I think it is unfortunate that we can’t get the last xml file. I think it would be a mistake to label that xml file the “xml file for rfcXXXX.” However, it *is* the xml file for some previous version. Making it available as that seems to solve EKR’s problem. There is a risk of error; when filing a bis document, the first version should have no diffs with the RFC. 

> As you say, asking the RFC editor to solve this problem right now seems like a really bad idea. So while in the abstract I think there is a low cost path to something useful, I am not suggesting th…

First of all - I totally agree that this is a bad time!

Secondly, thinking back over my own experience. Before I joined the IETF and as a new person, I found the process of creating a new draft felt very opaque, and the fact that I couldn't see the XML for documents that I already knew meant that I couldn't see if I was "doing it right". There were some example XML documents out there, but they were all either really old or missing the bits that I might want to duplicate in my own work.

It took years before I was comfortable that I had a working process - which involves using mmark and building XML documents from that. And I stole that process from somebody else, not from primary sources.

I'm very glad that the v3 workflow will lead to XML documents being a primary source, and hopefully more friendly to potential new authors trying to work out how to even get started.

> I’ve never had a problem getting one from them, when they existed.

This is super exclusionary. Most people don't ask, and don't feel that asking is an option.

When we talk about barriers to entry and turning people away, forcing people to ask to see the "source code" for how we work, in the preferred format for submission, is a definitely not welcoming. Being able to look at an RFC that you already know, and then see the source code that generated that RFC, would de-mystify our process to those looking from the outside.

"It's easy for me so it's not a problem" is an unfortunate response. 

Regards,

Bron.

--
 Bron Gondwana, CEO, Fastmail Pty Ltd
 [email-masked]

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 11 Oct 2019 16:20:23 -0400

Ekr,

> that's something else entirely.

 Heather already gave a number of good reasons why this would be a
distraction for the RPC at this point. We're all in agreement that the v2
publication process wasn't ideal. However, one of the goals of the v3
process was to fix the v2 process, and that it has. With v3, final XML that
represents the final RFC content will be available online for every RFC.

Is there some specific reason why you feel you need all of the v2 XML files
immediately?

Cheers,
Andy

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 11 Oct 2019 16:25:10 -0400

Bron,

I totally agree that it's much easier to program by example than starting
from scratch. In this case, however, since you're writing an Internet-Draft
rather than an RFC (and there are boilerplate and content differences
between the two), it's much better to use another I-D for example XML
rather than an RFC, and there's many I-D XML files available for download
by anyone in the datatracker.

Cheers,
Andy

On Fri, Oct 11, 2019 at 4:17 PM Bron Gondwana <[email-masked]>
wrote:

> Ted Lemon <[email-masked]>
> I think it is unfortunate that we can’t get the last xml file. I think it
> would be a mistake to label that xml file the “xml file for rfcXXXX.”
>  However, it *is* the xml file for some previous version. Making it
> available as that seems to solve EKR’s problem. There is a risk of error;
> when filing a bis document, the first version should have no diffs with the
> RFC.
>
>
>
>
> As you say, asking the RFC editor to solve this problem right now seems
> like a really bad idea.   So while in the abstract I think there is a low
> cost path to something useful, I am not suggesting that it can be done
> today.
>
>
> Secondly, thinking back over my own experience.  Before I joined the IETF
> and as a new person, I found the process of creating a new draft felt very
> opaque, and the fact that I couldn't see the XML for documents that I
> already knew meant that I couldn't see if I was "doing it right".  There
> were some example XML documents out there, but they were all either really
> old or missing the bits that I might want to duplicate in my own work.
>
> It took years before I was comfortable that I had a working process -
> which involves using mmark and building XML documents from that.  And I
> stole that process from somebody else, not from primary sources.
>
> I'm very glad that the v3 workflow will lead to XML documents being a
> primary source, and hopefully more friendly to potential new authors trying
> to work out how to even get started.
>
> Nothing is preventing you from getting the XML. It’s just that the RFC
> Editor has chosen to require you to get it through an email request rather
> than putting it up on a web site.
>
> When we talk about barriers to entry and turning people away, forcing
> people to ask to see the "source code" for how we work, in the preferred
> format for submission, is a definitely not welcoming.  Being able to look
> at an RFC that you already know, and then see the source code that
> generated that RFC, would de-mystify our process to those looking from the
> outside.
>

---

From: Christian Huitema <[email-masked]>
Date: Fri, 11 Oct 2019 13:28:40 -0700

On 10/11/2019 1:12 PM, Henrik Levkowetz wrote:
> ...
> I think the RFC Editor's responses on this has been perfectly clear,
> and ignoring them and throwing out fictitious reasons to refute is
> unhelpful.

Looking at this thread, I see the following:

1) When writing draft-rfcXXXX-bis-00, having the XML source of RFCXXXX
is quite helpful.

2) For many or most old RFC, the XML source does not actually generate
the final RFC text. For a fair proportion of old RFC, there is actually
no XML source.

3) If the XML source was published with the same visibility as TXT or
PDF, there is a risk that some people would use it instead of the
approved versions, which because of (2) above will create confusion on
the actual text of the RFC.

4) With V3, the final XML source does reflect the final RFC text.

Given that, it seems that the RFC Editor practice of providing the XML
text only upon demand is reasonable. If this is only an issue of writing
draft-RFC-bis, the overhead of email processing does not appear too large.

Do we have other use cases for the XML source, besides "writing
draft-rfcXXXX-bis-00" ?

-- Christian Huitema

---

From: "Bron Gondwana" <[email-masked]>
Date: Fri, 11 Oct 2019 16:35:05 -0400

Yeah, for sure if I'd already known all the terminology and where to look I would have been set. The probability that I had any clue what the difference between an internet draft and an RFC was back then is pretty low...

It's not that the information wasn't available, it's that there was a black box between the end product I was using and the way to go about creating one of those myself, and I certainly didn't discover my way through that easily.

Now that I've shepherded a few documents through the whole process and written s couple of drafts myself it's no longer scary, but I'm trying to remember my initial onboarding experience, and having to ask for documents would have been an insurmountable obstacle to discovery.

Cheers,

Bron.

On Fri, Oct 11, 2019, at 16:25, Andrew G. Malis wrote:
> Bron,
> 
> I totally agree that it's much easier to program by example than starting from scratch. In this case, however, since you're writing an Internet-Draft rather than an RFC (and there are boilerplate and content differences between the two), it's much better to use another I-D for example XML rather than an RFC, and there's many I-D XML files available for download by anyone in the datatracker.
> 
> Cheers,
> Andy
> 
> 
> On Fri, Oct 11, 2019 at 4:17 PM Bron Gondwana <[email-masked]> wrote:
>> __
>> 
>>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Fri, 11 Oct 2019 13:38:27 -0700

On Fri, Oct 11, 2019 at 1:17 PM Bron Gondwana <[email-masked]>
wrote:

> ...  Being able to look at an RFC that you already know, and then see the
> source code that generated that RFC, would de-mystify our process to those
> looking from the outside. ...
>

This is something of a tangent, but I'd point out that the XML isn't the
source code for a lot of RFCs and I-Ds. It's an intermediate format that a
couple different tools can generate. To help folks find the actual source
code, I've been copying Carsten Bormann's practice of including a "Note to
Readers" or "Contributing" section that links to the source repository. For
example, see
https://cbor-wg.github.io/CBORbis/draft-ietf-cbor-7049bis.html#rfc.abstract.
This isn't perfect and won't persist into the final RFC, but it's a step to
make things easier to navigate.

Jeffrey

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Fri, 11 Oct 2019 21:13:57 +0000

From: rfc-interest <[email-masked]> on behalf of Bron Gondwana <[email-masked]>
Date: Friday, October 11, 2019 at 4:17 PM
To: "[email-masked]" <[email-masked]>
Subject: Re: [rfc-i] Accessing RFCs in XML

Bron, I’d agree with you IF the RFC V2 xml documents being discussed were of any real use for your use case. However, they’re not.  Your use case is for creating a new internet draft, and you should be looking for the XML for internet drafts. Now making THOSE easily accessible WOULD be a “good thing”.

I just poked at the datatracker.ietf.org system and tools.ietf.org repos for some internet drafts that I KNOW were created using xml v2, and couldn’t find an easy way to grab the corresponding XML files. Yet they are all up there on the servers.

Henrik, what do you think about putting a link to the XML versions of the internet drafts that are present?

                Tony

---

From: Ronald Tse <[email-masked]>
Date: Fri, 11 Oct 2019 21:22:27 +0000

I am 100% for the idea that previous XML v2 should be provided.

Other than the very valid, previously stated argument that it would greatly help production of future -bis documents for XML v3, just the fact of allowing the fetching of XML v2 would open the doors for machine readability and referencability, as well as de-mystifying the process (echoing Bron).

A link to the XML version is all we ask for… It’s not an urgent task but would be great if it could be done one day.

Ron

_____________________________________

Ronald Tse
Ribose Inc.

On Oct 11, 2019, at 5:13 PM, HANSEN, TONY L <[email-masked]<mailto:[email-masked]>> wrote:

From: rfc-interest <[email-masked]<mailto:[email-masked]>> on behalf of Bron Gondwana <[email-masked]<mailto:[email-masked]>>
Date: Friday, October 11, 2019 at 4:17 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [rfc-i] Accessing RFCs in XML

On Fri, Oct 11, 2019, at 14:41, HANSEN, TONY L wrote:
From: rfc-interest <[email-masked]<mailto:[email-masked]>> on behalf of Ted Lemon <[email-masked]<mailto:[email-masked]>>
Date: Friday, October 11, 2019 at 2:13 PM
Subject: Re: [rfc-i] Accessing RFCs in XML

I just poked at the datatracker.ietf.org<http://datatracker.ietf.org/> system and tools.ietf.org<http://tools.ietf.org/> repos for some internet drafts that I KNOW were created using xml v2, and couldn’t find an easy way to grab the corresponding XML files. Yet they are all up there on the servers.

                Tony
_______________________________________________
rfc-interest mailing list
[email-masked]<mailto:[email-masked]>
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Ronald Tse <[email-masked]>
Date: Fri, 11 Oct 2019 21:27:26 +0000

And in case it wasn’t clear, RFCs in the XML v2 format are still useful. We have a tool called “rfc2mn” [1] for converting XML RFC v2 documents into the AsciiRFC format (which somehow is still stuck at AUTH for over a year…?), which could be used in generating v3 as well as to other SDO schemas.

[1] https://github.com/metanorma/rfc2mn

Ronald Tse
Ribose Inc.

On Oct 11, 2019, at 5:22 PM, Ronald Tse <[email-masked]<mailto:[email-masked]>> wrote:

Ron

Ronald Tse
Ribose Inc.

                Tony

---

From: Bob Hinden <[email-masked]>
Date: Fri, 11 Oct 2019 14:28:11 -0700

Christian,

> On Oct 11, 2019, at 1:28 PM, Christian Huitema <[email-masked]> wrote:
> 
> 

I agree.  I have done this for a number of bis documents, it was never a problem getting the XML from the RFC Editor via email.

BTW, the way I do this is to first create a baseline XML that produces an Internet Draft and then do rfcdiff between the published RFC text and the new bis draft to make sure it matches (except for the formatting difference between RFC and ID styles).   Once it matched, I published this as -00.  The first draft with the actual changes was -01 to make it easier to track the changes.

Bob

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Fri, 11 Oct 2019 21:32:20 +0000

From: rfc-interest <[email-masked]> on behalf of "HANSEN, TONY L" <[email-masked]>
Date: Friday, October 11, 2019 at 5:14 PM

                Tony

I guess I got unlucky in the sample I poked at. The datatracker and tools.ietf.org DOES include xml links for several other internet drafts that I looked at after sending the above email.

                Tony

PS. Thanks Jeffrey

---

From: "John Levine" <[email-masked]>
Date: 11 Oct 2019 21:48:47 -0400

In article <CABcZeBO3ruXajMguCs_A5Gaz+T2JU7oYownWYuk0HKQQSbO=[email-masked]> you write:
>retention, should be straightforward to quickly fulfill.

If the RSE says that they've had trouble with people misunderstanding
what the XML v2 files are, I'm inclined to believe her since she's a
lot closer to the production process than any of us are.

I don't see the point of this argument other than as a rather
unpleasant exercise in who can push who around.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 11 Oct 2019 22:30:27 -0400

In article <[email-masked]> you write:
>I just poked at the datatracker.ietf.org system and tools.ietf.org repos for some internet drafts that I KNOW were created using
>xml v2, and couldn’t find an easy way to grab the corresponding XML files. Yet they are all up there on the servers.

When I look at the datatracker page for some of my drafts, around the
middle of the page is a Formats line with a bunch of buttons one of
which is xml.  Click it and you get the xml version.  It looks like
the xml is more likely to be present if the draft isn't expired.

If you rsync the i-d's from ftp.rfc-editor.org::internet-drafts, the
xml is there, too.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 11 Oct 2019 22:32:56 -0400

See ticket 2814.  It's a new version of draft-levine-dnsextlang-12,
which is not very different from draft-levine-dnsextlang-11.

When I run -11 through the current xml2rfc, I get a python
traceback for which I do not see any obvious cause.  The -12
is updated so I think it's valid in both v2 and v3, works
in xml2rfc, but crashed the i-d upload server.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 12 Oct 2019 09:26:54 +0200

> 4) With V3, the final XML source does reflect the final RFC text.

But note that even when V3 XML is published, that XML will not be really
suitable for future revisions, because of all the inlining the preptool
step does. Either this needs to be undone, or the originally submitted
files need to preserved anyway.

Best regards, Julian

---

From: Randy Bush <[email-masked]>
Date: Sat, 12 Oct 2019 09:40:48 +0200

> lot closer to the production process than any of us are.

bingo

---

From: Christian Huitema <[email-masked]>
Date: Sat, 12 Oct 2019 00:49:13 -0700

> files need to preserved anyway.

Of course, there is a significant tension between "getting an easy to
edit XML" and "getting an XML that exactly produces the final text." But
then, most of the draft that I did lately were composed in markdown, not
XML. So "easy to edit" would mean "give me the markdown source of the
previous text". Of course, not everybody uses markdown. And there are
good reasons to believe that people writing an RFCbis 10 years from now
will be using some other popular format of the future. I suspect that
there are no 100% solutions there.

Maybe we should just ask Adobe. They are able to produce a word doc from
a PDF. They might be able to produce xml2rfc, markdown or other formats
as well...

-- Christian Huitema

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 12 Oct 2019 10:33:47 +0200

> files need to preserved anyway.

Good point.  When I redo the RFCXML to markdown converter (*) for v3, I could add some up conversion for publish-RFCXMLv3 to author-RFCXMLv3 in the process, if that can be done reasonably.

Or we could preserve the author-RFCXMLv3 (the same way we are discussing this for v2 now).

Grüße, Carsten

(*) This tool is not yet published because I need more time to finish it, but is available as a service — just send me your XML and I’ll apply the tool, fix a few more bugs in that, and ship you back the markdown.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 12 Oct 2019 12:41:09 +0200

> On Oct 12, 2019, at 04:30, John Levine <[email-masked]> wrote:
> 
> which is xml.  Click it and you get the xml version.  

… if the original submitter bothered to submit it, alongside the .txt (talking about the RFCXMLv2 situation here).

I usually don’t, because the actual source is at the github repo (in markdown), so providing it in the I-D repo (where I can’t put the markdown source) might confuse people to start editing the RFCXML (which confusion actually does happen often enough that I wrote an RFCXML to markdown upconverter).  Heather has a point, people do get confused by confusingly labeled files…

Of course, for the final version of the I-D that goes into the RFC editor queue, I do provide the RFCXML (unless asleep at the wheel).  The markdown input that was used to generate that is stashed into a comment in the RFCXML and can be extracted from that.  So even if the github repo goes away, the document source is preserved.

Somebody should write this up…

Grüße, Carsten

---

From: Russ Housley <[email-masked]>
Date: Sat, 12 Oct 2019 10:49:54 -0400

John:

Were you able to produce the .txt file locally and then upload it?

Russ

> On Oct 11, 2019, at 10:32 PM, John Levine <[email-masked]> wrote:
>

---

From: Michael Richardson <[email-masked]>
Date: Fri, 11 Oct 2019 21:32:54 +0200

Heather Flanagan <[email-masked]> wrote:
    > I can see a world where this is slightly more automated, in that the
    > person doing the request needs to click on some kind of
    > acknowledgement, but I suspect that will be equally annoying to those
    > who just want the files. I also think it’s easier to ignore a click
    > through than it is to ignore a human sending you email. Still, worth
    > discussing after the RPC is through the hurdle of v3 implementation.

I think just throw a tgz/zip file at Richard and he'll put it somewhere
that is clearly not rfc-editor.org.   Maybe he'll hack the abstract to insert
some words "THIS FILE IS NOT RFCXXXX. IT PRECEEDS AUTH48" or something.
Then it's out of your hair.

I think that the community would really like it available for doing "stuff"
on it.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 11 Oct 2019 23:21:49 +0200

On Oct 11, 2019, at 22:38, Jeffrey Yasskin <[email-masked]> wrote:
> 
> the XML isn't the source code for a lot of RFCs and I-Ds.

Hi Jeffrey,

good point.  For XML files that were generated by kramdown-rfc, the source is actually embedded in the XML in a comment and can be recovered using the tool kramdown-rfc-extract-markdown that comes with kramdown-rfc.  This of course no longer works when the XML has become the source, as with RFCXML edited by the RFC editor.

On the subject of this thread, I think most of us see the point that Heather has been making.
Opinions differ whether making the XML available with low threshold (and for bulk analysis) is actually a greater good (I do very much think so(*)) and how the objectives of the current restrictions can possibly be approximated by a simpler approach (e.g., appropriate file naming — certainly not RFCnnnn.xml).

Grüße, Carsten

(*) Both Julian and I have large collections of these XML files that we have sneaked out from the AUTH48 process, and for both of us these collections are actually rather useful.  At least my collection is not complete, though, and completeness would help finding out whether a specific RFC actually was edited in XML (this information could be made available separately, though) and what “tricks” were used in achieving a specific result, which may not be reconstructable from the last I-D (which only works if the XML for that is available).  Other useful information such as what version of xml2rfc was used during editing may no longer be available at all; this is maybe something that could be established for v3.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 11 Oct 2019 23:26:42 +0200

On Oct 11, 2019, at 23:22, Ronald Tse <[email-masked]> wrote:
> 
> allowing the fetching of XML v2 would open the doors for machine readability and referencability,

Right!  I forgot to make one point: 

For instance, RFC 8152 says:

   The collected CDDL can be extracted from the XML version of this
   document via the following XPath expression below.  (Depending on the
   XPath evaluator one is using, it may be necessary to deal with &gt;
   as an entity.)

   //artwork[@type='CDDL']/text()

Fortunately, with v3 this approach will become much easier.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Sat, 12 Oct 2019 17:37:32 +0200

> the final RFC text. For a fair proportion of old RFC, there is actually

And for some, there is XML source, but it's not easy to use (because of
missing included files, or general brokenness that was possible to
ignore with the TCL processor).

That said, if the XML is processable, it's relatively easy to check how
close it is to the publish RFC. Just run xml2rfc on it, and compare the
output with the published text version. That can be automated (and I
actually have done that).

Best regards, Julian

---

From: "John R Levine" <[email-masked]>
Date: 12 Oct 2019 12:37:28 -0400

> Were you able to produce the .txt file locally and then upload it?

That worked, although idnits complained that Trumansburg isn't a country.

R's,
John

---

From: Michael Richardson <[email-masked]>
Date: Sat, 12 Oct 2019 19:07:14 +0200

Ted Lemon <[email-masked]> wrote:
    > I think it is unfortunate that we can’t get the last xml file. I think
    > it would be a mistake to label that xml file the “xml file for
    > rfcXXXX.”  However, it is the xml file for some previous
    > version. Making it available as that seems to solve EKR’s

That's a good idea.
draft-ietf-foo-bar-24.xml

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 14 Oct 2019 17:47:48 +0200

> draft-ietf-foo-bar-24.xml

Umm.  That makes it look like a regular draft submission, which it isn't.

I think we need a new naming convention for this.

	Henrik

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 14 Oct 2019 17:52:44 +0200

editor-draft-rfc-8446.xml

or so.

Grüße, Carsten

> On Oct 14, 2019, at 17:47, Henrik Levkowetz <[email-masked]> wrote:
>

---

From: Richard Barnes <[email-masked]>
Date: Tue, 15 Oct 2019 14:27:10 -0400

FYI, if other people have need of these files, I have populated the
following repo with files collected by Julian.

https://github.com/bifurcation/pre-rfc-xml

Note: These were not provided to me by the RSE or RPC.  I understand that
Julian collected them by periodically rsync'ing the directory where AUTH48
work happens.

---

From: Bob Hinden <[email-masked]>
Date: Tue, 15 Oct 2019 15:40:09 -0700

Richard,

> On Oct 15, 2019, at 11:27 AM, Richard Barnes <[email-masked]> wrote:
> 
> FYI, if other people have need of these files, I have populated the following repo with files collected by Julian.
> 

I looked and got:

"Sorry, we had to truncate this directory to 1,000 files. 1,716 entries were omitted from the list.”

Bob

> 
> Note: These were not provided to me by the RSE or RPC.  I understand that Julian collected them by periodically rsync'ing the directory where AUTH48 work happens.
> 
> Over the past several years, it seems like many (most?) RFCs have been submitted and AUTH48'ed in XML.  Certainly all of the ones I've authored have been.  Yet, I can't seem to find the XML on rfc-editor.org.  Am I missing something?
> 
> Thanks,
> --Richard

---

From: Richard Barnes <[email-masked]>
Date: Tue, 15 Oct 2019 18:55:11 -0400

If you refresh, it should be better.  I moved the XML files into a
directory.  If you clone the repo, you'll get them all; it's just the GUI
that is truncated.

On Tue, Oct 15, 2019 at 6:40 PM Bob Hinden <[email-masked]> wrote:

> > Note: These were not provided to me by the RSE or RPC.  I understand
> that Julian collected them by periodically rsync'ing the directory where
> AUTH48 work happens.
> >

---

From: Bob Hinden <[email-masked]>
Date: Wed, 16 Oct 2019 08:20:44 -0700

Richard,

Refreshing didn’t change anything.  I don’t want a copy (aka clone), just wanted to see a list of all of the files.

Bob

> On Oct 15, 2019, at 3:55 PM, Richard Barnes <[email-masked]> wrote:
> 
> If you refresh, it should be better.  I moved the XML files into a directory.  If you clone the repo, you'll get them all; it's just the GUI that is truncated.
>

---

From: Richard Barnes <[email-masked]>
Date: Wed, 16 Oct 2019 11:55:11 -0400

I don't think the Github GUI lets you do that.  It's probably faster to
copy/paste `git clone https://github.com/bifurcation/pre-rfc-xml.git && ls
pre-rfc-xml/xml/` than it is to send me email asking how to make the GUI
work :)

On Wed, Oct 16, 2019 at 11:20 AM Bob Hinden <[email-masked]> wrote:

> Richard,
>
> Refreshing didn’t change anything.  I don’t want a copy (aka clone), just
> wanted to see a list of all of the files.
>
> Bob
>
>

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 16 Oct 2019 09:01:23 -0700

Richard,

Maybe ls -l > index.txt ?

Sent from my iPhone

> On 16 Oct 2019, at 8:20 am, Bob Hinden <[email-masked]> wrote:
> 
> Richard,
>

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 16 Oct 2019 12:21:47 -0400

In the end, you found a way to get what you could.  I'm not sure you
realize this, but there's been no explanation of your plans for these files
that were accessible and there was no way for the RFC Editor or production
center to understand that in order to prioritize your request as far as I
can tell from this thread.  There was no way for leadership bodies to also
evaluate your request and understand your plans as they have key objectives
to be met for the remainder of Heather's time as well as for production
center priorities.  It's not like these files go away with Heather, they
are part of the production center that is already dealing with a backlog.
Additionally, your screen scapes may not even be the latest XML versions
that the RFC production center holds.  If people use these, what
disclaimers will be offered?  This hack may backfire.  How about a
considered project proposal with objectives that can be evaluated and
prioritized?

I do hope when we get a new RFC Editor (and for the remainder of Heather's
term) that you offer more professional courtesy in requests.  I'm a bit
appalled on the behavior aspects of this thread and the ganging up by a few
people.  This is less than respectful or professional, which is the only
reason for my post.  Let's try to do better and be a more welcoming
community.  A few of you are in leadership roles and should be mindful of
that.

Best regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Ted Lemon <[email-masked]>
Date: Wed, 16 Oct 2019 12:27:37 -0400

On Oct 16, 2019, at 12:21 PM, Kathleen Moriarty <[email-masked]> wrote:
> I do hope when we get a new RFC Editor (and for the remainder of Heather's term) that you offer more professional courtesy in requests.  I'm a bit appalled on the behavior aspects of this thread and the ganging up by a few people.  This is less than respectful or professional, which is the only reason for my post.  Let's try to do better and be a more welcoming community.  A few of you are in leadership roles and should be mindful of that.

Well said, particularly the bit about scope and prioritization.  I’ve been kind of hornswoggled by the lack of consideration and respect in this thread.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 16 Oct 2019 19:24:33 +0200

On 16.10.2019 18:21, Kathleen Moriarty wrote:
> (...)  Additionally, your screen scapes may
> not even be the latest XML versions that the RFC production center
> holds. (...)

"Screen scapes"?

Best regards, Julian

---

From: Richard Barnes <[email-masked]>
Date: Wed, 16 Oct 2019 13:31:06 -0400

Hi Kathleen,

I think you've got things a bit backwards here.  As a community member, I
have no obligations here -- indeed, the whole point of open standards is
that they can be used without getting permission!  Rather, the RFC Editor
and RPC have a responsibility to provide a repository of RFCs and related
data -- a role that they are contracted and paid to do.

The dynamic you are observing here is my reaction to the RSE/RPC failing to
meet my expectations as a community member, at multiple levels.  First, my
expectation that the RSE/RPC would publish the XML files in the same way
they publish other non-authoritative versions of RFCs (e.g., PDFs).  And
second, my expectation that even if they didn't publish them, they would at
least have a sensible data management system so that they could be provided
with minimal effort.

Why didn't I provide the RSE/RPC with the information they would need to
prioritize work?  Because when this thread started, I didn't think work
would be involved.  And now that I have learned that it would be a project,
I don't think it's worth prioritizing.

--Richard

---

From: Richard Barnes <[email-masked]>
Date: Wed, 16 Oct 2019 13:44:27 -0400

Good idea.  Bob, here you go:

https://github.com/bifurcation/pre-rfc-xml/blob/master/xml-index.txt

On Wed, Oct 16, 2019 at 12:01 PM Mark Nottingham <[email-masked]> wrote:

> > Refreshing didn’t change anything.  I don’t want a copy (aka clone),
> just wanted to see a list of all of the files.
> >
> > Bob
> >
> >

---

From: Julian Reschke <[email-masked]>
Date: Wed, 16 Oct 2019 19:44:45 +0200

On 16.10.2019 19:24, Julian Reschke wrote:
>> (...)  Additionally, your screen scapes may not even be the latest XML
>> versions that the RFC production center holds. (...)
>
> "Screen scapes"?
>
> Best regards, Julian

To clarify, it's not the typo that confuses me, but the fact that these
files definitively are not the results of "scraping screens". These are
just archived copies of the files present on
<ftp://ftp.rfc-editor.org/in-notes/authors>.

Best regards, Julian

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 16 Oct 2019 13:44:33 -0400

Hi Richard,

> data -- a role that they are contracted and paid to do.

While you may be posting as a community member, you are on the ISOC board.
Your tone always matters.  The tone of leadership sets the standard for the
organization and this thread put it pretty low.  Reading the thread at one
time, it came across as a bullying effort among a few people several in
leadership role or recently out of them.  There was also no explanation for
the request and no stated or considered plans for the requested files, but
repeated requests for an ETA.

I have not read the full contract, so I will not comment on that.  I do
think a request like this should go through an approval process and should
require a considered project plan given their current constraints presented
to the entire community at the recent plenary sessions.  The issues have
been resolved for v3 as stated in the thread, so the tone really seems
unnecessary.

> The dynamic you are observing here is my reaction to the RSE/RPC failing
> to meet my expectations as a community member, at multiple levels.  First,
> my expectation that the RSE/RPC would publish the XML files in the same way

Sure and they heard you and fixed it for v3.  Perhaps you could have
enquired if there was a plan to retroactively resolve this for v2 and if
there was not proposed a plan for review and consideration?  I'd bet there
are moving parts within their work that may not come to mind for those of
us not in this role.  Respecting their work and expertise should be the
norm.  Additionally, you get more with honey then vinegar.

> I don't think it's worth prioritizing.

OK, it's good that you are stating that now.  It would have been nice to do
it without me prompting on professional behavior (it wasn't just one
person, so we all need to be cognizant).  They are the experts on their
work and should be approached in that manner for the RFC Editor, RPC, and
other functions vital to the IETF.

Best regards,
Kathleen

>> I do hope when we get a new RFC Editor (and for the remainder of
>> Heather's term) that you offer more professional courtesy in requests.  I'm
>> a bit appalled on the behavior aspects of this thread and the ganging up by
>> a few people.  This is less than respectful or professional, which is the
>> only reason for my post.  Let's try to do better and be a more welcoming
>>>> > Over the past several years, it seems like many (most?) RFCs have
>>>> been submitted and AUTH48'ed in XML.  Certainly all of the ones I've
>>>> authored have been.  Yet, I can't seem to find the XML on
>>>> rfc-editor.org.  Am I missing something?
>>>> >
>>>> > Thanks,
>>>> > --Richard

-- 

Best regards,
Kathleen

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 16 Oct 2019 14:22:36 -0400

I wonder where you think the commitment to provide the intermediate XMLs 
come from.  That is NOT part of the contracted deliverables.  It is 
something the RPC does to be helpful to authors.

Complaining that the RPC is doing a partial job of doing something that 
they are only doing as a favor to the community seems rather backwards 
to me.

Yours,
Joel

On 10/16/2019 1:31 PM, Richard Barnes wrote:
> Hi Kathleen,
> 
> I think you've got things a bit backwards here.  As a community member, 
> I have no obligations here -- indeed, the whole point of open standards 
> is that they can be used without getting permission!  Rather, the RFC 
> Editor and RPC have a responsibility to provide a repository of RFCs and 
> related data -- a role that they are contracted and paid to do.
> 
> to meet my expectations as a community member, at multiple levels.  
> First, my expectation that the RSE/RPC would publish the XML files in 
> the same way they publish other non-authoritative versions of RFCs 
> (e.g., PDFs).  And second, my expectation that even if they didn't 
> publish them, they would at least have a sensible data management system 
> so that they could be provided with minimal effort.
> 
> would be involved.  And now that I have learned that it would be a 
> project, I don't think it's worth prioritizing.
> 
> --Richard
> 
> On Wed, Oct 16, 2019 at 12:22 PM Kathleen Moriarty 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     realize this, but there's been no explanation of your plans for
>     these files that were accessible and there was no way for the RFC
>     Editor or production center to understand that in order to
>     prioritize your request as far as I can tell from this thread. 
>     There was no way for leadership bodies to also evaluate your request
>     and understand your plans as they have key objectives to be met for
>     the remainder of Heather's time as well as for production center
>     priorities.  It's not like these files go away with Heather, they
>     are part of the production center that is already dealing with a
>     backlog.  Additionally, your screen scapes may not even be the
>     latest XML versions that the RFC production center holds.  If people
>     use these, what disclaimers will be offered?  This hack may
>     backfire.  How about a considered project proposal with objectives
>     that can be evaluated and prioritized?
> 
>     Heather's term) that you offer more professional courtesy in
>     requests.  I'm a bit appalled on the behavior aspects of this thread
>     and the ganging up by a few people.  This is less than respectful or
>     professional, which is the only reason for my post.  Let's try to do
>     better and be a more welcoming community.  A few of you are in
>     leadership roles and should be mindful of that.
> 
>     Best regards,
>     Kathleen
> 
>         If you refresh, it should be better.  I moved the XML files into
>         a directory.  If you clone the repo, you'll get them all; it's
>         just the GUI that is truncated.
> 
>         On Tue, Oct 15, 2019 at 6:40 PM Bob Hinden <[email-masked]
>         <mailto:[email-masked]>> wrote:
> 
>             Richard,
> 
>              > FYI, if other people have need of these files, I have
>             populated the following repo with files collected by Julian.
>              >
>             "Sorry, we had to truncate this directory to 1,000 files.
>             1,716 entries were omitted from the list.”
> 
>             Bob
> 
>              >
>              > Note: These were not provided to me by the RSE or RPC.  I
>             understand that Julian collected them by periodically
>             rsync'ing the directory where AUTH48 work happens.
>              >
>              > Over the past several years, it seems like many (most?)
>             RFCs have been submitted and AUTH48'ed in XML.  Certainly
>             all of the ones I've authored have been.  Yet, I can't seem
>             to find the XML on rfc-editor.org <http://rfc-editor.org>. 
>             Am I missing something?
>              >
>              > Thanks,
>              > --Richard
>             <mailto:[email-masked]>

---

From: S Moonesamy <[email-masked]>
Date: Fri, 18 Oct 2019 01:32:27 -0700

Hi Kathleen, Richard,
At 10:44 AM 16-10-2019, Kathleen Moriarty wrote:
>I have not read the full contract, so I will not comment on that.  I 
>do think a request like this should go through an approval process 
>and should require a considered project plan given their current 
>constraints presented to the entire community at the recent plenary 
>sessions.  The issues have been resolved for v3 as stated in the 
>thread, so the tone really seems unnecessary.

I took a quick look at the contract.  The XMLs which Richard 
requested are not part of the deliverables.  An external observer 
would likely view the request as something that can be easily done as 
the XMLs might already be produced as part of the existing work 
flow.  Given that the person is external, he/she might not be fully 
aware of the constraints, e.g. it is not that easy to do, the pre-v3 
XML file might be inconsistent with the final output (RFC).

Regards,
S. Moonesamy