Subject: [rfc-i] The <tt> train wreck

From: Carsten Bormann <[email-masked]>
Date: Sat, 14 Aug 2021 01:13:32 +0200

The original RFCXML RFC 2629 did not have any elements to indicate emphasis (often rendered as italic/oblique and/or bold type).  More normatively in practice, XML2RFCv1 had »<spanx style=«, which provided text spans with the following properties:

| style   | nobreak | decorator | type      | tt |
|---------|---------|-----------|-----------|----|
| emph    |         | _         | em        | -  |
| strong  |         | *         | strong    | -  |
| nobreak | x       | none      | -         | -  |
| vbare   | x       | none      | -         | x  |
| verb    | x       | "         | -         | x  |
| vemph   | x       | _         | em        | x  |
| vstrong | x       | *         | strong    | x  |
| vdeluxe | x       | *_ and _* | em strong | x  |

The “style" column is the style attribute that could be given to spanx.

The "nobreak" column indicates whether a non-breaking semantics was intended (trying harder to keep pieces together around /@&|-+#%: characters).

For TXT, the “decorator" column indicates what character is used _around_ the span in plaintext rendering.  Note that only “nobreak" and “vbare” had no decorator.  Using »"« as a decorator for “verb” certainly was an unloved compromise that did, however, work well enough.

For HTML, I note that the nobreak functionality was commented out in my version of XML2RFCv1 (there seemed to be no easy way to translate it into HTML at the time).  The columns “type” and “tt” indicate the font to be used in HTML: “type” provides the variation of the base font, and “tt” indicates whether the base font is monospaced or not.

Much of this functionality was known only to people who actually looked into the source code of XML2RFC.  I haven’t checked XML2RFCv2, but it seems that some of these features should have survived into v2, but have decayed.  Since almost nobody cared about the HTML renderings (it was irrelevant for the actual RFC publishing), I’m not sure that full support was checked extensively — XML2RFCv1 was available and could be used by authors that did need the full functionality.

Enter v3.

I can’t find any horizontal no-breaking support (except for that which Unicode provides).  Unexplicably whole-document options like --table-hyphen-breaks were introduced to create tweaks that should have been applied for specific items.

Emph and strong were finally spread out into their own elements, <em and <strong.
These can be combined with each other and with monospaced font selection, so the latter was turned into its own element, <tt.
As <spanx style=“verb”> was the only supported form of monospacing in xml2rfcv2 at the time, <tt putatively was its replacement.

So we have the last two columns of my table above covered, but not the second and the third.

Little thinking was wasted about the plaintext rendering of these new span elements — after all, XML2RFCv2 had emph, strong, and verb, and these seemed to work.
In fact, the de-facto v3 manual https://xml2rfc.tools.ietf.org/xml2rfc-doc.html until today doesn’t mention plaintext rendering of these elements.

So, since RFC8650, v3 documents have been keyboarded and proof-read under the assumption that <em, <strong, and <tt are the replacements of the spanx styles “emph”, “strong”, and “verb”.

Note that these span elements not only set the font for each of the characters in the span; they also have a delimiting semantics.  This is obvious when decorators are used in the plaintext form, but also on the HTML side, the <tt element is rendered as a separate HTML element, which with its CSS styles makes sure <tt>a </tt> (note the space after the a) looks different from <tt>a</tt>.

The »"« decorator for what is now <tt continues to be unloved.
Since 2020-06-20, there is some weird code in XML2RFCv3 that guesses that these decorators are unwanted in certain table contexts; the fact that this code tends to guess wrong (of course!) already has led to a bug report [0].
Apparently this will be fixed by removing the decorators entirely [1].

So, after 400+ RFCs have been published under the assumption of (and proofread against) decorators enabling understanding the plaintext rendering of <tt, the meaning of <tt will be retroactively changed from <spanx style=“verb”> to <spanx style=“vbare”> (which hasn’t even been available for the decade most people used XML2RFCv2).

I have pointed out why the decorators are needed in certain cases [2].
As several people point out, there are also cases where the <tt decorators are unneeded or even somewhat ugly.
Which of these are the case depends on the authors’ intent with the <tt.
As that is not captured (as it used to be in verb vs. vbare), there is no hope to get this decision right in the two formatters, TXT and HTML/PDF.

TL;DR:

The decision to always remove the decorations from the TXT rendering of <tt> is wrong.
This is because unfortunately <tt> is broken, i.e., ill-conceived, for its application.
As this has been enshrined, there can be no backward-compatible “right thing", only repairs going forward.
(The decision also points out that the way we currently reach these decisions is broken; maybe we can revisit that particular point when we come up with a new decision structure.)

Grüße, Carsten

PS.: I have CCed this to rfc-markdown because there also is no good way in markdown to keyboard the distinction between a decorative <tt that can be ignored in plaintext and one that *needs* a representation (“decorators”):  Like this part of XML2RFCv3, markdown also was not designed to format into plaintext.  It would be nice if there were a way to create a workaround from kramdown-rfc, but the problem is that the single piece of XML that kramdown-rfc puts out needs to generate both the TXT and the HTML/PDF version, and there is no way in RFCXMLv3 to indicate the variant processing needed for TXT.

[0]: <https://mailarchive.ietf.org/arch/msg/xml2rfc/30tTnMMcJHCIH8t8-s_NVLZYrFg>
[1]: https://trac.ietf.org/trac/xml2rfc/ticket/600
[2]: E—mail that unfortunately only went to a subgroup of people discussing the issue and therefore isn’t archived; reproduced below.  It first discussed the need to resurrect some non-breaking semantics, and then (search for “txtquotes”) discussed the need for the author to actively choose between decorated (»txtquotes=“true”«) and undecorated (»txtquotes=“true”«) variants.

> On 7/8/21 1:33 AM, Carsten Bormann wrote:
>> I probably should add that on the authoring side, a variant of <code> with non-breaking semantics is needed.
>> But that can be done in the authoring tool (kramdown-rfc) by transliterating space, hyphen etc. into their non-breaking equivalents, so it probably doesn’t need support from xml2rfc.
[…]

I ran into this requirements (non-breaking semantics) when I converted the XML for RFC 6125, where the RFC-editor (I assume) had converted some, but not all hyphens in syntax snippets that were in quotes into non-breaking hyphens.
(Note that they didn’t bother with the at the time clumsy <spanx style=“verb”> but put in the quotes that would have been generated anyway from that directly, as that is equivalent with TXT-only production.)

 (or its equivalent) containing a "reg&nbhy;name".  (Matching only the
 "reg&nbhy;name" rule from <xref target='URI'/> limits verification to DNS
 domain names, thereby differentiating a URI&nbhy;ID from a

 A certificate for this service might include SRV-IDs of 
 "_xmpp&nbhy;client.im.example.org" and "_xmpp&nbhy;server.im.example.org” 
 (see <xref target='XMPP'/>), a DNS-ID of "im.example.org", and an XMPP-specific

Clearly, those quotes were desired in the TXT output here (but, I don’t think, would not have been in the HTML), so these all would be specified as <tt txtquotes=“true”> in my imagined new world.

On the matter of my suggested txtquotes bit, I also just looked into RFC 8949, because I’m familiar with it and it is a non-trivial document.

The instances of <tt>false… (true, …) would actually improve with txtquotes=“false".

The instances of <tt>0 and <tt>0.0 also work with txtquotes=“false".

The reference to

 the <tt>date-time</tt> production in <xref target="RFC3339" 

can be understood either way, but for txtquotes=“false” that understandability hinges on the production having a nominal name; if it were <tt txtquotes=“false" >second</tt>, this would not work at all.

Similar with

 doesn't match the <tt>URI-reference</tt> production, the string is invalid.</li>

Neutral for

 the encoded text string <tt>0x62c0ae</tt>

Getting more of a problem:

 interested in this information.  For example, <tt>_</tt> or <tt>_3</tt>.

Completely broken:
     <t indent="0" pn="section-appendix.c-5">Note that <tt>well_formed</tt>
     returns the major type for well-formed

Note that one is the function name and the other one is the property defined in this RFC.
(Yes, in this case the reader can guess which is which by one being snake_case and the other kebab-case.  But ouch.)
So to make the TXT acceptable, the XML and thus the HTML would need to be changed here.

In the RFCXMLv2 times, the authors could decide whether they wanted TXT quotes and just leave the decoration off if they didn’t.  But with better styling available on the HTML side, they’ll want to switch on <tt> and sometimes regret if that semantics is suppressed on the TXT side.

By the way, that same “switch off the fallback” bit would tremendously improve <em> and <strong> as well in certain cases, too:

  *  *SHA-384* and *SHA-512* hash functions are efficient for 64-bit
     hardware.

(I’ve seen much worse, but can’t find an example of that right now.  I did see one in the course of today!)

So maybe this would be ignore-in-plain-text=“true” instead of txtquotes=“false”.

Grüße, Carsten

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 16 Aug 2021 11:11:07 +1000

It seems like overloading this with three levels of semantics is the original sin.

decorations (italic, bold, monospace), quoting (_, *, "), line breaking

From my perspective, it would be good to control each independently.  With tags.  I don't care if that is different tags, attributes on a single tag, or some combination of that with some global flags to control it.  (Global flags => stylesheet?)

Regarding non-breaking options:

I personally find the current reliance on &nbsp; &nbhy; (and worse, &zwnj; of which we have one in RFC 9000) to be problematic.  It means that the text you copy is not the text you expect which can confuse all but the smartest searcher.  I would prefer to control this with tags.  If nothing else, it would be much more explicit and less error-prone.

With all the effort that went into making BCP 14 not wrap, I note that RFC 9000 wraps between BCP and 14.  Something that RFC 9087 doesn't do - at least for the text rendering (the &nbsp; is in the XML, but not the HTML, which suggests that xml2rfc is bleaching it incorrectly).

On Sat, Aug 14, 2021, at 09:13, Carsten Bormann wrote:
> The original RFCXML RFC 2629 did not have any elements to indicate 
> emphasis (often rendered as italic/oblique and/or bold type).  More 
> normatively in practice, XML2RFCv1 had »<spanx style=«, which provided 
> text spans with the following properties:
> 
> The "nobreak" column indicates whether a non-breaking semantics was 
> intended (trying harder to keep pieces together around /@&|-+#%: 
> characters).
> 
> For TXT, the “decorator" column indicates what character is used 
> _around_ the span in plaintext rendering.  Note that only “nobreak" and 
> “vbare” had no decorator.  Using »"« as a decorator for “verb” 
> certainly was an unloved compromise that did, however, work well enough.
> 
> For HTML, I note that the nobreak functionality was commented out in my 
> version of XML2RFCv1 (there seemed to be no easy way to translate it 
> into HTML at the time).  The columns “type” and “tt” indicate the font 
> to be used in HTML: “type” provides the variation of the base font, and 
> “tt” indicates whether the base font is monospaced or not.
> 
> Much of this functionality was known only to people who actually looked 
> into the source code of XML2RFC.  I haven’t checked XML2RFCv2, but it 
> seems that some of these features should have survived into v2, but 
> have decayed.  Since almost nobody cared about the HTML renderings (it 
> was irrelevant for the actual RFC publishing), I’m not sure that full 
> support was checked extensively — XML2RFCv1 was available and could be 
> used by authors that did need the full functionality.
> 
> Enter v3.
> 
> I can’t find any horizontal no-breaking support (except for that which 
> Unicode provides).  Unexplicably whole-document options like 
> --table-hyphen-breaks were introduced to create tweaks that should have 
> been applied for specific items.
> 
> These can be combined with each other and with monospaced font 
> selection, so the latter was turned into its own element, <tt.
> As <spanx style=“verb”> was the only supported form of monospacing in 
> xml2rfcv2 at the time, <tt putatively was its replacement.
> 
> Little thinking was wasted about the plaintext rendering of these new 
> span elements — after all, XML2RFCv2 had emph, strong, and verb, and 
> these seemed to work.
> In fact, the de-facto v3 manual 
> https://xml2rfc.tools.ietf.org/xml2rfc-doc.html until today doesn’t 
> mention plaintext rendering of these elements.
> 
> So, since RFC8650, v3 documents have been keyboarded and proof-read 
> under the assumption that <em, <strong, and <tt are the replacements of 
> the spanx styles “emph”, “strong”, and “verb”.
> 
> Note that these span elements not only set the font for each of the 
> characters in the span; they also have a delimiting semantics.  This is 
> obvious when decorators are used in the plaintext form, but also on the 
> HTML side, the <tt element is rendered as a separate HTML element, 
> which with its CSS styles makes sure <tt>a </tt> (note the space after 
> the a) looks different from <tt>a</tt>.
> 
> Since 2020-06-20, there is some weird code in XML2RFCv3 that guesses 
> that these decorators are unwanted in certain table contexts; the fact 
> that this code tends to guess wrong (of course!) already has led to a 
> bug report [0].
> So, after 400+ RFCs have been published under the assumption of (and 
> proofread against) decorators enabling understanding the plaintext 
> rendering of <tt, the meaning of <tt will be retroactively changed from 
> <spanx style=“verb”> to <spanx style=“vbare”> (which hasn’t even been 
> available for the decade most people used XML2RFCv2).
> 
> As several people point out, there are also cases where the <tt 
> decorators are unneeded or even somewhat ugly.
> As that is not captured (as it used to be in verb vs. vbare), there is 
> no hope to get this decision right in the two formatters, TXT and 
> HTML/PDF.
> 
> TL;DR:
> 
> (The decision also points out that the way we currently reach these 
> decisions is broken; maybe we can revisit that particular point when we 
> come up with a new decision structure.)
> 
> Grüße, Carsten
> 
> 
> PS.: I have CCed this to rfc-markdown because there also is no good way 
> in markdown to keyboard the distinction between a decorative <tt that 
> can be ignored in plaintext and one that *needs* a representation 
> (“decorators”):  Like this part of XML2RFCv3, markdown also was not 
> designed to format into plaintext.  It would be nice if there were a 
> way to create a workaround from kramdown-rfc, but the problem is that 
> the single piece of XML that kramdown-rfc puts out needs to generate 
> both the TXT and the HTML/PDF version, and there is no way in RFCXMLv3 
> to indicate the variant processing needed for TXT.
> 
> [0]: 
> <https://mailarchive.ietf.org/arch/msg/xml2rfc/30tTnMMcJHCIH8t8-s_NVLZYrFg>
> [2]: E—mail that unfortunately only went to a subgroup of people 
> discussing the issue and therefore isn’t archived; reproduced below.  
> It first discussed the need to resurrect some non-breaking semantics, 
> and then (search for “txtquotes”) discussed the need for the author to 
> actively choose between decorated (»txtquotes=“true”«) and undecorated 
> (»txtquotes=“true”«) variants.
> 
> 
> I ran into this requirements (non-breaking semantics) when I converted 
> the XML for RFC 6125, where the RFC-editor (I assume) had converted 
> some, but not all hyphens in syntax snippets that were in quotes into 
> non-breaking hyphens.
> (Note that they didn’t bother with the at the time clumsy <spanx 
> style=“verb”> but put in the quotes that would have been generated 
> anyway from that directly, as that is equivalent with TXT-only 
> production.)
> 
> Clearly, those quotes were desired in the TXT output here (but, I don’t 
> think, would not have been in the HTML), so these all would be 
> specified as <tt txtquotes=“true”> in my imagined new world.
> 
> On the matter of my suggested txtquotes bit, I also just looked into 
> RFC 8949, because I’m familiar with it and it is a non-trivial document.
> 
> can be understood either way, but for txtquotes=“false” that 
> understandability hinges on the production having a nominal name; if it 
> were <tt txtquotes=“false" >second</tt>, this would not work at all.
> 
> Similar with
> 
> (Yes, in this case the reader can guess which is which by one being 
> snake_case and the other kebab-case.  But ouch.)
> In the RFCXMLv2 times, the authors could decide whether they wanted TXT 
> quotes and just leave the decoration off if they didn’t.  But with 
> better styling available on the HTML side, they’ll want to switch on 
> <tt> and sometimes regret if that semantics is suppressed on the TXT 
> side.
> 
> 
> By the way, that same “switch off the fallback” bit would tremendously 
> improve <em> and <strong> as well in certain cases, too:
>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Aug 2021 08:30:28 +0200

> It seems like overloading this with three levels of semantics is the original sin.

Absolutely; and it's because of that chaos I chose not to document the
later "add-ons" in RFC 7749.

>  From my perspective, it would be good to control each independently.  With tags.  I don't care if that is different tags, attributes on a single tag, or some combination of that with some global f…

Yes, but we really need to understand which of these we really need.
Which brings us back to the question: "what purpose does the TXT version
serve?".

> I personally find the current reliance on &nbsp; &nbhy; (and worse, &zwnj; of which we have one in RFC 9000) to be problematic.  It means that the text you copy is not the text you expect which can…

I agree with that, but would ask first what we need the break control
for. Is this about avoiding line breaks, or about making whitespace in
prose significant? Both? These are different problems.

> With all the effort that went into making BCP 14 not wrap, I note that RFC 9000 wraps between BCP and 14.  Something that RFC 9087 doesn't do - at least for the text rendering (the &nbsp; is in the…

Avoiding line breaks in BCP14 markup is another sad story. If the intent
is not to break inside those consistently, then this is something the
formatter could do (and not require authors to modify the source text).

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Aug 2021 08:56:24 +0200

On 2021-08-16, at 08:30, Julian Reschke <[email-masked]> wrote:
> 
> what purpose does the TXT version
> serve?

That is indeed an interesting question, and one answer is in <https://mailarchive.ietf.org/arch/msg/tools-discuss/BK7yyQoSbhnbYDYsyzjk_J2vufg>

The question is, however, not so relevant to the discussion at hand.
After a decade of trying to suppress the TXT version, it is still considered desirable by many even after the transition that was meant to make it less relevant.  So I would expect that, independent of any outcome of the discussion of the question for the reasons, the TXT version is here to stay.

So, back to fixing the train wreck.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Mon, 16 Aug 2021 09:10:44 +0200

> So, back to fixing the train wreck.

I agree that it'll stay; the question is what are the use cases, and how
does this discussion relate to these.

Decoration in plain txt was removed because it was distracting (it was),
and thus people avoided font changes altogether. If there was a
consensus what the TXT version is needed for, this would help us in
deciding things like these.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Aug 2021 11:09:03 +0200

> It seems like overloading this with three levels of semantics is the original sin.

Right.  When the font change was split out from a single style attribute into bits (<em, <strong, <tt), the other semantics might have been split out as well.  (There are some 64 or so combinations…)

> decorations (italic, bold, monospace), quoting (_, *, "), line breaking

You are using different terms than I did, and decoration is easily confused with decorator, so let me propose yet another set of terms:

font-change (italic, bold, monospace)
txt-fallback (_, *, “)
no-breaking syntax (break-on-space, no-break? See below.)

> From my perspective, it would be good to control each independently.  With tags.  I don't care if that is different tags, attributes on a single tag, or some combination of that with some global fl…

The spanx markup was meant to delimit a span of text for a style change.  Having txt-fallback for that without the actual style change doesn’t make a lot of sense to me; this makes an attribute on the span to control txt-fallback look good.

No-breaking is actually useful without visual delimitation, so maybe this should be considered separately.

To me it does not make sense to remove the txt-fallback default=on from <tt but keep it in <em and <strong.  I think we need to design a way to make this work for all three.  (Note that the default fallbacks for <em and <strong don’t always work, so it would be good to be able to select a different one, as is also needed with <tt.  Compare the use of >false< and >true< in https://www.rfc-editor.org/rfc/rfc8949.html#name-diagnostic-notation — we used that sick notation because the default fallback for <tt was wrong (»false« and »“false”« are two different things in CBOR diagnostic notation), but there was no way to specify a different fallback, so we opted to always have the fallback characters in there, and then we were limited to ASCII.  Ouch.)

Global flags create all kinds of problems and are best avoided.

<aside markdown=“1”>
(CCing rfc-markdown again:) I shudder about the way to indicate the fallback preference in the markdown.  Maybe this can be made almost palatable with predefined ALDs, as in

`foo`
`bar`{:nf}

(Where nf is an abbreviation for “no fallback”.)

We could also invent some new syntax, of course (and we don’t need to limit the markdown input charset to ASCII for a more readable version of the above: »bar« maybe?).

Global flags are somewhat more excusable for a keyboarding syntax, but there still would need to be a way to compose text from different sources.

Note that the question which of the attribute values are default in the markdown is entirely orthogonal to the question which are the defaults in the XML; when in doubt, I prefer to keep backwards compatibility (which would mean the default should be fallback=_/*/" for <em, <strong, <tt).
</aside>

> I personally find the current reliance on &nbsp; &nbhy; (and worse, &zwnj; of which we have one in RFC 9000) to be problematic.  

I find a zero-width space (U+200B) on 0x0100-​0x01ff [look closely after the "-"!] in the table row “CRYPTO-ERROR”, is that what you mean? (zwsp U+200B ≠ zwnj U+200C, and I don’t think we need a lot of ligature control.)

My current view is that *introducing* break points into a span is something that the Unicode spaces do reasonably.  An editor with a reveal-mode does help (haven’t primed my emacs to deal with U+200B yet, though).

> It means that the text you copy is not the text you expect which can confuse all but the smartest searcher.  I would prefer to control this with tags.  If nothing else, it would be much more explicit and less error-prone.

So you would prefer 0x0100-<preferentially-break-here/>0x01ff or some such?

> With all the effort that went into making BCP 14 not wrap,

(Do you mean the Phrase “BCP 14”, which should have an nbsp in it, or do you mean <bcp14>MUST NOT</bcp14>?)

> I note that RFC 9000 wraps between BCP and 14.  Something that RFC 9087 doesn't do - at least for the text rendering (the &nbsp; is in the XML, but not the HTML, which suggests that xml2rfc is bleaching it incorrectly).

The boilerplate says “BCP 78” without no-break as well.
Note that RFC 9087 has six occurrences of “AS path”, only one of which is nbsp-protected (but the example pathes after three of them are).

Note that there are several aspects of horizontal no-breaking:

— turn blank space into no-break spaces etc.
— don’t allow breaking after characters such as / @ & | - + # % :
(— hyphenation no-breaking, which we don’t need as we don’t do hyphenation - or should we?)

Note that one constant source of spurious rfcdiff differences is the differences in breaking on »/«.  The default could be to never break on these, but allow WJ (U+2060) to enable breaking.  But then we are used to breaking on »-«…

Grüße, Carsten

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 17 Aug 2021 07:59:15 +1000

On Mon, Aug 16, 2021, at 19:09, Carsten Bormann wrote:
> I find a zero-width space (U+200B) on 0x0100-​0x01ff 

Ah, I forgot which it was, you are right.

> So you would prefer 0x0100-<preferentially-break-here/>0x01ff or some such?

Neither.  The table layout should handle hyphens as it would in regular flow, with an override needed to suppress that behaviour.  It appears as though hyphen/punctuation breaks are not considered when doing an initial pass on column width calculations.  That said, I appreciate just how challenging it is to do this sort of thing and don't see a fix as high priority, so I can tolerate the hack.

> do you mean <bcp14>MUST NOT</bcp14>?)

I meant the phrase "BCP 14".  In LaTeX source we would have "BCP~14".  I am ambivalent on whether to discourage breaking within bcp14 elements.  They seem OK with their current implementation.

> The boilerplate says “BCP 78” without no-break as well.

Yep.

> Note that RFC 9087 has six occurrences of “AS path”, only one of which 
> is nbsp-protected (but the example pathes after three of them are).

> hyphenation - or should we?)

I would settle for a control that discouraged breaking at the usual places (spaces and punctuation).  I would use that for URLs, RFC numbers (like "RFC 9000", for which breaking should be avoided), and Section references ("Section 7.1" is often cause for &nbsp; if constructed manually).

As for the rest, I generally only want the font change.  It appears as though you usually want a default of font+decoration.  That suggests a global switch in authoring tools might be worthwhile.  The XML can do whatever it likes.

---

From: Robert Sparks <[email-masked]>
Date: Sat, 21 Aug 2021 12:55:12 -0500

Tearing out the separate topic (as Carsten already identified).

On 8/16/21 2:10 AM, Julian Reschke wrote:
>> That is indeed an interesting question, and one answer is in 
>> <https://mailarchive.ietf.org/arch/msg/tools-discuss/BK7yyQoSbhnbYDYsyzjk_J2vufg>
>>
>> After a decade of trying to suppress the TXT version, it is still 
>> considered desirable by many even after the transition that was meant 
>> to make it less relevant.  So I would expect that, independent of any 
>> outcome of the discussion of the question for the reasons, the TXT 
>> version is here to stay.
>>

There have been several times we have had long threads where people have 
told us what they use the text formats for. They've been compelling, at 
least to me, for helping ensure the format continues to be produced.

I truly don't know what you're looking for when you say "consensus what 
the TXT version is needed for". Are you looking for a document that's 
gone through some formal consensus-gauging process that iterates these 
things? Are you just looking for an easy-to-find list of things that 
people have told us? Can you point more clearly at what you think we 
should have that would help?

I note that your choice of words "what the TXT version is needed for" 
frames things differently than "what the TXT version is used for". The 
difference will make the conversation less one of defending the format 
and more of exposing what's people actually need from it.

RjS

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 22 Aug 2021 09:33:45 +1200

On 22-Aug-21 05:55, Robert Sparks wrote:

<snip>

> and more of exposing what's people actually need from it.

This. We should also remember why we wanted all this change. In
summary, we wanted a rendered format more suitable for current usage
than fixed width plain text with fixed white space and hard returns
and page feeds, which was great for Diabolo printers.

If the only thing we'd wanted to change was "plain", we'd almost
certainly have picked PDF/A as the primary rendered format, which
allows fancy text and diagrams, and scaleability. But we also wanted
reflowable text, so we picked HTML as the primary rendered format.
(XML as the canonical format was a by-product, not a driver.)

Recent RFCs rendered in HTML look really nice, are easier to read,
and *much* easier to consult on a smart phone. Mission accomplished.

It's easy enough to cite details by section number or by URL
(even figures, e.g. https://www.rfc-editor.org/rfc/rfc8989#venn1).
So losing page numbers, a feature of .txt that I've often seen
mentioned, seems unimportant. This applies to drafts as much
as to RFCs.

The only use I have made of .txt on recent drafts is to run rfcdiff.
IMHO, rfcdiff is an essential tool for document development.
I can't see anything else that amounts to a need.

    Brian

---

From: "John Levine" <[email-masked]>
Date: 21 Aug 2021 18:27:03 -0400

>least to me, for helping ensure the format continues to be produced.

Early on I thought the text version was a barbarous relic, but as I
talked to more people I see that it remains useful mostly because we
(for a very broad version of we, not limited to the IETF) do not have
adequate tools for using non-text versions. We don't have xmldiff or
xml or html grep. Several people have said that when they're
commenting on a draft or I-D, they copy and paste rectangular regions
of text, which is generally impossible in HTML in a browser. (You can
copy, but try to select a rectangle.) To do anything with a PDF beyond
display or print it needs a graduate seminar in PDF-ology.

This tells me that we need to keep the text version and it needs to have the full
contents of the document, but not that it has to be particularly beautiful.  The HTML
is where you get the beautiful version.

R's,
John

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 22 Aug 2021 11:26:07 +1200

On 22-Aug-21 10:27, John Levine wrote:
> adequate tools for using non-text versions. We don't have xmldiff 

Have you tried rfcdiff on xml2rfc files? It works remarkably well
except for needing to wrap long lines. That could be fixed. But of
course what we really need is a diff between *rendered* html
files, and that is science fiction afaik.

> or
> copy, but try to select a rectangle.) 

Huh? I don't know how to select a rectangle from a .txt file which
is not full width. Also, the cases where you actually *need* to
preserve the layout will be artwork or code. You can select those
from the rendered HTML with no problem.

> To do anything with a PDF beyond

It depends a bit which PDF viewer you use, but with the most widely
distributed one, selecting and copying from PDF/A is indeed annoying
as it loses whitespace even in artwork or code.

(However, PDF has the advantage that it supports commenting.)

   Brian

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 21 Aug 2021 23:00:58 -0400

inline

> from the rendered HTML with no problem.

I'm one of those people. Call me lazy, but here are a couple of use cases:

* As you mention, rfcdiff of *rendered* text. The side by side format is 
very convenient for tracking the changes in a doc I follow and don't 
author. A diff of the xml would be next to useless to me.

Sometimes the side by side diff doesn't show enough context.In that case 
it is trivial to find the same place in the .txt.

Once I see something in the change that I want to comment on, I can copy 
a block of consecutive lines from the .txt and paste them into an email 
to give context to a comment I want to make. The copied content is 
conveniently indented to set it off from my comments. From that email 
others can easily find the same place in the .txt.

If there were convenient tools to do the same using html renderings then 
I might no longer be attached to the .txt.

	Thanks,
	Paul

---

From: Julian Reschke <[email-masked]>
Date: Sun, 22 Aug 2021 10:55:19 +0200

Am 21.08.2021 um 19:55 schrieb Robert Sparks:
> ...

I admit that I did not consider at all *how* to get to the list. What I
wanted to say is that if we need to tune the plain text output in the
future, it would be great to have a list of things to check for.

> and more of exposing what's people actually need from it.

Yep, that was the intention.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 22 Aug 2021 10:57:47 +0200

> from the rendered HTML with no problem.

Agreed.

That said, there are a few things that will continue to be complicated
for copy and paste:

- lists (because the list decorators are auto-generated in the HTML and
currently do not survive copy/paste)

- tables

Best regards, Julian

---

From: Michael Richardson <[email-masked]>
Date: Sun, 22 Aug 2021 11:47:56 -0400

jl> xml or html grep. Several people have said that when they're
    jl> commenting on a draft or I-D, they copy and paste rectangular regions
    jl> of text, which is generally impossible in HTML in a browser. (You can
    jl> copy, but try to select a rectangle.)

Not only is the way to comment, but it's also the best thing to paste into
your source code to articulate why you are doing something weird.
(Argue copyright vs debian in another thread)

    > from the rendered HTML with no problem.

Emacs can mark non-full-width parts to copy.
Academic papers are (still) doing two-column PDF, which is so annoying to read on
the screen in fit-width.  I wish I could reflow that.
It's also really hard to copy and paste from due to being unable to select a
single column.

    > (However, PDF has the advantage that it supports commenting.)

However, the ecosystem(s) to share the comments are lacking.
The HTML version can be matched with tools like https://web.hypothes.is/
equally well.

I've done this while reviewing, but we are long way (socially) from being
able to benefit from this.  (Some of you will remember Mozilla Group
Annotations...).  It might be that they have the technology do to HTMLdiff.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Leonard Rosenthol <[email-masked]>
Date: Sun, 22 Aug 2021 18:15:16 +0000

PDF Commenting is fully defined in the PDF standard of ISO 32000.  It is supported by pretty much every major viewer out there – from those built into browsers and other UAs to stand-alone clients on desktop, mobile and web platforms.

Additionally, Web Annotation standard on which products like Hypothes.is are built, fully supports PDFs (as does Hypothes.is, IIRC).

Not sure what else you think is missing, Michael?

Leonard

From: rfc-interest <[email-masked]> on behalf of Michael Richardson <[email-masked]>
Date: Sunday, August 22, 2021 at 11:49 AM
To: [email-masked] <[email-masked]>
Subject: Re: [rfc-i] What the text version is used for (was Re: The <tt> train wreck)

However, the ecosystem(s) to share the comments are lacking.
The HTML version can be matched with tools like https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fweb.hypothes.is%2F&amp;data=04%7C01%7Clrosenth%40adobe.com%7C87484dcfeaa443ff4eda08d965844037%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C637652441422430265%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=z679Nc2kLiP7l9JYoiFMsdG5%2F9iGXSRHCjehrC8eB44%3D&amp;reserved=0
equally well.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 22 Aug 2021 15:05:28 -0400

> Not sure what else you think is missing, Michael?

The social agreement on using it.

---

From: tom petch <[email-masked]>
Date: Sun, 22 Aug 2021 20:39:33 +0100

<snip>
> mentioned, seems unimportant. This applies to drafts as much

May be but when the section spans what would be 50 pages were there to 
be pages, then I find it somewhat difficult, especially when the same 
text e.g. 'leaf name', appears in multiple places.  I refer, of course, 
to YANG modules.  Perhaps YANG version 2.1 will introduce the concept of 
sections.

Tom Petch

> .

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 23 Aug 2021 08:36:07 +1200

> to YANG modules.  Perhaps YANG version 2.1 will introduce the concept of 

Shouldn't we be handling YANG modules differently, i.e. using some
system for handling diffs and issues in code? Handling them as if
they are English text make no sense.

   Brian

---

From: tom petch <[email-masked]>
Date: Mon, 23 Aug 2021 12:51:54 +0100

> they are English text make no sense.

Yes!  Most of the languages I have dealt with have the concept of 
sequence numbers, so that when you drop the card deck on the floor, you 
can sort it back into order, but that is now history!  I note that some 
AD reviews use line numbers for their comments, I think generated by ID 
nits.

The better authors of YANG, like the better authors of other languages, 
make extensive use of comments which can then be used for identifying 
'sections' (as well as providing a narrative that can be read to get an 
overview of the module).  Perhaps that needs formalising.

Tom Petch

---

From: tom petch <[email-masked]>
Date: Mon, 23 Aug 2021 12:59:18 +0100

> least to me, for helping ensure the format continues to be produced.

I was going to pick up on this but the thread seemed to move on, so 
thank you for returning to it.  txt is simple, straightforward, easy to 
use, universal.

txt is compact; not as compact as Shannon would allow but I have seen 
pdf in the IETF that are some 50 times the size of the txt version. Yes, 
computers get bigger and faster but not that much.

txt can be processed on every device I can ever recall having used, give 
or take the ASCII/EBCDIC conversion and some awareness of code pages (my 
dollar is your cent) and that applies to display, to editing and to 
printing.  It is simple, supported everywhere, can be used for interchange.

It encourages people to be clear in their language.  I have seen 
diagrams lately of the kind
I---I     I---I
I   I --->I   I
I---I     I---I
with no explanation of the meaning of the arrow and, when I sought 
clarification, it transpired that the meaning was the exact opposite of 
what I had assumed!  And this is a general issue, the richer the 
language that is used in a specification, the more comprehension is 
assumed in the reader. At a slight tangent, the logic of AND and OR are 
widely understood, NAND or NEQ less so even if they are arguably simpler.

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Aug 2021 14:31:16 +0200

> overview of the module).  Perhaps that needs formalising.

Would adding optional line numbers to artwork/sourcecode help?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Aug 2021 14:35:01 +0200

OK, I'll bite.

> computers get bigger and faster but not that much.

Example? (The only reasons I can think of are either scanned documents,
or documents with lots of graphics not present in the TXT version).

> printing.  It is simple, supported everywhere, can be used for interchange.

Well, depends on "processing". How well does your smart phone "process"
plain text formatted for printing on A4/letter?

> diagrams lately of the kind

That is a problem, but the example looks exactly like it was taken from
plain text :-).

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Mon, 23 Aug 2021 17:00:28 +0100

> Would adding optional line numbers to artwork/sourcecode help?

With YANG, I am not sure where they would go.  A lot of YANG modules 
(most?) bump up against the RFC line length limit because the language 
is heavily nested and the convention is to indent further each level of 
nesting; and there are path statements which, with such nesting, can 
exceed the line length limit on their own so a sequence number in 
columns 72-80 (e.g.) would not work.  (There is already an RFC8792 which 
provides a convention for dealing with long lines but I do not find that 
that makes for a document that is easy to read).

Rather inserting a comment line every 50 or so lines (or at the end of 
what used to be a page) such as
/* line 1250*/
would work for me.  An example of a YANG module comes in
draft-ietf-i2nsf-nsf-monitoring-data-model
which is 86pp of which 41 are the model itself.  This is an author whose 
used of comment lines is sparing but had no need to split long lines. 
The AD, Roman, comments are on the i2nsf list 25jun21; I can marry those 
that have no section numbers to the text but then I have studied and 
commented on the I-D previously.

Tom Petch

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Aug 2021 18:09:49 +0200

How about inserting schema tree identifiers?
Much more stable.

Grüße, Carsten

> On 2021-08-23, at 18:00, tom petch <[email-masked]> wrote:
> 
> Rather inserting a comment line every 50 or so lines (or at the end of what used to be a page) such as
> /* line 1250*/
> would work for me.

---

From: tom petch <[email-masked]>
Date: Mon, 23 Aug 2021 17:33:57 +0100

> How about inserting schema tree identifiers?

I do not understand.  The YANG is the schema, those 41 pages are the 
structure of the tree, the identifiers, their syntax and semantics.

Tom Petch

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Aug 2021 21:38:45 +0200

On 23. Aug 2021, at 18:33, tom petch <[email-masked]> wrote:
> I do not understand.  The YANG is the schema, those 41 pages are the structure of the tree, the identifiers, their syntax and semantics.

Yes, this was a bit terse.

What we essentially are discussing here is a way to point into a YANG specification.
Line numbers have been mentioned as one such way.
We do have a fragment identifier syntax for lines in (at least a plain) text file (RFC 5147).
But these are brittle.  Adding a line number every, say, 70 lines would also be disruptive.

However, a YANG spec is a structured document.  So why not employ the structure of that document as a fragment identifier syntax?
Adding a full schema tree identifier in certain places (left as exercise for the reader) would provide a nice way to point into the YANG spec.
(Of course, that identifier is also known when it is not added to the YANG spec, but with it added simple text searching would work.)
So what I’m proposing is defining a fragment identifier syntax for YANG specs plus a convention to include these (typically as part of a pretty-printing step) to enable text-based searching.

(And, as a side effect, having the full schema tree identifier handy would also help with reading through 40-page YANG specs.)

Grüße, Carsten

---

From: "John Levine" <[email-masked]>
Date: 23 Aug 2021 17:27:43 -0400

>or documents with lots of graphics not present in the TXT version).

For the RFCs published since we switched xml, the total text is 25MB, the html
is 59MB, and the PDFs are 120MB.  They're larger but it doesn't seem like a
qualitative difference to me.

>plain text formatted for printing on A4/letter?

Even on a stock Windows or Mac laptop, the tools to handle text files are pretty poor.

We have tools that only work on the text rendering of RFCs, most notably rfcdiff.
But that means we need to improve our tool set, not that we should be stuck with
fixed pitch text forever.

R's,
John

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 24 Aug 2021 11:34:14 +1200

...
> Would adding optional line numbers to artwork/sourcecode help?

That's better than nothing, for sure. However, I'm old enough to
remember that old-fashioned programmers would initially number
their Fortran statements 00100, 00200, 00300 etc., so that when
adding new statements to fix a bug, the numbers could stay in
order and the rest of the card deck didn't need to be renumbered.
Sequential numbering is not the perfect answer.

   Brian

---

From: tom petch <[email-masked]>
Date: Tue, 24 Aug 2021 09:48:38 +0100

> However, a YANG spec is a structured document.  So why not employ the structure of that document as a fragment identifier syntax?

Mmm, later last night I thought that that was what you meant.  It would 
be fine with SMI, a single root, world-wide tree with compact 
identifiers for every node (1.3.6.1. ....)

YANG is different.  It can have multiple top level nodes in every module 
while also, in a sense, having none as when the module augments 50 or 
more nodes in another module.  Thus in CCAMP and TEAS modules such as
  draft-ietf-ccamp-flexigrid-media-channel-yang-03
(an author who makes good use of comment statements)
you get
      augment "/te:te/te:tunnels/te:tunnel/"
            + "te:p2p-primary-paths/te:p2p-primary-path/"
            + "te:p2p-primary-reverse-path/"
            + "te:optimizations/te:algorithm/te:metric/"
            + "te:optimization-metric/te:explicit-route-exclude-objects/"
            + "te:route-object-exclude-object/te:type/te:label/"
            + "te:label-hop/te:te-label/te:technology" {
or
      augment "/te:te/te:tunnels/te:tunnel/"
            + "te:p2p-primary-paths/te:p2p-primary-path/"
            + "te:p2p-primary-reverse-path/"
            + "te:optimizations/te:algorithm/te:metric/"
            + "te:optimization-metric/te:explicit-route-include-objects/"
            + "te:route-object-include-object/te:type/te:label/"
            + "te:label-hop/te:te-label/te:technology" {
(spot the difference between the two) and the same augment can appear 
more than once so for uniqueness you need to know what comes next which 
may be an import from another module (as it is here); and you need to 
know the mapping of prefix (te:) to module name since it is module name 
that uniquely identifies the namespace.   Some authors think that they 
know better and invent their own prefix to refer to the module of 
another author so identifying a node in the tree can take 10 or more 
lines of text, the path, the mapping of prefix to module and so on.

And it gets worse.  You may have mastered the multi-root tree with 10 or 
more depths and inconsistent identifiers to arrive at a single node in 
the schema tree but then you need to find its definition in the 
documents. YANG allows 'subroutines', groupings which can be used 
anywhere and just as some programmers grossly overuse subroutines there 
is a growing fashion to make everything a grouping and place them at the 
start of a module, or in a separate common module, so you have to scroll 
back and forth multiple documents in order to locate the definition of 
this one leaf in the tree.

(In the module I cite there is a comment line for each augment and, like 
a page number, that provides a clear and quick locator but other authors 
have no comment lines.)

Like any language, YANG offers much scope for misuse.

Tom Petch

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Aug 2021 10:58:27 +0200

On 2021-08-24, at 10:48, tom petch <[email-masked]> wrote:
> 
> compact identifiers for every node 

We’ll have these soon with YANG SIDs.
So maybe this is the best replacement for a line number...

Grüße, Carsten

---

From: tom petch <[email-masked]>
Date: Tue, 24 Aug 2021 11:29:04 +0100

John

The I-D that caught my eye some time ago, but stayed in my mind, was
draft-mansfield-netmod-uml-to-yang
which appeared in the repository as a pdf of 2.826Mbyte. Copying the 
text and saving it as such the file is 80kbyte; ok, only a 35-fold 
increase and for an ephemeral I-D not an RFC.

I have no way of looking into a .pdf to see what has happened.  I know 
for example that an e-mail can grow tenfold when the sender includes the 
entire MS Office style sheet and I assume that similar things are 
possible with .pdf.

As you may gather, I am not an optimist.  'Smile things could be worse 
and I smiled, and Lo, things were worse'.  Sums up my experience of 
trying to contribute to the IETF over the past 21 months with a number 
of issues, some generated by the IETF, others of a more generic nature. 
  Were .txt to go I think I would write off the IETF as unsupportable:-(

Tom Petch

---

From: Julian Reschke <[email-masked]>
Date: Tue, 24 Aug 2021 12:34:10 +0200

> entire MS Office style sheet and I assume that similar things are

Maybe
<https://www.ietf.org/archive/id/draft-mansfield-netmod-uml-to-yang-02.pdf>?

That appears to contain bitmap graphics which clearly (:-) are not in
the plain text version.

Best regards, Julian

---

From: Michael Richardson <[email-masked]>
Date: Tue, 24 Aug 2021 14:26:14 -0400

> However, a YANG spec is a structured document.  So why not employ the
    > structure of that document as a fragment identifier syntax?

That works for valid YANG, and we should do that.
And the HTML should get anchors in... but that's into "sourcecode",
"artwork", etc.

For internet-drafts (what are they? who cares...)   where the YANG might not
validate (or might not validate with the version of pyang that the DT uses),
how does not reference the problem part?

    > (And, as a side effect, having the full schema tree identifier handy
    > would also help with reading through 40-page YANG specs.)

It would be nice if we could reflow the YANG and syntax highlight it.

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Aug 2021 20:39:05 +0200

> On 2021-08-24, at 20:26, Michael Richardson <[email-masked]> wrote:
> 
> 

I never understood why people are submitting drafts with formal descriptions that don’t even parse.  I’d suggest we stop doing that.

> It would be nice if we could reflow the YANG and syntax highlight it.

Syntax highlighting is one of the applications of type= (and one of the reasons why artwork needs to have a proper replacement for type=, given that type= means something different for artwork).

Reflowing:
I seem to remember that YANG specs are regularly pretty-printed (reformatted) before publishing them; what would “reflowing” add to that?  Reflowing comments (descriptions etc.)?  That would require comments to have a reflowable syntax (e.g., one of the markdown flavors).

Grüße, Carsten

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 24 Aug 2021 15:28:17 -0400

> I never understood why people are submitting drafts with formal descriptions that don’t even parse.  I’d suggest we stop doing that.

I can't speak for YANG, but for ABNF there are a reasons:

1) snippets of ABNF tend to be interspersed in text discussions. They 
sometimes are just examples, where the context is understood from the 
text. Other times a "complete" ABNF is introduced bit by bit across the 
document. In this case the snippets are are typically small, so they 
don't, by themselves, make referencing difficult.

2) very often (in my experience, most of the time) the ABNF in one 
document builds on the ABNF in one or more other documents. Comments in 
the ABNF or accompanying text say "See RFCnnn for the definition of 
FOO". It can take considerable work to pull together all the referenced 
pieces that are necessary for the ABNF to verify. (In this case the ABNF 
is well formed except that some terms are undefined.)

Also, I find problematic the idea of needing source-language-specific 
ways of constructing references to points within a document. It presumes 
that a person reviewing who needs to make or follow a reference will be 
familiar with the specific language mechanism. Oft times this is not a 
reasonable assumption.

For instance, I do my best to avoid any document that has YANG in its 
title, but sometimes I can't. (E.g., when doing a genart review.) I'm 
not going to be making any deep comments on the YANG, but I may still 
need to reference it. If so, I'm not going to be using a YANG-specific 
mechanism to do so. Cutting a a seemingly unique block of the document 
and pasting it into my review is easier and works for anything.

	Thanks,
	Paul

---

From: tjw ietf <[email-masked]>
Date: Tue, 24 Aug 2021 15:32:59 -0400

I’ve been thinking like we have bib references we should have a similar service for ABNF. 

(Who spent some time understanding the ABNF import refs for DMARC)

Sent from my iPhone

> On Aug 24, 2021, at 15:28, Paul Kyzivat <[email-masked]> wrote:
> 
> 1) snippets of ABNF tend to be interspersed in text discussions. They sometimes are just examples, where the context is understood from the text. Other times a "complete" ABNF is introduced bit by bit across the document. In this case the snippets are are typically small, so they don't, by themselves, make referencing difficult.
> 
> 2) very often (in my experience, most of the time) the ABNF in one document builds on the ABNF in one or more other documents. Comments in the ABNF or accompanying text say "See RFCnnn for the definition of FOO". It can take considerable work to pull together all the referenced pieces that are necessary for the ABNF to verify. (In this case the ABNF is well formed except that some terms are undefined.)
> 
> Also, I find problematic the idea of needing source-language-specific ways of constructing references to points within a document. It presumes that a person reviewing who needs to make or follow a reference will be familiar with the specific language mechanism. Oft times this is not a reasonable assumption.
> 
> For instance, I do my best to avoid any document that has YANG in its title, but sometimes I can't. (E.g., when doing a genart review.) I'm not going to be making any deep comments on the YANG, but I may still need to reference it. If so, I'm not going to be using a YANG-specific mechanism to do so. Cutting a a seemingly unique block of the document and pasting it into my review is easier and works for anything.
> 
>    Thanks,
>    Paul

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Aug 2021 21:36:34 +0200

> I’ve been thinking like we have bib references we should have a similar service for ABNF. 

Yep.  We need the same thing for CDDL (which is almost but not entirely unlike ABNF), so expect some drafts from the CBOR WG…

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 24 Aug 2021 21:52:37 +0200

Hi Paul,

Indeed, dealing with snippets (some of which are *deliberately* incorrect!) is not what I meant here.  Having a 40-page YANG spec in a document that didn’t have the benefit from some tool usage is not an efficient use of everyone’s time.

On 2021-08-24, at 21:28, Paul Kyzivat <[email-masked]> wrote:
> 
> I'm not going to be using a YANG-specific mechanism to do so.

Yes, one of the goals would be to have anchors in the source code the syntax of which is independent of the source code language (they would need to be in a comment of the specific source code language but stand for itself; see PoC below).  For YANG, these could be derived from schema identifiers or SIDs.  For ABNF, the production names are already good referencing anchors (that are probably quite accessible for non-aficionados, too), so it would not need to share that generic fragment identifier anchoring syntax.

> Cutting a a seemingly unique block of the document and pasting it into my review is easier and works for anything.

Reliable copy/paste is indeed one of the most important remaining functions that makes plaintext IETF documents indispensable.

Grüße, Carsten

PoC, assuming the syntax of anchors is “##” 1*DIGIT and YANG’s // comment syntax is being used:

   container system-state {                                //##1720

     container clock {                                     //##1721
       leaf current-datetime {                            
         type date-and-time;
       }

       leaf boot-datetime {                                //##1722
         type date-and-time;
       }
     }
   }

---

From: Michael Richardson <[email-masked]>
Date: Tue, 24 Aug 2021 21:13:18 -0400

> I never understood why people are submitting drafts with formal
    > descriptions that don’t even parse.  I’d suggest we stop doing that.

a) because trying to get help with fixing it.

b) because it has a feature that the *DT*'s copy of pyang doesn't parse
   (yet). But, the local version does.

c) because it's a ID about some part of yang that we are enhancing, so the
   YANG is a snippet, and isn't intended to be complete on it's own.

    > Syntax highlighting is one of the applications of type= (and one of the
    > reasons why artwork needs to have a proper replacement for type=, given
    > that type= means something different for artwork).

Yes.

    > Reflowing:
    > I seem to remember that YANG specs are regularly pretty-printed
    > (reformatted) before publishing them; what would “reflowing” add to
    > that?  Reflowing comments (descriptions etc.)?  That would require
    > comments to have a reflowable syntax (e.g., one of the markdown
    > flavors).

Yes, the descriptions.  It's a disaster coordinating what's in the YANG
description, and what's in the document, and keeping them in sync.  How much
detail do you put in each part?

---

From: Bob Hinden <[email-masked]>
Date: Tue, 24 Aug 2021 19:21:25 -0700

Hi,

> On Aug 21, 2021, at 3:27 PM, John Levine <[email-masked]> wrote:
> ….

I agree with this.    I use text file version of drafts quite a lot currently.  This includes:

- rfcdiff to compare two different versions, this includes:
   o After running xml2rfc to see the changes
   o Looking at a new text output in emacs (where I edit the xml)
   o Looking at the changes when a new draft is published
   o Emailing a link to a diff of two text versions
- Excerpts from text drafts paste into email as part of list discussions
- I even occasionally print a text version from emacs (ps-print) if
  I want to read and markup a paper version.

If there were equivalent tools that worked with one of the other formats (html or pdf), I might be able to migrate to using that.   As far as I can tell that doesn’t exist today.

Bob

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 25 Aug 2021 12:34:26 +1000

On Wed, Aug 25, 2021, at 12:21, Bob Hinden wrote:
> I use text file version of drafts quite a lot currently.  This includes:
> 
> If there were equivalent tools that worked with one of the other 
> formats (html or pdf), I might be able to migrate to using that.   As 
> far as I can tell that doesn’t exist today.

I will observe that with the exception of diffs, I use HTML for all of the things he lists.  I find that most of these use cases work better from HTML than text.  For instance, I find the added spacing and line breaks in the text format occasionally awkward for pasting quotations into other mediums.

Diffs of HTML really don't work well, so I find that plain, unpaginated text is best for that.  Using unpaginated text means that I can use ordinary diff tools rather than rfcdiff, which gives me a lot more choice.  That said, I do find the online service that diffs arbitrary URLs to be quite convenient and I do use diffs a lot.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Aug 2021 08:50:53 +0200

>    I want to read and markup a paper version.

The HTML is supposed to print well (I know that rfc2629.xslt's output does).

Best regards, Julian

---

From: tom petch <[email-masked]>
Date: Wed, 25 Aug 2021 12:49:54 +0100

>     YANG is a snippet, and isn't intended to be complete on it's own.

d) because the fault is in someone else's document that this is building 
on so get on with what we can while waiting for the author of the other 
document (months or longer).

Tom Petch

---

From: tom petch <[email-masked]>
Date: Wed, 25 Aug 2021 12:59:25 +0100

<snip>

> detail do you put in each part?

I keep forgetting the obvious:-(

The YANG module be it 5 pages or 50 is a separate document.  It MUST be 
plain text that can be extracted and circulate as an independent 
document.  The descriptions, the references, the URL and so on must make 
sense when it is not part of an RFC.  The descriptions must tell the 
user what an object is with a reference to where more can be found, 
sometimes the 'mothership' RFC, usually not.

My take is that it is like documenting code, the description are what a 
good programmer places in the code with the rest of the RFC providing 
context, objectives, options rejected and so on.

The issue is how to reference into this plain text be it for a query, 
comment, erratum and so on.  As I said before, YANG is not structured 
the way SMI is so a reference to a leaf may take 10 or so lines of YANG!

Comment lines, which good authors use (as with code), are more compact 
(if a little bit too free form).

Tom Petch

---

From: Bob Hinden <[email-masked]>
Date: Wed, 25 Aug 2021 09:10:29 -0700

Martin,

> On Aug 24, 2021, at 7:34 PM, Martin Thomson <[email-masked]> wrote:
> 

I agree.   Diffs appear to be the main reason to keep using text at this point in time.

Bob

---

From: Robert Sparks <[email-masked]>
Date: Wed, 25 Aug 2021 13:59:44 -0500

I remember EKR mentioning that he used the plain text version when doing 
reviews, as it loaded well into the code-review tool he was co-opting.

Are other people using the text this way?

RjS

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 25 Aug 2021 15:59:33 -0400

> I will observe that with the exception of diffs, I use HTML for all of the things he lists.  I find that most of these use cases work better from HTML than text.  For instance, I find the added spa…

While copy/paste in general works reasonably well with HTML, I'm a 
ludite in that I avoid using it for email, especially on IETF lists. I 
find it messes up the flow of quoting in extended conversations. So I 
want .txt snippets to put in my emails. Pasting HTLM as txt gives the 
worst of both.

	Thanks,
	Paul

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 25 Aug 2021 13:06:37 -0700

> reviews, as it loaded well into the code-review tool he was co-opting.
I've stopped using Phab so much, but yeah, that's what I did. I also just
do that with emacs.

I've been following this thread but tbh I'm having trouble understanding
what's at stake here. It seems like there are a lot of different uses of
the text format and we have tools that generate it. Are we really
considering deprecating it? Perhaps, we can just leave it more or less
as-is?

-Ekr

RjS
> On Aug 24, 2021, at 7:34 PM, Martin Thomson <[email-masked]> <[email-masked]> wrote:
>
> rfc-interest mailing [email-masked]://www.rfc-editor.org/mailman/listinfo/rfc-interest
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 25 Aug 2021 22:23:50 +0200

On 25. Aug 2021, at 22:06, Eric Rescorla <[email-masked]> wrote:
> 
> I've been following this thread but tbh I'm having trouble understanding what's at stake here. It seems like there are a lot of different uses of the text format and we have tools that generate it. Are we really considering deprecating it? Perhaps, we can just leave it more or less as-is?

Indeed: Clearly, plaintext is not going away.

> is where you get the beautiful version.

The root of this discussion was the situation we have created around the <tt> element.
This has recently led to an incompatible change (ticket 600) to its rendering that entirely seems to be motivated by making the plaintext more beautiful.

Quoting you again:

> Perhaps, we can just leave it more or less as-is?

This.
*We need to back out the change made in ticket 600 before it gets deployed*.

Grüße, Carsten

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 25 Aug 2021 13:26:21 -0700

Sounds like we're on the same page. I believe that the plain text is about
as good as it's going to get and so mostly not breaking it seems like the
requirement.

-Ekr

On Wed, Aug 25, 2021 at 1:23 PM Carsten Bormann <[email-masked]> wrote:

> This has recently led to an incompatible change (ticket 600) to its
> rendering that entirely seems to be motivated by making the plaintext more
> beautiful.
>
> Quoting you again:
>

---

From: Robert Sparks <[email-masked]>
Date: Wed, 25 Aug 2021 15:56:35 -0500

Changing the subject again, because this is really now about tt, and the 
other thread I do want to see continue.

The principle of not breaking the plain text doesn't really help with 
this one - it's a new v3 artifact, and there are complaints about how 
<tt> has been rendering in v3 plain text to date.

Ticket 300 doesn't really capture the history of the issue that led to 
the change well. https://trac.ietf.org/trac/xml2rfc/ticket/647 does a 
little better - there Martin makes the request to have the quotes 
removed. We got to it, I think, from v2v3 turning v2 <spanx 
style=“verb”> into v3 <tt>.

John did a survey of what the RFCs published so far and found that the 
would have been legible without the quotes, so we moved ahead to do that.

As you can see Carsten doesn't think that's the right thing to do. He's 
pointed to some cases that could be constructed where not having the 
quotes would hurt, but it looks like those could be avoided. He's also 
proposed that we add more format-specific layout tweaking attributes, 
which is something the whole v3 effort has worked to avoid.

I'm not trying to argue for any outcome here - just trying to shed more 
light on what this part of the conversation is about.

RjS

On 8/25/21 3:26 PM, Eric Rescorla wrote:
> Sounds like we're on the same page. I believe that the plain text is 
> about as good as it's going to get and so mostly not breaking it seems 
> like the requirement.
>
> -Ekr
>
>
> On Wed, Aug 25, 2021 at 1:23 PM Carsten Bormann <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     On 25. Aug 2021, at 22:06, Eric Rescorla <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > I've been following this thread but tbh I'm having trouble
>     understanding what's at stake here. It seems like there are a lot
>     of different uses of the text format and we have tools that
>     generate it. Are we really considering deprecating it? Perhaps, we
>     can just leave it more or less as-is?
>
>     This has recently led to an incompatible change (ticket 600) to
>     its rendering that entirely seems to be motivated by making the
>     plaintext more beautiful.
>
>     Quoting you again:
>

---

From: Robert Sparks <[email-masked]>
Date: Wed, 25 Aug 2021 16:04:53 -0500

> Indeed: Clearly, plaintext is not going away.

I feel the same, but I don't believe everyone in the conversation does. 
It feels to me that there are people who believe it should be a goal to 
work to turn the text format off. Am I wrong?

---

From: Lars Eggert <[email-masked]>
Date: Thu, 26 Aug 2021 13:58:07 +0300

On 2021-8-25, at 21:59, Robert Sparks <[email-masked]> wrote:
> I remember EKR mentioning that he used the plain text version when doing reviews, as it loaded well into the code-review tool he was co-opting.
> 

I do.

Thanks,
Lars

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 Aug 2021 20:24:40 +0200

On 2021-08-25, at 22:56, Robert Sparks <[email-masked]> wrote:
> 
> Changing the subject again, because this is really now about tt, and the other thread I do want to see continue.
> 
> The principle of not breaking the plain text doesn't really help with this one - it's a new v3 artifact, and there are complaints about how <tt> has been rendering in v3 plain text to date.

<tt is the direct replacement of <spanx style=“verb”.
The only thing “new” about <tt is that it combines with <em and <strong, which together build a bitset of three bits that can be set or reset (<sub/<sup are also roughly in the same set, but are more than bits).

<tt sailed through the v3 process because it is that direct replacement; it is otherwise hard to understand why this would go into the same bucket with <em and <strong, which are named in a more “semantic” way.
(The main application of <tt is called <code in html, but there are also <kbd, <samp, <var, <dfn, <cite, some of which are closer to how we use <em.)

Reviewers of my specs have repeatedly noted that the plaintext-fallback for <em or <strong is jarring.  We aren’t we “fixing” (amputating the plaintext-fallback for) these as well?

Inventing new processing rules for <tt without even considering <em and <strong strikes me as another instance of the weird way v3 ripened.

I think the discussion we had in the other thread was useful to recognize that plaintext is here to stay, and that we will *not* fall for beautifying the plaintext.

>> > is where you get the beautiful version.

So having ugly plaintext fallbacks is OK as long as these are consistent, so a reader can create a mental image of what was meant.  We already have damaged that consistency for <sub and <sup for certain “simple” cases, as well as for table cells that carry only a <tt.  This is the path we should *not* go further.

Completely removing from the plaintext form the information that is embodied by the <tt tags is so outlandish that I indeed struggled arguing against it.

Of course, we could deprecate <tt entirely (and there are probably good arguments for that), but if we want to keep it, we can’t tell authors “you can use it, but the plaintext version won’t show the full contents of the document” (in particular not retroactively, with documents that actually discuss the plaintext rendering and will be starting to lie about themselves after the change is deployed).

(The proper fix is to give the authors a way to influence the fallback, but that seemed to be hard to get consensus for with the previously dominating view that plaintext will just vanish.)

Grüße, Carsten

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 27 Aug 2021 10:52:12 +1000

On Fri, Aug 27, 2021, at 04:24, Carsten Bormann wrote:
> Reviewers of my specs have repeatedly noted that the plaintext-fallback 
> for <em or <strong is jarring.  We aren’t we “fixing” (amputating the 
> plaintext-fallback for) these as well?

Sounds like a good idea to me.
 
I think that you have the wrong end of this.  I can agree that the text form can be ugly or uglier than HTML, but we have to acknowledge the process that authors will use.

Authors will look at the output (text or html or both) and adjust their input to get the outcome they desire.  Inconsistency in presentation between formats makes this process unreliable.  If authors only look at one format, that will dictate their inputs and the other will suffer.

If quotes are desired, then it is reasonable and appropriate to have authors type quotes into to the source.  That leads to less reliance on the automatic decoration and pushes toward fewer decorations for different semantic elements.  It means that the ugliness innate in text renderings is less obvious differentiation for different "decoration" elements (<tt, <em, <strong, <aside, ...).

---

From: "John Levine" <[email-masked]>
Date: 26 Aug 2021 21:36:33 -0400

It appears that Martin Thomson  <[email-masked]> said:
>Authors will look at the output (text or html or both) and adjust their input to get the outcome they desire.  Inconsistency in presentation between
>formats makes this process unreliable.  If authors only look at one format, that will dictate their inputs and the other will suffer.
>
>If quotes are desired, then it is reasonable and appropriate to have authors type quotes into to the source.  That leads to less reliance on the
>automatic decoration and pushes toward fewer decorations for different semantic elements.  It means that the ugliness innate in text renderings is less
>obvious differentiation for different "decoration" elements (<tt, <em, <strong, <aside, ...).

After thinking about it for a while, this is where I ended up, too.
The <tt> tag was clearly a mistake, both because no two authors agree
about where to use it, and because there is no way to render it in
text that's already fixed pitch without adding confusion. (See
yesterday's IANA message asking about whether to include the quotes.)
If you really care about font changes and superscripts and the like,
we have two other lovely renderings you can use.

This is also a hint that it is not a great idea to write drafts in a
way that needs fancy typography to make sense. Beyond the text
rendering issues, it's also an issue for screen readers and other
accessibility tools.

R's,
John