Subject: [rfc-i] Unicode in xml2rfc v3

From: Lars Eggert <[email-masked]>
Date: Tue, 1 Dec 2020 19:35:13 +0200

Hi,

we're at the moment revising RFC8312, the CUBIC congestion controller for TCP. There is quite a bit of math in RFC8312, and even more in the papers that describe CUBIC, some of which we're planning to roll into RFC8312bis.

That math uses a bunch of greek letters, which we needed to ASCIIfy for RFC8312. One thing I was planning to do for the bis was to reduce the difference between the paper and the RFC, by using the greek letters in the RFC. Since, you know, xml2rfc v3 is supposed to handle Unicode.

Except, as most of you probably know but I didn't, you can't just use Unicode. You need to wrap it in a <u> tag, and it must be rendered with "num" in the format.

Which means that while I can write a formula in kramdown-rfc2629:

~~~ math
W_{est} = W_{est} + α_{aimd} * \frac{segments_{acked}}{cwnd}
~~~

and have it automatically rendered in SVG as (via tex2svg):

and in ASCII (via asciitex) as

                       segments
                               acked
W    = W    + α     * -------------
 est    est    aimd        cwnd

but I cannot use "α<sub>aimd</sub>" in the text of the RFC - it gets rendered as "&#945;_(aimd)".

And using "<u>α</u><sub>aimd</sub>" gets rendered as the even uglier ""α" (GREEK SMALL LETTER ALPHA, U+03B1)_(aimd)". I can of course play with the format string, but since there is no way to not at least use "char", I have basically no option here.

I had really been hoping that v3 would enable better math. Unicode is not just for names...

Thanks,
Lars

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 1 Dec 2020 18:58:07 +0100

Hi Lars,

The way to go, given the current limitations, is to provide the equations
within <artwork>.  This would work both for unicode text artwork (with the
limitation that it will use a monospaced font) and svg artwork, if you
can produce that from the math formula input.

I have an idea of how to improve math support substantially, and I strongly
believe it's needed and appropriate, but for now the approach above is what
is available.

Looking to the future, I also mentioned Math support in the new work section
of my implementation notes:
https://tools.ietf.org/html/draft-levkowetz-xml2rfc-v3-implementation-notes-11#section-5.1

Best regards,

	Henrik

On 2020-12-01 18:35, Lars Eggert wrote:
> Hi,
> 
> we're at the moment revising RFC8312, the CUBIC congestion controller for TCP. 
> There is quite a bit of math in RFC8312, and even more in the papers that 
> describe CUBIC, some of which we're planning to roll into RFC8312bis.
> 
> That math uses a bunch of greek letters, which we needed to ASCIIfy for RFC8312. 
> One thing I was planning to do for the bis was to reduce the difference between 
> the paper and the RFC, by using the greek letters in the RFC. Since, you know, 
> xml2rfc v3 is supposed to handle Unicode.
> 
> Except, as most of you probably know but I didn't, you can't just use Unicode. 
> You need to wrap it in a <u> tag, and it must be rendered with "num" in the format.
> 
> And using "<u>α</u><sub>aimd</sub>" gets rendered as the even uglier ""α" (GREEK 
> SMALL LETTER ALPHA, U+03B1)_(aimd)". I can of course play with the format 
> string, but since there is no way to not at least use "char", I have basically 
> no option here.
>

---

From: Lars Eggert <[email-masked]>
Date: Tue, 1 Dec 2020 20:22:25 +0200

Hi,

> can produce that from the math formula input.

math in figures works fine, actually. See the examples I had in the original email.

The problem is that one cannot use the same Greek variable names from such math figures in the body text.

Thanks,
Lars

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 1 Dec 2020 20:02:06 +0100

> The problem is that one cannot use the same Greek variable names from such math figures in the body text.

Well, with the current toolset, you can.

https://mailarchive.ietf.org/arch/msg/xml2rfc/48tS5guJqVpjGUkvPeD7K2aViis/

Grüße, Carsten

---

From: Lars Eggert <[email-masked]>
Date: Wed, 2 Dec 2020 08:35:39 +0200

Hi,

> https://mailarchive.ietf.org/arch/msg/xml2rfc/48tS5guJqVpjGUkvPeD7K2aViis/

thanks! Could I maybe ask that we start an FAQ or HOWTO for kramdown-rfc2629 to capture some of these not-so-obvious tricks and tips? Or add a canonical example Markdown document that uses them all?

Also, using the {{{α}}} trick doesn't seem to work in a list. This source:

- set {{{α}}}<sub>aimd</sub> to 1 once W<sub>est</sub> reaches W_max (#2)

gets converted to:

<li>set <contact fullname="α"/><sub>aimd</sub> to 1 once W<sub>est</sub> reaches W_max (#2)</li>

and throws this error:

draft-eggert-tcpm-rfc8312bis.xml(2402): Error: Element li has extra content: contact, at /rfc/back/section[2]/section[1]/ul/li[6]/contact

Is there another workaround?

Thanks,
Lars

---

From: Lars Eggert <[email-masked]>
Date: Wed, 2 Dec 2020 08:51:35 +0200

Hi,

> draft-eggert-tcpm-rfc8312bis.xml(2402): Error: Element li has extra content: contact, at /rfc/back/section[2]/section[1]/ul/li[6]/contact

and outside a list context, the error is:

/Users/lars/Documents/IETF/rfc8312bis/draft-eggert-tcpm-rfc8312bis.xml(667): Error: Expected an asciiFullname= attribute to match the unicode fullname value

So the {{{α}}} doesn't seem to work at all?

Thanks,
Lars

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 07:57:23 +0100

> On 2020-12-02, at 07:35, Lars Eggert <[email-masked]> wrote:
> 
> Hi,
> 

I’m long overdue with starting an I-D about kramdown-rfc2629.
This particular trick has been recently invented though/

> draft-eggert-tcpm-rfc8312bis.xml(2402): Error: Element li has extra content: contact, at /rfc/back/section[2]/section[1]/ul/li[6]/contact

Hmm, that is an xml2rfc bug.

<contact/> has only been added to <section/> (block-level) and <t/>, not to <li/>, which has its own copy of the content model of <t/>.  There are a few more elements like <name/>, <blockquote/>, <dt/>, <dd/>, <xref/>, <cref/>, <tt/>, <strong/>, <em/>, <sub/>, <sup/>, <preamble/>, <postamble/>, <th/>, <td/>, <annotation/>, <refcontent/> that need this fix.

> Is there another workaround?

Put the content of <li/> into a <t/>.  Unfortunately, that appears to be removed in the v2-to-v3 converter inside xml2rfc, so that would need to be fixed (or the previous fix applied).

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 08:15:39 +0100

> So the {{{α}}} doesn't seem to work at all?

Right.  Since α is not recognized by xml2rfc as Latin, it expects an asciiFullname.  So this trick only works with what xml2rfc deems to be Latin (which happens to have been where I needed it).  Too bad.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 08:22:47 +0100

> Right.  Since α is not recognized by xml2rfc as Latin, it expects an asciiFullname.  So this trick only works with what xml2rfc deems to be Latin (which happens to have been where I needed it).  To…

Err,

set {{{α}}}{: asciiFullname=""}<sub>aimd</sub> to 1 once W<sub>est</sub> reaches W_max (#2)

I think I’ll have to wire that into {{{…}}}...

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 08:31:57 +0100

On 2020-12-02, at 08:22, Carsten Bormann <[email-masked]> wrote:
> I think I’ll have to wire that into {{{…}}}…

Oh, there already is syntax for this.

set {{{α}{}}}<sub>aimd</sub> to 1 once W<sub>est</sub> reaches W_max (#2)

{{{Voilà}}}.

Sorry for the bracefest…

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 09:27:34 +0100

pip3 install unicodeit

set $$α_{aim}$$ to 1 once $$W_{est}$$ reaches $$W_{max}$$ and $$ a < b $$
➔
   set αₐᵢₘ to 1 once Wₑₛₜ reaches Wₘₐₓ and a < b

Well, let’s call that a spike(*).

But keyboarding $$\alpha$$ may be liked better than {{{α}{}}}.
And maybe ultimately a version of unicodeit can be persuaded to put in some <sub>/<sup>.

So I think some form of this might find its way into 1.3.18.

Grüße, Carsten

(*) I actually had to delete the d from aimd because unicodeit doesn’t know a subscript form of that letter, only a e h i j k l m n o p r s t u v x are supported.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 2 Dec 2020 12:14:21 +0100

> Is there another workaround?

Don't abuse <contact> for things that are not contacts.

If you disagree with the constraints implemented in xml2rfc, by all
means discuss them with the tool author and the RSE.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 2 Dec 2020 12:17:02 +0100

> <contact/> has only been added to <section/> (block-level) and <t/>, not to <li/>, which has its own copy of the content model of <t/>.  There are a few more elements like <name/>, <blockquote/>, <…

On the other hand it's not clear at all, whether *contact* information
is needed in all of these places (I do agree with <li>, and there's
already a ticket about that:
<https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/519>, opened
seven months ago).

> ...

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 13:22:24 +0100

> <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/519>, opened

As long as (span level) <contact/> is our only way to have the name of a person:

<section><name>History: the <contact fullname=“Gössner”/> draft

<blockquote>We have implemented <contact fullname=“Gössner”/>’s original specification but ran into

<dt><contact fullname=“Gössner”/> interpretation:

<dd>This interpretation differs from the original proposal by <contact fullname=“Gössner”/> in that

<xref target=“https://goessner.net/articles/JsonPath/”>Original <contact fullname=“Gössner”/> draft</xref>

(Stopping here; similar obvious cases can easily be found for all the other elements.)

All these could occur in a future version of draft-goessner-dispatch-jsonpath or a similar document.

Proof by lack of imagination is rarely the best guidance :-)

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Wed, 2 Dec 2020 13:30:00 +0100

> Proof by lack of imagination is rarely the best guidance :-)

And it helps to have concrete examples.

I will stick with my opinion that <contact> should be reverted to its
original definition (block content, for formatting similar as authors),
and that we should just allow non-ASCII in prose (with the production
center having an eye an abuse).

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 13:55:54 +0100

> center having an eye an abuse).

Completely agree.

Until that change happens, we have the workaround, which is marred by the incomplete integration of <contact/> into the grammar.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 16:37:57 +0100

> Completely agree.

Well, maybe with a twist.

I think that a way to identify a name (e.g., in an acknowledgements section) is a good thing on the XML side, without any influence towards the various readable representations.

E.g., see the acknowledgements section in https://github.com/cbor-wg/CBORbis/blob/1747eb6f436f679ab4587f498832c079847cc1f9/rfc8949-to-be.xml#L3933

Whether that element needs to be a variation of “contact”, I don’t know.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Wed, 02 Dec 2020 10:50:34 -0500

draft-ietf-cellar-ffv1 also uses a lot of math.
The process used by the authors involves markdown, but via mmark.

https://github.com/FFmpeg/ffv1

I can't recall if they have greek letters, but they definitely have other
math, including, think the Floor symbol.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Dec 2020 17:04:52 +0100

On 2020-12-02, at 16:50, Michael Richardson <[email-masked]> wrote:
> Signed PGP part

This is all done in SVG, which is not a problem with RFCXMLv3.

Grüße, Carsten

---

From: Lars Eggert <[email-masked]>
Date: Wed, 16 Dec 2020 17:29:13 +0200

>> center having an eye an abuse).

Agree as well.

> Until that change happens, we have the workaround, which is marred by the incomplete integration of <contact/> into the grammar.

And other places, as we just discussed on GitHub today.

It's ridiculous that I can't just write α when I mean α.

(Sorry, I'm grumpy.)

Lars

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 16 Dec 2020 18:51:06 +0100

> (Sorry, I'm grumpy.)

If you want to know what happens when I am grumpy:

http://www.tzi.de/~cabo/contact-hack.sh

Enjoy (doesn’t fix the version used at submission time, though)

Grüße, Carsten