Subject: Re: [rfc-i] [xml2rfc] use of sourcecode type

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Jul 2020 17:10:18 +0200

A similar problem is giving examples that are intentionally bad in order to demonstrate a kind of error.

I typically tag them with a type that is derived from the one I would give for real code, e.g., “CDDLx” for a bad CDDL example.  I think it would be good to agree on some way to indicate this.

A related problem is that often several code blocks combine to one valid instance of CDDL, for example see Figure 1, 2, 3 in RFC 8428.  There is no way to say that Figure 1 and 2 combine into a valid instance, and so do Figure 1 and 3, but not any other combination.

And, by the way, those type tags are conventionally lower-cased, but this is not made very explicit; you have to infer that from the list in Section 2.48.4 of RFC 7991 or the RFC editor’s updated copy of that list:

https://www.rfc-editor.org/materials/sourcecode-types.txt

(Ha, this doesn’t even have “cddl” in it; I’m not sure how this is updated and whether there shouldn’t really be an IANA registry for these.)

Grüße, Carsten

> On 2020-07-21, at 16:36, Paul Kyzivat <[email-masked]> wrote:
> 
> I have a question about specification of type in sourcecode elements:
> 
> In RFC4566bis there are many examples that have fragments of SDP. But they aren't compliant to SDP syntax, since it requires that many things be present - that are intentionally omitted from these examples.
> 
> Is it valid to tag these with type="SDP"?
> 
> (In sip we had a similar problem. There is a mime-type message/sip, but we sometimes also return fragments of sip in error messages. We ended up defining a separate message/sipfrag mime-type for this.)
> 
> 	Thanks,
> 	Paul
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 21 Jul 2020 11:48:13 -0400

> A similar problem is giving examples that are intentionally bad in order to demonstrate a kind of error.

Good point.

> I typically tag them with a type that is derived from the one I would give for real code, e.g., “CDDLx” for a bad CDDL example.  I think it would be good to agree on some way to indicate this.

I agree that we should have some agreed upon way to do this.

Perhaps a "+xyz" suffix, with some agreed up xyz values.

> A related problem is that often several code blocks combine to one valid instance of CDDL, for example see Figure 1, 2, 3 in RFC 8428.  There is no way to say that Figure 1 and 2 combine into a val…

I'm also interested in this. I believe an obvious solution to this is 
via the "name" attribute. All the ones with the same name should be 
gathered together.

A problem I have with both name and type is that they are invisible in 
the human readable formats. They provide semantic information that may 
be of interest to a reader. Perhaps they could be available in html via 
a popup?

> (Ha, this doesn’t even have “cddl” in it; I’m not sure how this is updated and whether there shouldn’t really be an IANA registry for these.)

For these to be useful for any sort of automated processing I think they 
should be standardized. I agree with an IANA registry.

If we wanted to allow unstandardized usage there could be X- prefixes, 
but we have banned those many other places.

	Thanks,
	Paul

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 21 Jul 2020 18:15:08 +0200

> via the "name" attribute. All the ones with the same name should be 

Yes.  That's already mentioned in RFC7991: 

  https://tools.ietf.org/html/rfc7991#section-2.48.2

and also in the documentation for the current xml2rfc release:

  https://xml2rfc.tools.ietf.org/xml2rfc-doc.html#name-name-attribute-4

Regards,

	Henrik

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 23 Jul 2020 11:37:35 +1000

In rust documentation, there is a flag for example code that says "this code is invalid".  That is separate from the (implied) type.

I don't see it being necessary to fix the "this is invalid, but it is if you add it to this other thing".  But that might be fixed with an excerpt flag that says "this is an excerpt (from X?) and therefore not valid on its own".

As for IANA maintaining a registry of types, sure.  I had no idea that this was a thing until the tls-syntax tag was added by the editor to one of my drafts.

On Wed, Jul 22, 2020, at 01:10, Carsten Bormann wrote:
> I typically tag them with a type that is derived from the one I would 
> give for real code, e.g., “CDDLx” for a bad CDDL example.  I think it 
> would be good to agree on some way to indicate this.
> 
> A related problem is that often several code blocks combine to one 
> valid instance of CDDL, for example see Figure 1, 2, 3 in RFC 8428.  
> There is no way to say that Figure 1 and 2 combine into a valid 
> instance, and so do Figure 1 and 3, but not any other combination.
> 
> And, by the way, those type tags are conventionally lower-cased, but 
> this is not made very explicit; you have to infer that from the list in 
> Section 2.48.4 of RFC 7991 or the RFC editor’s updated copy of that 
> list:
> 
> (Ha, this doesn’t even have “cddl” in it; I’m not sure how this is 
> updated and whether there shouldn’t really be an IANA registry for 
> these.)
> 
> Grüße, Carsten
> 
>

---

From: "John Levine" <[email-masked]>
Date: 22 Jul 2020 22:45:52 -0400

In article <[email-masked]> you write:
>I don't see it being necessary to fix the "this is invalid, but it is if you add it to this other
>thing".  But that might be fixed with an excerpt flag that says "this is an excerpt (from X?) and
>therefore not valid on its own".

If people can come up with a list of conventional code types, we can encourage people to use them.

>As for IANA maintaining a registry of types, sure.  I had no idea that this was a thing until the
>tls-syntax tag was added by the editor to one of my drafts.

Once again, no.  The RSE keeps a list of conventional artwork types, not IANA.  See RFC 7991, sec 2.5.7.

R's,
John

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 23 Jul 2020 13:23:26 +1000

> not IANA.  See RFC 7991, sec 2.5.7.

We documented this format in an RFC because we thought that interoperability matters.  I don't think that the RSE gets to be above that.

---

From: "John R Levine" <[email-masked]>
Date: 22 Jul 2020 23:40:46 -0400

> We documented this format in an RFC because we thought that 
> interoperability matters.  I don't think that the RSE gets to be above 
> that.

That section of RFC 7991 says:

2.5.7.  "type" Attribute

    Specifies the type of the artwork.  The value of this attribute is
    free text with certain values designated as preferred.

    The preferred values for <artwork> types are:

    o  ascii-art

    o  binary-art

    o  call-flow

    o  hex-dump

    o  svg

    The RFC Series Editor will maintain a complete list of the preferred
    values on the RFC Editor web site, and that list is expected to be
    updated over time.  Thus, a consumer of v3 XML should not cause a
    failure when it encounters an unexpected type or no type is
    specified.  The table will also indicate which type of art can appear
    in plain-text output (for example, type="svg" cannot).

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly
_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 23 Jul 2020 06:15:11 +0200

>   o  svg

In an authoring system such as kramdown-rfc, you get to use source-code (e.g., some form of UML) to generate artwork (right now limited to our SVG subset, or plaintext (“ASCII”) art).

The existing vocabulary cannot represent this.
But that is maybe mostly OK, because at the author-publisher interface, the source code (i.e., editable form) is no longer needed.
Until it is.

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 23 Jul 2020 00:22:31 -0400

> But that is maybe mostly OK, because at the author-publisher interface, the source code (i.e., editable form) is no longer needed.

This list of artwork types isn't cast in stone.  If we need new ones, we 
can add them.

Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 23 Jul 2020 06:35:53 +0200

> On 2020-07-23, at 06:22, John R Levine <[email-masked]> wrote:
> 
> This list of artwork types isn't cast in stone.  If we need new ones, we can add them.

If we want to add the artwork source code to the artwork types, we need to identify them as “needing processing” — presenting them as plaintext (“ASCII”) art may make more or less sense, but they really aren’t.

So we have

		TXT	HTML	
Plaintext art	x	x
SVG		- (*)	x
Source code	? (**)	p

Where x means “can be presented in” and p means “needs processing for”.
(With tools such as ditaa and goat, the difference between source code and plaintext art diminished.)  (**) Some tools are able to generate fallback TXT, notably plantuml for some of its input forms. (*) This is the place where artset was intended to help.  We may see all three, source code, fallback plaintext, and SVG in an artset...

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Mon, 27 Jul 2020 13:32:56 +0200

Am 21.07.2020 um 17:10 schrieb Carsten Bormann:
> ...

Picking a somewhat random message here to reply to... :-)

There are multiple use cases to consider:

1) tagging for the purpose of validation

2) tagging for the purpose of syntax coloring
(<https://greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html#xml2rfc-ext-html-pretty-print>)

3) hightlighting that something is a "bad" example

(and maybe more)

re 1) it is quite common that ABNF is spread over multiple parts of the
document; each of these is supposed to be syntactically correct, but
would fail validation if that includes checking for undefined rules

re 2) it might be desirable to do syntax coloring even on incorrect input

re 3) might be nice to have some color to indicate that some example is
incorrect/discouraged (and yes, the color would need to be just an
additional hint)

Best regards, Julian

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 Jul 2020 11:29:47 -0400

On 7/27/20 7:32 AM, Julian Reschke wrote:
>> this is not made very explicit; you have to infer that from the list 
>> in Section 2.48.4 of RFC 7991 or the RFC editor’s updated copy of that 
>> list:
>>

Giving all the pieces the same name will allow them to be accumulated 
into a single file for validation. This can potentially resolve some of 
the undefined rules. But to the best of my recollection I have never 
seen abnf in an ietf document that was complete on its own. Every one 
references rules defined somewhere else - usually in some other ietf 
document. These references all all informal, either stated in text 
before or after the abnf, or within the abnf as a comment or as a 
prose-val (e.g., <foo defined in RFC666>).

But I think *that* is a problem that we shouldn't attempt to solve here. 
IMO an abnf file that is intended to be syntactically correct and 
complete other than references to external rules should probably be 
tagged with type "abnf".

ABNF is particularly forgiving regarding statement ordering. So it may 
take further study to decide if/when marking abnf as a fragment makes sense.

When to a fragment modifier is much clearer when applied to sdp or sip or c.

	Thanks,
	Paul

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 27 Jul 2020 17:43:36 +0200

> […] I have never seen abnf in an ietf document that was complete on its own. Every one references rules defined somewhere else - usually in some other ietf document. These references all all informal, either stated in text before or after the abnf, or within the abnf as a comment or as a prose-val (e.g., <foo defined in RFC666>).

Well, I have started to recommend doing complete ABNF instead of ill-defined references.  E.g., see it in draft-ietf-core-dev-urn-07.txt:

   The above Augmented Backus-Naur Form (ABNF) copies the DIGIT and
   ALPHA rules originally defined in [RFC5234], exactly as defined
   there.

The general problem of composing a grammar out of parts coming from different places is now being discussed in CBOR (for CDDL), and I would expect the results copy over to ABNF as CDDL is so similar to ABNF.

> But I think *that* is a problem that we shouldn't attempt to solve here. IMO an abnf file that is intended to be syntactically correct and complete other than references to external rules should probably be tagged with type "abnf".
> 
> ABNF is particularly forgiving regarding statement ordering. So it may take further study to decide if/when marking abnf as a fragment makes sense.

If you aren’t supposed to use it on its own.
As with “bad” ABNF (CDDL, …); I would suggest a “;bad” qualifier for these.

Grüße, Carsten

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 Jul 2020 12:18:08 -0400

>     ALPHA rules originally defined in [RFC5234], exactly as defined

This can be quite impractical in some common usage.

At least in the sip world it is pretty common to have things like this:

(base-document):

message = foo / bar / baz / message-extention
message-extension = generic-message-format

(extension-document):

; message-header and message defined in base-document
new-message = message-header other stuff
message =/ new-message

where message-header, foo, bar, and baz may take hundreds of lines of 
abnf to define. Even in less extreme cases it can be difficult to trace 
all the dependencies on a rule whose definition you want to copy. And it 
may well involve tracing further dependencies to other documents.

I have in the past proposed and discussed (on the abnf-discuss list) 
some extensions to the abnf syntax to specify references to rules 
defined externally. They didn't get much interest at the time. But in 
any case they weren't easily and reliably automatable as long as the 
abnf had to be scraped out of a txt document. That might be more 
practical now.

	Thanks,
	Paul

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 27 Jul 2020 18:23:17 +0200

> This can be quite impractical in some common usage.

Then please don’t do it…

> new-message = message-header other stuff

Got it!

We’ll make sure we cover that use case in the CDDL discussion as well.

Grüße, Carsten

> 
> where message-header, foo, bar, and baz may take hundreds of lines of abnf to define. Even in less extreme cases it can be difficult to trace all the dependencies on a rule whose definition you want to copy. And it may well involve tracing further dependencies to other documents.
> 
> I have in the past proposed and discussed (on the abnf-discuss list) some extensions to the abnf syntax to specify references to rules defined externally. They didn't get much interest at the time. But in any case they weren't easily and reliably automatable as long as the abnf had to be scraped out of a txt document. That might be more practical now.
> 
> 	Thanks,
> 	Paul

---

From: Julian Reschke <[email-masked]>
Date: Mon, 27 Jul 2020 22:32:18 +0200

> prose-val (e.g., <foo defined in RFC666>).

Yes, prose values are a way to achieve this. See the HTTP RFCs.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 27 Jul 2020 22:39:36 +0200

Achieve what?  Achieve ABNF that can’t be processed by machine?  Yeah, that works :-)

Sent from mobile, sorry for terse

> On 27. Jul 2020, at 22:32, Julian Reschke <[email-masked]> wrote:
> 
> achieve this

---

From: Julian Reschke <[email-masked]>
Date: Tue, 28 Jul 2020 05:29:33 +0200

> Achieve what?  Achieve ABNF that can’t be processed by machine?  Yeah,

Achieve an ABNF that can be validated to be complete, as far as possible.

And if we establish a micro syntax for the prose production (essentially
implementing include functionality), we might even get further.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 28 Jul 2020 07:56:19 +0200

> implementing include functionality), we might even get further.

Now we are talking.

As I said in another thread, we need to solve the problem for CDDL anyway (which now can contain ABNF <draft-bormann-cbor-cddl-control-01>), so maybe we can target coming up with something that will work for ABNF on its own as well (not necessarily exactly the same solution).

Grüße, Carsten