Subject: Re: [Cbor] Chair review of cbor-file-magic 08

From: Michael Richardson <[email-masked]>
Date: Wed, 02 Mar 2022 11:23:49 -0500

Christian =?iso-8859-1?Q?Ams=FCss?= <[email-masked]> wrote:
    > On Wed, Feb 23, 2022 at 02:08:07PM -0500, Michael Richardson wrote:
    >> > * 5.3: At latest here, I'd have expected to read something about the
    >> > information that is lost when using this tag with 2.2. Maybe there
    >> > should be a line (not for IANA but for users, so maybe in a different
    >> > section) that outlines when this works with (non)CBOR content formats
    >> > and with which encodings.
    >>
    >> I've tried how to understand this comment, but I'm failing.
    >> Which information is lost?

    > I've phrased this badly: The ct encoded in the tag gives information
    > about the (structured) media type and the content coding. In the 2.2 and
    > 2.3 use cases, the information in the content coding is ... discarded
    > (should be the word to use rather than lost), if it is acceptable that a
    > ct for application/mine+cbor@gzip is also used in this tag. Or the
    > content coding is required to be identity, in which case any
    > non-identity content coding would be ruled out (just like any non-CBOR
    > media types are ruled out).

I'm even more lost now.

    >> >   I think I understand now that the 5.3 tags can be used with either
    >> > 2.2, 2.3 or C (55899, 55800 and 55801), but
    >>
    >> good...
    >>
    >> >   * with 55899 the ct tags can only be used if the content format is a
    >> > CBOR content format and coding is identity (and then tags a value)
    >>
    >> 55899 marks CBOR, so the content had better be CBOR.

    > any tag tags CBOR, so the content needs to be CBOR.

Yes, we are using CBOR to create the marks, so yes, this is trivially true.
The marks do not all tag CBOR.

At this point, I feel like adding 55801 and the CT stuff has just confused
what was a really really simple proposal, and I wish we hadn't gone there.

    > I'm just saying that the formalities of defining the tags are complex
    > (because 55799(ct1234(X)) can take anything for X, whereas
    > 55801(ct1234(X)) takes precisely a bytestring ('BOR'). The text in the
    > IANA section 5.3 reflect that correctly, and I haven't found text that
    > sets the rules (rather than implying them).

I'll look again to see if I can clarify this better.

    >> > * (Maybe security) considerations: 3.2 talks of trivially concatenated
    >> > data items. If CBOR Tag Sequence is used and the data gets literally
    >> > "concatenated", (say, `cat *.cborseq > outfile`), one might wind up
    >> > with self-described tags in the middle of the sequence. (As we do with
    >> > BOMs).
    >>
    >> I have added:
    >> https://github.com/cbor-wg/cbor-magic-number/commit/23c4176796d83e4d69ba56ab8a9b7f45cd691272

    > That sounds like accepting such chains; is that really desirable? It
    > sounds really messy to me.

Yes, it is sometimes appropriate to accept such chains.
Consider a file containing a series (bag) of certificates.

    >> I think that you are having a problem with A.1 wrapping a CBOR thing in
    >> 55801?

    > I don't care about what is in a 55801, but the CDDL is confusing because
    > it is unspecific about which use case it is.

    > Either this tag is for use with 55800/55801, then it can be `senml-cbor
    > = #6.1668546672('BOR')` right away.

    > Or it is for use with 55799, then it needs to be `senml-cbor =
    > #6.1668546672([*record])` or whatever we use for SenML.

I think that we shouldn't use senml-cbor in the example.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 5 Mar 2022 02:14:03 +0100

Hi Christian,

I didn’t originally follow this thread, so I’m going to go through this sequentially.

> On 2022-02-22, at 14:19, Christian Amsüss <[email-masked]> wrote:
> 
> Hello file-magic authors,
> 
> as part of my shepherd write-up I'm reviewing the altest file-magic
> document as it is passing working group last call. All these comments
> are (presumably) editorial in nature and do not alter that outcome.
> 
> * "for each major type of object that they might store on disk": It is
>  not used as a proper name, but "major type" might ring the bell of
>  "first 3 bits" of CBOR, which AIU is not intended here.
> 
>  (Suggestion: "for each kind of object that they might store on disk").

(Already in MCR’s PR.)

> * 5.3: At latest here,

IANA Considerations text is not really meant to have technical content, so I wouldn’t want to add text here.

> I'd have expected to read something about the
>  2.2, 2.3 or C (55899, 55800 and 55801),

The 5.3 tags can be used for a lot of other purposes.  For instance, we forgot to define CBOR tags for SenML.  This is now remedied by virtue of having defined Content-Format numbers.

> but

Right.
Actually, the CBOR data item in the content format is used as the tag content of the ct tag.  (5.3 does not mention this case, sigh.)

>  * with 55900 the ct tags can only be used if the content format is a
>    CBOR-sequence content format and coding is identity (and then tags
>    a byte string)

The tag content is ‘BOR’.

>  * with 55901 the ct tags can always be used, and tag a byte string
>    (which is also encoded according to the coding that is part of the
>    ct)

Again, the tag is on byte string ‘BOR’.

>  If all these should be allowed, I think this should be spelled out
>  somewhere -- otherwise the examples appear to be conflicting.

Yes.
I made a significant editorial round to get this captured.

Please see: https://github.com/cbor-wg/cbor-magic-number/pull/17

>  with self-described tags in the middle of the sequence. (As we do
>  with BOMs).

Yes.

So it is better to take off the cbor-magic label before concatenating.
More text in the above PR.

>  Have BOMs caused issues in the past?

(Does it hurt when you hit your head with a hammer?)

> If so, would you care for grave
>  words we can haul against those who repeat inner-BOMs with CBOR?

This is not a BOM, please don’t say that word in polite company.
I think that the wording in the PR is describing the need and when it occurs.

> * A.1: This appears to be in direct conflict with the 2.2.1 example
>  (where the same tag is used with a non-bstr content). By the above the
>  limitation to bstr may still make sense, but only in the context of a
>  55800 or 55801 tag (but there, the tagged value is always not only
>  bstr but even precisely 'BOR'). Maybe the example would be clearer if
>  it would reference one of 2.2, 2.3 and appendix C.

I put some more explanation in the draft, but maybe looking at the examples again is next.
> 
> Best regards
> Christian
> 
> -- 
> To use raw power is to make yourself infinitely vulnerable to greater powers.
>  -- Bene Gesserit axiom

---

From: Michael Richardson <[email-masked]>
Date: Sat, 05 Mar 2022 14:39:37 -0500

> Please see: https://github.com/cbor-wg/cbor-magic-number/pull/17

Thank you for your changes.
I've merged them, and I'll publish a new draft, even if not perfect, now,
before the deadline.  If we need more, then so be it.

If the Shepherd is not persuaded, then I'll try an in-person editing session
:-)

Okay, -09 is posted.

As I read the diffs, I am concerned that some IESG reviewers may start askinq
questions about "CBOR Protocols", and I wonder if we have explained it well
enough elsewhere.

---

From: Christian Amsüss <[email-masked]>
Date: Mon, 7 Mar 2022 12:38:49 +0100

Hello Michael, Carsten,

thanks for the update, it does address the semantic concerns.

Two things were left but minor enough to be addressed in the next phase:

* In Section 5.3 (IANA / CBOR Tags for CoAP Content-Format Numbers), it
  still says "Date item: byte string"

  (when the rest of the document is now clear that it depends on the
  the context and (when in 55799) the individual content format).

* The introduction of Appendix A ("CBOR Tags for CoAP Content Formats")
  says:

  > Often, there is a need to identify a media type (or content type,
  > i.e., media type optionally used with parameters) that applies to
  > the contents of a byte string in a CBOR data item.

  Of all the things we describe how these 64k tags could be used,
  annotating a byte string is none of them -- and I think the sentence
  just needs an editorial nudge.

  (Incidentally, that is one variation which I might want to use in
  embedded representations in CoRAL, but now is most definitely not the
  time ask for things being added, especially when I'm not perfectly
  sure yet I'll need them).

I'll update the shepherd review after the cut-off.

Best regards,
Christian

-- 
This may seem a bit weird, but that's okay, because it is weird.
  -- perldata(1) about perl variables

---

From: Michael Richardson <[email-masked]>
Date: Mon, 07 Mar 2022 16:08:43 -0500

> still says "Date item: byte string"

In the other two IANA cases it says "tagged byte string"
In the CF case, the *tag* we agreed applies to the byte string 'BOR'

In the case of 55800 (5.1) and 55801 (5.2), the newly allocated tags apply to
a tagged byte stream.  I.e.:
  #6.55800<#6.12345678<BOR>>

while for the CF tags, it's not the outer piece that we are allocating,
but the inner part (12345678).  So it's correct: it tags 'BOR', not #6.foo<BOR>

    > just needs an editorial nudge.

How about this:

diff --git a/cbor-file-magic.mkd b/cbor-file-magic.mkd
index 3db8f84..7939740 100644
--- a/cbor-file-magic.mkd
+++ b/cbor-file-magic.mkd
@@ -401,16 +401,13 @@ Reference:

 # CBOR Tags for CoAP Content Formats {#ct-tags}

-Often, there is a need to identify a media type (or content type,
-i.e., media type optionally used with parameters) that applies to the
-contents of a
-byte string in a CBOR data item.
-
 {{Section 5.10.3 of -coap}} defines the concept of a Content-Format,
 which is a short 16-bit unsigned integer that identifies a specific
 content type (media type plus optionally parameters), optionally
 together with a content encoding.

+Outside of a CoAP transfer it may be necessary to identify the contents of the CoAP payload when it is on disk, in firmware, or when debugging.
+

 This specification allocates CBOR tag numbers 1668546560 (0x63740000) to
 1668612095 (0x6374FFFF) for the tagging of representations of specific
 content formats.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 7 Mar 2022 22:21:17 +0100

> but the inner part (12345678).  So it's correct: it tags 'BOR', not #6.foo<BOR>

The tags allocated in 5.3 are described in Appendix A.
This should make clear that the data item is actually /any/.
(Byte string normally and in cases of 55800 and 55801, but /any/ in case of tag 55799.)

> +Outside of a CoAP transfer it may be necessary to identify the contents of the CoAP payload when it is on disk, in firmware, or when debugging.

This almost works for me, except that it somehow relates this to CoAP transfer.
The Content-Format concept comes from CoAP, but is not limited to CoAP transfer (e.g., RFC 9165).

How about:

> +Outside of a transfer protocol that indicates the Content-Format for a representation, it may be necessary to identify the Content-Format of the representation when it is on disk, in firmware, or when debugging.
> +

Or some such.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Mon, 07 Mar 2022 17:40:54 -0500

>> +Outside of a transfer protocol that indicates the Content-Format for a representation, it may be necessary to identify the Content-Format of the representation when it is on disk, in firmware…

I was about to submit -10 with the changes I proposed, because draft
deadline, but I've now cancelled that, and submitted with your suggestion.

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 9 Mar 2022 16:45:28 +0100

Hello Michael, Carsten, CBOR,

The document has been submitted to the IESG. The shepherd writeup
contains a note that about three items need minor few-word changes,
these are:

There is a "CBOR objects" in the abstract,
you wanted to do something about that.

> (Byte string normally and in cases of 55800 and 55801, but /any/ in case of tag 55799.)

I think it is confusing to users if the registry describes the value for
one case that is more restrictive than the general value.

Would it make sense to put "depends on context" in there? (It's not like
the registry is CDDL typed).

Speaking of the registry: The shepherd writeup form makes a point of
asking that the used registry be identified, and 'Concise Binary Object
Representation (CBOR) Tags' doe snot show up once in the IANA
considerations. (It talks of 'registering CBOR tags', which is obvious
to us with the context, but not necessarily to an editor who does not
have all the lists indexed by abbreviations).

BR
c

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 9 Mar 2022 16:56:30 +0100

> have all the lists indexed by abbreviations).

Yes, I normally do a full reference to a registry (and have one to IANA.core-parameters); no idea why this isn’t in here.

The full kramdown-rfc liturgy:

Registry {{cbor-tags (CBOR Tags)<IANA.cbor-tags}} of {{!IANA.cbor-tags}}

(I need to invent special syntax for that…)

The registry (sometimes called subregistry (*)) is actually called “CBOR Tags”, btw.; but the web page that combines the registries (sometimes called registry) is called “Concise Binary Object Representation (CBOR) Tags”.

https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml#cbor-tags

Grüße, Carsten

(*) No, I don’t know why RFC 8126 didn’t nail down one consistent terminology here.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 9 Mar 2022 17:09:55 +0100

> you wanted to do something about that.

Now https://github.com/cbor-wg/cbor-magic-number/pull/18

Grüße, Carsten