Subject: [Cbor] eliding CBOR diagnostic notation binary contents

From: Michael Richardson <[email-masked]>
Date: Fri, 04 Feb 2022 13:12:43 -0500

In draft-ietf-anima-constrained-voucher we want to show examples, but some of
the hex strings are too long to fit easily, and there is little point in
wasting space for them.
(We include base64 encodings of the entire CBOR object)

please see https://github.com/anima-wg/constrained-voucher/pull/217/files

we try to consistently use / comment / in the diagnostic format, but we also
do:
   change from:
         h'1234',         // request_voucher
   to:
         h'<request_voucher>',  / byte string as detailed below /

I'm a little bit uncomfortable with <foobar> in the hex.
While h'1234' was syntactically valid, it was completely invalid contents.

Is there another choice?
Does this call out for some additional syntax for diagnostic format?
Something that is officially invalid, but that can be parsed and complained
about more clearly?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 4 Feb 2022 20:14:45 +0100

> Does this call out for some additional syntax for diagnostic format?

Additional syntax for elisions makes a lot of sense to me.
Not sure there needs to be a h’…’ around that (why not b64’…’, by the way?).
We also probably want to put in elisions for a map or array, e.g.,

[1, 2, …]

or

{
  a: b;
  c: d;
  …
}

Being able to indicate more type information about what is being elided borders on reinventing CDDL, but maybe there is a place for such a marriage.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Fri, 04 Feb 2022 15:29:57 -0500

> Not sure there needs to be a h’…’ around that (why not b64’…’, by the

b64() would disingenious, I think.
As for why h'...',  I might want to do h'30...AB' providing just a tiny bit of context.

I agree that going beyond that is verging on CDDL.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 8 Feb 2022 17:58:36 +0100

https://tzi.de/~cabo/cbor-elision.pdf

Let’s talk about that tomorrow.

Grüße, Carsten

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 9 Feb 2022 13:41:28 +0100

> https://tzi.de/~cabo/cbor-elision.pdf

Interesting, though I'm not sure I've made up my mind about it yet (then
again, shouldn't before presentation and discussion anyway).

The slides draw a connection to EDN, and I wonder: Why can EDN be purely
on the diagnostic side, while this would need an extra tag? Could
diag2cbor just leave all `...` empty (or replace them with Lorem Ipsum)
and cbor2diag just never generate any dots by default, possibly
recognize Lorem Ipsum with similar heurstics it uses, or generate dots
when explicitly asked to?

Or, conversely, would it be useful to have a tag there as well, even if
it is (as 888 is) only used when something like
"debug-roundtrip-friendly CDDL" is used?

And if neither, what sets them apart?

BR
Christian

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 9 Feb 2022 13:52:13 +0100

> on the diagnostic side, while this would need an extra tag? 

That is indeed a good question.

On one hand, one wants to use dt’2022-02-09T15:00Z’ to be processed as a number 1644418800.  On the other hand, when showing CBOR, 1644418800 is not detectable as something that should be shown as dt’2022-02-09T15:00Z’ — there would need to be an annotation that makes the diagnostic writer do so, and maybe we indeed need tags for annotations (which, in some cases, would however need to be presented as a tag when they themselves are the subject of diagnostics).

The elision is a bit different, as the intention is not to have an application specific notation for some data item, but to indicate that the data item is not known/not provided.  As proposed, a tag can be used for representing that, which can be used to make the relationship between the tag and the elision syntax be bidirectional.  Except when you actually want to look at the tag in detail…

The cbor-diag tools now have half a dozen flags (see also the similar flags in cbor.me).  We probably don’t want to proliferate them too much, and they are less useful as global flags than they would be as annotations…  At some point cbor-diag will be able to use a data definition (such as in CDDL) to add annotations to the CBOR items kept separately; that will then be easier to use.

Grüße, Carsten

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 23 Feb 2022 14:54:21 +0100

> On one hand, one wants to use dt’2022-02-09T15:00Z’ to be processed as
> a number 1644418800.  On the other hand, when showing CBOR, 1644418800
> is not detectable as something that should be shown as
> dt’2022-02-09T15:00Z’ — there would need to be an annotation that
> makes the diagnostic writer do so, and maybe we indeed need tags for
> annotations .

A particular reason why I'd consider tags for EDN (strawman: the pair
#6.88889([tstr, any]), eg. 88889("dt", 1644418800), and #6.88887([tstr,
tstr]), eg. 88887("dt", "2022-02-09T15:00Z")) useful is that while there
are many tools for working with CBOR, there are few for working with
diagnostic notation.

The tags could thus be used by application specific CBOR tools (which
would *know* which data items are best represented how) to embellish
CBOR before it gets sent to a more generic cbor2diag tool.

In the opposite direction, the tagged form (esp. the tstr/tstr one) be
a tool usable by the developers of application extensions to experiment
with the literal conversion without any of the few (AFAIK: single)
diag2cbor tools picking up that particular application-extension.

> [on EDN] (which, in some cases, would however need to be presented as
> a tag when they themselves are the subject of diagnostics)
>
> [on elision] Except when you actually want to look at the tag in detail…

I'd assume that both cases can be handled under the same set of
switches; something like `--debug-edn` would both leave tag 888 be and
show 88887 / 88889 as the tags they are, whereas the default would be to
pretty-print them.

BR
c

---

From: Michael Richardson <[email-masked]>
Date: Wed, 23 Feb 2022 19:18:16 -0500

I find having a tag for elided stuff really weird :-)
It's a higher level of meta, and maybe that argues for having a tag to
indicate that :-)

From my point of view, what I want from tooling is a way to say which fields
I want to elide when I process my (raw) CBOR into diagnostic notation.
I'd be doing this when building my internet-draft.

I think that I also want something processing diagnostic notation to not
process the elided notation literally.  I gather that the proposal is to put
these tags where the elidation is.  I am very concerned about people copy and
pasting elided examples into to code and getting surprised.

So I rather would like something that fails.

    >> makes the diagnostic writer do so, and maybe we indeed need tags for

    ca> A particular reason why I'd consider tags for EDN (strawman: the pair
    ca> #6.88889([tstr, any]), eg. 88889("dt", 1644418800), and
    ca> #6.88887([tstr, tstr]), eg. 88887("dt", "2022-02-09T15:00Z")) useful
    ca> is that while there are many tools for working with CBOR, there are
    ca> few for working with diagnostic notation.

...

    ca> I'd assume that both cases can be handled under the same set of
    ca> switches; something like `--debug-edn` would both leave tag 888 be
    ca> and show 88887 / 88889 as the tags they are, whereas the default
    ca> would be to pretty-print them.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 24 Feb 2022 02:06:50 +0100

> So I rather would like something that fails.

Actually processing elisions should be a special flag.
That should address your concern.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Wed, 23 Feb 2022 20:21:37 -0500

> Actually processing elisions should be a special flag.  That should
    > address your concern.

Yes, that works for me.