Subject: [Cbor] Record proposal

From: Carsten Bormann <[email-masked]>
Date: Wed, 5 May 2021 07:04:44 +0200

Kris Zyp put together a proposal for adding tags for records:

https://github.com/kriszyp/cbor-records

Note that this has some of the problems alluded to in…

https://www.rfc-editor.org/rfc/rfc8949.html#section-3.4-10

…which could be solved here (usage mostly in arrays) by defining a partial order that the usage of the tags defined on the fly would need to follow.

As Kris pointed out, my visceral reaction (below) has the limitation that it would only work for homogeneous arrays of records (and you have to ascertain homogeneity of the whole array before tagging).

Let’s have a quick discussion in the meeting later today.

Grüße, Carsten

JSON example:

[
   { "name": "one", "value": 1 },
   { "name": "two", "value": 2 },
   { "name": "three", "value": 3 }
]

Potential representations:

TBD([[“name”, “value”],
    [“one”, 1], [“two", 2], [“three", 3]])
(“CSV style”)

TBD([[“name”, “value”],
    [[“one”, 1], [“two", 2], [“three", 3]]])
(“Explicit”)

TBD([[“name”, “value”],
    [“one”, 1, “two", 2, “three", 3]])
(“Tight”)

TBD([[“name”, “value”],
    “one”, 1, “two", 2, “three", 3])
(“Too tight”)

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 5 May 2021 13:16:48 +0200

> Let’s have a quick discussion in the meeting later today.

Thanks, taken up on the agenda.

(And as much as I dislike being "the person to whom everything looks
like a nail", I'll probably ask about whether this fits in the general
model of packed CBOR, with the first entity setting up a single table
entry, and then the entries expanding a [] to a {}).

BR
c

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 5 May 2021 14:50:17 +0200

> entry, and then the entries expanding a [] to a {}).

There are two potential aspects to the proposed tag:

* a more compact representation (which is all that cbor-packed is about)

* semantic indication that a specific kind of record is being used

Proposed Tag 105 currently does not have a place for further semantic indications, but one could be added.

By the way, cbor-packed turns the example I gave in the referenced email into

51([["value", "name"], [], [], 
   [{simple(1): "one", simple(0): 1}, 
    {simple(1): "two", simple(0): 2}, {simple(1): "three", simple(0): 3}]])

Encoding-wise, the last array looks like this:

      83                  # array(3)
         a2               # map(2)
            e1            # primitive(1)
            63            # text(3)
               6f6e65     # "one"
            e0            # primitive(0)
            01            # unsigned(1)
         a2               # map(2)
            e1            # primitive(1)
            63            # text(3)
               74776f     # "two"
            e0            # primitive(0)
            02            # unsigned(2)
         a2               # map(2)
            e1            # primitive(1)
            65            # text(5)
               7468726565 # "three"
            e0            # primitive(0)
            03            # unsigned(3)

So the overhead here is one map head and two simple values per row.
(Of course, that assumes that one-byte simple values are still available in the greater context this is in.)

Even with a form of circumfix compression (e.g., mapping tables with parameters [1]), this is hard to beat encoding wise.
The record proposal as is takes four bytes per row (1+2 tag, 1 array).
This can be optimized significantly further only by amortizing the tag over more than one row, as my “CSV style” does, but that requires homogeneity.

Grüße, Carsten

[1]: https://datatracker.ietf.org/doc/draft-bormann-lpwan-cbor-template/