Subject: [Suit] Mechanism to start with CBOR, leaving C-Struct for future extension (2 bytes cost - magic 0x82 0xF6 and 0x82 0x82)

From: Alexander Pelov <[email-masked]>
Date: Thu, 8 Nov 2018 18:50:00 +0700

Hi all,

The discussion today on ONE vs TWO formats, got me thinking on the way to actually split the process. 
I think it's better to have ONE thing that works, and then provide a second option compatible with the first.

Here are a couple of thoughts that are independent of the other threads (the "Why"). A couple of thoughts on the "How" to start with ONE, and add second when (if) necessary.

The idea is to have a basic format that is a valid CBOR container, which can be simply parsed by anything (even a C-Struct). 
The cost is 2 Bytes for a CBOR manifest, and 10 bytes for a binary manifest. You can have both in the same Manifest container.

The requirements would be:
- Have a full-featured bootloader (with CBOR/COSE knowledge) be able to always update with SUIT Manifest
- Have a simplified / non-CBOR-aware bootloader be able to understand a SUIT Manifest - and know when it fits it format. 
  -> We can imagine in the future there may be more than one C-Struct representations. How does the non-CBOR bootloader know that it has indeed the correct format?

Thus: 
Have a leading 2 bytes that allow any parser (CBOR or non-CBOR) to understand the format and:
1) abort if they cannot parse the full payload
2) parse correctly if it is for them

In this idea, the current SUIT Manifest is encapsulated in a very simple outer shell, which is static (and thus understandable by anyone with 0 CBOR knowledge).
That is:

The magic 2 byte header can be:
0x82 0xF6 - CBOR-only Manifest (the SUIT Manifest follows) (a total of 2 bytes are "wasted")
0x82 0x82 - Other Manifest format present (with several bytes after to encode the data) (a total of 10 bytes are "wasted")

How does that work?
** A) CBOR-unaware parser: 

A future CBOR un-aware parser can then continue the parsing :
0x82 0x82 0xVV 0xC2 0x9A 0xXX 0xX1 0xX2 0xX3 0xX4 OTHER-ENCODING-BYTES 0xXZ 0xXZ ....

Where:
- 0xVV - One byte to determine the format of the Other-Encoded-Manifest-Format It MUST be 0x00 - 0x17. Allocated by IANA. (for example, 0x00 could be MCUboot C-Struct, 0x01 something else, etc.)
- 0xX1-0xX4 are 4 leading opaque bytes to be ignored.
- OTHER-ENCODING-BYTES - the contents of the Manifest to be read by the binary parser. The length MUST be encoded in the format, as there WILL be trailing bytes (with variable length)
- 0xXZ, 0xXZ ... are an undefined number of trailing bytes to be ignored

** B) CBOR-aware parser:
The above "Magic" sequence resolves to a valid CBOR description of the form:
[binary_manifest, CBOR_manifest]

If binary_manifest is null, there is only a CBOR manifest.
If CBOR_manifest is null, there is only a binary manifest.
If both are present, they MUST represent the SAME information, e.g. the binary manifest REFLECTS 100% what the CBOR Manifest contains.

Note that the bytes 0x82 0xVV 0xC2 0x9A  0xX1 0xX2 0xX3 0xX4 resolve to a valid CBOR of the type :
[version, bignum(max size: 4GB)] - and the binary manifest is contained in the bignum.

*************

Of course, it is up to the group to decide the "Why". This is the "How", which can be improved (e.g. register a new Tag for the binary manifest for example). 

For me, the way ahead is:
1) Focus only on CBOR Manifest. 
  1.1) Make sure the optional attributes allow for mini-environments

2) When the whole thing is sent to the IESG, IF there is interest, start standardization work on binary formats that are potential subsets of the CBOR Manifest.

Cheers
Alexander

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 8 Nov 2018 19:31:03 +0700

> 0x82 0x82 0xVV 0xC2 0x9A 0xXX 0xX1 0xX2 0xX3 0xX4 OTHER-ENCODING-BYTES 0xXZ 0xXZ ....

I can’t quite parse this (bignum tag on array of lots of elements?).

Of course a bespoke binary header can be carried within CBOR (preferably as a byte string, without needing to mask it as a bignum), and the addition or stripping of an “envelope” like 0x82 0x2A (for a 10-byte binary string) does not require a lot of lines of code.
(See Section 3.2 of draft-ietf-core-multipart-ct for an example how this can be used the other way around, with non-fixed values/lengths.)

The problem here really is that the bespoke byte string loses the extensibility inherent in CBOR.
(We may not care about the additional safety, for code that will be highly robustness tested as well.)
That may be OK for some applications, in which case I’d say go ahead…  
The real CBOR representation of the structure follows in the second array element, after all, and the users of the bespoke binary representation simply don’t get to see the extensions.

But in general this is mostly a diversion distracting from getting the information model right…

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Thu, 08 Nov 2018 19:43:08 +0700

Thank Alexander, that's a very cool proposal.

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 8 Nov 2018 19:49:44 +0700

Hi Carsten ,

> I can’t quite parse this (bignum tag on array of lots of elements?).

Yep, the idea is to have a fixed number of bytes that allows for a non-cbor
parser to skip to their payload , while in the same time keeping it a valid
cbor construct.

This, a SUIT manifest can be parsed by anyone, even if only to understand
"that's not for me"

It is important that the binary blob that contains the non-cbor manifest is
in a fixed (and big enough) cbor container. That avoids the need for the
binary parser to parse any cbor.

Essentially, the algorithm is:
Read first two bytes, if 0x82 0x82 then :
  Read third byte, if equal to my format version :
  Skip 7 bytes
  Load binary manifest (must have length indicator)
  Skip padding

> Of course a bespoke binary header can be carried within CBOR (preferably
> as a byte string, without needing to mask it as a bignum), and the addition
> or stripping of an “envelope” like 0x82 0x2A (for a 10-byte binary string)
> does not require a lot of lines of code.
> (See Section 3.2 of draft-ietf-core-multipart-ct for an example how this
> can be used the other way around, with non-fixed values/lengths.)
>
> The real CBOR representation of the structure follows in the second array
> element, after all, and the users of the bespoke binary representation
> simply don’t get to see the extensions.
>

Completely agree on both points.

What I wanted to point out is that there is a way to ensure extensibility
and thus not block on binary formats .

Just focus on CBOR (+the semantics and all that). Worst case scenario,
that's a 2 byte overhead on all SUIT manifests.

Alex

---

From: Martin Pagel <[email-masked]>
Date: Fri, 9 Nov 2018 01:49:33 +0000

Alex,
Very clever idea! I like the “magic bytes” to distinguish between the formats…
But the increased size kind of defeats the purpose to have a simple tight format which can be easily loaded and interpreted in memory.

I think the manifest format decision should be left to the platform vendor who also develops the update agent (status tracker or communicator). The Update Server will need to know anyway what options a particular platform requires as certain platforms will require specific image types, signature formats, instructions etc. We talked about the fact that every platform won’t support all options. Then the server might as well also generate CBOR or Binary manifest formats and not both.
Martin

From: Suit <[email-masked]> On Behalf Of Alexander Pelov
Sent: Thursday, November 8, 2018 4:50 AM
To: Carsten Bormann <[email-masked]>
Cc: [email-masked]
Subject: Re: [Suit] Mechanism to start with CBOR, leaving C-Struct for future extension (2 bytes cost - magic 0x82 0xF6 and 0x82 0x82)

Hi Carsten ,
On Thu, Nov 8, 2018, 7:31 PM Carsten Bormann <[email-masked]<mailto:[email-masked]> wrote:
> 0x82 0x82 0xVV 0xC2 0x9A 0xXX 0xX1 0xX2 0xX3 0xX4 OTHER-ENCODING-BYTES 0xXZ 0xXZ ....

I can’t quite parse this (bignum tag on array of lots of elements?)..

Yep, the idea is to have a fixed number of bytes that allows for a non-cbor parser to skip to their payload , while in the same time keeping it a valid cbor construct.

This, a SUIT manifest can be parsed by anyone, even if only to understand "that's not for me"

It is important that the binary blob that contains the non-cbor manifest is in a fixed (and big enough) cbor container. That avoids the need for the binary parser to parse any cbor.

What I wanted to point out is that there is a way to ensure extensibility and thus not block on binary formats .

Just focus on CBOR (+the semantics and all that). Worst case scenario, that's a 2 byte overhead on all SUIT manifests.

Alex

Grüße, Carsten