Subject: [icnrg] FLIC annotations

From: Marc Mosko <[email-masked]>
Date: Mon, 3 Aug 2020 18:07:23 +0000

This is one of a couple of emails on FLIC topics from today’s call.  This email focuses on the ability to annotate hash pointers in a FLIC manifest with metadata, such as size or pointer type (e.g. manifest, data), or decoding information (e.g. fec layers or video information).  On the call, we discussed three ways of doing it, which I summarize here.

Option 1: Next to the pointer (as it is written in -02)

A HashGroup can be a plain Pointer group or an Annotated Pointer group.  The format of each annotation TLV could be defined later, as it is its own TLV type inside an annotated group.

  1.  In a plain pointer group, there is one TLV that contains an array of hash values.
  2.  In an annotated pointer group, There is a TLV that contains a list of sub-TLVs, one for each pointer called a PointerBlock.  Inside that PointerBlock is an optional annotation TLV and a Pointer TLV containing a single hash value.

Option 2: Parallel Groups in one manifest

The supposed benefit of this is that a reader only needs to understand the HashGroup and could ignore the annotations.  Annotations structure could be defined later.  It leverages all the work on the basic manifest and a parser is the same for all the other things.

  1.  Define a HashGroup to be of the Plain type (#1 above) – an array of hash values.
  2.  Define other types of groups for annotations.  This moves the annotations to parallel groups, but still bundled within a single Manifest.

Option 3: Separate Data/Content objects within the FLIC world

One could always do separate metadata objects, but in this case there would be a specific mechanism within FLIC to inform the reader that a separate data structure exists and how to fetch it.  The format of the annotation object is wide open, but one could use the basic structure of FLIC.  A FLIC manifest would have a means to indicate to the reader that one or more parallel annotation trees exist and how to find them.

One possibility is to use the idea of Option 2 and move those annotation groups out of the plain FLIC manifest into an annotation manifest.

On a historical note, the earlier drafts on CCNx chunking (https://tools.ietf.org/html/draft-mosko-icnrg-ccnxchunking-01) included the idea of having the metadata in a separate object.  This was a 1-for-1 parallel scheme where the metadata for a chunk used a name tweak to point to the metadata.  This is an old concept from the early CCNx days of naming sidecar data.  The metadata option was dropped from the last revision (-02) of ccnxchunking to keep it simpler.

Option N: others?
The 3 above are what were brought up on the call.  Are there other options we should consider?

Security Considerations
The way the -02 FLIC draft is written, each Manifest object has a single encryption wrapper.  If one wants, for example, separate encryption for annotations from manifests, one needs to put the annotations in a separate object.  This most cleanly fits with Option #3.  One could change the encryption to work at the Group level to accommodate #2, but that will be pretty complicated pretty fast.

Marc