Subject: Re: [icnrg] On FLIC and ISO manifest differences

From: <[email-masked]>
Date: Tue, 1 Dec 2015 04:13:38 +0000

On 11/30/15, 3:48 PM, "icnrg on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

>
>(ManifestMsg 
>    (HashGroup 
>        (MetaData (Blocksize 2048) (Totalsize 4001)) 
>        (DataPtr 0xabcd....) 
>        (DataPtr 0x2334....) 
>    ) 
>    (HashGroup 
>        (MetaData (Totalsize 64987)) 
>        (MfstPtr 0x8772....) 
>    ) 
>    (HashGroup 
>        (MetaData (Blocksize 2048) (Totalsize 2999)) 
>        (DataPtr 0xbcde....) 
>        (DataPtr 0x3345....) 
>    ) 
>) 
>
>Note that this trick of adding a separate HashGroup consumes the same 
>number of encoded bytes as in ISO. But we save many more bytes in the 
>normal HashGroup case because DataPtr directly leads to the 32 digest 
>bytes instead of having internal structure. 

I’m not sure why you claim that it consumes the same number of bytes since we haven’t defined an encoding yet.

Not to mention that it’s probably not true that it needs the same number of bytes. It would probably consume more bytes if you are encoding an array of MfstPrts (couldn’t you spell this out?) each one inside a HashGroup.

Also, I’m unsure that you save more bytes in a normal HashGroup. (Purely for the fact of being in a HashGroup, as opposed to lack of features, which could lead to byte savings).

>To summarize, we believe the ISO approach of mixing named and nameless 
>objects *in the manifest tree* should not be done for single object 
>manifests. This is not to say this approach won’t be useful for the 
>“advanced manifest” to come later.

Forcing all objects to be nameless does sound nice and it simplifies stuff.  However, it also means that you have to host all the data yourself… all the time.  I don’t want to impose that restriction.

Nacho

--
Nacho (Ignacio) Solis
Protocol Architect
Principal Scientist
Palo Alto Research Center (PARC)
+1(650)812-4458
[email-masked]

---

From: [email-masked]
Date: Mon, 30 Nov 2015 23:20:15 -0800 (PST)

Hi Nacho,

On Tue, 1 Dec 2015, [email-masked] wrote:

> Forcing all objects to be nameless does sound nice and it simplifies 
> stuff.  However, it also means that you have to host all the data 
> yourself… all the time.  I don’t want to impose that restriction.

Picking this one first: would you like to have partial storage of an ISO 
involving multiple hosting sites, instead of storing the full set at a 
single site?

This is supported in FLIC by having a "locator entry" in a HashGroup's 
metadata. In the example below all chunks would be fetched from where 
the root manifest was fetched (/parc/west) except for the manifest with 
the metadata locator entry: this manifest and all its descendent nodes 
will be fetched at /parc/east.

(ManifestMsg   # retrieved e.g. at /parc/west
    (HashGroup
        (DataPtr 0xabcd....)
        (DataPtr 0x2334....)
    )
    (HashGroup
        (MetaData (Locator /parc/east))
        (MfstPtr 0x8772....)
    )
   ...
)

But I would advise against such a practice because you nail down 
(hardcode) part of an ISO to a locator, which means that you cannot 
switch the hoster of the *whole* ISO anymore by just putting the root 
manifest elsewhere. Switching then comes at the price of recomputing 
several manifest nodes.

The second point - byte count - is minor, I should have left it out at 
this stage of the discussion. But the observation is simply this: If in 
ISO each pointer can have a metadata entry, a dataptr needs at least two 
levels of TLVs.

(dataptr
   (hashvalue OCTET[32])
   (metadata
      (entry1 ...)
   )
)

which holds even if no metadata section is present. While in FLIC, it's 
always

(dataptr OCTET[32])

You ask specifically for manifest pointers with size information: here 
we have equality. Using CCNx encoding when showing byte counts:

ISO:
(mfstptr                  2+2
   (hashvalue OCTET[32])   2+2+32
   (metadata               2+2
      (size ...)           2+2+ intLen
   )
)

FLIC:
(HashGroup                2+2
   (metadata               2+2
     (size ...)            2+2+ intLen
   )
   (mfstptr OCTET[32])     2+2+32
)

ISO could save 2+2 bytes if all metadata entries would be at the same 
level as the hashvalue entry. (But so could FLIC).

best, c

> I’m not sure why you claim that it consumes the same number of bytes 
> since we haven’t defined an encoding yet.
>
> Not to mention that it’s probably not true that it needs the same 
> number of bytes. It would probably consume more bytes if you are 
> encoding an array of MfstPrts (couldn’t you spell this out?) each one 
> inside a HashGroup.
>
> Also, I’m unsure that you save more bytes in a normal HashGroup. 
> (Purely for the fact of being in a HashGroup, as opposed to lack of 
> features, which could lead to byte savings).
>
>
> Forcing all objects to be nameless does sound nice and it simplifies 
> stuff.  However, it also means that you have to host all the data 
> yourself… all the time.  I don’t want to impose that restriction.
>
> Nacho
>
>
>
> --
> Nacho (Ignacio) Solis
> Protocol Architect
> Principal Scientist
> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg