Subject: Re: [icnrg] icnrg Digest, Vol 101, Issue 8

From: Ken Calvert <[email-masked]>
Date: Sat, 8 Aug 2020 15:19:24 -0400

Responding to a couple of points:

DaveO said:

> I’m still digesting this. Consider my comments more as questions, although all of you who know me also are aware of my predilection to couch questions as objections.
> 
> I also admit to not really understanding it completely after a couple of read-throughs. Some of this may be due to having to purge my translation cache of my usual understanding of some of the terms you have chosen. Feel free to respond, ponder for why the explanation isn’t getting through well to me, or just ignore.

I won't try to respond to your points (I'm already way behind), but I think maybe you and Christian are closer together than your reactions might indicate.
For example, I interpreted his "virtual blob" in exactly the way you suggest.

Marc said:

> FLIC describes a single file.  No.  By definition, FLIC must always describe two files: the manifest and the data.

I think you mean to say "FLIC specification must always describe two types" here, right?

> ...

> Inlining: Yes!  One use case we had for this was when someone wants to re-publish a manifest object (e.g. a content distributor wants to point its subscribers to local caches via locators), it would encapsulate the original root manifest in its re-written one so the consumer can verify in one step that they (a) got the thing they were asking for (from the interior manifest) and that someone they trust for such things (e.g. their ISP) tells them a good place to fetch it from.

Hmmm. This is different from my understanding of the "inlining" discussion.  As I said in my previous message, I interpreted it like what some filesystems do for inodes of small files: put application data directly in the inode/manifest structure.  I think the latter is a bad idea, for reasons I mentioned earlier.  OTOH, what you describe here, encapsulating metadata in other metadata, seems like just a normal thing you might do with a metadata-in-manifest facility.

Ken

---

From: Marc Mosko <[email-masked]>
Date: Sun, 9 Aug 2020 22:51:35 +0000

>
  >On 8/8/20, 12:19 PM, "icnrg on behalf of Ken Calvert" <[email-masked] on behalf of calvert=[email-masked]> wrote:
  >

Well, the Manifest describes how to assemble all the pieces of the manifest into the correct order such that one can derive the correct order of data pointers from it.  The manifest is, at its base, an ordered list of pointers to data, plus a little extra for security context and maybe an annotation.  To me, that is the manifest file.  I could give you a single file (say json or binary) that conveys the same exact information as gets encoded in the manifest tree.  The manifest tree exists because we want to limit the size of the data transfer unit.

But this is a semantic point not really that important.  We could call it the manifest type or whatever.  My point is there's two things being conveyed: the manifest (data pointers) and the data itself.

  >    Hmmm. This is different from my understanding of the "inlining" discussion.  As I said in my previous message, I interpreted it like what some filesystems do for inodes of small files: put app…

Ok, I understand the distinction.  At some early point, it was decided that a Manifest should not carry application data in its payload.  I think that's fine.  

  >
  >    Ken
  >
  >    _______________________________________________
  >    icnrg mailing list
  >    [email-masked]
  >    https://www.irtf.org/mailman/listinfo/icnrg
  >

---

From: Ken Calvert <[email-masked]>
Date: Sun, 9 Aug 2020 19:26:52 -0400

> But this is a semantic point not really that important.  We could call it the manifest type or whatever.  My point is there's two things being conveyed: the manifest (data pointers) and the data it…

My point was that I'm not sure the term "file" has a precise definition in this context - or at least, I'm not sure that I understand it the way you were using it.
(Perhaps this was why Christian introduced the "blob" terminology.  Networking has now been around long enough that many otherwise-useful terms have baggage that renders them problematic.)

But I agree that it's a minor point!

Ken

---

From: [email-masked]
Date: Sun, 9 Aug 2020 22:55:52 -0700 (PDT)

Thanks for the discussion among us four. I'm not super happy about how 
it goes so I will interject a meta contribution. The reason may be that 
I start to see FLIC as a patch for something that an information centric 
network should have had from the beginning, namely ample place to put 
content _of any size_ and stream content of any size (duration) while 
our discussion over and over nitpicks about how to stuff things into 
packets .. oops, wrong word.

My take is that the discussion should be on size, random access and streaming.

- streaming? Not really as FLIC will only cover playback, cannot handle novel content
- random access? Will remain a library hack forever (1000 wrong ways), the network only cares about piecemeal delivery
- size? vblobs try to save at least this

I would like to first see the "surrogate API" that FLIC probably 
implicitly tries to invent, starting from the producer side:

- simplest case is content copy and producer wants to help with speedup via parallel fetch. Let producer prepare metadata where suitable *pickup places* are, instead of byte-level random access.
- segmented data like DVD chapters? Use several FLICS (and an independent draft for directory-like content
- byte-level seek? Hm, too much stress with NDN/CCNx chunking. Or use NFN at producer-side where we have access to the flat content on disk, unchunked.
- republishing content under a new prefix: should we even talk about this? Not worth an API in FLIC: see directory-like content where it's natural that content can appear under a new name.

and so on. The "library destiny" of FLIC is currently a missguided 
invitation to try to make everybody happy - let's trim that down. My 
guess is that for NDN/CCNx (but also other networks), only full 
content-copy or streaming tasks will remain in the long run (plus 
interactive access i.e. NFN). Let's go for the minimal case: a FLIC 
tailored for content copy via DFS, which at the same time is good for 
streaming finite content. Or do you really want to work towards a 
glorious FLIC-for-everything library, for fun or fame? I think that 
"improvements" beyond above minimalist program will not be seen as such, 
nor be necessary.

Best, c