Subject: Re: [icnrg] Pushing Payloaded Interests in ccnx[messages|semantics]

From: "Dirk Kutscher" <[email-masked]>
Date: Mon, 01 Oct 2018 13:37:44 +0200

Hi Marc,

Thanks for that — interesting and relevant input!

> Interest Data in pull operations is extremely common.  In HTTP, for 
> example, the Query String is essentially Interest Data.  In ICN (at 
> least NDN and CCN), I think most every app beyond those that fetch 
> static data uses some form of Interest Data (either in the name or a 
> payload).  Even “phone home” apps that send an Interest to solicit 
> and Interest to pull data use Interest Payload to tell the other side 
> what to ask for.  Maybe someone on the NDN team with all their apps 
> could do a survey, but I expect that the vast majority of apps use 
> some form of Interest Data (in names).

So, we have been debating about how much of the application layer 
request data should go into an CCN/NDN Interest message. In the 
“Communication Patterns for Web Interaction…” paper and in the 
RICE protocol (that could be a basis for some HTTP-like protocol), we 
decided to actually NOT include all the possible application-layer data 
(parameters for the called method etc.) into the initiating Interest 
message. One reason was avoiding server overloading attacks (server 
producers would be forced to parse every Interest), and another one was 
adhering to flow balance and assumption in congestion control schemes 
(that use Interest sending rates to regulate the flow of Data messages).

Obviously, this comes at a cost of half an RTT (or more). On the other 
hand, this might be somehow amortized by enabling caching of larger 
input data  (e.g., for local re-transmissions) and by having a more 
robust system (where all the data in conveyed in congestion-controlled 
Data messages).

In the spirit of that approach, you would only use Interest Data for 
information that is absolutely required to process the Interest message 
“on the CCN/NDN” layer”, and use one more level of indirection for 
everything else.

Thinking further, asynchronous notifications (for triggering a pull) 
could be dealt with in different ways. In Cenk’s IoT scenario, they 
are using link-local notifications (could be seen an a “management 
extension”). In other application scenarios, you may be able to 
address the application requirements by an AJAX-like polling-based 
surrogate for data push. It’s not the most efficient approach, but 
could be done without deviating from CCN/NDN Interest/Data semantics and 
may actually be OK for most applications. Depending on the scenario 
(multiparty communication comes to mind), other benefits such as caching 
and implicit multicast distribution again may amortize the overhead.

Another design question in this context would be how orthogonal the used 
mechanisms are with respect to other extensions that manipulate 
forwarder FIB state and require special forwarder logic (producer 
mobility support such as MapME or KITE for example). RICE installs 
additional temporary FIB state, too…

Cheers,
Dirk

> One use case we wanted to capture in the CCNx 1.0 spec, which Mark 
> Stapp brought up [3], was REST-like operation of ICN.  In this case, 
> the payload is not a push, but is actually an echo of state from the 
> service back to the service and will be modified again and a new state 
> sent back next time.  So, having the service need to (a) echo the old 
> state back in the name and (b) send new state back in the payload 
> seems excessive.  This was another motivation for the IPID and 
> Interest Payload, unrelated to push.
>
> The Voice over CCN paper used a paradigm that is close to establishing 
> a bi-directional flow controlled by Interests.  I took this a few 
> steps further in a white paper on bi-directional streams [1], which 
> was originally written in 2013 [2] and put up on arxiv in 2017.  This 
> becomes very similar to having a two-way TCP stream between users flow 
> controlled by Interests, where the Interests are naming blocks and the 
> payload of those blocks can be whatever the two parties want.  Chris 
> Wood and I then used a mechanism like this for encrypted streams where 
> the Content Object payloads are carrying inner encrypted Interest and 
> ContentObjects, somewhat like an ICN-based TLS wrapper (the outer 
> names are the tunnel and the inner names are the actual interest/data 
> traffic).
>
> Marc
>
> [1] https://arxiv.org/abs/1707.04738
> [2] 
> https://github.com/PARC/CCNxReports/blob/master/2014/4.5%20CCNx%201.0%20Bidirectional%20Streams.pdf
> [3] https://dl.acm.org/citation.cfm?doid=2660129.2660152
>
>
>
> On Sep 28, 2018, at 8:11 AM, Cenk Gündoğan 
> <mail=2Bietf=[email-masked]<mailto:mail=2Bietf=[email-masked]>> 
> wrote:
>
> Hello Marc, Dirk, *,
>
> indeed, also thanks from my side for this great explanation. These
> historical facts really help to grasp the motivation for some of the
> design choices in the CCNx spec. Using an IPID is a reasonable way to
> reduce forwarding state for payloaded Interests .. and given the
> "echoing" of names, IPIDs surely are a much cleaner approach than
> encoding content into names.
>
> As Dirk already explained, I was referring to the purely pull-based
> operation of CCNx/NDN. While reading the drafts, I had the impression
> that concepts for a pull-based operation (Interest-data exchange,...)
> and concepts for a push-based operation (payloaded Interest, 
> 0-lifetime
> Interests) were mixed together (stucture-wise). I personally would 
> have
> expected a conceptually cleaner distinction between both paradigms in
> the docs.
>
> FWIW, I re-read the sections in question after your explanations. The
> mechanisms for payloaded Interests seem to be isolated enough and are
> mostly independent of other parts of the text. So, IMO (and with 
> another
> mindset than before), the docs are in good shape and provide a nice
> reference for CCNx.
>
> Cheers,
> Cenk
>
> On 18-09-28 10:54:05, Dirk Kutscher wrote:
> Hi Cenk and Marc,
>
> Thanks for bringing that up, and thanks to Marc for explaining the
> “historic” background.
>
> At the ICN conference and Interim in Boston, we had quite interesting
> discussions around “what abstractions does ICN need and how should 
> existing
> relevant interactions models be implemented in ICN”.
>
> There was a panel on “Augmenting NDN beyond the pull-based data 
> delivery:
> Mechanisms and Insights” with Christian, Dave, Karen, Marie-Jose, 
> and Lixia
> and we also had a few papers and demos that touched upon ‘push’ 
> and related
> questions.
>
> For example:
>
> - HoPP: Publish-Subscribe for the Constrained IoT (Cenk et al.)
> - RICE: Remote Method Invocation in ICN
>
> FWIW, one opinion in that context suggested that ICN (CCNx/NDN style) 
> offers
> new ways to address certain communication requirements, so that you do 
> not
> have to directly copy existing mechanisms (e.g., pushing unsolicited 
> Data)
> in all cases (if you can afford additional RTs for example). Instead,
> maintaining principles such as Interest/Data semantics (with data 
> primarily
> in Data messages only), flow balance, consumer anonymity could/should 
> be
> emphasized.
>
> Now (with chair hat on), regarding the CCNx specs, ICNRG wanted to get 
> these
> published in order to document these particular specifications and to
> provide a reference for more experiments etc.
>
> It’s not a problem if some design decisions or mechanisms raise 
> questions or
> are eventually deemed unnecessary (just in case somebody is 
> wondering).
>
> Best regards,
> Dirk
>
>
>
>
>
>
>
> On 27 Sep 2018, at 20:57, 
> [email-masked]<mailto:[email-masked]> wrote:
>
> Thank you for your questions on the drafts.  I did not follow the 
> recent
> meeting, so I do not know the reference you make to "resonances of the
> past days."  Here is some more insight into the sections you mention.
>
> In addition to the sections you point out, another relevant item is
> Semantics Sec. 3.2, "Interest Payload ID" (IPID).  The IPID is 
> important
> in the context of using an Interest payload, as I will explain below.
>
> Ever since the earliest days of CCNx (see, for example, the 2009 
> "Voice
> over CCN" paper), before NDN existed, the concept of pushing data in 
> an
> Interest existed.   At that point, it was always put in the name, 
> which
> method NDN adopted and has used extensively.  It was always understood
> by myself and the other draft authors that the name-based method would
> continue to exist and be used in CCNx.  We did not see a reason to
> specifically standardize that process.  NDN has a document about how 
> to
> put signed payloads in an Interest name, similar to informal methods
> used in early CCNx (it's been formalized to the "command interest" 
> now).
> My opinion is those methods are still valid and would continue to be
> used in CCNx.
>
> Putting large payloads, however, in a name has performance 
> consequences,
> as it must be stored in the PIT and parsed as part of forwarding.   
> The
> data must also be _echoed_completely_ in any response because the
> responding Data object must match (or extend) the name of the 
> Interest.
> I would expect that an actual deployment would need to have rules on 
> the
> maximum name size due to its implications to memory and packet
> processing.
>
> It also appears that large names imply that one must use hop-by-hop
> fragmentation, because one likely needs the whole name to make
> forwarding decisions (I use wiggle words here because I don't think 
> this
> is proven, but seems likely).  We tried exploring ways of doing
> end-to-end fragmentation where even the name could span fragments so
> long as the routable portion is in the first fragment, but these are
> cumbersome and likely have security issues.  If one can have a short 
> (1
> fragment) name plus payload, one could use other forms of 
> fragmentation
> than hop-by-hop (e.g. [1,2]).
>
> Due to those limitations of putting non-trivial amounts of data in a
> name, we decided to allow a payload field in Interests.  It also means
> that the packet envelope of an Interest and a Content Object is, for 
> all
> purposes, identical.  This means that if a user wishes to send data in
> an Interest, it can use the same payload and signing mechanisms as a
> Content Object rather than make a custom signed data structure to 
> append
> to a name.  Because a forwarder only multiplexes Interest responses
> based on the name and ignores Interest payload, it is important to 
> have
> a principled way for a user to generate unique Interest names for a
> payload, and thus we introduced the IPID.  The IPID is essential in 
> the
> name for an abbreviated placeholder for the actual payload without
> having the same memory pressure on the PIT or the same response bloat 
> as
> a Content Object responding to an Interest with payload.
>
> The 0 interest lifetime option is exactly as you describe it: send
> notifications without incurring unnecessary forwarder overhead.  This 
> is
> an example of the CCNx 1.0 drafts trying to be explicit about corner
> cases, where the prior CCNx spec was silent.  What should you do with 
> a
> 0 lifetime Interest?  Drop it?  Drop it and send a NAK?  Forward it, 
> but
> never send the response back because the PIT entry is pre-expired?    
> We
> chose to allow a 0-lifetime Interest to propagate but not elicit a
> response.
>
> Having explained this, I believe we probably need a rule that if a
> forwarder _decrements_ an InterestLifetime to 0, it must send an
> InterestReturn and not treat it like a 0-lifetime Interest.
>
> The NDN spec behaves, I believe, the same as our explicit "0 interest
> lifetime" but without specifically saying it.  The packet 
> specification
> does not say the Interest Lifetime must be positive -- in fact, it is
> explicitly a "nonNegativeInteger" representing milli-seconds..  The 
> NFD
> developer guide (Rev 10) also does not say anything that I could find
> that requires the lifetime to be positive.  It clearly says that an
> in-record and out-record expire when the Interest Lifetime elapses,
> which for a 0 value I assume is immediately.  I have not looked at the
> source code to see if there is some undocumented behavior.  The NFD
> guide is explicit if the InterestLifetime is missing, but that is
> different than being present with a 0 value.   Based on my parsing of
> the NFD guide, it would accept a 0-interest lifetime, create an
> in-record and out-record (that would be immediately expired) and 
> forward
> the Interest.  But I'm not an expert of NFD and it is a bit hard for 
> me
> to extract protocol behavior from the document.
>
> Marc
>
> [1] Ghali, Cesar, Ashok Narayanan, David Oran, Gene Tsudik, and
> Christopher A. Wood. "Secure fragmentation for content-centric
> networks." In Network Computing and Applications (NCA), 2015 IEEE 14th
> International Symposium on, pp. 47-56. IEEE, 2015.
>
> [2] Mosko, Marc, and Christopher A. Wood. "Secure fragmentation for
> content centric networking." In Mobile Ad Hoc and Sensor Systems 
> (MASS),
> 2015 IEEE 12th International Conference on, pp. 506-512. IEEE, 2015.
>
>
> ________________________________________
> From: icnrg [[email-masked]<mailto:[email-masked]>] on 
> behalf of Cenk Gündoğan
> [mail=2Bietf=[email-masked]<mailto:mail=2Bietf=[email-masked]>]
> Sent: Thursday, September 27, 2018 9:13 AM
> To: [email-masked]<mailto:[email-masked]>
> Subject: [icnrg] Pushing Payloaded Interests in 
> ccnx[messages|semantics]
>
> Hello all,
>
> while working on ICN-LoWPAN, I carefully re-read the ccnxsemantics [1]
> and ccnxmessages [2] drafts. Thanks to the authors for this
> well-structured CCNx reference; they are easy to digest. However, I
> came across two features I'd like to point out:
>
> 1) Interests may contain payload
>
> Section 3.6 of [2] displays the header structure of a general CCNx
> message and it contains an "Optional Payload TLV (T_PAYLOAD)". Ergo, a
> T_PAYLOAD TLV might also be included in a CCNx Interest message, and
> still be syntactically correct. This is also backed by the ABNF in 
> [3].
>
> 2) Interests with a lifetime of 0
>
> Section 3.4.1 of [2] contains:
> "A value of 0 (encoded as 1 byte %x00) indicates the Interest does not
> elicit a Content Object response.  It should still be forwarded, but 
> no
> reply is expected and a forwarder could skip creating a PIT entry."
>
> Both primitives combined allow for pushing payloaded Interests without
> even setting up PIT state on a forwarding path. Given the resonances 
> of
> the past days, describing these features in [1] and [2] feels IMO a
> little misplaced.
>
> What were the incentives to include them into the drafts?
>
> Cheers,
> Cenk
>
> [1] https://tools.ietf.org/html/draft-irtf-icnrg-ccnxsemantics-09
> [2] https://tools.ietf.org/html/draft-irtf-icnrg-ccnxmessages-08
> [3]
> https://tools.ietf.org/html/draft-irtf-icnrg-ccnxsemantics-09#section-2..1
>
> --
> Cenk Gündoğan
>
> Hamburg University of Applied Sciences
> Dept. of Computer Science / Internet Technologies Group
> Berliner Tor 7, 20099 Hamburg, Germany
> Fon: +49 40 42875 - 8426
> Mail: [email-masked]
> Web: https://www.inet.haw-hamburg.de/
>
> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg
>
> _______________________________________________
> icnrg mailing list
> [email-masked]<mailto:[email-masked]>
> https://www.irtf.org/mailman/listinfo/icnrg
>
>
> --
> Cenk Gündoğan
>
> Hamburg University of Applied Sciences
> Dept. of Computer Science / Internet Technologies Group
> Berliner Tor 7, 20099 Hamburg, Germany
> Fon: +49 40 42875 - 8426
> Mail: 
> [email-masked]<mailto:[email-masked]>
> Web: https://www.inet.haw-hamburg.de/
> _______________________________________________
> icnrg mailing list
> [email-masked]<mailto:[email-masked]>
> https://www.irtf.org/mailman/listinfo/icnrg

> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg

---

From: Luca Muscariello <[email-masked]>
Date: Mon, 1 Oct 2018 14:59:18 +0200

> robust system (where all the data in conveyed in congestion-controlled

When putting HTTP on top of CCNx and hICN we decided to keep
a clear separation between the application and transport layers.
The implementation available in https://git.fd.io in the cicn project
makes use of producer and consumer sockets that are used to
send or recv data.

An HTTP GET with client cookies and payload must be reverse pulled by the
HTTP server.
Same thing for and HTTP POST.

For other application level protocols such as RTP there is no request to
map into
interests. At leat one cannot be tempted to map non isomorphic things
together
such as application level requests to interests.

The assumption is that the application must be able to bind meaningful
resource names
such as HTTP resources and RTP media sources (A/V) and streams.
This also assumes name mapping and/or resolution between the two layers.

Dirk, if you think about zero-RTT transport such as TCP fast open,
I would tend to re-use the Interest payload just as Marc described.
E.g. as a fast open of the transaction where the Interest payload
accelerates opening,
This makes sense for STREAM like sockets where the open is essentially used
to exchange
one or two manifests (in both directions) or to send the first MTU of data.

For DATAGRAM like sockets such as Cenk’s IoT scenario
where a sensor might need to notify data to some IoT GW, I would still
prefer
the data consumer (e.g. IoT GW) to pull as it is the one that must be able
to
schedule several READ and store data locally.

In between the two examples, real-time data streaming protocols (Kafka, RPC
etc.) are useful examples to
observe that flow balance in the transport is v v  useful to manage
asynchronous streams of
data with built-in back-pressure (aka rcv flow control) w/o redoing that
into the app.
Said in a different way, I would think twice before pushing data if you
don't know if the app can read and store that now.

> could be dealt with in different ways. In c, they