Subject: Re: [icnrg] icnrg Digest, Vol 97, Issue 1

From: Ken Calvert <[email-masked]>
Date: Tue, 7 Apr 2020 16:49:06 -0400

Dave and Dirk (and all) -

Thanks for writing up the reflexive forwarding scheme!  The draft describes it clearly.  (I have some minor presentation/typo comments that I'll send separately.)
I like the approach of having a separate FIB for this purpose!
> Since this is a fairly major change/enhancement to the CCNx forwarding model, it really needs a close review to help us figure out:
> 
> Are we crazy?
No more so than the rest of us, I think. :-)
> Is the complexity/functionality tradeoff appropriate?
I think that is the big question, and I'm still mulling it.  See below for some general comments.
> Did we miss anything, particularly any corner cases we should have considered
One thing I found myself wondering about is the definition of "next/previous hop".  All the diagrams and explanations seem to implicitly assume it's a single node.  What happens if a forwarder broadcasts a regular interest and it is received (and forwarded on) by multiple next hops? Is it worth mentioning that possibility so you can say explicitly that it's fine (or not)?

Along those lines, if a "next hop" is not identified with a specific machine, it seems like anyone that can get a packet to the upstream (toward the server) interface can blow away the reflexive FIB entry by sending a bogus Data packet that matches the interest.  But that is also true for PIT entries, so maybe this problem has a known solution (apologies for not knowing it).

My other comment is that the PIT entry lifetime-lengthening algorithm seems pretty complicated.  Why wouldn't it be sufficient to just put it on the consumer to set the lifetime of the original interest to be longer, and have forwarders admit a somewhat higher max lifetime for interests that carry a reflexive name prefix?  The application has to know whether the server might request information from it, so that seems cleaner, but I may be missing something.  I get nervous about per-packet state that can be renewed (see https://dl.acm.org/doi/10.1145/964725.633051 <https://dl.acm.org/doi/10.1145/964725.633051>).
> Which of the alternative approaches to backward/forward compatibility we describe seems the right one?
It seems to me there are two aspects to consider and they are mostly orthogonal:

(i) Enable neighboring forwarders to know each others' capabilities so they can construct and use routes that fully support reflexive forwarding (if they exist).  You posit two solutions to this, version numbers and a (new, to be designed) capabiliites-exchange protocol.

(ii) What should happen when an interest encounters a "gap" in support for reflexive forwarding? This can still happen, even with a solution to (i), and it can happen in the middle of the network. ISTM a solution to (i) would just (at best) shift who gets the "no route" error. In that case the consumer can get it when an interest containing a reflexive name reaches a forwarder that supports it, but which doesn't know a compliant route to the producer.  In every other case the producer can still get "no route to destination".   Apologies if I am missing something.
> Are there more use cases that we ought to document? We were hoping to have one or two to include for state synchronization, but have deferred that for now in the interest of getting the spec out for review.
> We’d like to talk about this at the ICNRG Interim meeting on April 20, so p[lease sets aside some time to read/eview/think about between now and then
> 
> 

I look forward to the discussion!

Ken

---

From: "David R. Oran" <[email-masked]>
Date: Wed, 08 Apr 2020 09:47:43 -0400

These are super-useful comments - many thanks. Also looking forward to 
getting the typos/presentation comments so we can re-spin the draft.

some thoughts below.

On 7 Apr 2020, at 16:49, Ken Calvert wrote:

> Thanks for writing up the reflexive forwarding scheme!  The draft 
> describes it clearly.  (I have some minor presentation/typo comments 
> that I'll send separately.)
>> Since this is a fairly major change/enhancement to the CCNx 
>> forwarding model, it really needs a close review to help us figure 
>> out:
>>
>> Are we crazy?
> One thing I found myself wondering about is the definition of 
> "next/previous hop".  All the diagrams and explanations seem to 
> implicitly assume it's a single node.  What happens if a forwarder 
> broadcasts a regular interest and it is received (and forwarded on) by 
> multiple next hops? Is it worth mentioning that possibility so you can 
> say explicitly that it's fine (or not)?
>
The fact that you mention it indicates we probably ought to say more. I 
think we’re fine in terms of things working both correctly and not 
violating any principle of least astonishment. There are two sub-cases 
of multi-next hop behavior; regular multi-path (where the split traffic 
reconverges further upstream) and multi-destination (where it doesn’t 
and the Interest reaches multiple producers).

For multi-path, since the Interests that converge upstream carry 
identical reflexive Interest name TLVs, they will get aggregated. Only 
one FIB entry will be created, pointing back to the previous hop(s) the 
Interest arrived on. The forwarder might, just as for any other 
Interest, decide to either do single or multi-path forwarding of that 
reflexive interest. If sent multi-path in parallel, these also will 
reconverge on the inverse path and get aggregated.

For multi-destination, reflexive Interests might get issued by multiple 
producers, but they will carry the same reflexive name prefix and hence 
be forwarded over the reflexive FIB entries until reaching the join 
point, at which they will get aggregated and thus handled identically to 
any other Interest(s) subject to aggregation.

We’ll craft some language like what’s written above and add it to 
the section on interest aggregation.

> Along those lines, if a "next hop" is not identified with a specific 
> machine, it seems like anyone that can get a packet to the upstream 
> (toward the server) interface can blow away the reflexive FIB entry by 
> sending a bogus Data packet that matches the interest.  But that is 
> also true for PIT entries, so maybe this problem has a known solution 
> (apologies for not knowing it).
>
On-path forwarders can do anything, including poisoning anything 
downstream. The only remedy in CCNx is to utilize content-hash 
restrictions and key restrictions that will bypass malicious forwarders 
if possible. I don’t think there’s anything new here.

> My other comment is that the PIT entry lifetime-lengthening algorithm 
> seems pretty complicated.  Why wouldn't it be sufficient to just put 
> it on the consumer to set the lifetime of the original interest to be 
> longer, and have forwarders admit a somewhat higher max lifetime for 
> interests that carry a reflexive name prefix?  The application has to 
> know whether the server might request information from it, so that 
> seems cleaner, but I may be missing something.  I get nervous about 
> per-packet state that can be renewed (see 
> https://dl.acm.org/doi/10.1145/964725.633051 
> <https://dl.acm.org/doi/10.1145/964725.633051>).
Gee, I remember that CCR paper! Very nice - predates the work on tiny 
packet programs too I think.
You may very well be right that we picked the wrong tradeoff and the 
complexity and the additional hazard that has to be dealt with by 
consumers is not justified when the consumer can just inflate the 
original interest lifetime. Our concern was that consumers will have no 
motivation to even try to guess a good interest lifetime since they 
don’t know how many additional RTTs may be involved, and pick very 
large values, causing long recovery times on Interest loss. There’s a 
whole line of reasoning about these tradeoffs in the work my group did 
on timers for ICN back a few years ago. If the probability of 
**undetected** Interest loss is vanishing small (which it can be if you 
have robust hop-by-hop reliability mechanisms) then inflated interest 
lifetimes are not a big concern. However, we don’t have these 
mechanisms now for all link types, and for IP tunnels the loss 
probability can be pretty high. I’ve attached a copy of the 
presentation for refresh peoples’ memories.

Bottom line is that we’d really like more feedback on whether the cure 
of having forwarders inflate the interest lifetime is worse than the 
disease of consumers guessing.

> (i) Enable neighboring forwarders to know each others' capabilities so 
> they can construct and use routes that fully support reflexive 
> forwarding (if they exist).  You posit two solutions to this, version 
> numbers and a (new, to be designed) capabiliites-exchange protocol.
>
I’d love to have a capabilities negotiation protocol. Any takers to 
define one (that really uses one-hop Interest-Data exchanges and not 
some push-mode hello protocol)?

> (ii) What should happen when an interest encounters a "gap" in support 
> for reflexive forwarding? This can still happen, even with a solution 
> to (i), and it can happen in the middle of the network.
> ISTM a solution to (i) would just (at best) shift who gets the "no 
> route" error.
I’m not sure “No Route” is the right error in this case, but 
that’s a secondary consideration.

> In that case the consumer can get it when an interest containing a 
> reflexive name reaches a forwarder that supports it, but which doesn't 
> know a compliant route to the producer.
Right, perhaps no route is the right error, but this needs more thought.

> In every other case the producer can still get "no route to 
> destination".   Apologies if I am missing something.
There are a few reasons why a producer might get no route on a reflexive 
interest:
1) The original Interest timed out
2) The network broke (possibly via a transient)
3) Forwarder bug
I suspect all of these basically devolve to the producer wanting to just 
abandon what he was doing and try to do an Interest Return on the 
original Interest with an appropriate error (not sure which one to use - 
this case might just wind being a generic IE.AFU).

>> Are there more use cases that we ought to document? We were hoping to 
>> have one or two to include for state synchronization, but have 
>> deferred that for now in the interest of getting the spec out for 
>> review.
>> We’d like to talk about this at the ICNRG Interim meeting on April 
>> 20, so p[lease sets aside some time to read/eview/think about between 
>> now and then
>>
>>
>

DaveO

---

From: Ken Calvert <[email-masked]>
Date: Thu, 9 Apr 2020 12:04:34 -0400

Let me say that I believe the benefits of this reflexive forwarding scheme, in terms of keeping interests small and free of client-identifying information, are worth the added complexity - which is maybe not as significant as I first thought.
> You may very well be right that we picked the wrong tradeoff and the complexity and the additional hazard that has to be dealt with by consumers is not justified when the consumer can just inflate the original interest lifetime. Our concern was that consumers will have no motivation to even try to guess a good interest lifetime since they don’t know how many additional RTTs may be involved, and pick very large values, causing long recovery times on Interest loss. There’s a whole line of reasoning about these tradeoffs in the work my group did on timers for ICN back a few years ago. If the probability of undetected Interest loss is vanishing small (which it can be if you have robust hop-by-hop reliability mechanisms) then inflated interest lifetimes are not a big concern. However, we don’t have these mechanisms now for all link types, and for IP tunnels the loss probability can be pretty high. I’ve attached a copy of the presentation for refresh peoples’ memories.
> 
I was originally thinking that routers would (for engineering reasons) clamp interest lifetimes in their PITs in any case, and that the application would have a good idea of the number of additional round-trips.  Would it be reasonable to specify a design limit for this mechanism?  Say, enough for two round trips plus some transmit time - enough to get a manifest plus a limited amount of content. Seems like there are reasonable alternatives that applications needing a lot of info from the consumer could make use of, if Interest lifetimes are known to be absolutely bounded.

The other place where I had a (second order) comment is Section 4 - specifically, the enumeration of cases for contents of the Reflexive Name Prefix TLV.  First, allowing an option and then recommending against it (#4, multiple Reflexive Name TLVs) seems like not a good idea, especially for a mechanism that needs to be supported along the entire path.  If there's a compelling use case that requires this, it would be good to describe it explicitly.  Otherwise, it seems like a single TLV containing a reflexive prefix + optional consumer-defined suffix should suffice.

Cheers,
Ken

---

From: "David R. Oran" <[email-masked]>
Date: Thu, 09 Apr 2020 13:15:55 -0400

On 9 Apr 2020, at 12:04, Ken Calvert wrote:

> Let me say that I believe the benefits of this reflexive forwarding 
> scheme, in terms of keeping interests small and free of 
> client-identifying information, are worth the added complexity - which 
> is maybe not as significant as I first thought.
>> original interest lifetime. Our concern was that consumers will have 
>> no motivation to even try to guess a good interest lifetime since 
>> they don’t know how many additional RTTs may be involved, and pick 
>> very large values, causing long recovery times on Interest loss. 
>> There’s a whole line of reasoning about these tradeoffs in the work 
>> my group did on timers for ICN back a few years ago. If the 
>> probability of undetected Interest loss is vanishing small (which it 
>> can be if you have robust hop-by-hop reliability mechanisms) then 
>> inflated interest lifetimes are not a big concern. However, we 
>> don’t have these mechanisms now for all link types, and for IP 
>> tunnels the loss probability can be pretty high. I’ve attached a 
>> copy of the presentation for refresh peoples’ memories.
>>
> I was originally thinking that routers would (for engineering reasons) 
> clamp interest lifetimes in their PITs in any case, and that the 
> application would have a good idea of the number of additional 
> round-trips.  Would it be reasonable to specify a design limit for 
> this mechanism?  Say, enough for two round trips plus some transmit 
> time - enough to get a manifest plus a limited amount of content. 
> Seems like there are reasonable alternatives that applications needing 
> a lot of info from the consumer could make use of, if Interest 
> lifetimes are known to be absolutely bounded.
>
That may be, in which case the added complexity is not warranted. I’m 
interested on what others (particularly people who have implemented 
high-speed forwarders) have to say about this tradeoff. My personal view 
on this tradeoff is highly dependent on how frequent undetected interest 
loss is, since that makes the recovery penalty from over-estimating 
interest lifetime potentially very severe.

> The other place where I had a (second order) comment is Section 4 - 
> specifically, the enumeration of cases for contents of the Reflexive 
> Name Prefix TLV.  First, allowing an option and then recommending 
> against it (#4, multiple Reflexive Name TLVs) seems like not a good 
> idea, especially for a mechanism that needs to be supported along the 
> entire path.  If there's a compelling use case that requires this, it 
> would be good to describe it explicitly.  Otherwise, it seems like a 
> single TLV containing a reflexive prefix + optional consumer-defined 
> suffix should suffice.
>
We left the option there so that we haven’t boxed ourselves in if a 
use case appears. If we decide to explicitly disallow more than one 
reflexive interest TLV, there’s a convenient T_MALFORMED_INTEREST 
error that can be returned. Would obviously like more input on this 
issue as well.

> _______________________________________________
> icnrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/icnrg

DaveO