Subject: Re: [dtn] Custody

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 2 Jun 2016 20:54:40 +0000

Carlo, just to quibble with one point in your analysis: a triangular network (more generally, a triangular network "hop") does not necessarily make a chain of reliable CLAs impossible.  It only means that the only way for the CLA for such a hop to be reliable is for it to utilize the topological triangle itself, as a bespoke convergence-layer protocol.  In your A/B/C topology, for example, C can simply send positive and negative CL acknowledgments to A rather than to B.  This seems pretty reasonable to me: the purpose of convergence layer adaptation has always been to insulate BP from the eccentricities of the underlying networks that it has to span.

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Carlo Caini
Sent: Monday, May 23, 2016 3:13 AM
To: Eric Travis <[email-masked]>; Will Ivancic <[email-masked]>
Cc: [email-masked]
Subject: Re: [dtn] Custody

Dear all,
          I do not want to abuse of your patience, but maybe a comparison of custody vs reliable CLAs may be useful.
First, let me stress once again  than "full" reliability requires an ack from the destination; thus a chain of reliable CLAs or a chain of Custodians (doing re-transmissions and storage in permanent memory) cannot provide this level of reliability.
Does this mean that a chain of either reliable CLAs or Custodians are useless? The answer is no. Below a few reasons.
Does a chain of reliable CLAs is equivalent to a custody chain? No again.

I am sorry for the length; I tried hard to be as brief as possible, but also exhaustive; the topic is intrinsically complex. Last I could be wrong (the topic is complex). Please, consider all considerations below as my personal understanding of the topic.

Carlo

...

Unidirectional links (a chain of reliable CLAs is impossible, while a custody chain is) Let me assume a triangular network, A (source), B (intermediate node), C (destination). All links are reliable but the B-->C, which is unidirectional, thus unreliable by definition. All links have a high bandwidth but A-C, which forces the routing select the two-hop path A-B->C for big data. The fact that the A-B link is reliable means nothing, as is the NEXT B->C link is not (and cannot). If B acts as custodian, you can have automatic re-transmissions from B. B can still have custody acceptance from C via A.

Raliable CLAs chain vs Custody chain
Reliability of CLAs and Custody are both useful to assure a partial reliability (i.e. to increase the chances of successful final delivery without retransmissions from the sources). CLAs reliability and Custody are different, complementary and not alternative.

The case of unidirectional links shows a situation where a chain of reliable CLAs is not possible independently from the willingness of node owners, while a chain of Custodians is.

---

From: Carlo Caini <[email-masked]>
Date: Fri, 3 Jun 2016 13:23:39 +0200

Dear Scott,
       I am sorry but I am not fully convinced yet. In a
heterogeneous networks, node A, B and C can belong to different
entities. Your solution requires that:
1) it exist a CL that receives data from a node and sends ACKs to
another, if I have well understood. I would say that neither TCPCL or
LTPCL with RED can, at least in the current implementations.
2) (assumed that this CL is existent) C "know" that it should use it,
and b) be willing to do a thing that is neither defined nor even
mentioned in the BP specs. It seems barely possible, unless C manager
belongs to the same entity as A and B (e-g. private network).
Custody can be more effective, especially if coupled with CRC checks.

Let me change a little the example: the source (A) is a lunar
satellite, which has to transfer data to the MCC (C). It can do this,
for large bundles, only indirectly via a terrestrial receiving only
gateway B. The CL on B cannot do anything, as it is not in direct
contact with A; the CL on C must send its ACKs to B obviously. I
cannot see any alternative to custody (either on B or C) here.

I agree with you that the purpose of lower layers is to insulate
upper layers from the underlying problems (as this is the rationale
of layering). However, even if they try hard to do this, sometimes
they just cannot fully conceal the problems. This is why TCP's
end-to-end reliability is not, and cannot be theoretically
equivalent, to a chain of reliable link layers (e.g. internal
hardware&software failures, power offs, etc. cannot be covered).

Custody cannot offer end-to-end reliability either. However, it can
offer something intermediate between  hop-by-hop (i.e. CL's)
reliability and end-to-end reliability. Let me stress once again that
it can cover multiple hops, while CLs cannot. Thus the level of
reliability cannot be theoretically the same.

That said, I fully agree with you that Custody has its own limits
(e.g. how to set the re transmission time is a challenge just to
recall one). You have highlighted them very clearly in the other
mail. However, my personal opinion is that it is preferable an
optional custody with limits than nothing. Custody is an option both
for the source and for the intermediate nodes. You can avoid to set
it and also avoid to take it. But a custody without a minimum set of
services defined in the specs would prevent all other people who
consider it useful from using it in an effective way, unless in their
own private network.

Yours,
    Carlo

At 22:54 02/06/2016, Burleigh, Scott C (312B) wrote:
>Carlo, just to quibble with one point in your analysis: a triangular
>network (more generally, a triangular network "hop") does not
>necessarily make a chain of reliable CLAs impossible.  It only means
>that the only way for the CLA for such a hop to be reliable is for
>it to utilize the topological triangle itself, as a bespoke
>convergence-layer protocol.  In your A/B/C topology, for example, C
>can simply send positive and negative CL acknowledgments to A rather
>than to B.  This seems pretty reasonable to me: the purpose of
>convergence layer adaptation has always been to insulate BP from the
>eccentricities of the underlying networks that it has to span.
>
>Scott
>
>           I do not want to abuse of your patience, but maybe a
> comparison of custody vs reliable CLAs may be useful.
>First, let me stress once again  than "full" reliability requires an
>ack from the destination; thus a chain of reliable CLAs or a chain
>of Custodians (doing re-transmissions and storage in permanent
>memory) cannot provide this level of reliability.
>Does this mean that a chain of either reliable CLAs or Custodians
>are useless? The answer is no. Below a few reasons.
>I am sorry for the length; I tried hard to be as brief as possible,
>but also exhaustive; the topic is intrinsically complex. Last I
>could be wrong (the topic is complex). Please, consider all
>considerations below as my personal understanding of the topic.
>
>Carlo
>
>...
>
>Unidirectional links (a chain of reliable CLAs is impossible, while
>a custody chain is) Let me assume a triangular network, A (source),
>B (intermediate node), C (destination). All links are reliable but
>the B-->C, which is unidirectional, thus unreliable by definition.
>All links have a high bandwidth but A-C, which forces the routing
>select the two-hop path A-B->C for big data. The fact that the A-B
>link is reliable means nothing, as is the NEXT B->C link is not (and
>cannot). If B acts as custodian, you can have automatic
>re-transmissions from B. B can still have custody acceptance from C via A.
>
>Reliability of CLAs and Custody are both useful to assure a partial
>reliability (i.e. to increase the chances of successful final
>delivery without retransmissions from the sources). CLAs reliability
>and Custody are different, complementary and not alternative.
>
>The case of unidirectional links shows a situation where a chain of
>reliable CLAs is not possible independently from the willingness of
>node owners, while a chain of Custodians is.

5x1000 AI GIOVANI RICERCATORI
DELL'UNIVERSITÀ DI BOLOGNA
Codice Fiscale: 80007010376

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Fri, 3 Jun 2016 16:13:30 +0000

Hi, Carlo.  I certainly agree that neither the existing TCPCL or LTPCL adapters could handle the triangular network hop you described; as I said, a "bespoke" convergence-layer adapter specifically designed for this configuration (or perhaps more generally designed for this class of configurations) would be needed.  Developing such a CLA would take a bit of time and money, and so it is tempting simply to try to use BP custody transfer to accomplish the same thing.  I think this is in general a mistake because custody transfer is such an inefficient ARQ system, for the reasons I outlined in my earlier email, but this is a point on which we can agree to disagree.

And certainly even if this hypothetical triangular CLA (let's call it "XCL" for the moment) existed it would be necessary to configure it to operate properly in this specific case.  But there's nothing new there: we always have to do that with every CLA.  In some cases that configuration must be managed and in some cases it may be automatic, thanks to (for example) neighbor discovery, but it always has to be done.

You're right, clearly XCL would have to do a thing that is neither defined nor even mentioned in the BP specs, but that's also true of the LTP and TCP CLAs; that's the whole point of the convergence-layer concept.

And I don't think the ownership of the various entities is a significant issue: the Internet has many routers and hosts that are owned by different entities, yet we usually manage to configure them to interoperate correctly.

In your revised example I don't see any difference from the original case.  If B is receive-only then it can send neither XCL acknowledgments nor custody signals back to A.  If C is able to send custody signals back to A then it is just as able to send XCL acknowledgments back to A.  Custody transfer offers no advantage apart from making it unnecessary to develop XCL - which is a real advantage, but it's one that I think is more than offset by the inefficiency of custodial retransmission.

I also don't see much benefit in custody transfer's ability to acknowledge reception across multiple hops.  The only way to defend against the loss of data in retransmission buffers due to hardware failure (or whatever) anywhere in the end-to-end path is to require a true end-to-end acknowledgment.  The pseudo-end-to-end acknowledgments of custody transfer can accelerate recovery from such a failure in some cases, so if these failures are frequent then maybe custodial retransmission can be cost-justified.  In that case, though, I think your network has got problems that need to be addressed at a more fundamental level.

The one feature of BP custody transfer that I think is truly valuable is the one that is already clearly spelled out in RFC5050: in the event that the forwarding of a bundle reaches a dead end (a node that can't forward it any further, for whatever reason), we reduce network traffic somewhat if the dead-end node can send a small custody refusal signal - rather than the entire bundle - back to the point in the network at which a different path might be chosen.  I'm all for retaining BP custody transfer, for this reason alone.  But I think it's a mistake to rely on it for anything else.

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]] 
Sent: Friday, June 3, 2016 4:24 AM
To: Burleigh, Scott C (312B) <[email-masked]>; Eric Travis <[email-masked]>; Will Ivancic <[email-masked]>
Cc: [email-masked]
Subject: RE: [dtn] Custody

Dear Scott,
       I am sorry but I am not fully convinced yet. In a heterogeneous networks, node A, B and C can belong to different entities. Your solution requires that:
1) it exist a CL that receives data from a node and sends ACKs to another, if I have well understood. I would say that neither TCPCL or LTPCL with RED can, at least in the current implementations.
2) (assumed that this CL is existent) C "know" that it should use it, and b) be willing to do a thing that is neither defined nor even mentioned in the BP specs. It seems barely possible, unless C manager belongs to the same entity as A and B (e-g. private network).
Custody can be more effective, especially if coupled with CRC checks.

Let me change a little the example: the source (A) is a lunar satellite, which has to transfer data to the MCC (C). It can do this, for large bundles, only indirectly via a terrestrial receiving only gateway B. The CL on B cannot do anything, as it is not in direct contact with A; the CL on C must send its ACKs to B obviously. I cannot see any alternative to custody (either on B or C) here.

I agree with you that the purpose of lower layers is to insulate upper layers from the underlying problems (as this is the rationale of layering). However, even if they try hard to do this, sometimes they just cannot fully conceal the problems. This is why TCP's end-to-end reliability is not, and cannot be theoretically equivalent, to a chain of reliable link layers (e.g. internal hardware&software failures, power offs, etc. cannot be covered).

Custody cannot offer end-to-end reliability either. However, it can offer something intermediate between  hop-by-hop (i.e. CL's) reliability and end-to-end reliability. Let me stress once again that it can cover multiple hops, while CLs cannot. Thus the level of reliability cannot be theoretically the same.

That said, I fully agree with you that Custody has its own limits (e.g. how to set the re transmission time is a challenge just to recall one). You have highlighted them very clearly in the other mail. However, my personal opinion is that it is preferable an optional custody with limits than nothing. Custody is an option both for the source and for the intermediate nodes. You can avoid to set it and also avoid to take it. But a custody without a minimum set of services defined in the specs would prevent all other people who consider it useful from using it in an effective way, unless in their own private network.

Yours,
    Carlo

At 22:54 02/06/2016, Burleigh, Scott C (312B) wrote:
>that the only way for the CLA for such a hop to be reliable is for it 
>to utilize the topological triangle itself, as a bespoke 
>convergence-layer protocol.  In your A/B/C topology, for example, C can 
>simply send positive and negative CL acknowledgments to A rather than 
>to B.  This seems pretty reasonable to me: the purpose of convergence 
>layer adaptation has always been to insulate BP from the eccentricities 
>of the underlying networks that it has to span.
>
>Scott
>
>ack from the destination; thus a chain of reliable CLAs or a chain of 
>Custodians (doing re-transmissions and storage in permanent
>Does this mean that a chain of either reliable CLAs or Custodians are 
>useless? The answer is no. Below a few reasons.
>I am sorry for the length; I tried hard to be as brief as possible, but 
>also exhaustive; the topic is intrinsically complex. Last I could be 
>wrong (the topic is complex). Please, consider all considerations below 
>as my personal understanding of the topic.
>
>Carlo
>
>...
>
>Unidirectional links (a chain of reliable CLAs is impossible, while a 
>custody chain is) Let me assume a triangular network, A (source), B 
>(intermediate node), C (destination). All links are reliable but the 
>B-->C, which is unidirectional, thus unreliable by definition.
>select the two-hop path A-B->C for big data. The fact that the A-B link 
>is reliable means nothing, as is the NEXT B->C link is not (and 
>reliability (i.e. to increase the chances of successful final delivery 
>without retransmissions from the sources). CLAs reliability and Custody 
>are different, complementary and not alternative.
>

---

From: Carlo Caini <[email-masked]>
Date: Fri, 3 Jun 2016 20:40:48 +0200

Dear Scott,
        Just a clarification concerning the revised example:
At 18:13 03/06/2016, Burleigh, Scott C (312B) wrote:
>In your revised example I don't see any difference from the original
>case.  If B is receive-only then it can send neither XCL
>acknowledgments nor custody signals back to A.

I do apologize. I was maybe unclear in the text. A->B is
unidirectional, so B cannot send anything back directly as you have
pointed out. However, it can send custody acceptance indirectly to A,
via C (B-C-A). Martin Lulf, a student of TUM (Munich) and I did this
with ION some years ago, and worked fine. Actually, I was impressed
by the elegancy of the solution (the idea was by Martin, that had to
face this problem in the premises of his university), and this is why
I am so passionate (i.e. biased in favor of custody if you like)
about the custody. The problem was that integrity had to be checked,
not to take in custody a corrupted bundle, which was not possible
without CRCs. Now we have them.

To send back XCL ACKs to A in the same way, i.e. from B to A via C,
you would need the new XCL not only on C, as in the previous example,
but also on B.
A second additional problem is how to transfer these A->B XCL ACKs
from B to C. They are not bundles, so they cannot be moved by a CL from B to C.
Maybe you would need to encapsulate them in bundles in B first, so
that they can be then transferred by a CL from B to C. Here they
could be de-encapsulated and sent back to A by XCL on C. Maybe you
have a better solution than mine.
Anyway, this does not change the rationale of your arguments against
custody. I have added these comments just to better explain to all
why the new example seemed to me more challenging.
Yours,
    Carlo

>If C is able to send custody signals back to A then it is just as
>able to send XCL acknowledgments back to A.  Custody transfer offers
>no advantage apart from making it unnecessary to develop XCL - which
>is a real advantage, but it's one that I think is more than offset
>by the inefficiency of custodial retransmission.

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Fri, 3 Jun 2016 19:28:11 +0000

Ah, okay, I think I see what you were getting at.  You're right, the B->C->A custody signal path is an elegant solution.  I can think of a few alternatives that I think would work as nicely, without suffering the problems that are innate in BP custody transfer, but that's something to explore at another time.

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]] 
Sent: Friday, June 3, 2016 11:41 AM
To: Burleigh, Scott C (312B) <[email-masked]>; [email-masked]
Subject: Re: [dtn] Custody

Dear Scott,
>acknowledgments nor custody signals back to A.

I do apologize. I was maybe unclear in the text. A->B is unidirectional, so B cannot send anything back directly as you have pointed out. However, it can send custody acceptance indirectly to A, via C (B-C-A). Martin Lulf, a student of TUM (Munich) and I did this with ION some years ago, and worked fine. Actually, I was impressed by the elegancy of the solution (the idea was by Martin, that had to face this problem in the premises of his university), and this is why I am so passionate (i.e. biased in favor of custody if you like) about the custody. The problem was that integrity had to be checked, not to take in custody a corrupted bundle, which was not possible without CRCs. Now we have them.

To send back XCL ACKs to A in the same way, i.e. from B to A via C, you would need the new XCL not only on C, as in the previous example, but also on B.
A second additional problem is how to transfer these A->B XCL ACKs from B to C. They are not bundles, so they cannot be moved by a CL from B to C.
Maybe you would need to encapsulate them in bundles in B first, so that they can be then transferred by a CL from B to C. Here they could be de-encapsulated and sent back to A by XCL on C. Maybe you have a better solution than mine.
Anyway, this does not change the rationale of your arguments against custody. I have added these comments just to better explain to all why the new example seemed to me more challenging.
Yours,
    Carlo

>If C is able to send custody signals back to A then it is just as able 
>to send XCL acknowledgments back to A.  Custody transfer offers no 
>advantage apart from making it unnecessary to develop XCL - which is a 
>real advantage, but it's one that I think is more than offset by the 
>inefficiency of custodial retransmission.