Subject: [dtn] Bundle custody transfer and reliable CLs

From: "Sipos, Brian J." <[email-masked]>
Date: Thu, 24 Mar 2022 20:38:52 +0000

All,

There was some brief discussion during the BIBE presentation about custody
transfer concept and CL mechanisms. This is also an open topic in the CCSDS
drafting of BPv7 standardization. I would like to add some additional points
for thought in how a reliable convergence layer can relate to the concept of
custody transfer for agents on either side of the transfer.

Overall, there still seems to be some vagueness about what "custody" means
(in the sense of a service level agreement) between peers exchanging
bundles. My understanding is that "custody" means the custodial node is
willing to make some kind of effort to keep the bundle moving toward its
destination until the bundle lifetime expires.

The current RFC 9174 document is silent about what exactly an XFER_ACK
segment with END flag set (an END ACK) means from the perspective of the BP
agent and what is guaranteed about the transferred bundle. This provides an
opportunity for a follow-on clarification of END ACK semantics for the TCPCL
entity and for the BP agent. Two potential ways of making the behavior more
well-defined:

1.      A network-specific profile of TCPCL could simply mandate that any
node accepts custody by sending an END ACK. This would simply be a condition
of conformance to the profile in a controlled network. This could be done
immediately without any change elsewhere, but needs out-of-band
coordination.

2.      One or more (quite simple) extension types can be defined for TCPCL
to allow an entity to expose its END ACK behavior (RX) and desire (TX):

a.      A session extension can allow an entity to assert what its sent END
ACK means for received transfers. The value in this is to allow the peer
entity to adjust behavior depending on the capability (e.g. use BIBE if the
next-hop doesn't take END ACK custody), including possibly refusing to
establish a session with (or refusing to send bundles to) a peer that does
not take custody via END ACK.

b.      Additionally, a transfer extension can allow a sender to assert its
custody desire on a per-bundle basis (signaling that some bundles need
custody transfer while others do not). The value in this is to allow the
receiving entity to optimize its behavior based on whether or not custody is
needed for a bundle; though I don't know how much benefit this would be.

The possible values enumerated by the session extension would be something
like:

.        Custody is not taken at END ACK

.        Custody is taken at END ACK

And if there is a transfer extension defined, a the session extension could
indicate:

.        Reception behavior is unconditional

.        Reception behavior can be overridden per-transfer based on the
sender's desire

These changes would all be backward compatible in the sense that a default
policy would be in place in the absence of this extension item. And all of
this is an independent mechanism from BIBE for a custody transfer to take
place; both this mechanism and BIBE have their own costs, benefits, and side
effects of such a transfer.

Trying to make almost-there-already capabilities more obvious,

Brian S.

---

From: <[email-masked]>
Date: Thu, 24 Mar 2022 15:03:35 -0700

Brian, there was some further discussion of custody transfer in this
morning's meeting of the CCSDS Space Internetworking Systems DTN Working
Group as well.  A couple of notes:

It's important to remember that we are talking about state machines here,
not people.  Anthropomorphizing the DTN architecture is tempting but
treacherous.  BP nodes have no will; they cannot take responsibility; they
cannot promise to do anything.  All they do is behave, ideally in a fashion
that conforms to the protocol specifications to which they were allegedly
developed.

Also, any given node may have been designed with malign intent or
implemented with errors.  Stating a requirement in a protocol specification
does not ensure its satisfaction; what it does is give a node's human (maybe
eventually AI) network operators a means of assessing the behavior of
another node and possibly taking some sort of out-of-band administrative
action in defense against that behavior as needed.

You're right, the term "custody" is not defined for BPv7.  It is still
widely used to refer to some behavior that future users of DTN for space
flight operations state will be very important, but for which the
requirements are not yet clearly established.  It is starting to look like
the BP-based ARQ in the most recent BIBE draft, while needed (I think), is
not exactly what people mean by "custody transfer."  So it may become useful
to define an additional BP extension (TBD) that we label with this term.

TCPCL enhancements along the lines you propose here may very well be
valuable; I don't know, need to think about them some more.  But I would say
the BPv7 specification already contains language (in 5.4, Step 5 and the
following two paragraphs, and in 7.2, second bullet point) constraining the
sort of convergence-layer reliability that I think is indispensable,
regardless of what we call it.

Scott

From: dtn <[email-masked]> On Behalf Of Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To: [email-masked]
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

*	Custody is not taken at END ACK
*	Custody is taken at END ACK

*	Reception behavior is unconditional
*	Reception behavior can be overridden per-transfer based on the
sender's desire

Brian S.

---

From: William Ivancic <[email-masked]>
Date: Thu, 24 Mar 2022 20:14:00 -0400

“Custody” is  a bundle level concept, not transport.  Way back when, “Custody” meant that the node taking “Custody” would try it’s best to ensure the bundle was forwarded either to another “Custody” node or to eventually the destination node.  The idea, as I recall, was this would allow the original bundle source to clear its memory of that bundle.  

For space operations, I don’t think the operations people were ever comfortable with this concept.

//Will

From: dtn <[email-masked]> on behalf of <[email-masked]>
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Brian, there was some further discussion of custody transfer in this morning’s meeting of the CCSDS Space Internetworking Systems DTN Working Group as well.  A couple of notes:

It’s important to remember that we are talking about state machines here, not people.  Anthropomorphizing the DTN architecture is tempting but treacherous.  BP nodes have no will; they cannot take responsibility; they cannot promise to do anything.  All they do is behave, ideally in a fashion that conforms to the protocol specifications to which they were allegedly developed.

Also, any given node may have been designed with malign intent or implemented with errors.  Stating a requirement in a protocol specification does not ensure its satisfaction; what it does is give a node’s human (maybe eventually AI) network operators a means of assessing the behavior of another node and possibly taking some sort of out-of-band administrative action in defense against that behavior as needed.

You’re right, the term “custody” is not defined for BPv7.  It is still widely used to refer to some behavior that future users of DTN for space flight operations state will be very important, but for which the requirements are not yet clearly established.  It is starting to look like the BP-based ARQ in the most recent BIBE draft, while needed (I think), is not exactly what people mean by “custody transfer.”  So it may become useful to define an additional BP extension (TBD) that we label with this term.

TCPCL enhancements along the lines you propose here may very well be valuable; I don’t know, need to think about them some more.  But I would say the BPv7 specification already contains language (in 5.4, Step 5 and the following two paragraphs, and in 7.2, second bullet point) constraining the sort of convergence-layer reliability that I think is indispensable, regardless of what we call it.

Scott

All,

There was some brief discussion during the BIBE presentation about custody transfer concept and CL mechanisms. This is also an open topic in the CCSDS drafting of BPv7 standardization. I would like to add some additional points for thought in how a reliable convergence layer can relate to the concept of custody transfer for agents on either side of the transfer.

Overall, there still seems to be some vagueness about what “custody” means (in the sense of a service level agreement) between peers exchanging bundles. My understanding is that “custody” means the custodial node is willing to make some kind of effort to keep the bundle moving toward its destination until the bundle lifetime expires.

The current RFC 9174 document is silent about what exactly an XFER_ACK segment with END flag set (an END ACK) means from the perspective of the BP agent and what is guaranteed about the transferred bundle. This provides an opportunity for a follow-on clarification of END ACK semantics for the TCPCL entity and for the BP agent. Two potential ways of making the behavior more well-defined:

1.       A network-specific profile of TCPCL could simply mandate that any node accepts custody by sending an END ACK. This would simply be a condition of conformance to the profile in a controlled network. This could be done immediately without any change elsewhere, but needs out-of-band coordination.

2.       One or more (quite simple) extension types can be defined for TCPCL to allow an entity to expose its END ACK behavior (RX) and desire (TX):

a.       A session extension can allow an entity to assert what its sent END ACK means for received transfers. The value in this is to allow the peer entity to adjust behavior depending on the capability (e.g. use BIBE if the next-hop doesn’t take END ACK custody), including possibly refusing to establish a session with (or refusing to send bundles to) a peer that does not take custody via END ACK.

b.      Additionally, a transfer extension can allow a sender to assert its custody desire on a per-bundle basis (signaling that some bundles need custody transfer while others do not). The value in this is to allow the receiving entity to optimize its behavior based on whether or not custody is needed for a bundle; though I don’t know how much benefit this would be.

The possible values enumerated by the session extension would be something like:

·         Custody is not taken at END ACK

·         Custody is taken at END ACK

And if there is a transfer extension defined, a the session extension could indicate:

·         Reception behavior is unconditional

·         Reception behavior can be overridden per-transfer based on the sender’s desire

These changes would all be backward compatible in the sense that a default policy would be in place in the absence of this extension item. And all of this is an independent mechanism from BIBE for a custody transfer to take place; both this mechanism and BIBE have their own costs, benefits, and side effects of such a transfer.

Brian S.

_______________________________________________ dtn mailing list [email-masked] https://www.ietf.org/mailman/listinfo/dtn

---

From: [email-masked]
Date: Fri, 25 Mar 2022 11:11:12 +0100

(cross-posting to CCSDS DTN mailinglist as it seems to be of high 
relevance to the on-going discussions in the DTN WG).

I think we clearly need to distinguish between (at least) two different 
types of 'custody transfer' according to where it is implemented:

(a) BPv6 - like custody transfer which has been a function of the BPA / AA 
Administrative element
(b) BIBE-like custody transfer which is implemented in the CLA

The main differences I see are:
- In (a) the decision whether to take custody or not can be taken on more 
available information, eg timely availability of a route toward a next 
hop, available storage, policies (eg, checking Bundle Integrity Blocks), 
... In (b), the CLA may accept custody and the BPA would just decide to 
discard the bundle for whatever reason (of course, there could be 
interfaces to make such information available to the CLA but this would 
somehow 'blur' the architecture.
- In (b), the node to take custody needs to be explicitly addressed (it's 
the BIBE destination) while in (a) any node could take custody.

Requirements regarding custody transfer I would see for space missions 
are:

1) Assertion of a high probability that a bundle will reach its 
destination once a hop has accepted custody (which would allow the 
forwarding node to release storage).
The meaning of 'high probability' does really depend on the mission data 
return requirements. While for some missions it may be close to 100%, 
other mission may be ready to accept higher data loss in favour of 
timeliness (eg, certain types of Earth Observation missions).

2) Forwarding bundles without knowing which node will take custody.
In particular with high data rates and optical direct-to-Earth downlinks 
we may have situations where the spacecraft may not know the actual next 
hop is sending to but may want to get a confirmation that the bundle has 
been received on ground. With high-data rates, bundles might already be 
prepared and encoded in frames and be sitting in some buffers within 
optical terminal because the data rates on the on-board buses would not 
allow to generate and send in real time. Use of optical direct-to-Earth 
links may be opportunistic and we may not know in advance how much data 
will go down. So, addressing a specific DTN node in a ground station 
becomes unpractical (if DTN nodes in ground station would share the same 
anycast eID it may be possible but BIBE is currently limited to singleton 
endpoints).

It is clear that these requirements cannot be solved by protocol 
specification as Scott pointed out below but will also require that 
implementing nodes conform to certain behaviours. This will not be 
possible for an open, Internet-like DTN network (where we can only try to 
take defensive actions). For space missions I would still expect limited, 
tightly-controlled network for some time to come (maybe becoming 'trusted 
zones' in a larger network). For these, we should have protocol mechanism 
which can support above requirements (while being aware that some of these 
mechanisms will not work in a fully open DTN).

Finally, 'custody transfer' seems to be always related to timer-base 
re-transmission. However, I think there are other options as well, like:
- command-based re-transmission: an explicit command is sent to a DTN node 
to re-transmit all bundles for which custody has been requested but no 
signal has been received
- sequence-based re-transmission: in some situations it might be possible 
(using additional extension blocks) to detect which bundles have not been 
received by inspecting the received custody signals and re-send the ones 
for which no custody signal have been received
Again, this would likely not work in an open DTN but only in (very) 
limited, controlled networks. But this doesn't make such mechanisms less 
useful (although I would prefer to have something more generic if 
possible) and we should address it in the IETF (if there is general 
interest) and/or the CCSDS (if it is for near-to-medium term space mission 
use cases only). 

Regarding reliable CLs: I currently don't see the point of a reliable CL 
taking custody because it would be (only) type (b) custody which is 
basically just a confirmation that the bundle has been received and this 
can already be assumed by the fact that it is a reliable CL.

Regards,
Felix

From:   "William Ivancic" <[email-masked]>
To:     <[email-masked]>, "'Sipos, Brian J.'" 
<[email-masked]>, <[email-masked]>
Date:   25/03/2022 01:16
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs
Sent by:        "dtn" <[email-masked]>

?Custody? is  a bundle level concept, not transport.  Way back when, 
?Custody? meant that the node taking ?Custody? would try it?s best to 
ensure the bundle was forwarded either to another ?Custody? node or to 
eventually the destination node.  The idea, as I recall, was this would 
allow the original bundle source to clear its memory of that bundle.  
 
For space operations, I don?t think the operations people were ever 
comfortable with this concept.
 
//Will

Brian, there was some further discussion of custody transfer in this 
morning?s meeting of the CCSDS Space Internetworking Systems DTN Working 
Group as well.  A couple of notes:
 
It?s important to remember that we are talking about state machines here, 
not people.  Anthropomorphizing the DTN architecture is tempting but 
treacherous.  BP nodes have no will; they cannot take responsibility; they 
cannot promise to do anything.  All they do is behave, ideally in a 
fashion that conforms to the protocol specifications to which they were 
allegedly developed.
 
Also, any given node may have been designed with malign intent or 
implemented with errors.  Stating a requirement in a protocol 
specification does not ensure its satisfaction; what it does is give a 
node?s human (maybe eventually AI) network operators a means of assessing 
the behavior of another node and possibly taking some sort of out-of-band 
administrative action in defense against that behavior as needed.
 
You?re right, the term ?custody? is not defined for BPv7.  It is still 
widely used to refer to some behavior that future users of DTN for space 
flight operations state will be very important, but for which the 
requirements are not yet clearly established.  It is starting to look like 
the BP-based ARQ in the most recent BIBE draft, while needed (I think), is 
not exactly what people mean by ?custody transfer.?  So it may become 
useful to define an additional BP extension (TBD) that we label with this 
term.
 
TCPCL enhancements along the lines you propose here may very well be 
valuable; I don?t know, need to think about them some more.  But I would 
say the BPv7 specification already contains language (in 5.4, Step 5 and 
the following two paragraphs, and in 7.2, second bullet point) 
constraining the sort of convergence-layer reliability that I think is 
indispensable, regardless of what we call it.
 
Scott

All,
There was some brief discussion during the BIBE presentation about custody 
transfer concept and CL mechanisms. This is also an open topic in the 
CCSDS drafting of BPv7 standardization. I would like to add some 
additional points for thought in how a reliable convergence layer can 
relate to the concept of custody transfer for agents on either side of the 
transfer.
 
Overall, there still seems to be some vagueness about what ?custody? means 
(in the sense of a service level agreement) between peers exchanging 
bundles. My understanding is that ?custody? means the custodial node is 
willing to make some kind of effort to keep the bundle moving toward its 
destination until the bundle lifetime expires.
 
The current RFC 9174 document is silent about what exactly an XFER_ACK 
segment with END flag set (an END ACK) means from the perspective of the 
BP agent and what is guaranteed about the transferred bundle. This 
provides an opportunity for a follow-on clarification of END ACK semantics 
for the TCPCL entity and for the BP agent. Two potential ways of making 
the behavior more well-defined:
 
1.       A network-specific profile of TCPCL could simply mandate that any 
node accepts custody by sending an END ACK. This would simply be a 
condition of conformance to the profile in a controlled network. This 
could be done immediately without any change elsewhere, but needs 
out-of-band coordination.
2.       One or more (quite simple) extension types can be defined for 
TCPCL to allow an entity to expose its END ACK behavior (RX) and desire 
(TX):
a.       A session extension can allow an entity to assert what its sent 
END ACK means for received transfers. The value in this is to allow the 
peer entity to adjust behavior depending on the capability (e.g. use BIBE 
if the next-hop doesn?t take END ACK custody), including possibly refusing 
to establish a session with (or refusing to send bundles to) a peer that 
does not take custody via END ACK.
b.      Additionally, a transfer extension can allow a sender to assert 
its custody desire on a per-bundle basis (signaling that some bundles need 
custody transfer while others do not). The value in this is to allow the 
receiving entity to optimize its behavior based on whether or not custody 
is needed for a bundle; though I don?t know how much benefit this would 
be.
 
The possible values enumerated by the session extension would be something 
like:
·         Custody is not taken at END ACK
·         Custody is taken at END ACK
And if there is a transfer extension defined, a the session extension 
could indicate:
·         Reception behavior is unconditional
·         Reception behavior can be overridden per-transfer based on the 
sender?s desire
These changes would all be backward compatible in the sense that a default 
policy would be in place in the absence of this extension item. And all of 
this is an independent mechanism from BIBE for a custody transfer to take 
place; both this mechanism and BIBE have their own costs, benefits, and 
side effects of such a transfer.
 
Trying to make almost-there-already capabilities more obvious,
Brian S.
_______________________________________________ dtn mailing list 
[email-masked] https://www.ietf.org/mailman/listinfo/dtn 
_______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dtn

---

From: "Sipos, Brian J." <[email-masked]>
Date: Fri, 25 Mar 2022 11:45:55 +0000

Felix,

Thank you for this feedback. There is a misinterpretation of what I (and
maybe also Scott via BIBE) am suggesting about what happens during reliable
reception. The idea isn't that two different things happen, it's that it's
the same BP agent custody acceptance criteria just that some CLs can provide
intrinsic signaling of that acceptance while other CLs have no means to
signal that specific feedback.

While IETF doesn't (normally) specify internal APIs, the CCSDS documents do
and this can help here. Currently 734.2-B-1 does not actually define a
BPA-CLA interface API, but it seems like the rough interface looks like:

.        Send.request to the CLA

.        Send.response from the CLA

.        Receive.indication from the CLA

What I am suggesting is that the Receive side could be changed from the
asynchronous "I just got this transfer. Here it is, thanks." to a
synchronous "I just got this transfer. Will you accept it?" similar to:

.        Receive.indication from the CLA

.        Receive.response to the CLA

The TCPCL and LTPCL already provide the negative response over-the-wire
(TCPCL reception can send XFER_REFUSE at any time before the END ACK, and
LTPCL can send "Cancel from the block receiver" similarly) there is
currently just no specific formal definition of what, from the BPA side, the
positive acknowledgement is required to mean. For example "If the transfer
is not canceled by the receiver and the final ACK is sent, the transferred
bundle SHALL be completely and positively received within the BP agent's
forwarding or delivery queue." 

As Scott mentioned earlier, custody isn't an anthropomorphization of the
BPA, it's a specific behavior, and it seems like by acknowledging that the
bundle was received into the queue for delivery/forwarding the agent has
"accepted" it. If the intent of custody is that it's a more
restricted/reserved resource pool (e.g. my forwarding queue is size X but of
that only Y (with Y<X) is reserved for bundles over which I have custody)
then that's a local agent management issue, not an over-the-wire signaling
issue. The BPA has still positively accepted the bundle and some CLAs can
communicate this back to the sending agent synchronously.

Thanks for any further clarification,

Brian S.

From: dtn <[email-masked]> On Behalf Of [email-masked]
Sent: Friday, March 25, 2022 6:11 AM
To: [email-masked]; [email-masked]
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

(cross-posting to CCSDS DTN mailinglist as it seems to be of high relevance
to the on-going discussions in the DTN WG). 

The main differences I see are: 
- In (a) the decision whether to take custody or not can be taken on more
available information, eg timely availability of a route toward a next hop,
available storage, policies (eg, checking Bundle Integrity Blocks), ... In
(b), the CLA may accept custody and the BPA would just decide to discard the
bundle for whatever reason (of course, there could be interfaces to make
such information available to the CLA but this would somehow 'blur' the
architecture. 
- In (b), the node to take custody needs to be explicitly addressed (it's
the BIBE destination) while in (a) any node could take custody. 

Requirements regarding custody transfer I would see for space missions are: 

1) Assertion of a high probability that a bundle will reach its destination
once a hop has accepted custody (which would allow the forwarding node to
release storage). 
The meaning of 'high probability' does really depend on the mission data
return requirements. While for some missions it may be close to 100%, other
mission may be ready to accept higher data loss in favour of timeliness (eg,
certain types of Earth Observation missions). 

2) Forwarding bundles without knowing which node will take custody. 
In particular with high data rates and optical direct-to-Earth downlinks we
may have situations where the spacecraft may not know the actual next hop is
sending to but may want to get a confirmation that the bundle has been
received on ground. With high-data rates, bundles might already be prepared
and encoded in frames and be sitting in some buffers within optical terminal
because the data rates on the on-board buses would not allow to generate and
send in real time. Use of optical direct-to-Earth links may be opportunistic
and we may not know in advance how much data will go down. So, addressing a
specific DTN node in a ground station becomes unpractical (if DTN nodes in
ground station would share the same anycast eID it may be possible but BIBE
is currently limited to singleton endpoints). 

It is clear that these requirements cannot be solved by protocol
specification as Scott pointed out below but will also require that
implementing nodes conform to certain behaviours. This will not be possible
for an open, Internet-like DTN network (where we can only try to take
defensive actions). For space missions I would still expect limited,
tightly-controlled network for some time to come (maybe becoming 'trusted
zones' in a larger network). For these, we should have protocol mechanism
which can support above requirements (while being aware that some of these
mechanisms will not work in a fully open DTN). 

Finally, 'custody transfer' seems to be always related to timer-base
re-transmission. However, I think there are other options as well, like: 
- command-based re-transmission: an explicit command is sent to a DTN node
to re-transmit all bundles for which custody has been requested but no
signal has been received 
- sequence-based re-transmission: in some situations it might be possible
(using additional extension blocks) to detect which bundles have not been
received by inspecting the received custody signals and re-send the ones for
which no custody signal have been received 
Again, this would likely not work in an open DTN but only in (very) limited,
controlled networks. But this doesn't make such mechanisms less useful
(although I would prefer to have something more generic if possible) and we
should address it in the IETF (if there is general interest) and/or the
CCSDS (if it is for near-to-medium term space mission use cases only). 

Regarding reliable CLs: I currently don't see the point of a reliable CL
taking custody because it would be (only) type (b) custody which is
basically just a confirmation that the bundle has been received and this can
already be assumed by the fact that it is a reliable CL. 

Regards, 
Felix 

From:        "William Ivancic" < <mailto:[email-masked]>
[email-masked]> 
To:        < <mailto:[email-masked]> [email-masked]>, "'Sipos,
Brian J.'" < <mailto:[email-masked]> [email-masked]>, <
<mailto:[email-masked]> [email-masked]> 
Date:        25/03/2022 01:16 
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs 
Sent by:        "dtn" < <mailto:[email-masked]> [email-masked]> 

  _____  

"Custody" is  a bundle level concept, not transport.  Way back when,
"Custody" meant that the node taking "Custody" would try it's best to ensure
the bundle was forwarded either to another "Custody" node or to eventually
the destination node.  The idea, as I recall, was this would allow the
original bundle source to clear its memory of that bundle.  

For space operations, I don't think the operations people were ever
comfortable with this concept.

//Will

From: dtn < <mailto:[email-masked]> [email-masked]> on behalf of
< <mailto:[email-masked]> [email-masked]>
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" < <mailto:[email-masked]>
[email-masked]>, < <mailto:[email-masked]> [email-masked]>
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Scott

From: dtn < <mailto:[email-masked]> [email-masked]> On Behalf Of
Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To:  <mailto:[email-masked]> [email-masked]
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

Brian S.

_______________________________________________ dtn mailing list
<mailto:[email-masked]> [email-masked]
<https://www.ietf.org/mailman/listinfo/dtn>
https://www.ietf.org/mailman/listinfo/dtn
_______________________________________________
dtn mailing list
 <mailto:[email-masked]> [email-masked]
 <https://www.ietf.org/mailman/listinfo/dtn>
https://www.ietf.org/mailman/listinfo/dtn

---

From: "Stuart William Card" <[email-masked]>
Date: Fri, 25 Mar 2022 09:52:28 -0400

One cavil: a BP node could "take responsibility... promise..." via a smart
contract, although I would hesitate to allow current blockchain based code
to access precious DTN bandwidth. I am actually serious about this, but
not near term, as distributed ledger technology is even less mature than
DTN.

> fashion
> specification
> CCSDS
> BP
> condition
> is
> could

-- 
Stuart W. Card, VP & Chief Scientist
Critical Technologies Inc. (CTI)
<[email-masked]>

---

From: Carlo Caini <[email-masked]>
Date: Fri, 25 Mar 2022 15:51:08 +0000

Dear William,
         very very well said! Layers are essential to understand the differences. Custody is (or better it was in RFC5050) at bundle layer, not at transport. Retransmission at Transport are hop-by-hop, bundle retransmission triggered by the lack of a custody acceptance in RFC 5050 were not (the destination of the retranmitted bundle in RFC 5050 was the original destination end-point, not a proximate node).
The current custodian in RFC5050 could get rid of the bundle only either:
1) after having received the custody acceptance by another node along the path to destination (again, not necessarily a proximate node) 
2) or after bundle expiration.
In both custody on and off cases, the bundle had to be kept in a local DB. However, a custodian could not delete a bundle of which he had accepted the custody after simply receiving a positive confirmation from a reliable protcol (TCPCL or LTP red) as for non custodial bundles. It had to wait for a confirmation by a peer (the custody acceptance) i.e. by a BP agent on another node, i.e. from an entity at the same layer.
Yours,
   Carlo
________________________________________
Da: dtn [[email-masked]] per conto di William Ivancic [[email-masked]]
Inviato: venerdì 25 marzo 2022 01:14
A: [email-masked]; 'Sipos, Brian J.'; [email-masked]
Oggetto: Re: [dtn] Bundle custody transfer and reliable CLs

//Will

Scott

All,

---

From: <[email-masked]>
Date: Fri, 25 Mar 2022 10:07:08 -0700

Hi, Brian.  Actually I think Felix's analysis is pretty much spot-on.  We
really are talking about two different behaviors, which respond to two
different sets of requirements.

The "custody transfer" procedures that I proposed in the BIBE draft are very
specifically aimed at defining a reliable convergence-layer adapter that
happens to use BP as its underlying convergence-layer protocol.  There is a
requirement for this capability: Keith Scott has often pointed out that
there are scenarios in which reliable transmission between nodes is required
but no reliable transmission protocol is available, e.g., when the sender's
communication capability is temporarily unidirectional.  These are not
hypothetical; MITRE's customers must at times operate in such environments,
and some space flight missions and other IoT systems could be similarly
constrained.  Under these conditions, the mechanism by which NAKs and ACKs
are returned to the sender may function at a later time and/or be unrelated
to the mechanism by which the sender transmitted the data.  There might be
better standardized protocols than BP for supporting these kinds of
scenarios, but none leap to mind.

The "custody transfer" procedures for which Felix proposes requirements are
different.  Since there is no need for timeout-triggered retransmission
(retransmission is instead triggered by command or by negative
acknowledgment), there is no need for accurate estimation of the round-trip
time; therefore there is no need for the sender to know which node will
issue the responding (positive) custody acknowledgment, exactly as required.
I think of it as a resource management system rather than an ARQ system.  A
mechanism very much like BPv6 custody transfer will work fine.

I would propose that we term the latter procedures "custody transfer" and
proceed to standardize them, while renaming the former something like
"BPARQ".

I don't think there's any need to impose any additional requirements on CL
protocols, TCPCL or other, to satisfy the requirements.  These are separate
things.  Let's keep them separate and support them separately and clearly.

Scott

From: dtn <[email-masked]> On Behalf Of Sipos, Brian J.
Sent: Friday, March 25, 2022 4:46 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Felix,

*	Send.request to the CLA
*	Send.response from the CLA
*	Receive.indication from the CLA

*	Receive.indication from the CLA
*	Receive.response to the CLA

Brian S.

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of
[email-masked] <mailto:[email-masked]> 
Sent: Friday, March 25, 2022 6:11 AM
To: [email-masked] <mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> 
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

---

From: Mehmet Adalier <[email-masked]>
Date: Fri, 25 Mar 2022 10:21:13 -0700

I believe Scott’s analysis below succinctly articulates the difference between the two approaches and I agree that they should be kept separate. For my intended use cases, approach 1 (BIBE/BPARQ?) is what I would need to use. I’ll be happy to contribute to this approach and prototype.

Best

mehmet

From: SIS-DTN <[email-masked]> on behalf of "sburleig.sb--- via SIS-DTN" <[email-masked]>
Reply-To: <[email-masked]>
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>, <[email-masked]>, <[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Hi, Brian.  Actually I think Felix’s analysis is pretty much spot-on.  We really are talking about two different behaviors, which respond to two different sets of requirements.

The “custody transfer” procedures that I proposed in the BIBE draft are very specifically aimed at defining a reliable convergence-layer adapter that happens to use BP as its underlying convergence-layer protocol.  There is a requirement for this capability: Keith Scott has often pointed out that there are scenarios in which reliable transmission between nodes is required but no reliable transmission protocol is available, e.g., when the sender’s communication capability is temporarily unidirectional.  These are not hypothetical; MITRE’s customers must at times operate in such environments, and some space flight missions and other IoT systems could be similarly constrained.  Under these conditions, the mechanism by which NAKs and ACKs are returned to the sender may function at a later time and/or be unrelated to the mechanism by which the sender transmitted the data.  There might be better standardized protocols than BP for supporting these kinds of scenarios, but none leap to mind.

The ”custody transfer” procedures for which Felix proposes requirements are different.  Since there is no need for timeout-triggered retransmission (retransmission is instead triggered by command or by negative acknowledgment), there is no need for accurate estimation of the round-trip time; therefore there is no need for the sender to know which node will issue the responding (positive) custody acknowledgment, exactly as required.  I think of it as a resource management system rather than an ARQ system.  A mechanism very much like BPv6 custody transfer will work fine.

I would propose that we term the latter procedures “custody transfer” and proceed to standardize them, while renaming the former something like “BPARQ”.

I don’t think there’s any need to impose any additional requirements on CL protocols, TCPCL or other, to satisfy the requirements.  These are separate things.  Let’s keep them separate and support them separately and clearly.

Scott

Felix,

Thank you for this feedback. There is a misinterpretation of what I (and maybe also Scott via BIBE) am suggesting about what happens during reliable reception. The idea isn’t that two different things happen, it’s that it’s the same BP agent custody acceptance criteria just that some CLs can provide intrinsic signaling of that acceptance while other CLs have no means to signal that specific feedback.

While IETF doesn’t (normally) specify internal APIs, the CCSDS documents do and this can help here. Currently 734.2-B-1 does not actually define a BPA—CLA interface API, but it seems like the rough interface looks like:
Send.request to the CLA
Send.response from the CLA
Receive.indication from the CLA

What I am suggesting is that the Receive side could be changed from the asynchronous “I just got this transfer. Here it is, thanks.” to a synchronous “I just got this transfer. Will you accept it?” similar to:
Receive.indication from the CLA
Receive.response to the CLA

The TCPCL and LTPCL already provide the negative response over-the-wire (TCPCL reception can send XFER_REFUSE at any time before the END ACK, and LTPCL can send “Cancel from the block receiver” similarly) there is currently just no specific formal definition of what, from the BPA side, the positive acknowledgement is required to mean. For example “If the transfer is not canceled by the receiver and the final ACK is sent, the transferred bundle SHALL be completely and positively received within the BP agent’s forwarding or delivery queue.” 

As Scott mentioned earlier, custody isn’t an anthropomorphization of the BPA, it’s a specific behavior, and it seems like by acknowledging that the bundle was received into the queue for delivery/forwarding the agent has “accepted” it. If the intent of custody is that it’s a more restricted/reserved resource pool (e.g. my forwarding queue is size X but of that only Y (with Y<X) is reserved for bundles over which I have custody) then that’s a local agent management issue, not an over-the-wire signaling issue. The BPA has still positively accepted the bundle and some CLAs can communicate this back to the sending agent synchronously.

Brian S.

(cross-posting to CCSDS DTN mailinglist as it seems to be of high relevance to the on-going discussions in the DTN WG). 

I think we clearly need to distinguish between (at least) two different types of 'custody transfer' according to where it is implemented: 

(a) BPv6 - like custody transfer which has been a function of the BPA / AA Administrative element 
(b) BIBE-like custody transfer which is implemented in the CLA 

The main differences I see are: 
- In (a) the decision whether to take custody or not can be taken on more available information, eg timely availability of a route toward a next hop, available storage, policies (eg, checking Bundle Integrity Blocks), ... In (b), the CLA may accept custody and the BPA would just decide to discard the bundle for whatever reason (of course, there could be interfaces to make such information available to the CLA but this would somehow 'blur' the architecture. 
- In (b), the node to take custody needs to be explicitly addressed (it's the BIBE destination) while in (a) any node could take custody. 

1) Assertion of a high probability that a bundle will reach its destination once a hop has accepted custody (which would allow the forwarding node to release storage). 
The meaning of 'high probability' does really depend on the mission data return requirements. While for some missions it may be close to 100%, other mission may be ready to accept higher data loss in favour of timeliness (eg, certain types of Earth Observation missions). 

2) Forwarding bundles without knowing which node will take custody. 
In particular with high data rates and optical direct-to-Earth downlinks we may have situations where the spacecraft may not know the actual next hop is sending to but may want to get a confirmation that the bundle has been received on ground. With high-data rates, bundles might already be prepared and encoded in frames and be sitting in some buffers within optical terminal because the data rates on the on-board buses would not allow to generate and send in real time. Use of optical direct-to-Earth links may be opportunistic and we may not know in advance how much data will go down. So, addressing a specific DTN node in a ground station becomes unpractical (if DTN nodes in ground station would share the same anycast eID it may be possible but BIBE is currently limited to singleton endpoints). 

It is clear that these requirements cannot be solved by protocol specification as Scott pointed out below but will also require that implementing nodes conform to certain behaviours. This will not be possible for an open, Internet-like DTN network (where we can only try to take defensive actions). For space missions I would still expect limited, tightly-controlled network for some time to come (maybe becoming 'trusted zones' in a larger network). For these, we should have protocol mechanism which can support above requirements (while being aware that some of these mechanisms will not work in a fully open DTN). 

Finally, 'custody transfer' seems to be always related to timer-base re-transmission. However, I think there are other options as well, like: 
- command-based re-transmission: an explicit command is sent to a DTN node to re-transmit all bundles for which custody has been requested but no signal has been received 
- sequence-based re-transmission: in some situations it might be possible (using additional extension blocks) to detect which bundles have not been received by inspecting the received custody signals and re-send the ones for which no custody signal have been received 
Again, this would likely not work in an open DTN but only in (very) limited, controlled networks. But this doesn't make such mechanisms less useful (although I would prefer to have something more generic if possible) and we should address it in the IETF (if there is general interest) and/or the CCSDS (if it is for near-to-medium term space mission use cases only). 

Regarding reliable CLs: I currently don't see the point of a reliable CL taking custody because it would be (only) type (b) custody which is basically just a confirmation that the bundle has been received and this can already be assumed by the fact that it is a reliable CL. 

Regards, 
Felix 

From:        "William Ivancic" <[email-masked]> 
To:        <[email-masked]>, "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]> 
Date:        25/03/2022 01:16 
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs 
Sent by:        "dtn" <[email-masked]> 

//Will

Scott

All,

Brian S.

_______________________________________________ dtn mailing list [email-masked] https://www.ietf.org/mailman/listinfo/dtn _______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dtn

_______________________________________________ SIS-DTN mailing list [email-masked] https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn

---

From: "Dr. Keith L Scott" <[email-masked]>
Date: Mon, 28 Mar 2022 13:00:36 +0000

The question remains: do we need some form of reliability above the CLA layer?  As Scott points out, BIBE is ‘just’ a reliable CLA that uses bundles as its transport mechanism.  That allows BIBE to function as a reliable CLA in environments where other CLAs that require bidirectional connectivity cannot, but viewed from the standpoint of the sending and receiving BIBE endpoints, it’s just another reliable CLA.

The not-often-stated assumption here is that bundle nodes that receive bundles via reliable CLAs do in fact take on the responsibilities of what was traditionally referred to as ‘custody transfer’ – i.e. following whatever procedures are necessary to ensure that the received bundle reaches its destination (e.g. store it until a forward path becomes available, attempt retransmission to the next hop until there’s an indication that the next hop has (via a reliable CLA in this context) received the bundle, etc.).  I.e. this approach assumes that receipt of a bundle • accepting ‘custody’ of it.  Without this assumption, the service provided by BP is essentially the same best-effort service that IP provides, which I think is less than what we want, particularly for space missions. So I think that in the BPv7 context, all bundles are considered ‘custodial’.

If we consider cases where there may be congestion (contention for storage space at nodes), this means that when congestion happens at a node, the only course of action available to the node will be to refuse new incoming bundles (presumably because the receiving CLAs stop accepting them).

There are at least two things to consider here:

  1.  What if I have a bundle node that is capable of forwarding but that has quite minimal storage?  By receiving a bundle, this node is committing to storing that bundle until it can be reliably forwarded.  It seems like there could be cases where this is really inefficient, especially if the reliable forwarding is over something like BIBE where the RTT to get an acknowledgement could be high.  In BPv6, such a node could simply forward a custody-requesting bundle without actually taking custody, sort of like a ‘transit node’ in a BIBE tunnel.  In this case, it might be able to achieve a higher throughput at the expense of NOT accepting the storage requirements from the current custodians.
  2.  Even if a congested node has a lot of storage (but still becomes congested), in BPv6 there was the notion that the node could drop (probably lower-priority) non-custodial bundles in favor of (probably higher-priority) custodial bundles.  We don’t currently have any notion of priority in BPv7, but if we ever want to admit the possibility that a bundle node might drop a bundle due to congestion, it seems like the assumption that receipt • ‘custodial’ acceptance constrains us.  In the BPv7 ‘receipt is (custody) acceptance’ model, the node would have to refuse new bundles.

This might be right thing for some CONOPS.  It would impose backpressure (at DTN timescales) on the network, eventually to the bundle sources.  The same thing would happen with BPv6 and custodial bundles, the difference being that a BPv6 node would have the option of dropping non-custodial bundles to accommodate newer (again, presumably higher-priority) bundles.

I’ll readily admit that calculating a (good) retransmission timer value in the case where a node does NOT know if the proximate (or even which) downstream node will take custody is difficult or impossible for some networks.  BIBE still has a bit of this problem, especially if the path to the BIBE destination is long, as the sender won’t necessarily know the path the BIBE bundles will take, but it is at least more constrained than the completely open case.

If we want to have the option of dropping lower-priority bundles that have already been received and are being stored at a node, we’ll need an extension block to mark priority, fine.  We could then create rules that operate at the BP layer to drop bundles when congestion occurs according to their priority markings and address #2 above. I suppose in this case the ‘reliability’ is still at the CLA layer and the decision-making process on whether or not to drop an incoming bundle is at the BP layer.

  *   That might not address #1 above but maybe #1 is sufficiently rare (or non-existent) that we decide to ignore it.
  *   Regardless, this has the disadvantage that the transmitting node would believe that the bundle was accepted (because it would have been, by the receiving CLA) event though it then got dropped by the BP layer at the receiving node.

                                --keith

From: "[email-masked]" <[email-masked]> on behalf of "[email-masked]" <[email-masked]>
Reply-To: Mehmet Adalier <[email-masked]>
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" <[email-masked]>, Felix Flentge <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Best
mehmet

Scott

Felix,

From: dtn <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Friday, March 25, 2022 6:11 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

Regards,
Felix

From:        "William Ivancic" <[email-masked]<mailto:[email-masked]>>
To:        <[email-masked]<mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>
Date:        25/03/2022 01:16
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs
Sent by:        "dtn" <[email-masked]<mailto:[email-masked]>>
________________________________

//Will

From: dtn <[email-masked]<mailto:[email-masked]>> on behalf of <[email-masked]<mailto:[email-masked]>>
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Scott

From: dtn <[email-masked]<mailto:[email-masked]>> On Behalf Of Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

•         Custody is not taken at END ACK

•         Custody is taken at END ACK

•         Reception behavior is unconditional

•         Reception behavior can be overridden per-transfer based on the sender’s desire

Brian S.

_______________________________________________ dtn mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn _______________________________________________
dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn
_______________________________________________ SIS-DTN mailing list [email-masked] https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn

---

From: [email-masked]
Date: Mon, 28 Mar 2022 16:53:45 +0200

Yes, I do think there is a need for reliability mechanisms above the CLA 
Layer. BIBE as a generic reliable CLA allows to address the case of 
reliability for uni-directional links (although in this case the setting 
of the timers may already get difficult) but it is currently not 
addressing:
- non-timer based re-transmission (which might be very specific to a 
certain node and deployment and therefore might be better addressed as 
part of the BPA / AA)
- forwarding a bundle without explicitly addressing a (singleton) next hop 
(eg, an opportunistic downlink where some signal about reception or 
delivery is uplinked at some later time).

I agree that we somehow can consider all received bundles as 'custodial' 
as a node should attempt to store and forward them. I also think that the 
typical use of custody transfer for BPv6 has been probably more to be 
informed about the reception (or deletion) of a bundle then whatever 
(additional ???) responsibility would be taken by the receiving node. So, 
maybe we should abandon the term 'custody' altogether. However, there are 
certainly situations where a sending node would like to get information 
about reception/delivery/deletion/forwarding of a bundle at another node 
in an operationally viable and efficient way (so, not using the current 
Bundle Status Reports).

Regards,
Felix

From:   "Dr. Keith L Scott" <[email-masked]>
To:     "Mehmet Adalier" <[email-masked]>, 
"[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" 
<[email-masked]>, "[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>, "[email-masked]" 
<[email-masked]>
Date:   28/03/2022 15:11
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs

The question remains: do we need some form of reliability above the CLA 
layer?  As Scott points out, BIBE is ‘just’ a reliable CLA that uses 
bundles as its transport mechanism.  That allows BIBE to function as a 
reliable CLA in environments where other CLAs that require bidirectional 
connectivity cannot, but viewed from the standpoint of the sending and 
receiving BIBE endpoints, it’s just another reliable CLA.
 
The not-often-stated assumption here is that bundle nodes that receive 
bundles via reliable CLAs do in fact take on the responsibilities of what 
was traditionally referred to as ‘custody transfer’ – i.e. following 
whatever procedures are necessary to ensure that the received bundle 
reaches its destination (e.g. store it until a forward path becomes 
available, attempt retransmission to the next hop until there’s an 
indication that the next hop has (via a reliable CLA in this context) 
received the bundle, etc.).  I.e. this approach assumes that receipt of a 
bundle �� accepting ‘custody’ of it.  Without this assumption, the service 
provided by BP is essentially the same best-effort service that IP 
provides, which I think is less than what we want, particularly for space 
missions. So I think that in the BPv7 context, all bundles are considered 
‘custodial’.

If we consider cases where there may be congestion (contention for storage 
space at nodes), this means that when congestion happens at a node, the 
only course of action available to the node will be to refuse new incoming 
bundles (presumably because the receiving CLAs stop accepting them). 
 
There are at least two things to consider here:
1.      What if I have a bundle node that is capable of forwarding but 
that has quite minimal storage?  By receiving a bundle, this node is 
committing to storing that bundle until it can be reliably forwarded.  It 
seems like there could be cases where this is really inefficient, 
especially if the reliable forwarding is over something like BIBE where 
the RTT to get an acknowledgement could be high.  In BPv6, such a node 
could simply forward a custody-requesting bundle without actually taking 
custody, sort of like a ‘transit node’ in a BIBE tunnel.  In this case, it 
might be able to achieve a higher throughput at the expense of NOT 
accepting the storage requirements from the current custodians.
2.      Even if a congested node has a lot of storage (but still becomes 
congested), in BPv6 there was the notion that the node could drop 
(probably lower-priority) non-custodial bundles in favor of (probably 
higher-priority) custodial bundles.  We don’t currently have any notion of 
priority in BPv7, but if we ever want to admit the possibility that a 
bundle node might drop a bundle due to congestion, it seems like the 
assumption that receipt �� ‘custodial’ acceptance constrains us.  In the 
BPv7 ‘receipt is (custody) acceptance’ model, the node would have to 
refuse new bundles.
This might be right thing for some CONOPS.  It would impose backpressure 
(at DTN timescales) on the network, eventually to the bundle sources.  The 
same thing would happen with BPv6 and custodial bundles, the difference 
being that a BPv6 node would have the option of dropping non-custodial 
bundles to accommodate newer (again, presumably higher-priority) bundles.
 
I’ll readily admit that calculating a (good) retransmission timer value in 
the case where a node does NOT know if the proximate (or even which) 
downstream node will take custody is difficult or impossible for some 
networks.  BIBE still has a bit of this problem, especially if the path to 
the BIBE destination is long, as the sender won’t necessarily know the 
path the BIBE bundles will take, but it is at least more constrained than 
the completely open case.

If we want to have the option of dropping lower-priority bundles that have 
already been received and are being stored at a node, we’ll need an 
extension block to mark priority, fine.  We could then create rules that 
operate at the BP layer to drop bundles when congestion occurs according 
to their priority markings and address #2 above. I suppose in this case 
the ‘reliability’ is still at the CLA layer and the decision-making 
process on whether or not to drop an incoming bundle is at the BP layer. 
That might not address #1 above but maybe #1 is sufficiently rare (or 
non-existent) that we decide to ignore it.
Regardless, this has the disadvantage that the transmitting node would 
believe that the bundle was accepted (because it would have been, by the 
receiving CLA) event though it then got dropped by the BP layer at the 
receiving node.
 
                                --keith

From: "[email-masked]" 
<[email-masked]> on behalf of 
"[email-masked]" <[email-masked]>
Reply-To: Mehmet Adalier <[email-masked]>
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" 
<[email-masked]>, Felix Flentge <[email-masked]>, 
"[email-masked]" <[email-masked]>, "[email-masked]" 
<[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs
 
I believe Scott’s analysis below succinctly articulates the difference 
between the two approaches and I agree that they should be kept separate. 
For my intended use cases, approach 1 (BIBE/BPARQ?) is what I would need 
to use. I’ll be happy to contribute to this approach and prototype.
 
Best
mehmet
 
From: SIS-DTN <[email-masked]> on behalf of 
"sburleig.sb--- via SIS-DTN" <[email-masked]>
Reply-To: <[email-masked]>
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>, 
<[email-masked]>, <[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs
 
Hi, Brian.  Actually I think Felix’s analysis is pretty much spot-on.  We 
really are talking about two different behaviors, which respond to two 
different sets of requirements.
 
The “custody transfer” procedures that I proposed in the BIBE draft are 
very specifically aimed at defining a reliable convergence-layer adapter 
that happens to use BP as its underlying convergence-layer protocol. There 
is a requirement for this capability: Keith Scott has often pointed out 
that there are scenarios in which reliable transmission between nodes is 
required but no reliable transmission protocol is available, e.g., when 
the sender’s communication capability is temporarily unidirectional. These 
are not hypothetical; MITRE’s customers must at times operate in such 
environments, and some space flight missions and other IoT systems could 
be similarly constrained.  Under these conditions, the mechanism by which 
NAKs and ACKs are returned to the sender may function at a later time 
and/or be unrelated to the mechanism by which the sender transmitted the 
data.  There might be better standardized protocols than BP for supporting 
these kinds of scenarios, but none leap to mind.
 
The ”custody transfer” procedures for which Felix proposes requirements 
are different.  Since there is no need for timeout-triggered 
retransmission (retransmission is instead triggered by command or by 
negative acknowledgment), there is no need for accurate estimation of the 
round-trip time; therefore there is no need for the sender to know which 
node will issue the responding (positive) custody acknowledgment, exactly 
as required.  I think of it as a resource management system rather than an 
ARQ system.  A mechanism very much like BPv6 custody transfer will work 
fine.
 
I would propose that we term the latter procedures “custody transfer” and 
proceed to standardize them, while renaming the former something like 
“BPARQ”.
 
I don’t think there’s any need to impose any additional requirements on CL 
protocols, TCPCL or other, to satisfy the requirements.  These are 
separate things.  Let’s keep them separate and support them separately and 
clearly.
 
Scott

Felix,
Thank you for this feedback. There is a misinterpretation of what I (and 
maybe also Scott via BIBE) am suggesting about what happens during 
reliable reception. The idea isn’t that two different things happen, it’s 
that it’s the same BP agent custody acceptance criteria just that some CLs 
can provide intrinsic signaling of that acceptance while other CLs have no 
means to signal that specific feedback.
 
While IETF doesn’t (normally) specify internal APIs, the CCSDS documents 
do and this can help here. Currently 734.2-B-1 does not actually define a 
BPA—CLA interface API, but it seems like the rough interface looks like:
Send.request to the CLA
Send.response from the CLA
Receive.indication from the CLA
 
What I am suggesting is that the Receive side could be changed from the 
asynchronous “I just got this transfer. Here it is, thanks.” to a 
synchronous “I just got this transfer. Will you accept it?” similar to:
Receive.indication from the CLA
Receive.response to the CLA
 
The TCPCL and LTPCL already provide the negative response over-the-wire 
(TCPCL reception can send XFER_REFUSE at any time before the END ACK, and 
LTPCL can send “Cancel from the block receiver” similarly) there is 
currently just no specific formal definition of what, from the BPA side, 
the positive acknowledgement is required to mean. For example “If the 
transfer is not canceled by the receiver and the final ACK is sent, the 
transferred bundle SHALL be completely and positively received within the 
BP agent’s forwarding or delivery queue.” 
 
As Scott mentioned earlier, custody isn’t an anthropomorphization of the 
BPA, it’s a specific behavior, and it seems like by acknowledging that the 
bundle was received into the queue for delivery/forwarding the agent has 
“accepted” it. If the intent of custody is that it’s a more 
restricted/reserved resource pool (e.g. my forwarding queue is size X but 
of that only Y (with Y<X) is reserved for bundles over which I have 
custody) then that’s a local agent management issue, not an over-the-wire 
signaling issue. The BPA has still positively accepted the bundle and some 
CLAs can communicate this back to the sending agent synchronously.

 

Regards, 
Felix 

From:        "William Ivancic" <[email-masked]> 
To:        <[email-masked]>, "'Sipos, Brian J.'" <
[email-masked]>, <[email-masked]> 
Date:        25/03/2022 01:16 
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs 
Sent by:        "dtn" <[email-masked]> 

“Custody” is  a bundle level concept, not transport.  Way back when, 
“Custody” meant that the node taking “Custody” would try it’s best to 
ensure the bundle was forwarded either to another “Custody” node or to 
eventually the destination node.  The idea, as I recall, was this would 
allow the original bundle source to clear its memory of that bundle. 
 
For space operations, I don’t think the operations people were ever 
comfortable with this concept.
 
//Will

Brian, there was some further discussion of custody transfer in this 
morning’s meeting of the CCSDS Space Internetworking Systems DTN Working 
Group as well.  A couple of notes:
 
It’s important to remember that we are talking about state machines here, 
 
node’s human (maybe eventually AI) network operators a means of assessing 
 
You’re right, the term “custody” is not defined for BPv7.  It is still 
widely used to refer to some behavior that future users of DTN for space 
flight operations state will be very important, but for which the 
requirements are not yet clearly established.  It is starting to look like 
the BP-based ARQ in the most recent BIBE draft, while needed (I think), is 
not exactly what people mean by “custody transfer.”  So it may become 
useful to define an additional BP extension (TBD) that we label with this 
term.
 
valuable; I don’t know, need to think about them some more.  But I would 
 
Scott

All,
 
Overall, there still seems to be some vagueness about what “custody” means 
(in the sense of a service level agreement) between peers exchanging 
bundles. My understanding is that “custody” means the custodial node is 
willing to make some kind of effort to keep the bundle moving toward its 
destination until the bundle lifetime expires.

if the next-hop doesn’t take END ACK custody), including possibly refusing 
is needed for a bundle; though I don’t know how much benefit this would 
be.
 
sender’s desire
 
Trying to make almost-there-already capabilities more obvious,
Brian S.
_______________________________________________ dtn mailing list 
[email-masked] https://www.ietf.org/mailman/listinfo/dtn 
_______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dtn
_______________________________________________ SIS-DTN mailing list 
[email-masked] 
https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn

---

From: <[email-masked]>
Date: Mon, 28 Mar 2022 11:44:04 -0700

I agree with everything Felix says here (except perhaps abandonment of the
term "custodial", which seems to have a lot of resonance in the DTN
community).

It is true that computed retransmission timeout intervals will not
necessarily be absolutely accurate even in BIBE/CT, but at least there are
ways to limit the error.  Methods for bundle delivery time estimation based
on contact plan analysis were studied in detail and published almost 10
years ago: given known source and destination node ID and a bundle
origination time and bundle size, we can estimate the time of arrival of the
original bundle and consequently the time of arrival of its acknowledgment
bundle, even over asymmetric paths.

Certainly we don't have the final answer to BP node congestion.  QoS may
well play a role, and we are starting to think about that in DTN WG.  BPv7
does support the concept of "paring" excessive traffic by imposing lifetime
overrides that result in immediate bundle expiration, a somewhat manageable
mechanism for dropping bundles as necessary.

But I don't think we need to give up on flow control altogether.  When TCP
is used at the convergence layer, a receiving node that is facing congestion
can simply stop reading on the socket and let TCP flow control
back-propagate.  When LTP is used at the convergence layer, a receiving node
that is facing congestion can begin canceling sessions; this will result in
CL protocol failure at the sender, bouncing the affected bundles back up to
the BPA where reforwarding procedures can be initiated (admittedly an
implementation matter, at least for now).  Neither entails bundle reception
that requires the receiving BPA to make a decision.  Personally, I think
building on these kinds of capabilities to address congestion at the
convergence layer would be more powerful than demanding more functionality
from custody transfer.

Scott

From: [email-masked] <[email-masked]> 
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; Mehmet Adalier
<[email-masked]>; [email-masked];
[email-masked]
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable
CLs

Yes, I do think there is a need for reliability mechanisms above the CLA
Layer. BIBE as a generic reliable CLA allows to address the case of
reliability for uni-directional links (although in this case the setting of
the timers may already get difficult) but it is currently not addressing: 
- non-timer based re-transmission (which might be very specific to a certain
node and deployment and therefore might be better addressed as part of the
BPA / AA) 
- forwarding a bundle without explicitly addressing a (singleton) next hop
(eg, an opportunistic downlink where some signal about reception or delivery
is uplinked at some later time). 

I agree that we somehow can consider all received bundles as 'custodial' as
a node should attempt to store and forward them. I also think that the
typical use of custody transfer for BPv6 has been probably more to be
informed about the reception (or deletion) of a bundle then whatever
(additional ???) responsibility would be taken by the receiving node. So,
maybe we should abandon the term 'custody' altogether. However, there are
certainly situations where a sending node would like to get information
about reception/delivery/deletion/forwarding of a bundle at another node in
an operationally viable and efficient way (so, not using the current Bundle
Status Reports). 

Regards, 
Felix 

From:        "Dr. Keith L Scott" <[email-masked] <mailto:[email-masked]>
To:        "Mehmet Adalier" <[email-masked]
<mailto:[email-masked]> >, "[email-masked]
<mailto:[email-masked]> " <[email-masked]
<mailto:[email-masked]> >, "'Sipos, Brian J.'"
<[email-masked] <mailto:[email-masked]> >,
"[email-masked] <mailto:[email-masked]> "
<[email-masked] <mailto:[email-masked]> >, "[email-masked]
<mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >,
"[email-masked] <mailto:[email-masked]> "
<[email-masked] <mailto:[email-masked]> > 
Date:        28/03/2022 15:11 
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and
reliable CLs 

  _____  

The question remains: do we need some form of reliability above the CLA
layer?  As Scott points out, BIBE is 'just' a reliable CLA that uses bundles
as its transport mechanism.  That allows BIBE to function as a reliable CLA
in environments where other CLAs that require bidirectional connectivity
cannot, but viewed from the standpoint of the sending and receiving BIBE
endpoints, it's just another reliable CLA.

The not-often-stated assumption here is that bundle nodes that receive
bundles via reliable CLAs do in fact take on the responsibilities of what
was traditionally referred to as 'custody transfer' - i.e. following
whatever procedures are necessary to ensure that the received bundle reaches
its destination (e.g. store it until a forward path becomes available,
attempt retransmission to the next hop until there's an indication that the
next hop has (via a reliable CLA in this context) received the bundle,
etc.).  I.e. this approach assumes that receipt of a bundle ==> accepting
'custody' of it.  Without this assumption, the service provided by BP is
essentially the same best-effort service that IP provides, which I think is
less than what we want, particularly for space missions. So I think that in
the BPv7 context, all bundles are considered 'custodial'.

If we consider cases where there may be congestion (contention for storage
space at nodes), this means that when congestion happens at a node, the only
course of action available to the node will be to refuse new incoming
bundles (presumably because the receiving CLAs stop accepting them).  

1.        What if I have a bundle node that is capable of forwarding but
that has quite minimal storage?  By receiving a bundle, this node is
committing to storing that bundle until it can be reliably forwarded.  It
seems like there could be cases where this is really inefficient, especially
if the reliable forwarding is over something like BIBE where the RTT to get
an acknowledgement could be high.  In BPv6, such a node could simply forward
a custody-requesting bundle without actually taking custody, sort of like a
'transit node' in a BIBE tunnel.  In this case, it might be able to achieve
a higher throughput at the expense of NOT accepting the storage requirements
from the current custodians. 
2.        Even if a congested node has a lot of storage (but still becomes
congested), in BPv6 there was the notion that the node could drop (probably
lower-priority) non-custodial bundles in favor of (probably higher-priority)
custodial bundles.  We don't currently have any notion of priority in BPv7,
but if we ever want to admit the possibility that a bundle node might drop a
bundle due to congestion, it seems like the assumption that receipt -->
'custodial' acceptance constrains us.  In the BPv7 'receipt is (custody)
acceptance' model, the node would have to refuse new bundles. 

This might be right thing for some CONOPS.  It would impose backpressure (at
DTN timescales) on the network, eventually to the bundle sources.  The same
thing would happen with BPv6 and custodial bundles, the difference being
that a BPv6 node would have the option of dropping non-custodial bundles to
accommodate newer (again, presumably higher-priority) bundles.

I'll readily admit that calculating a (good) retransmission timer value in
the case where a node does NOT know if the proximate (or even which)
downstream node will take custody is difficult or impossible for some
networks.  BIBE still has a bit of this problem, especially if the path to
the BIBE destination is long, as the sender won't necessarily know the path
the BIBE bundles will take, but it is at least more constrained than the
completely open case.

If we want to have the option of dropping lower-priority bundles that have
already been received and are being stored at a node, we'll need an
extension block to mark priority, fine.  We could then create rules that
operate at the BP layer to drop bundles when congestion occurs according to
their priority markings and address #2 above. I suppose in this case the
'reliability' is still at the CLA layer and the decision-making process on
whether or not to drop an incoming bundle is at the BP layer.  

*	That might not address #1 above but maybe #1 is sufficiently rare
(or non-existent) that we decide to ignore it. 
*	Regardless, this has the disadvantage that the transmitting node
would believe that the bundle was accepted (because it would have been, by
the receiving CLA) event though it then got dropped by the BP layer at the
receiving node.

                                --keith

From: "[email-masked]
<mailto:[email-masked]> "
<[email-masked]
<mailto:[email-masked]> > on behalf of
"[email-masked] <mailto:[email-masked]> "
<[email-masked] <mailto:[email-masked]> >
Reply-To: Mehmet Adalier <[email-masked]
<mailto:[email-masked]> >
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked] <mailto:[email-masked]> "
<[email-masked] <mailto:[email-masked]> >, "'Sipos, Brian J.'"
<[email-masked] <mailto:[email-masked]> >, Felix Flentge
<[email-masked] <mailto:[email-masked]> >, "[email-masked]
<mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >,
"[email-masked] <mailto:[email-masked]> "
<[email-masked] <mailto:[email-masked]> >
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable
CLs

I believe Scott's analysis below succinctly articulates the difference
between the two approaches and I agree that they should be kept separate.
For my intended use cases, approach 1 (BIBE/BPARQ?) is what I would need to
use. I'll be happy to contribute to this approach and prototype.

Best

mehmet

From: SIS-DTN <[email-masked]
<mailto:[email-masked]> > on behalf of "sburleig.sb---
via SIS-DTN" <[email-masked] <mailto:[email-masked]> >
Reply-To: <[email-masked] <mailto:[email-masked]> >
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" <[email-masked]
<mailto:[email-masked]> >, <[email-masked]
<mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >,
<[email-masked] <mailto:[email-masked]> >
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable
CLs

Scott

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of
Sipos, Brian J.
Sent: Friday, March 25, 2022 4:46 AM
To: [email-masked] <mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> 
Subject: Re: [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Felix,

Brian S.

From: dtn < <mailto:[email-masked]> [email-masked]> On Behalf Of
<mailto:[email-masked]> [email-masked]
Sent: Friday, March 25, 2022 6:11 AM
To:  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]>
[email-masked]
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

Regards, 
Felix 

  _____  

//Will

From: dtn <[email-masked] <mailto:[email-masked]> > on behalf of
<[email-masked] <mailto:[email-masked]> >
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" <[email-masked]
<mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Scott

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of
Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To: [email-masked] <mailto:[email-masked]> 
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

Brian S.

[email-masked] <mailto:[email-masked]>

_______________________________________________ SIS-DTN mailing list
[email-masked] <mailto:[email-masked]>
<https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn>
https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn

---

From: "scott" <[email-masked]>
Date: Mon, 28 Mar 2022 19:17:13 -0400

Noob lurker here, so please feel free to ignore... :)

> By receiving a bundle, this node is
> committing to storing that bundle until it can be reliably
> forwarded.

> (probably lower-priority) non-custodial bundles in favor of
> (probably higher-priority) custodial bundles.

Couldn't these be seen as attack vectors?

scott

---

From: "Dr. Keith L Scott" <[email-masked]>
Date: Tue, 29 Mar 2022 14:42:54 +0000

Well, yes, but there is more at play.  For example:

If security is used, the receiving node might have a policy stating that it requires a signed previous-hop block in the bundle.  Sending bogus bundles (ones without PHBs or PHBs signed by a sender that the receiver doesn't know of/like) to such a node wouldn't work (the CLA would accept them, but the 'assumption' that the receiving node would take custody would be overridden by the security policy).

		--keith

On 3/28/22, 7:18 PM, "dtn on behalf of scott" <[email-masked] on behalf of [email-masked]> wrote:

    scott

---

From: scott <[email-masked]>
Date: Tue, 29 Mar 2022 05:14:31 -1000

Thanks for the clarification.

scott

---

From: "Sanchez Net, Marc (US 332H)" <[email-masked]>
Date: Tue, 29 Mar 2022 20:09:26 +0000

I also agree that having reliability mechanisms above the CLA is necessary. For example, the Mars Relay Network (MRN) uses the Prox-1 protocol on the proximity links, which has a built-in Go-Back-N ARQ mechanism, so it is technically a reliable CLA. Yet, they routinely miss data on the return direction and both the relay orbiter and rover missions have accountability mechanisms that look for missing data.

When I talk to people in the MRN, their biggest desire from DTN is to avoid the problem above. In their words, they want a mechanism so that the rover unequivocally knows that a complete data product has made it to the orbiter.

-----------------------------------------------------------------------------------------------------------------------
Marc Sanchez Net
Telecommunications Engineer
Jet Propulsion Laboratory
Office: (818) 354-1650<tel:(818)%20393-5840> | Email: [email-masked]<mailto:[email-masked]>
-----------------------------------------------------------------------------------------------------------------------

From: SIS-DTN <[email-masked]> On Behalf Of sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To: [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; [email-masked]
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

I agree with everything Felix says here (except perhaps abandonment of the term "custodial", which seems to have a lot of resonance in the DTN community).

It is true that computed retransmission timeout intervals will not necessarily be absolutely accurate even in BIBE/CT, but at least there are ways to limit the error.  Methods for bundle delivery time estimation based on contact plan analysis were studied in detail and published almost 10 years ago: given known source and destination node ID and a bundle origination time and bundle size, we can estimate the time of arrival of the original bundle and consequently the time of arrival of its acknowledgment bundle, even over asymmetric paths.

Certainly we don't have the final answer to BP node congestion.  QoS may well play a role, and we are starting to think about that in DTN WG.  BPv7 does support the concept of "paring" excessive traffic by imposing lifetime overrides that result in immediate bundle expiration, a somewhat manageable mechanism for dropping bundles as necessary.

But I don't think we need to give up on flow control altogether.  When TCP is used at the convergence layer, a receiving node that is facing congestion can simply stop reading on the socket and let TCP flow control back-propagate.  When LTP is used at the convergence layer, a receiving node that is facing congestion can begin canceling sessions; this will result in CL protocol failure at the sender, bouncing the affected bundles back up to the BPA where reforwarding procedures can be initiated (admittedly an implementation matter, at least for now).  Neither entails bundle reception that requires the receiving BPA to make a decision.  Personally, I think building on these kinds of capabilities to address congestion at the convergence layer would be more powerful than demanding more functionality from custody transfer.

Scott

From: [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Mehmet Adalier <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Yes, I do think there is a need for reliability mechanisms above the CLA Layer. BIBE as a generic reliable CLA allows to address the case of reliability for uni-directional links (although in this case the setting of the timers may already get difficult) but it is currently not addressing:
- non-timer based re-transmission (which might be very specific to a certain node and deployment and therefore might be better addressed as part of the BPA / AA)
- forwarding a bundle without explicitly addressing a (singleton) next hop (eg, an opportunistic downlink where some signal about reception or delivery is uplinked at some later time).

I agree that we somehow can consider all received bundles as 'custodial' as a node should attempt to store and forward them. I also think that the typical use of custody transfer for BPv6 has been probably more to be informed about the reception (or deletion) of a bundle then whatever (additional ???) responsibility would be taken by the receiving node. So, maybe we should abandon the term 'custody' altogether. However, there are certainly situations where a sending node would like to get information about reception/delivery/deletion/forwarding of a bundle at another node in an operationally viable and efficient way (so, not using the current Bundle Status Reports).

Regards,
Felix

From:        "Dr. Keith L Scott" <[email-masked]<mailto:[email-masked]>>
To:        "Mehmet Adalier" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Date:        28/03/2022 15:11
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs
________________________________

The question remains: do we need some form of reliability above the CLA layer?  As Scott points out, BIBE is 'just' a reliable CLA that uses bundles as its transport mechanism.  That allows BIBE to function as a reliable CLA in environments where other CLAs that require bidirectional connectivity cannot, but viewed from the standpoint of the sending and receiving BIBE endpoints, it's just another reliable CLA.

The not-often-stated assumption here is that bundle nodes that receive bundles via reliable CLAs do in fact take on the responsibilities of what was traditionally referred to as 'custody transfer' - i.e. following whatever procedures are necessary to ensure that the received bundle reaches its destination (e.g. store it until a forward path becomes available, attempt retransmission to the next hop until there's an indication that the next hop has (via a reliable CLA in this context) received the bundle, etc.).  I.e. this approach assumes that receipt of a bundle ==> accepting 'custody' of it.  Without this assumption, the service provided by BP is essentially the same best-effort service that IP provides, which I think is less than what we want, particularly for space missions. So I think that in the BPv7 context, all bundles are considered 'custodial'.

1.        What if I have a bundle node that is capable of forwarding but that has quite minimal storage?  By receiving a bundle, this node is committing to storing that bundle until it can be reliably forwarded.  It seems like there could be cases where this is really inefficient, especially if the reliable forwarding is over something like BIBE where the RTT to get an acknowledgement could be high.  In BPv6, such a node could simply forward a custody-requesting bundle without actually taking custody, sort of like a 'transit node' in a BIBE tunnel.  In this case, it might be able to achieve a higher throughput at the expense of NOT accepting the storage requirements from the current custodians.
2.        Even if a congested node has a lot of storage (but still becomes congested), in BPv6 there was the notion that the node could drop (probably lower-priority) non-custodial bundles in favor of (probably higher-priority) custodial bundles.  We don't currently have any notion of priority in BPv7, but if we ever want to admit the possibility that a bundle node might drop a bundle due to congestion, it seems like the assumption that receipt --> 'custodial' acceptance constrains us.  In the BPv7 'receipt is (custody) acceptance' model, the node would have to refuse new bundles.

I'll readily admit that calculating a (good) retransmission timer value in the case where a node does NOT know if the proximate (or even which) downstream node will take custody is difficult or impossible for some networks.  BIBE still has a bit of this problem, especially if the path to the BIBE destination is long, as the sender won't necessarily know the path the BIBE bundles will take, but it is at least more constrained than the completely open case.

If we want to have the option of dropping lower-priority bundles that have already been received and are being stored at a node, we'll need an extension block to mark priority, fine.  We could then create rules that operate at the BP layer to drop bundles when congestion occurs according to their priority markings and address #2 above. I suppose in this case the 'reliability' is still at the CLA layer and the decision-making process on whether or not to drop an incoming bundle is at the BP layer.

                                --keith

From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> on behalf of "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Reply-To: Mehmet Adalier <[email-masked]<mailto:[email-masked]>>
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, Felix Flentge <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

I believe Scott's analysis below succinctly articulates the difference between the two approaches and I agree that they should be kept separate. For my intended use cases, approach 1 (BIBE/BPARQ?) is what I would need to use. I'll be happy to contribute to this approach and prototype.

Best

mehmet

From: SIS-DTN <[email-masked]<mailto:[email-masked]>> on behalf of "sburleig.sb--- via SIS-DTN" <[email-masked]<mailto:[email-masked]>>
Reply-To: <[email-masked]<mailto:[email-masked]>>
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Hi, Brian.  Actually I think Felix's analysis is pretty much spot-on.  We really are talking about two different behaviors, which respond to two different sets of requirements.

The "custody transfer" procedures that I proposed in the BIBE draft are very specifically aimed at defining a reliable convergence-layer adapter that happens to use BP as its underlying convergence-layer protocol.  There is a requirement for this capability: Keith Scott has often pointed out that there are scenarios in which reliable transmission between nodes is required but no reliable transmission protocol is available, e.g., when the sender's communication capability is temporarily unidirectional.  These are not hypothetical; MITRE's customers must at times operate in such environments, and some space flight missions and other IoT systems could be similarly constrained.  Under these conditions, the mechanism by which NAKs and ACKs are returned to the sender may function at a later time and/or be unrelated to the mechanism by which the sender transmitted the data.  There might be better standardized protocols than BP for supporting these kinds of scenarios, but none leap to mind.

The "custody transfer" procedures for which Felix proposes requirements are different.  Since there is no need for timeout-triggered retransmission (retransmission is instead triggered by command or by negative acknowledgment), there is no need for accurate estimation of the round-trip time; therefore there is no need for the sender to know which node will issue the responding (positive) custody acknowledgment, exactly as required.  I think of it as a resource management system rather than an ARQ system.  A mechanism very much like BPv6 custody transfer will work fine.

I would propose that we term the latter procedures "custody transfer" and proceed to standardize them, while renaming the former something like "BPARQ".

I don't think there's any need to impose any additional requirements on CL protocols, TCPCL or other, to satisfy the requirements.  These are separate things.  Let's keep them separate and support them separately and clearly.

Scott

From: dtn <[email-masked]<mailto:[email-masked]>> On Behalf Of Sipos, Brian J.
Sent: Friday, March 25, 2022 4:46 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Felix,

Thank you for this feedback. There is a misinterpretation of what I (and maybe also Scott via BIBE) am suggesting about what happens during reliable reception. The idea isn't that two different things happen, it's that it's the same BP agent custody acceptance criteria just that some CLs can provide intrinsic signaling of that acceptance while other CLs have no means to signal that specific feedback.

While IETF doesn't (normally) specify internal APIs, the CCSDS documents do and this can help here. Currently 734.2-B-1 does not actually define a BPA-CLA interface API, but it seems like the rough interface looks like:

What I am suggesting is that the Receive side could be changed from the asynchronous "I just got this transfer. Here it is, thanks." to a synchronous "I just got this transfer. Will you accept it?" similar to:

The TCPCL and LTPCL already provide the negative response over-the-wire (TCPCL reception can send XFER_REFUSE at any time before the END ACK, and LTPCL can send "Cancel from the block receiver" similarly) there is currently just no specific formal definition of what, from the BPA side, the positive acknowledgement is required to mean. For example "If the transfer is not canceled by the receiver and the final ACK is sent, the transferred bundle SHALL be completely and positively received within the BP agent's forwarding or delivery queue."

As Scott mentioned earlier, custody isn't an anthropomorphization of the BPA, it's a specific behavior, and it seems like by acknowledging that the bundle was received into the queue for delivery/forwarding the agent has "accepted" it. If the intent of custody is that it's a more restricted/reserved resource pool (e.g. my forwarding queue is size X but of that only Y (with Y<X) is reserved for bundles over which I have custody) then that's a local agent management issue, not an over-the-wire signaling issue. The BPA has still positively accepted the bundle and some CLAs can communicate this back to the sending agent synchronously.

Brian S.

Regards,
Felix

"Custody" is  a bundle level concept, not transport.  Way back when, "Custody" meant that the node taking "Custody" would try it's best to ensure the bundle was forwarded either to another "Custody" node or to eventually the destination node.  The idea, as I recall, was this would allow the original bundle source to clear its memory of that bundle.

For space operations, I don't think the operations people were ever comfortable with this concept.

//Will

Brian, there was some further discussion of custody transfer in this morning's meeting of the CCSDS Space Internetworking Systems DTN Working Group as well.  A couple of notes:

It's important to remember that we are talking about state machines here, not people.  Anthropomorphizing the DTN architecture is tempting but treacherous.  BP nodes have no will; they cannot take responsibility; they cannot promise to do anything.  All they do is behave, ideally in a fashion that conforms to the protocol specifications to which they were allegedly developed.

Also, any given node may have been designed with malign intent or implemented with errors.  Stating a requirement in a protocol specification does not ensure its satisfaction; what it does is give a node's human (maybe eventually AI) network operators a means of assessing the behavior of another node and possibly taking some sort of out-of-band administrative action in defense against that behavior as needed.

You're right, the term "custody" is not defined for BPv7.  It is still widely used to refer to some behavior that future users of DTN for space flight operations state will be very important, but for which the requirements are not yet clearly established.  It is starting to look like the BP-based ARQ in the most recent BIBE draft, while needed (I think), is not exactly what people mean by "custody transfer."  So it may become useful to define an additional BP extension (TBD) that we label with this term.

TCPCL enhancements along the lines you propose here may very well be valuable; I don't know, need to think about them some more.  But I would say the BPv7 specification already contains language (in 5.4, Step 5 and the following two paragraphs, and in 7.2, second bullet point) constraining the sort of convergence-layer reliability that I think is indispensable, regardless of what we call it.

Scott

All,

Overall, there still seems to be some vagueness about what "custody" means (in the sense of a service level agreement) between peers exchanging bundles. My understanding is that "custody" means the custodial node is willing to make some kind of effort to keep the bundle moving toward its destination until the bundle lifetime expires.

a.       A session extension can allow an entity to assert what its sent END ACK means for received transfers. The value in this is to allow the peer entity to adjust behavior depending on the capability (e.g. use BIBE if the next-hop doesn't take END ACK custody), including possibly refusing to establish a session with (or refusing to send bundles to) a peer that does not take custody via END ACK.

b.      Additionally, a transfer extension can allow a sender to assert its custody desire on a per-bundle basis (signaling that some bundles need custody transfer while others do not). The value in this is to allow the receiving entity to optimize its behavior based on whether or not custody is needed for a bundle; though I don't know how much benefit this would be.

*         Reception behavior can be overridden per-transfer based on the sender's desire

Brian S.

_______________________________________________ dtn mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn<https://urldefense.us/v3/__https:/www.ietf.org/mailman/listinfo/dtn__;!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq$> _______________________________________________
dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn<https://urldefense.us/v3/__https:/www.ietf.org/mailman/listinfo/dtn__;!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq$>

_______________________________________________ SIS-DTN mailing list [email-masked]<mailto:[email-masked]> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn<https://urldefense.us/v3/__https:/mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn__;!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6Wcylx8mv$>

---

From: <[email-masked]>
Date: Tue, 29 Mar 2022 16:22:06 -0700

Marc, I certainly agree that this is a serious problem, but I don't agree
that it motivates development of a reliability mechanism above the CLA.  (I
believe a number of other requirements do that.)  Prox-1 may be technically
a reliable CLA, but since it routinely misses data it's apparently not the
right one to use for this environment.  Why are people in MRN opposed to
using LTP, which is a CCSDS standard that has been in continuous operational
use in low Earth orbit (on ISS) since 2016?

Sorry, this discussion is likely not of general interest to DTN WG; we
should take it to another thread.

Scott

From: Sanchez Net, Marc (US 332H) <[email-masked]> 
Sent: Tuesday, March 29, 2022 1:09 PM
To: [email-masked]; [email-masked]; 'Dr. Keith L Scott'
<[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: RE: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody
transfer and reliable CLs

I also agree that having reliability mechanisms above the CLA is necessary.
For example, the Mars Relay Network (MRN) uses the Prox-1 protocol on the
proximity links, which has a built-in Go-Back-N ARQ mechanism, so it is
technically a reliable CLA. Yet, they routinely miss data on the return
direction and both the relay orbiter and rover missions have accountability
mechanisms that look for missing data.

When I talk to people in the MRN, their biggest desire from DTN is to avoid
the problem above. In their words, they want a mechanism so that the rover
unequivocally knows that a complete data product has made it to the orbiter.

----------------------------------------------------------------------------
-------------------------------------------

Marc Sanchez Net

Office:  <tel:(818)%20393-5840> (818) 354-1650 | Email:
<mailto:[email-masked]> [email-masked]

----------------------------------------------------------------------------
-------------------------------------------

From: SIS-DTN <[email-masked]
<mailto:[email-masked]> > On Behalf Of sburleig.sb--- via
SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To: [email-masked] <mailto:[email-masked]> ; 'Dr. Keith L
Scott' <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked]
<mailto:[email-masked]> >; [email-masked]
<mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer
and reliable CLs

Scott

From:  <mailto:[email-masked]> [email-masked] <
<mailto:[email-masked]> [email-masked]> 
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott < <mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]>
[email-masked]>;  <mailto:[email-masked]> [email-masked]; Mehmet Adalier
< <mailto:[email-masked]> [email-masked]>;
<mailto:[email-masked]> [email-masked];
<mailto:[email-masked]> [email-masked]
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable
CLs

Regards, 
Felix 

From:        "Dr. Keith L Scott" < <mailto:[email-masked]>
[email-masked]> 
To:        "Mehmet Adalier" < <mailto:[email-masked]>
[email-masked]>, " <mailto:[email-masked]>
[email-masked]" < <mailto:[email-masked]>
[email-masked]>, "'Sipos, Brian J.'" <
<mailto:[email-masked]> [email-masked]>, "
<mailto:[email-masked]> [email-masked]" <
<mailto:[email-masked]> [email-masked]>, "
<mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, "
<mailto:[email-masked]> [email-masked]" <
<mailto:[email-masked]> [email-masked]> 
Date:        28/03/2022 15:11 
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and
reliable CLs 

  _____  

 

 

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

_______________________________________________ dtn mailing list
[email-masked] <mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn
<https://urldefense.us/v3/__https:/www.ietf.org/mailman/listinfo/dtn__;!!PvB
Dto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCy
m6WUzbi2bq$>  _______________________________________________
dtn mailing list
 <mailto:[email-masked]> [email-masked]
 
<https://urldefense.us/v3/__https:/www.ietf.org/mailman/listinfo/dtn__;!!PvB
Dto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCy
m6WUzbi2bq$> https://www.ietf.org/mailman/listinfo/dtn

_______________________________________________ SIS-DTN mailing list
[email-masked] <mailto:[email-masked]>
https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn
<https://urldefense.us/v3/__https:/mailman.ccsds.org/cgi-bin/mailman/listinf
o/sis-dtn__;!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thP
yLbcOdehXgutIckCym6Wcylx8mv$>

---

From: "Chaoua, Rachid (GSFC-4500)" <[email-masked]>
Date: Wed, 30 Mar 2022 16:39:14 +0000

Marc's statement "the rover unequivocally knows that a complete data product has made it to the orbiter" reflects the sentiments I've heard within the Near Space Network when discussing DTN and Bundle Protocol. It is a core requirement for GSFC to confirm the next hop (bop) has custody of a data payload prior to that data being released from the source (or previous bop). This is requirement for certain data flow types not all.

Regarding the statement (the LTP part), "Why are people in MRN opposed to using LTP". Our (NSN/GSFC) stance (captured in the Draft LunaNet Interoperability Specification, https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20Specification%20Final.pdf, p.10) is that we will initially support various CLAs: TCPCL, UDPCL, LTPCL, or Encapsulation (future CLAs will be assessed and supportability determined by the NSN). Since we are planning on using various service providers, we are leaving the CLs used up to them. Regardless of the CL implemented, there is a need for a custody transfer option at the bundle layer. Relying on BIBE CL only works on nodes that support BIBE CL.  That gives me some concern.

Cheers!

From: dtn <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, March 29, 2022 7:22 PM
To: Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked]>; [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Marc, I certainly agree that this is a serious problem, but I don't agree that it motivates development of a reliability mechanism above the CLA.  (I believe a number of other requirements do that.)  Prox-1 may be technically a reliable CLA, but since it routinely misses data it's apparently not the right one to use for this environment.  Why are people in MRN opposed to using LTP, which is a CCSDS standard that has been in continuous operational use in low Earth orbit (on ISS) since 2016?

Sorry, this discussion is likely not of general interest to DTN WG; we should take it to another thread.

Scott

From: Sanchez Net, Marc (US 332H) <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, March 29, 2022 1:09 PM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

From: SIS-DTN <[email-masked]<mailto:[email-masked]>> On Behalf Of sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To: [email-masked]<mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

Regards,
Felix

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards,
Felix

//Will

Scott

All,

Brian S.

_______________________________________________ dtn mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fdtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=k%2F3uv%2B8kTA25lenkOtDhpbQxi43lX30qVx%2BdyjkZHVk%3D&reserved=0> _______________________________________________
dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fdtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=k%2F3uv%2B8kTA25lenkOtDhpbQxi43lX30qVx%2BdyjkZHVk%3D&reserved=0>

_______________________________________________ SIS-DTN mailing list [email-masked]<mailto:[email-masked]> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fmailman.ccsds.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsis-dtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6Wcylx8mv%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gkjxxJBrXxMT2YODEHEErMPPkwa0WmljSO7Y4Z%2Bcius%3D&reserved=0>

---

From: <[email-masked]>
Date: Wed, 30 Mar 2022 10:50:21 -0700

Hi, Rachid.  Confirming that the next BP node in the end-to-end path - the
next bundle hop (bop) - has received a bundle is precisely what a reliable
convergence-layer protocol does.  Now, it may be the case that the received
bundle will not be forwarded any further:

*	The bundle may be malformed or otherwise corrupted, therefore
deleted.
*	The bundle may be too large to fit into available storage, therefore
dropped.

In a well-managed network these errors will be rare, but I agree that for
critical transmissions they need to be reported.  That is why I fully
endorse Felix's concept of adding to BP a mechanism for sending aggregated
bundle acknowledgments.  But that mechanism is not a substitute for a
technically sound retransmission protocol at the convergence layer, for
reasons we've already discussed at length; it's an augmentation.

I also agree that reliance on BIBE ARQ will work only on nodes that support
BIBE ARQ.  Similarly, reliance on custody transfer - aggregated bundle
acknowledgments at the bundle layer - will work only on nodes that support
custody transfer.  This is a deficiency that is common to all protocols that
I know of.

Scott

From: Chaoua, Rachid (GSFC-4500) <[email-masked]> 
Sent: Wednesday, March 30, 2022 9:39 AM
To: [email-masked]; Sanchez Net, Marc (JPL-332H)[JPL Employee]
<[email-masked]>; [email-masked]; 'Dr. Keith L Scott'
<[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody
transfer and reliable CLs

Marc's statement "the rover unequivocally knows that a complete data product
has made it to the orbiter" reflects the sentiments I've heard within the
Near Space Network when discussing DTN and Bundle Protocol. It is a core
requirement for GSFC to confirm the next hop (bop) has custody of a data
payload prior to that data being released from the source (or previous bop).
This is requirement for certain data flow types not all. 

Regarding the statement (the LTP part), "Why are people in MRN opposed to
using LTP". Our (NSN/GSFC) stance (captured in the Draft LunaNet
Interoperability Specification,
https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20
Specification%20Final.pdf, p.10) is that we will initially support various
CLAs: TCPCL, UDPCL, LTPCL, or Encapsulation (future CLAs will be assessed
and supportability determined by the NSN). Since we are planning on using
various service providers, we are leaving the CLs used up to them.
Regardless of the CL implemented, there is a need for a custody transfer
option at the bundle layer. Relying on BIBE CL only works on nodes that
support BIBE CL.  That gives me some concern. 

Cheers!

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of
[email-masked] <mailto:[email-masked]> 
Sent: Tuesday, March 29, 2022 7:22 PM
To: Sanchez Net, Marc (JPL-332H)[JPL Employee]
<[email-masked] <mailto:[email-masked]> >;
[email-masked] <mailto:[email-masked]> ; 'Dr. Keith L Scott'
<[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked]
<mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody
transfer and reliable CLs

Scott

From: Sanchez Net, Marc (US 332H) < <mailto:[email-masked]>
[email-masked]> 
Sent: Tuesday, March 29, 2022 1:09 PM
To:  <mailto:[email-masked]> [email-masked];
<mailto:[email-masked]> [email-masked]; 'Dr. Keith L Scott' <
<mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]>
[email-masked]>;  <mailto:[email-masked]> [email-masked]
Subject: RE: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody
transfer and reliable CLs

 

Marc Sanchez Net

From: SIS-DTN < <mailto:[email-masked]>
[email-masked]> On Behalf Of sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To:  <mailto:[email-masked]> [email-masked]; 'Dr. Keith L
Scott' < <mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]>
[email-masked]>;  <mailto:[email-masked]>
[email-masked];  <mailto:[email-masked]> [email-masked]
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer
and reliable CLs

Scott

Regards, 
Felix 

  _____  

 

 

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

---

From: Mehmet Adalier <[email-masked]>
Date: Wed, 30 Mar 2022 11:03:57 -0700

Naïve question.

Regarding “It is a core requirement for GSFC to confirm the next hop (bop) has custody of a data payload prior to that data being released from the source (or previous bop).”

If we are only talking about the fact that the sent “data blob” made it to the next hop:

Since we are talking about LunaNet, why can’t we do this at the Data Link Layer (AOS or USLP)? One can program AOS/USLP to re-transmit if needed.  The sending node can get confirmation that the receiving node physically received the data. Is this covered under the CLA?

Otherwise, I would suggest that the notion of “did the receiving BP node get my bundle, did it successfully process it, is willing to forward it, and will forward it , and will really not drop it if it can’t see the next node for a while” is performed by an application that sits above the BP instance and has an understanding of the specific mission operational needs.

mA

From: dtn <[email-masked]> on behalf of <[email-masked]>
Date: Wednesday, March 30, 2022 at 10:50 AM
To: "'Chaoua, Rachid (GSFC-4500)'" <[email-masked]>, "'Sanchez Net, Marc (JPL-332H)[JPL Employee]'" <[email-masked]>, <[email-masked]>, "'Dr. Keith L Scott'" <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Hi, Rachid.  Confirming that the next BP node in the end-to-end path – the next bundle hop (bop) – has received a bundle is precisely what a reliable convergence-layer protocol does.  Now, it may be the case that the received bundle will not be forwarded any further:
The bundle may be malformed or otherwise corrupted, therefore deleted.
The bundle may be too large to fit into available storage, therefore dropped.
In a well-managed network these errors will be rare, but I agree that for critical transmissions they need to be reported.  That is why I fully endorse Felix’s concept of adding to BP a mechanism for sending aggregated bundle acknowledgments.  But that mechanism is not a substitute for a technically sound retransmission protocol at the convergence layer, for reasons we’ve already discussed at length; it’s an augmentation.

I also agree that reliance on BIBE ARQ will work only on nodes that support BIBE ARQ.  Similarly, reliance on custody transfer – aggregated bundle acknowledgments at the bundle layer – will work only on nodes that support custody transfer.  This is a deficiency that is common to all protocols that I know of.

Scott

From: Chaoua, Rachid (GSFC-4500) <[email-masked]> 
Sent: Wednesday, March 30, 2022 9:39 AM
To: [email-masked]; Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked]>; [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Marc’s statement “the rover unequivocally knows that a complete data product has made it to the orbiter” reflects the sentiments I’ve heard within the Near Space Network when discussing DTN and Bundle Protocol. It is a core requirement for GSFC to confirm the next hop (bop) has custody of a data payload prior to that data being released from the source (or previous bop). This is requirement for certain data flow types not all. 

Regarding the statement (the LTP part), “Why are people in MRN opposed to using LTP”. Our (NSN/GSFC) stance (captured in the Draft LunaNet Interoperability Specification, https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20Specification%20Final.pdf, p.10) is that we will initially support various CLAs: TCPCL, UDPCL, LTPCL, or Encapsulation (future CLAs will be assessed and supportability determined by the NSN). Since we are planning on using various service providers, we are leaving the CLs used up to them. Regardless of the CL implemented, there is a need for a custody transfer option at the bundle layer. Relying on BIBE CL only works on nodes that support BIBE CL.  That gives me some concern. 

Cheers!

Marc, I certainly agree that this is a serious problem, but I don’t agree that it motivates development of a reliability mechanism above the CLA.  (I believe a number of other requirements do that.)  Prox-1 may be technically a reliable CLA, but since it routinely misses data it’s apparently not the right one to use for this environment.  Why are people in MRN opposed to using LTP, which is a CCSDS standard that has been in continuous operational use in low Earth orbit (on ISS) since 2016?

Scott

To: [email-masked]; [email-masked]; 'Dr. Keith L Scott' <[email-masked]>

Marc Sanchez Net

Office: (818) 354-1650 | Email: [email-masked]

I agree with everything Felix says here (except perhaps abandonment of the term “custodial”, which seems to have a lot of resonance in the DTN community).

Certainly we don’t have the final answer to BP node congestion.  QoS may well play a role, and we are starting to think about that in DTN WG.  BPv7 does support the concept of “paring” excessive traffic by imposing lifetime overrides that result in immediate bundle expiration, a somewhat manageable mechanism for dropping bundles as necessary.

But I don’t think we need to give up on flow control altogether.  When TCP is used at the convergence layer, a receiving node that is facing congestion can simply stop reading on the socket and let TCP flow control back-propagate.  When LTP is used at the convergence layer, a receiving node that is facing congestion can begin canceling sessions; this will result in CL protocol failure at the sender, bouncing the affected bundles back up to the BPA where reforwarding procedures can be initiated (admittedly an implementation matter, at least for now).  Neither entails bundle reception that requires the receiving BPA to make a decision.  Personally, I think building on these kinds of capabilities to address congestion at the convergence layer would be more powerful than demanding more functionality from custody transfer.

Scott

Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; Mehmet Adalier <[email-masked]>; [email-masked]; [email-masked]

Regards, 
Felix 

From:        "Dr. Keith L Scott" <[email-masked]> 
To:        "Mehmet Adalier" <[email-masked]>, "[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]> 
Date:        28/03/2022 15:11 
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs 

The not-often-stated assumption here is that bundle nodes that receive bundles via reliable CLAs do in fact take on the responsibilities of what was traditionally referred to as ‘custody transfer’ – i.e. following whatever procedures are necessary to ensure that the received bundle reaches its destination (e.g. store it until a forward path becomes available, attempt retransmission to the next hop until there’s an indication that the next hop has (via a reliable CLA in this context) received the bundle, etc.).  I.e. this approach assumes that receipt of a bundle è accepting ‘custody’ of it.  Without this assumption, the service provided by BP is essentially the same best-effort service that IP provides, which I think is less than what we want, particularly for space missions. So I think that in the BPv7 context, all bundles are considered ‘custodial’.

1.        What if I have a bundle node that is capable of forwarding but that has quite minimal storage?  By receiving a bundle, this node is committing to storing that bundle until it can be reliably forwarded.  It seems like there could be cases where this is really inefficient, especially if the reliable forwarding is over something like BIBE where the RTT to get an acknowledgement could be high.  In BPv6, such a node could simply forward a custody-requesting bundle without actually taking custody, sort of like a ‘transit node’ in a BIBE tunnel.  In this case, it might be able to achieve a higher throughput at the expense of NOT accepting the storage requirements from the current custodians. 
2.        Even if a congested node has a lot of storage (but still becomes congested), in BPv6 there was the notion that the node could drop (probably lower-priority) non-custodial bundles in favor of (probably higher-priority) custodial bundles.  We don’t currently have any notion of priority in BPv7, but if we ever want to admit the possibility that a bundle node might drop a bundle due to congestion, it seems like the assumption that receipt à ‘custodial’ acceptance constrains us.  In the BPv7 ‘receipt is (custody) acceptance’ model, the node would have to refuse new bundles. 

 

If we want to have the option of dropping lower-priority bundles that have already been received and are being stored at a node, we’ll need an extension block to mark priority, fine.  We could then create rules that operate at the BP layer to drop bundles when congestion occurs according to their priority markings and address #2 above. I suppose in this case the ‘reliability’ is still at the CLA layer and the decision-making process on whether or not to drop an incoming bundle is at the BP layer.  
That might not address #1 above but maybe #1 is sufficiently rare (or non-existent) that we decide to ignore it. 
Regardless, this has the disadvantage that the transmitting node would believe that the bundle was accepted (because it would have been, by the receiving CLA) event though it then got dropped by the BP layer at the receiving node.

                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

//Will

Scott

All,

Brian S.

---

From: <[email-masked]>
Date: Wed, 30 Mar 2022 11:23:51 -0700

Hi, Mehmet.  Some remarks in-line below.

Scott

From: Mehmet Adalier <[email-masked]> 
Sent: Wednesday, March 30, 2022 11:04 AM
To: [email-masked]; 'Chaoua, Rachid (GSFC-4500)' <[email-masked]>; 'Sanchez Net, Marc (JPL-332H)[JPL Employee]' <[email-masked]>; [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Naïve question.

I think this is asking a little too much of the link-layer protocols.  AOS/USLP won’t suspend timeout-driven frame retransmission when the contact plan indicates that the link will be lost for the next 90 minutes, for example.  You could write an AOS/USLP convergence-layer adapter that did this, but then you’d just be reinventing LTP.

I think it’s better to do this in a BP extension rather than in a separate application above BP.  We normally expect applications to be operating only at the ends of the end-to-end path, not at intermediate forwarding points (in effect, routers).  I think it makes more sense to implement this capability within the administrative element of the node’s application agent, using administrative records to convey the acknowledgments; it’s still technically an application but it’s an intra-BP application, maybe a little like ICMP.

mA

From: dtn <[email-masked] <mailto:[email-masked]> > on behalf of <[email-masked] <mailto:[email-masked]> >
Date: Wednesday, March 30, 2022 at 10:50 AM
To: "'Chaoua, Rachid (GSFC-4500)'" <[email-masked] <mailto:[email-masked]> >, "'Sanchez Net, Marc (JPL-332H)[JPL Employee]'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, "'Dr. Keith L Scott'" <[email-masked] <mailto:[email-masked]> >
Cc: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

*	The bundle may be malformed or otherwise corrupted, therefore deleted.
*	The bundle may be too large to fit into available storage, therefore dropped.

Scott

From: Chaoua, Rachid (GSFC-4500) <[email-masked] <mailto:[email-masked]> > 
Sent: Wednesday, March 30, 2022 9:39 AM
To: [email-masked] <mailto:[email-masked]> ; Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

 

Cheers!

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of [email-masked] <mailto:[email-masked]> 
Sent: Tuesday, March 29, 2022 7:22 PM
To: Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: Sanchez Net, Marc (US 332H) < <mailto:[email-masked]> [email-masked]> 
Sent: Tuesday, March 29, 2022 1:09 PM
To:  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked]; 'Dr. Keith L Scott' < <mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]> [email-masked]>;  <mailto:[email-masked]> [email-masked]
Subject: RE: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Marc Sanchez Net

Office:  <tel:(818)%20393-5840> (818) 354-1650 | Email:  <mailto:[email-masked]> [email-masked]

From: SIS-DTN < <mailto:[email-masked]> [email-masked]> On Behalf Of sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To:  <mailto:[email-masked]> [email-masked]; 'Dr. Keith L Scott' < <mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]> [email-masked]>;  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked]
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From:  <mailto:[email-masked]> [email-masked] < <mailto:[email-masked]> [email-masked]> 
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott < <mailto:[email-masked]> [email-masked]>
Cc: 'Sipos, Brian J.' < <mailto:[email-masked]> [email-masked]>;  <mailto:[email-masked]> [email-masked]; Mehmet Adalier < <mailto:[email-masked]> [email-masked]>;  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked]
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Regards, 
Felix 

From:        "Dr. Keith L Scott" < <mailto:[email-masked]> [email-masked]> 
To:        "Mehmet Adalier" < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, "'Sipos, Brian J.'" < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]> 
Date:        28/03/2022 15:11 
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs 

  _____  

The not-often-stated assumption here is that bundle nodes that receive bundles via reliable CLAs do in fact take on the responsibilities of what was traditionally referred to as ‘custody transfer’ – i.e. following whatever procedures are necessary to ensure that the received bundle reaches its destination (e.g. store it until a forward path becomes available, attempt retransmission to the next hop until there’s an indication that the next hop has (via a reliable CLA in this context) received the bundle, etc.).  I.e. this approach assumes that receipt of a bundle ==> accepting ‘custody’ of it.  Without this assumption, the service provided by BP is essentially the same best-effort service that IP provides, which I think is less than what we want, particularly for space missions. So I think that in the BPv7 context, all bundles are considered ‘custodial’.

1.        What if I have a bundle node that is capable of forwarding but that has quite minimal storage?  By receiving a bundle, this node is committing to storing that bundle until it can be reliably forwarded.  It seems like there could be cases where this is really inefficient, especially if the reliable forwarding is over something like BIBE where the RTT to get an acknowledgement could be high.  In BPv6, such a node could simply forward a custody-requesting bundle without actually taking custody, sort of like a ‘transit node’ in a BIBE tunnel.  In this case, it might be able to achieve a higher throughput at the expense of NOT accepting the storage requirements from the current custodians. 
2.        Even if a congested node has a lot of storage (but still becomes congested), in BPv6 there was the notion that the node could drop (probably lower-priority) non-custodial bundles in favor of (probably higher-priority) custodial bundles.  We don’t currently have any notion of priority in BPv7, but if we ever want to admit the possibility that a bundle node might drop a bundle due to congestion, it seems like the assumption that receipt --> ‘custodial’ acceptance constrains us.  In the BPv7 ‘receipt is (custody) acceptance’ model, the node would have to refuse new bundles. 

 

                                --keith

From: "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> > on behalf of "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >
Reply-To: Mehmet Adalier <[email-masked] <mailto:[email-masked]> >
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >, "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, Felix Flentge <[email-masked] <mailto:[email-masked]> >, "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >, "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Best

mehmet

From: SIS-DTN <[email-masked] <mailto:[email-masked]> > on behalf of "sburleig.sb--- via SIS-DTN" <[email-masked] <mailto:[email-masked]> >
Reply-To: <[email-masked] <mailto:[email-masked]> >
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of Sipos, Brian J.
Sent: Friday, March 25, 2022 4:46 AM
To: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: Re: [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Felix,

Brian S.

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of [email-masked] <mailto:[email-masked]> 
Sent: Friday, March 25, 2022 6:11 AM
To: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

Regards, 
Felix 

From:        "William Ivancic" < <mailto:[email-masked]> [email-masked]> 
To:        < <mailto:[email-masked]> [email-masked]>, "'Sipos, Brian J.'" < <mailto:[email-masked]> [email-masked]>, < <mailto:[email-masked]> [email-masked]> 
Date:        25/03/2022 01:16 
Subject:        Re: [dtn] Bundle custody transfer and reliable CLs 
Sent by:        "dtn" < <mailto:[email-masked]> [email-masked]> 

  _____  

//Will

From: dtn <[email-masked] <mailto:[email-masked]> > on behalf of <[email-masked] <mailto:[email-masked]> >
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Scott

From: dtn <[email-masked] <mailto:[email-masked]> > On Behalf Of Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To: [email-masked] <mailto:[email-masked]> 
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

*         Reception behavior can be overridden per-transfer based on the sender’s desire

Brian S.

_______________________________________________ dtn mailing list [email-masked] <mailto:[email-masked]>  https://www.ietf.org/mailman/listinfo/dtn _______________________________________________
dtn mailing list
 <mailto:[email-masked]> [email-masked]
https://www.ietf.org/mailman/listinfo/dtn

_______________________________________________ SIS-DTN mailing list [email-masked] <mailto:[email-masked]>  https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn 

_______________________________________________ dtn mailing list [email-masked] <mailto:[email-masked]>  https://www.ietf.org/mailman/listinfo/dtn

---

From: <[email-masked]>
Date: Thu, 31 Mar 2022 08:00:25 +0000

On top of what shared by Scott, please also note that CCSDS AOS specifications do not support retransmissions. USLP does it in the form of COP-1 and COP-P (as also part of TC and Proximity-1 CCSDS Standards), but clearly more effective selective ARQ schemes as those available from LTP are desirable.

Best Regards,

Tomaso

————————————————————————
Deutsches Zentrum für Luft- und Raumfahrt (DLR)
German Aerospace Center
Institute of Communications and Navigation | Satellite Networks | Oberpfaffenhofen | 82234 Wessling | Germany
Tomaso de Cola, Ph.D.
Telefon +49 8153 28-2156 | Telefax  +49 8153 28-2844 | [email-masked]<mailto:[email-masked]>
http://www.dlr.de/kn/institut/abteilungen/san

From: dtn <[email-masked]> On Behalf Of [email-masked]
Sent: Mittwoch, 30. März 2022 20:24
To: 'Mehmet Adalier' <[email-masked]>; 'Chaoua, Rachid (GSFC-4500)' <[email-masked]>; 'Sanchez Net, Marc (JPL-332H)[JPL Employee]' <[email-masked]>; [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: Mehmet Adalier <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, March 30, 2022 11:04 AM
To: [email-masked]<mailto:[email-masked]>; 'Chaoua, Rachid (GSFC-4500)' <[email-masked]<mailto:[email-masked]>>; 'Sanchez Net, Marc (JPL-332H)[JPL Employee]' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Naïve question.

mA

From: dtn <[email-masked]<mailto:[email-masked]>> on behalf of <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, March 30, 2022 at 10:50 AM
To: "'Chaoua, Rachid (GSFC-4500)'" <[email-masked]<mailto:[email-masked]>>, "'Sanchez Net, Marc (JPL-332H)[JPL Employee]'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, "'Dr. Keith L Scott'" <[email-masked]<mailto:[email-masked]>>
Cc: "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: Chaoua, Rachid (GSFC-4500) <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, March 30, 2022 9:39 AM
To: [email-masked]<mailto:[email-masked]>; Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Cheers!

From: dtn <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Tuesday, March 29, 2022 7:22 PM
To: Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked]<mailto:[email-masked]>>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

Scott

Regards,
Felix

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards,
Felix

//Will

Scott

All,

Brian S.

_______________________________________________ SIS-DTN mailing list [email-masked]<mailto:[email-masked]> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fmailman.ccsds.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsis-dtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6Wcylx8mv%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gkjxxJBrXxMT2YODEHEErMPPkwa0WmljSO7Y4Z%2Bcius%3D&reserved=0>
_______________________________________________ dtn mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn

---

From: [email-masked]
Date: Thu, 31 Mar 2022 10:37:07 +0200

Well,

I would say all a reliable CLA would do is to confirm to the forwarding 
CLA that it has received all data that has been forwarded. I wouldn't 
expect the CLA to check whether this data is a bundle or something else.
This data will be provided to the BPA. Now:
a) if it is corrupted, the bundle node may not be able to do anything with 
it and just delete it.
b) the bundle node might be able to read correct bundles --> in this case 
we might want to report reception

After this:
c) the bundle node may forward the bundle --> we might want to report 
forwarding
d) the bundle node delivers the bundle --> we might want to report 
delivery
e) the bundle node deletes the bundle (expired, security checks fail, no 
storage, ...) --> we might want to report deletion

(The Compressed Bundle Reporting mechanism is about making these reporting 
more efficient.)

The report receiving node may do something wrt to the reports received 
(delete data, re-transmit data, ...). This might be application specific 
or we might want to specify some behaviour (which could be a 
re-transmission mechanism). 

Having said this, I agree that re-transmission at the convergence layer is 
the preferred mechanism if possible (eg, for bi-directional links) and 
that bundle re-transmission should only happen in exceptional (whatever 
this means) cases, e..g. if we have reliable convergence layers but maybe 
loose a bundle at the end of a contact.

Regards,
Felix

From:   <[email-masked]>
To:     "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked]>, 
"'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" 
<[email-masked]>, <[email-masked]>, "'Dr. Keith L 
Scott'" <[email-masked]>
Cc:     "'Sipos, Brian J.'" <[email-masked]>, [email-masked]
Date:   30/03/2022 19:52
Subject:        Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle 
custody transfer and reliable CLs
Sent by:        "dtn" <[email-masked]>

Hi, Rachid.  Confirming that the next BP node in the end-to-end path ? the 
next bundle hop (bop) ? has received a bundle is precisely what a reliable 
convergence-layer protocol does.  Now, it may be the case that the 
received bundle will not be forwarded any further:
The bundle may be malformed or otherwise corrupted, therefore deleted.
The bundle may be too large to fit into available storage, therefore 
dropped.
In a well-managed network these errors will be rare, but I agree that for 
critical transmissions they need to be reported.  That is why I fully 
endorse Felix?s concept of adding to BP a mechanism for sending aggregated 
bundle acknowledgments.  But that mechanism is not a substitute for a 
technically sound retransmission protocol at the convergence layer, for 
reasons we?ve already discussed at length; it?s an augmentation.
 
I also agree that reliance on BIBE ARQ will work only on nodes that 
support BIBE ARQ.  Similarly, reliance on custody transfer ? aggregated 
bundle acknowledgments at the bundle layer ? will work only on nodes that 
support custody transfer.  This is a deficiency that is common to all 
protocols that I know of.
 
Scott
 
From: Chaoua, Rachid (GSFC-4500) <[email-masked]> 
Sent: Wednesday, March 30, 2022 9:39 AM
To: [email-masked]; Sanchez Net, Marc (JPL-332H)[JPL Employee] 
<[email-masked]>; [email-masked]; 'Dr. Keith L 
Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
Marc?s statement ?the rover unequivocally knows that a complete data 
product has made it to the orbiter? reflects the sentiments I?ve heard 
within the Near Space Network when discussing DTN and Bundle Protocol. It 
is a core requirement for GSFC to confirm the next hop (bop) has custody 
of a data payload prior to that data being released from the source (or 
previous bop). This is requirement for certain data flow types not all. 
 
Regarding the statement (the LTP part), ?Why are people in MRN opposed to 
using LTP?. Our (NSN/GSFC) stance (captured in the Draft LunaNet 
Interoperability Specification, 
https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20Specification%20Final.pdf
, p.10) is that we will initially support various CLAs: TCPCL, UDPCL, 
LTPCL, or Encapsulation (future CLAs will be assessed and supportability 
determined by the NSN). Since we are planning on using various service 
providers, we are leaving the CLs used up to them. Regardless of the CL 
implemented, there is a need for a custody transfer option at the bundle 
layer. Relying on BIBE CL only works on nodes that support BIBE CL.  That 
gives me some concern. 

Cheers!

From: dtn <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, March 29, 2022 7:22 PM
To: Sanchez Net, Marc (JPL-332H)[JPL Employee] <
[email-masked]>; [email-masked]; 'Dr. Keith L Scott' 
<[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
Marc, I certainly agree that this is a serious problem, but I don?t agree 
that it motivates development of a reliability mechanism above the CLA. (I 
believe a number of other requirements do that.)  Prox-1 may be 
technically a reliable CLA, but since it routinely misses data it?s 
apparently not the right one to use for this environment.  Why are people 
in MRN opposed to using LTP, which is a CCSDS standard that has been in 
continuous operational use in low Earth orbit (on ISS) since 2016?

Scott
 
To: [email-masked]; [email-masked]; 'Dr. Keith L Scott' <
 
I also agree that having reliability mechanisms above the CLA is 
necessary. For example, the Mars Relay Network (MRN) uses the Prox-1 
protocol on the proximity links, which has a built-in Go-Back-N ARQ 
mechanism, so it is technically a reliable CLA. Yet, they routinely miss 
data on the return direction and both the relay orbiter and rover missions 
have accountability mechanisms that look for missing data.
 
When I talk to people in the MRN, their biggest desire from DTN is to 
avoid the problem above. In their words, they want a mechanism so that the 
rover unequivocally knows that a complete data product has made it to the 
orbiter. 

From: SIS-DTN <[email-masked]> On Behalf Of 
sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To: [email-masked]; 'Dr. Keith L Scott' <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; 
[email-masked]
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer 
and reliable CLs
 
I agree with everything Felix says here (except perhaps abandonment of the 
term ?custodial?, which seems to have a lot of resonance in the DTN 
community).
 
It is true that computed retransmission timeout intervals will not 
necessarily be absolutely accurate even in BIBE/CT, but at least there are 
ways to limit the error.  Methods for bundle delivery time estimation 
based on contact plan analysis were studied in detail and published almost 
10 years ago: given known source and destination node ID and a bundle 
origination time and bundle size, we can estimate the time of arrival of 
the original bundle and consequently the time of arrival of its 
acknowledgment bundle, even over asymmetric paths.
 
Certainly we don?t have the final answer to BP node congestion.  QoS may 
well play a role, and we are starting to think about that in DTN WG.  BPv7 
does support the concept of ?paring? excessive traffic by imposing 
lifetime overrides that result in immediate bundle expiration, a somewhat 
manageable mechanism for dropping bundles as necessary.
 
But I don?t think we need to give up on flow control altogether.  When TCP 
is used at the convergence layer, a receiving node that is facing 
congestion can simply stop reading on the socket and let TCP flow control 
back-propagate.  When LTP is used at the convergence layer, a receiving 
node that is facing congestion can begin canceling sessions; this will 
result in CL protocol failure at the sender, bouncing the affected bundles 
back up to the BPA where reforwarding procedures can be initiated 
(admittedly an implementation matter, at least for now).  Neither entails 
bundle reception that requires the receiving BPA to make a decision. 
Personally, I think building on these kinds of capabilities to address 
congestion at the convergence layer would be more powerful than demanding 
more functionality from custody transfer.
 
Scott
 
From: [email-masked] <[email-masked]> 
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott <[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; Mehmet 
Adalier <[email-masked]>; [email-masked]; 
[email-masked]
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs

Regards, 
Felix 

From:        "Dr. Keith L Scott" <[email-masked]> 
To:        "Mehmet Adalier" <[email-masked]>, "
[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" <
[email-masked]>, "[email-masked]" <[email-masked]>, 
"[email-masked]" <[email-masked]>, "[email-masked]" <
[email-masked]> 
Date:        28/03/2022 15:11 
Subject:        Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs 

The question remains: do we need some form of reliability above the CLA 
layer?  As Scott points out, BIBE is ?just? a reliable CLA that uses 
bundles as its transport mechanism.  That allows BIBE to function as a 
reliable CLA in environments where other CLAs that require bidirectional 
connectivity cannot, but viewed from the standpoint of the sending and 
receiving BIBE endpoints, it?s just another reliable CLA.
 
The not-often-stated assumption here is that bundle nodes that receive 
bundles via reliable CLAs do in fact take on the responsibilities of what 
was traditionally referred to as ?custody transfer? ? i.e. following 
whatever procedures are necessary to ensure that the received bundle 
reaches its destination (e.g. store it until a forward path becomes 
available, attempt retransmission to the next hop until there?s an 
indication that the next hop has (via a reliable CLA in this context) 
received the bundle, etc.).  I.e. this approach assumes that receipt of a 
bundle è accepting ?custody? of it.  Without this assumption, the service 
provided by BP is essentially the same best-effort service that IP 
provides, which I think is less than what we want, particularly for space 
missions. So I think that in the BPv7 context, all bundles are considered 
?custodial?.

custody, sort of like a ?transit node? in a BIBE tunnel.  In this case, it 
higher-priority) custodial bundles.  We don?t currently have any notion of 
assumption that receipt à ?custodial? acceptance constrains us.  In the 
BPv7 ?receipt is (custody) acceptance? model, the node would have to 
 
I?ll readily admit that calculating a (good) retransmission timer value in 
the case where a node does NOT know if the proximate (or even which) 
downstream node will take custody is difficult or impossible for some 
networks.  BIBE still has a bit of this problem, especially if the path to 
the BIBE destination is long, as the sender won?t necessarily know the 
path the BIBE bundles will take, but it is at least more constrained than 
the completely open case.

already been received and are being stored at a node, we?ll need an 
the ?reliability? is still at the CLA layer and the decision-making 
  
                                --keith

From: "[email-masked]" <
[email-masked]> on behalf of "[email-masked]
" <[email-masked]>
Reply-To: Mehmet Adalier <[email-masked]>
Date: Friday, March 25, 2022 at 1:21 PM
To: "[email-masked]" <[email-masked]>, "'Sipos, Brian J.'" <
[email-masked]>, Felix Flentge <[email-masked]>, "
[email-masked]" <[email-masked]>, "[email-masked]" <
[email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and 
reliable CLs
 
I believe Scott?s analysis below succinctly articulates the difference 
between the two approaches and I agree that they should be kept separate. 
For my intended use cases, approach 1 (BIBE/BPARQ?) is what I would need 
to use. I?ll be happy to contribute to this approach and prototype.
 
Best
mehmet

Hi, Brian.  Actually I think Felix?s analysis is pretty much spot-on.  We 
really are talking about two different behaviors, which respond to two 
different sets of requirements.
 
The ?custody transfer? procedures that I proposed in the BIBE draft are 
very specifically aimed at defining a reliable convergence-layer adapter 
that happens to use BP as its underlying convergence-layer protocol. There 
is a requirement for this capability: Keith Scott has often pointed out 
that there are scenarios in which reliable transmission between nodes is 
required but no reliable transmission protocol is available, e.g., when 
the sender?s communication capability is temporarily unidirectional. These 
are not hypothetical; MITRE?s customers must at times operate in such 
environments, and some space flight missions and other IoT systems could 
be similarly constrained.  Under these conditions, the mechanism by which 
NAKs and ACKs are returned to the sender may function at a later time 
and/or be unrelated to the mechanism by which the sender transmitted the 
data.  There might be better standardized protocols than BP for supporting 
these kinds of scenarios, but none leap to mind.
 
The ?custody transfer? procedures for which Felix proposes requirements 
 
I would propose that we term the latter procedures ?custody transfer? and 
proceed to standardize them, while renaming the former something like 
?BPARQ?.
 
I don?t think there?s any need to impose any additional requirements on CL 
protocols, TCPCL or other, to satisfy the requirements.  These are 
separate things.  Let?s keep them separate and support them separately and 
clearly.
 
Scott

Felix,
Thank you for this feedback. There is a misinterpretation of what I (and 
maybe also Scott via BIBE) am suggesting about what happens during 
reliable reception. The idea isn?t that two different things happen, it?s 
that it?s the same BP agent custody acceptance criteria just that some CLs 
can provide intrinsic signaling of that acceptance while other CLs have no 
means to signal that specific feedback.
 
While IETF doesn?t (normally) specify internal APIs, the CCSDS documents 
do and this can help here. Currently 734.2-B-1 does not actually define a 
BPA?CLA interface API, but it seems like the rough interface looks like:
Send.request to the CLA 
Send.response from the CLA 
Receive.indication from the CLA
  
What I am suggesting is that the Receive side could be changed from the 
asynchronous ?I just got this transfer. Here it is, thanks.? to a 
synchronous ?I just got this transfer. Will you accept it?? similar to:
Receive.indication from the CLA 
Receive.response to the CLA
  
The TCPCL and LTPCL already provide the negative response over-the-wire 
(TCPCL reception can send XFER_REFUSE at any time before the END ACK, and 
LTPCL can send ?Cancel from the block receiver? similarly) there is 
currently just no specific formal definition of what, from the BPA side, 
the positive acknowledgement is required to mean. For example ?If the 
transfer is not canceled by the receiver and the final ACK is sent, the 
transferred bundle SHALL be completely and positively received within the 
BP agent?s forwarding or delivery queue.? 
 
As Scott mentioned earlier, custody isn?t an anthropomorphization of the 
BPA, it?s a specific behavior, and it seems like by acknowledging that the 
bundle was received into the queue for delivery/forwarding the agent has 
?accepted? it. If the intent of custody is that it?s a more 
restricted/reserved resource pool (e.g. my forwarding queue is size X but 
of that only Y (with Y<X) is reserved for bundles over which I have 
custody) then that?s a local agent management issue, not an over-the-wire 
signaling issue. The BPA has still positively accepted the bundle and some 
CLAs can communicate this back to the sending agent synchronously.

 

Regards, 
Felix 

?Custody? is  a bundle level concept, not transport.  Way back when, ?
Custody? meant that the node taking ?Custody? would try it?s best to 
ensure the bundle was forwarded either to another ?Custody? node or to 
eventually the destination node.  The idea, as I recall, was this would 
allow the original bundle source to clear its memory of that bundle.  

//Will

 

 
 
Scott

All,

---

From: "Dr. Keith L Scott" <[email-masked]>
Date: Thu, 31 Mar 2022 12:23:41 +0000

I do think the kernel of the issue under discussion is whether the rover is willing to release resources based on a reliable CLA confirming receipt, or if it (the rover) should wait for some signaling from the BP layer.  The former implies an assumption that successful receipt by the (reliable) CLA  “custody transfer” to the receiving node and that the receiving node will hold onto the bundle until it can be reliably transferred (via a reliable CLA) to the next hop.  Without the assumption that every node takes custody, BP is a best-effort service and I think that’s too loose a service for what we want in space.

The way I read RFC9171, a bundle node that attempts to send a bundle that results in one or more of the CLAs reporting failure to forward MAY attempt to retransmit the bundle (or, presumably, it MAY not).  [The text after step 5 in section 5.4].  Depending on settings, some sort of signal SHOULD / could be generated noting the deletion of the bundle, but it would be lost regardless.

So, even if forwarding is attempted over a reliable CLA (e.g. TCPCLv4), if that fails for some reason, bundles can be lost.  Even if the space community modifies the processing rules to mandate that bundles that fail CLA forwarding ARE retransmitted, such a change can only apply to the portion of the path that we control.  If we end up buying commercial DTN routers for use in the ground, they could very well stick to the MAY clause and drop bundles for which the initial CLA transmission failed (I think).

To signaling (or at least being able to signal) all sorts of interesting events per Felix’s email: yeah, we should be able to do that.

                                --keith

 

From: Felix Flentge <[email-masked]>
Date: Thursday, March 31, 2022 at 4:37 AM
To: <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>, dtn <[email-masked]>, Keith Scott <[email-masked]>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Well, 

I would say all a reliable CLA would do is to confirm to the forwarding CLA that it has received all data that has been forwarded. I wouldn't expect the CLA to check whether this data is a bundle or something else. 
This data will be provided to the BPA. Now: 
a) if it is corrupted, the bundle node may not be able to do anything with it and just delete it. 
b) the bundle node might be able to read correct bundles --> in this case we might want to report reception 

After this: 
c) the bundle node may forward the bundle --> we might want to report forwarding 
d) the bundle node delivers the bundle --> we might want to report delivery 
e) the bundle node deletes the bundle (expired, security checks fail, no storage, ...) --> we might want to report deletion 

(The Compressed Bundle Reporting mechanism is about making these reporting more efficient.) 

The report receiving node may do something wrt to the reports received (delete data, re-transmit data, ...). This might be application specific or we might want to specify some behaviour (which could be a re-transmission mechanism). 

Having said this, I agree that re-transmission at the convergence layer is the preferred mechanism if possible (eg, for bi-directional links) and that bundle re-transmission should only happen in exceptional (whatever this means) cases, e..g. if we have reliable convergence layers but maybe loose a bundle at the end of a contact. 

Regards, 
Felix 

To:        "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked]>, "'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" <[email-masked]>, <[email-masked]>, "'Dr. Keith L Scott'" <[email-masked]> 

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

 

 

                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

//Will

Scott

All,

Brian S.

_______________________________________________ SIS-DTN mailing list [email-masked] https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn _______________________________________________
dtn mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dtn

---

From: Marc Blanchet <[email-masked]>
Date: Thu, 31 Mar 2022 08:46:18 -0400

> Le 31 mars 2022 à 08:23, Dr. Keith L Scott <[email-masked]> a écrit :
> 
>  

It will always be a best-effort service, because, even if you have mandatory custody, there are no guarantee whatsoever that a node will be able to conform to that custody, because a node may become out of resources (disk full, memory full, …), the node may, after receiving the bundle it has custody, not being able to communicate anymore with the network, etc…: there are many many scenarios (discussed in length for quite a long time) where bundles will be dropped even in a “full” custody network scenario.  Custody is a useful feature for space but it is a preferred service: "please do the very best you can to forward that bundle to the next hop”.

> So, even if forwarding is attempted over a reliable CLA (e.g. TCPCLv4), if that fails for some reason, bundles can be lost.  Even if the space community modifies the processing rules to mandate that bundles that fail CLA forwarding ARE retransmitted, such a change can only apply to the portion of the path that we control. 

To me, a deployment requirement is a profile where you add to an existing base standard something like: “feature X is required for our use case”.  Pretty straightforward and applies to all protocols, to all specifications: has been done for a large set of IETF standards. A good way to handle these use cases is to have a base standard and a series of optional features. Some communities/deployement define their profiles by making mandatory some features and require vendors to support that feature for their deployment.

> If we end up buying commercial DTN routers for use in the ground, they could very well stick to the MAY clause and drop bundles for which the initial CLA transmission failed (I think).

I think your last sentence is out of scope of this discussion. As written above, this is for the specific community to make sure that feature X is implemented by vendors, like any other feature one wants. It has to go into your RFP for acquiring products.

Marc.

> From: Felix Flentge <[email-masked] <mailto:[email-masked]>>
> To: <[email-masked] <mailto:[email-masked]>>
> Cc: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>, dtn <[email-masked] <mailto:[email-masked]>>, Keith Scott <[email-masked] <mailto:[email-masked]>>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked] <mailto:[email-masked]>>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked] <mailto:[email-masked]>>
> From:        <[email-masked] <mailto:[email-masked]>> 
> To:        "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked] <mailto:[email-masked]>>, "'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>, "'Dr. Keith L Scott'" <[email-masked] <mailto:[email-masked]>> 
> Cc:        "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, [email-masked] <mailto:[email-masked]> 
> Date:        30/03/2022 19:52 
> Sent by:        "dtn" <[email-masked] <mailto:[email-masked]>>
>  
> 
> From: Chaoua, Rachid (GSFC-4500) <[email-masked] <mailto:[email-masked]>> 
> To: [email-masked] <mailto:[email-masked]>; Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]>>
> Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>
> Regarding the statement (the LTP part), “Why are people in MRN opposed to using LTP”. Our (NSN/GSFC) stance (captured in the Draft LunaNet Interoperability Specification, https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20Specification%20Final.pdf <https://esc.gsfc.nasa.gov/static-files/Draft%20LunaNet%20Interoperability%20Specification%20Final.pdf>, p.10) is that we will initially support various CLAs: TCPCL, UDPCL, LTPCL, or Encapsulation (future CLAs will be assessed and supportability determined by the NSN). Since we are planning on using various service providers, we are leaving the CLs used up to them. Regardless of the CL implemented, there is a need for a custody transfer option at the bundle layer. Relying on BIBE CL only works on nodes that support BIBE CL.  That gives me some concern. 
>  
>  
> Cheers!
>  
>  
> From: dtn <[email-masked] <mailto:[email-masked]>> On Behalf Of [email-masked] <mailto:[email-masked]>
> To: Sanchez Net, Marc (JPL-332H)[JPL Employee] <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]>>
> Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>
> From: Sanchez Net, Marc (US 332H) <[email-masked] <mailto:[email-masked]>> 
> To: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]>>
> Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>
> Office: (818) 354-1650 <tel:(818)%20393-5840> | Email: [email-masked] <mailto:[email-masked]>
> From: SIS-DTN <[email-masked] <mailto:[email-masked]>> On Behalf Of sburleig.sb--- via SIS-DTN
> To: [email-masked] <mailto:[email-masked]>; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]>>
> Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> From: [email-masked] <mailto:[email-masked]> <[email-masked] <mailto:[email-masked]>> 
> To: Dr. Keith L Scott <[email-masked] <mailto:[email-masked]>>
> Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; Mehmet Adalier <[email-masked] <mailto:[email-masked]>>; [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> From:        "Dr. Keith L Scott" <[email-masked] <mailto:[email-masked]>> 
> To:        "Mehmet Adalier" <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>> 
> Date:        28/03/2022 15:11 
> From: "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>> on behalf of "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
> Reply-To: Mehmet Adalier <[email-masked] <mailto:[email-masked]>>
> To: "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, Felix Flentge <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
> From: SIS-DTN <[email-masked] <mailto:[email-masked]>> on behalf of "sburleig.sb--- via SIS-DTN" <[email-masked] <mailto:[email-masked]>>
> Reply-To: <[email-masked] <mailto:[email-masked]>>
> To: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>
> From: dtn <[email-masked] <mailto:[email-masked]>> On Behalf Of Sipos, Brian J.
> To: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> From: dtn <[email-masked] <mailto:[email-masked]>> On Behalf Of [email-masked] <mailto:[email-masked]>
> To: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> From:        "William Ivancic" <[email-masked] <mailto:[email-masked]>> 
> To:        <[email-masked] <mailto:[email-masked]>>, "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>> 
> Date:        25/03/2022 01:16 
> Sent by:        "dtn" <[email-masked] <mailto:[email-masked]>>
>  
> From: dtn <[email-masked] <mailto:[email-masked]>> on behalf of <[email-masked] <mailto:[email-masked]>>
> To: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]>>, <[email-masked] <mailto:[email-masked]>>
> From: dtn <[email-masked] <mailto:[email-masked]>> On Behalf Of Sipos, Brian J.
> _______________________________________________ dtn mailing list [email-masked] <mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/dtn <https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fdtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=k%2F3uv%2B8kTA25lenkOtDhpbQxi43lX30qVx%2BdyjkZHVk%3D&reserved=0>_______________________________________________
> dtn mailing list
> https://www.ietf.org/mailman/listinfo/dtn <https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fdtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6WUzbi2bq%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=k%2F3uv%2B8kTA25lenkOtDhpbQxi43lX30qVx%2BdyjkZHVk%3D&reserved=0>
> _______________________________________________ SIS-DTN mailing list [email-masked] <mailto:[email-masked]> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn <https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fmailman.ccsds.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsis-dtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6Wcylx8mv%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gkjxxJBrXxMT2YODEHEErMPPkwa0WmljSO7Y4Z%2Bcius%3D&reserved=0> _______________________________________________
> dtn mailing list
> https://www.ietf.org/mailman/listinfo/dtn <https://www.ietf.org/mailman/listinfo/dtn>_______________________________________________
> dtn mailing list
> https://www.ietf.org/mailman/listinfo/dtn <https://www.ietf.org/mailman/listinfo/dtn>

---

From: <[email-masked]>
Date: Thu, 31 Mar 2022 08:10:23 -0700

I think we are converging on a few principles:

1.	If all of the CLAs (usually only one, but there could be parallel transmissions) report to the BPA that forwarding failed, then the sending node may or may not try again, possibly on a different path and/or relying on different CLA support; if not, it may or may not release the retransmission resources occupied by the bundle.  This is a matter for implementation decision, which might vary depending on the size, QoS, time-to-live, etc. of the bundle.
2.	If successful forwarding is reported from the convergence layer, then at that time the sending node may or may not release retransmission resources occupied by the bundle.  This is again a matter for implementation decision, which might vary depending on the size, QoS, time-to-live, etc. of the bundle.
3.	If, following the reporting of successful or unsuccessful forwarding results from the convergence layer, the sending node does not release the retransmission resources occupied by the bundle, then that release of resources may (or may not) instead occur upon reception of a Compressed Bundle Report (or whatever we decide to call it; effectively, a custody signal) from the receiving node or from some other node further downstream, in the event that the bundle was flagged for this behavior.  The absence of such an acknowledgment may or may not result in either automatic (timeout-driven) or managed re-forwarding of the bundle.  Again these are implementation matters.
4.	If all else fails, expiration of the bundle’s time-to-live will eventually and unconditionally result in the release of the retransmission resources occupied by the bundle.

What we’ve currently got in the BPv7 specification already supports all of the above except #3; I think that new flagging/signaling mechanism is the augmentation that we need to focus on.

Scott

From: Dr. Keith L Scott <[email-masked]> 
Sent: Thursday, March 31, 2022 5:24 AM
To: [email-masked]; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; dtn <[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

I do think the kernel of the issue under discussion is whether the rover is willing to release resources based on a reliable CLA confirming receipt, or if it (the rover) should wait for some signaling from the BP layer.  The former implies an assumption that successful receipt by the (reliable) CLA --> “custody transfer” to the receiving node and that the receiving node will hold onto the bundle until it can be reliably transferred (via a reliable CLA) to the next hop.  Without the assumption that every node takes custody, BP is a best-effort service and I think that’s too loose a service for what we want in space.

 

                                --keith

 

From: Felix Flentge <[email-masked] <mailto:[email-masked]> >
Date: Thursday, March 31, 2022 at 4:37 AM
To: <[email-masked] <mailto:[email-masked]> >
Cc: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, dtn <[email-masked] <mailto:[email-masked]> >, Keith Scott <[email-masked] <mailto:[email-masked]> >, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked] <mailto:[email-masked]> >, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Well, 

After this: 

Regards, 
Felix 

From:        <[email-masked] <mailto:[email-masked]> > 
To:        "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked] <mailto:[email-masked]> >, "'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, "'Dr. Keith L Scott'" <[email-masked] <mailto:[email-masked]> > 
Cc:        "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, [email-masked] <mailto:[email-masked]>  
Date:        30/03/2022 19:52 
Subject:        Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs 
Sent by:        "dtn" <[email-masked] <mailto:[email-masked]> > 

  _____  

Scott

 

Cheers!

Scott

From: Sanchez Net, Marc (US 332H) <[email-masked] <mailto:[email-masked]> > 
Sent: Tuesday, March 29, 2022 1:09 PM
To: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> ; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: RE: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Marc Sanchez Net

From: SIS-DTN <[email-masked] <mailto:[email-masked]> > On Behalf Of sburleig.sb--- via SIS-DTN
Sent: Monday, March 28, 2022 11:44 AM
To: [email-masked] <mailto:[email-masked]> ; 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: [EXTERNAL] Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> > 
Sent: Monday, March 28, 2022 7:54 AM
To: Dr. Keith L Scott <[email-masked] <mailto:[email-masked]> >
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; Mehmet Adalier <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Regards, 
Felix 

  _____  

 

 

                                --keith

From: " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]> on behalf of " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>
Reply-To: Mehmet Adalier < <mailto:[email-masked]> [email-masked]>
Date: Friday, March 25, 2022 at 1:21 PM
To: " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, "'Sipos, Brian J.'" < <mailto:[email-masked]> [email-masked]>, Felix Flentge < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Best

mehmet

From: SIS-DTN < <mailto:[email-masked]> [email-masked]> on behalf of "sburleig.sb--- via SIS-DTN" < <mailto:[email-masked]> [email-masked]>
Reply-To: < <mailto:[email-masked]> [email-masked]>
Date: Friday, March 25, 2022 at 10:07 AM
To: "'Sipos, Brian J.'" < <mailto:[email-masked]> [email-masked]>, < <mailto:[email-masked]> [email-masked]>, < <mailto:[email-masked]> [email-masked]>, < <mailto:[email-masked]> [email-masked]>
Subject: Re: [Sis-dtn] [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: dtn < <mailto:[email-masked]> [email-masked]> On Behalf Of Sipos, Brian J.
Sent: Friday, March 25, 2022 4:46 AM
To:  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked]
Subject: Re: [dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Felix,

Brian S.

From: dtn < <mailto:[email-masked]> [email-masked]> On Behalf Of  <mailto:[email-masked]> [email-masked]
Sent: Friday, March 25, 2022 6:11 AM
To:  <mailto:[email-masked]> [email-masked];  <mailto:[email-masked]> [email-masked]
Subject: [EXT] Re: [dtn] Bundle custody transfer and reliable CLs

Regards, 
Felix 

  _____  

//Will

From: dtn < <mailto:[email-masked]> [email-masked]> on behalf of < <mailto:[email-masked]> [email-masked]>
Date: Thursday, March 24, 2022 at 6:04 PM
To: "'Sipos, Brian J.'" < <mailto:[email-masked]> [email-masked]>, < <mailto:[email-masked]> [email-masked]>
Subject: Re: [dtn] Bundle custody transfer and reliable CLs

Scott

From: dtn < <mailto:[email-masked]> [email-masked]> On Behalf Of Sipos, Brian J.
Sent: Thursday, March 24, 2022 1:39 PM
To:  <mailto:[email-masked]> [email-masked]
Subject: [dtn] Bundle custody transfer and reliable CLs

All,

Brian S.

_______________________________________________ dtn mailing list  <mailto:[email-masked]> [email-masked] https://www.ietf.org/mailman/listinfo/dtn _______________________________________________
dtn mailing list
 <mailto:[email-masked]> [email-masked]
https://www.ietf.org/mailman/listinfo/dtn

_______________________________________________ SIS-DTN mailing list  <mailto:[email-masked]> [email-masked] https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn _______________________________________________
dtn mailing list
[email-masked] <mailto:[email-masked]> 
 <https://www.ietf.org/mailman/listinfo/dtn> https://www.ietf.org/mailman/listinfo/dtn

---

From: <[email-masked]>
Date: Thu, 31 Mar 2022 08:19:11 -0700

Sorry, I just realized that the “if not” clause in #1 below could be ambiguous.  What I meant there was “if the sending node does not try again.”

Scott

From: [email-masked] <[email-masked]> 
Sent: Thursday, March 31, 2022 8:10 AM
To: 'Dr. Keith L Scott' <[email-masked]>; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; 'dtn' <[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' <[email-masked]>
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: Dr. Keith L Scott <[email-masked] <mailto:[email-masked]> > 
Sent: Thursday, March 31, 2022 5:24 AM
To: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> 
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; dtn <[email-masked] <mailto:[email-masked]> >; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked] <mailto:[email-masked]> >; 'Chaoua, Rachid(GSFC-4500)' <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

 

                                --keith

 

Well, 

After this: 

Regards, 
Felix 

  _____  

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

  _____  

 

 

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

---

From: Mehmet Adalier <[email-masked]>
Date: Thu, 31 Mar 2022 09:18:56 -0700

Scott,

Thanks for summarizing this. 

Beyond the specific implementation styles, describing/specifying a comprehensive “Compressed Bundle Report (or whatever we decide to call it; effectively, a custody signal)” with pre-defined and user definable fields (i.e., extensible) sounds like the first step we should take collectively.

I’d also like to further probe this suggestion: “I think it makes more sense to implement this capability within the administrative element of the node’s application agent, using administrative records to convey the acknowledgments; it’s still technically an application but it’s an intra-BP application, maybe a little like ICMP.”

So, looks like the two items we need to consider are: a) how do we best define an extensible “Compressed Bundle Report” and b) how do we process it in a standard way (i.e., intra-bp application) within BP spec

In my opinion, these two should be independent of the mission operational/business aspects of the node (QoS, priority, pay-per-xfer) which can be handled outside the BP spec.

Thoughts?

mA

From: dtn <[email-masked]> on behalf of <[email-masked]>
Date: Thursday, March 31, 2022 at 8:19 AM
To: "'Dr. Keith L Scott'" <[email-masked]>, <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, 'dtn' <[email-masked]>, <[email-masked]>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

I think we are converging on a few principles:
If all of the CLAs (usually only one, but there could be parallel transmissions) report to the BPA that forwarding failed, then the sending node may or may not try again, possibly on a different path and/or relying on different CLA support; if not, it may or may not release the retransmission resources occupied by the bundle.  This is a matter for implementation decision, which might vary depending on the size, QoS, time-to-live, etc. of the bundle.
If successful forwarding is reported from the convergence layer, then at that time the sending node may or may not release retransmission resources occupied by the bundle.  This is again a matter for implementation decision, which might vary depending on the size, QoS, time-to-live, etc. of the bundle.
If, following the reporting of successful or unsuccessful forwarding results from the convergence layer, the sending node does not release the retransmission resources occupied by the bundle, then that release of resources may (or may not) instead occur upon reception of a Compressed Bundle Report (or whatever we decide to call it; effectively, a custody signal) from the receiving node or from some other node further downstream, in the event that the bundle was flagged for this behavior.  The absence of such an acknowledgment may or may not result in either automatic (timeout-driven) or managed re-forwarding of the bundle.  Again these are implementation matters.
If all else fails, expiration of the bundle’s time-to-live will eventually and unconditionally result in the release of the retransmission resources occupied by the bundle.

Scott

 

I do think the kernel of the issue under discussion is whether the rover is willing to release resources based on a reliable CLA confirming receipt, or if it (the rover) should wait for some signaling from the BP layer.  The former implies an assumption that successful receipt by the (reliable) CLA à “custody transfer” to the receiving node and that the receiving node will hold onto the bundle until it can be reliably transferred (via a reliable CLA) to the next hop.  Without the assumption that every node takes custody, BP is a best-effort service and I think that’s too loose a service for what we want in space.

 

                                --keith

 

Well, 

After this: 

Regards, 
Felix 

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

 

 

                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

//Will

Scott

All,

Brian S.

---

From: "Dr. Keith L Scott" <[email-masked]>
Date: Thu, 31 Mar 2022 16:25:29 +0000

I think I agree, with the possible extension that if #3 is postulating some sort of custody signal the “If…does not release resources” can be strengthened a bit if we so choose.  That is, if we define how the extra signaling works, we have the opportunity to define / constrain behaviors of implementations that implement that signaling.  So if we define some sort of BP-layer signaling for ‘custody’ (using compressed bundle reports or otherwise), we might say that nodes implementing that signaling SHALL NOT release resources until (receipt of a positive custody signal OR  appropriate rules to address the case that no other node in the system implements the signaling, etc.).

                                --keith

From: "[email-masked]" <[email-masked]>
Date: Thursday, March 31, 2022 at 11:19 AM
To: Keith Scott <[email-masked]>, Felix Flentge <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, "[email-masked]" <[email-masked]>, 'dtn' <[email-masked]>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked]>
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

Scott

From: Dr. Keith L Scott <[email-masked]<mailto:[email-masked]>>
Sent: Thursday, March 31, 2022 5:24 AM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Cc: 'Sipos, Brian J.' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; dtn <[email-masked]<mailto:[email-masked]>>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked]<mailto:[email-masked]>>; 'Chaoua, Rachid(GSFC-4500)' <[email-masked]<mailto:[email-masked]>>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

                                --keith

From: Felix Flentge <[email-masked]<mailto:[email-masked]>>
Date: Thursday, March 31, 2022 at 4:37 AM
To: <[email-masked]<mailto:[email-masked]>>
Cc: "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, dtn <[email-masked]<mailto:[email-masked]>>, Keith Scott <[email-masked]<mailto:[email-masked]>>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked]<mailto:[email-masked]>>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Well,

After this:

Regards,
Felix

From:        <[email-masked]<mailto:[email-masked]>>
To:        "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked]<mailto:[email-masked]>>, "'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" <[email-masked]<mailto:[email-masked]>>, <[email-masked]<mailto:[email-masked]>>, "'Dr. Keith L Scott'" <[email-masked]<mailto:[email-masked]>>
Cc:        "'Sipos, Brian J.'" <[email-masked]<mailto:[email-masked]>>, [email-masked]<mailto:[email-masked]>
Date:        30/03/2022 19:52
Subject:        Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs
Sent by:        "dtn" <[email-masked]<mailto:[email-masked]>>
________________________________

Scott

Cheers!

Scott

Marc Sanchez Net

Scott

Regards,
Felix

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards,
Felix

//Will

Scott

All,

Brian S.

_______________________________________________ SIS-DTN mailing list [email-masked]<mailto:[email-masked]> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.us%2Fv3%2F__https%3A%2Fmailman.ccsds.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fsis-dtn__%3B!!PvBDto6Hs4WbVuu7!epFXV3_S7aEW7PDxgWNBW8LKuki9s1NTL4sa8CVsC2thPyLbcOdehXgutIckCym6Wcylx8mv%24&data=04%7C01%7Crachid.chaoua%40nasa.gov%7C5d5bb21035074c676e4608da11db3786%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637841930483903637%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=gkjxxJBrXxMT2YODEHEErMPPkwa0WmljSO7Y4Z%2Bcius%3D&reserved=0> _______________________________________________
dtn mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/dtn

---

From: <[email-masked]>
Date: Thu, 31 Mar 2022 10:03:41 -0700

Fine with me.  I wanted not to over-constrain the idea at this early stage of definition, but if that sort of mandate is okay with everybody I am for it.

Scott

From: Dr. Keith L Scott <[email-masked]> 
Sent: Thursday, March 31, 2022 9:25 AM
To: [email-masked]; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; 'dtn' <[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

                                --keith

From: "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >
Date: Thursday, March 31, 2022 at 11:19 AM
To: Keith Scott <[email-masked] <mailto:[email-masked]> >, Felix Flentge <[email-masked] <mailto:[email-masked]> >
Cc: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, "[email-masked] <mailto:[email-masked]> " <[email-masked] <mailto:[email-masked]> >, 'dtn' <[email-masked] <mailto:[email-masked]> >, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked] <mailto:[email-masked]> >, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked] <mailto:[email-masked]> >
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

From: [email-masked] <mailto:[email-masked]>  <[email-masked] <mailto:[email-masked]> > 
Sent: Thursday, March 31, 2022 8:10 AM
To: 'Dr. Keith L Scott' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Cc: 'Sipos, Brian J.' <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> ; 'dtn' <[email-masked] <mailto:[email-masked]> >; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked] <mailto:[email-masked]> >; 'Chaoua, Rachid(GSFC-4500)' <[email-masked] <mailto:[email-masked]> >
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

 

 

                                --keith

 

Well, 

After this: 

Regards, 
Felix 

  _____  

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

  _____  

 

 

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

---

From: <[email-masked]>
Date: Thu, 31 Mar 2022 10:14:36 -0700

Mehmet, I think those two work items are the right ones, but the words “within BP spec” make me uneasy.  I don’t think we should try to modify the BP spec itself to include this capability.  DTN WG is not chartered to publish a new BP RFC, and in any case there is no need: BP is designed for extensibility, and I think this new capability can readily be defined in a supplemental specification that identifies (for example) an additional bundle processing flag and an additional administrative record.

Scott

From: Mehmet Adalier <[email-masked]> 
Sent: Thursday, March 31, 2022 9:19 AM
To: [email-masked]; 'Dr. Keith L Scott' <[email-masked]>; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <[email-masked]>; 'dtn' <[email-masked]>; [email-masked]; 'Chaoua, Rachid(GSFC-4500)' <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott,

Thoughts?

mA

From: dtn <[email-masked] <mailto:[email-masked]> > on behalf of <[email-masked] <mailto:[email-masked]> >
Date: Thursday, March 31, 2022 at 8:19 AM
To: "'Dr. Keith L Scott'" <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >
Cc: "'Sipos, Brian J.'" <[email-masked] <mailto:[email-masked]> >, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked] <mailto:[email-masked]> >, 'dtn' <[email-masked] <mailto:[email-masked]> >, <[email-masked] <mailto:[email-masked]> >, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked] <mailto:[email-masked]> >
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

Scott

 

 

                                --keith

 

Well, 

After this: 

Regards, 
Felix 

  _____  

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

  _____  

 

 

                                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

  _____  

//Will

Scott

All,

Brian S.

---

From: Mehmet Adalier <[email-masked]>
Date: Thu, 31 Mar 2022 10:22:25 -0700

Scott, sorry. The “within BP spec” words were poorly chosen. I agree that we can do this in a supplemental specification and would be great if we could also somehow include it in the appropriate color CCSDS book.

From: dtn <[email-masked]> on behalf of <[email-masked]>
Date: Thursday, March 31, 2022 at 10:14 AM
To: 'Mehmet Adalier' <[email-masked]>, "'Dr. Keith L Scott'" <[email-masked]>, <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, "'Sanchez Net, Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, <[email-masked]>, 'dtn' <[email-masked]>, "'Chaoua, Rachid(GSFC-4500)'" <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody transfer and reliable CLs

Scott

Scott,

Thoughts?

mA

Scott

Scott

 

 

                                --keith

 

Well, 

After this: 

Regards, 
Felix 

Scott

 

Cheers!

Scott

Marc Sanchez Net

Scott

Regards, 
Felix 

 

 

                --keith

Best

mehmet

Scott

Felix,

Brian S.

Regards, 
Felix 

//Will

Scott

All,

Brian S.

---

From: [email-masked]
Date: Fri, 1 Apr 2022 08:28:03 +0200

I think we should distinguish between the signalling, the re-transmission 
and (maybe) custody:

1) CBR (or whatever better name we can come up with)
If a node implement CBR, it just means that it is able to send compressed 
status reports if requested by a specific extension block. This is about 
defining an extension block and administrative records.

2) CBR-R (CBR-based re-transmission)
It would mean the a node receiving CBR status reports implements a 
mechanism to re-send bundles for which it did not receive a reception 
report (this is basically Keith's 'SHALL NOT release resources part'). 
This is about defining node behaviour and there could be different 
variants (sequence-based, timer-based, command-based, ...)

3) CBR-C (CBR-based Custody Transfer)
Not sure, whether we would require this. The difference to 2) could be 
that a receiving node would also be able to send a custody signal (if 
requested in the extension block) in addition to the reception signal. The 
meaning of the custody signal could be beyond the reception signal, eg:
- the bundle is sitting safely in bundle storage
- the receiving node is implementing CBR-R and will add/replace the CBR 
extension block by a new one where it will request custody for the next 
hop
- based on local information, the receiving node will be able to forward 
the bundle towards its destination (there is/will be a contact to a next 
hop)
- ...

Regards,
Felix

From:   <[email-masked]>
To:     "'Dr. Keith L Scott'" <[email-masked]>, <[email-masked]>
Cc:     "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>, 
"'dtn'" <[email-masked]>, "'Sanchez Net, Marc\(JPL-332H\)[JPL 
Employee]'" <[email-masked]>, "'Chaoua, 
Rachid\(GSFC-4500\)'" <[email-masked]>
Date:   31/03/2022 19:05
Subject:        RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle 
custody transfer and reliable CLs

Fine with me.  I wanted not to over-constrain the idea at this early stage 
of definition, but if that sort of mandate is okay with everybody I am for 
it.
 
Scott
 
From: Dr. Keith L Scott <[email-masked]> 
Sent: Thursday, March 31, 2022 9:25 AM
To: [email-masked]; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; 'dtn' 
<[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' 
<[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' 
<[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
I think I agree, with the possible extension that if #3 is postulating 
some sort of custody signal the ?If?does not release resources? can be 
strengthened a bit if we so choose.  That is, if we define how the extra 
signaling works, we have the opportunity to define / constrain behaviors 
of implementations that implement that signaling.  So if we define some 
sort of BP-layer signaling for ?custody? (using compressed bundle reports 
or otherwise), we might say that nodes implementing that signaling SHALL 
NOT release resources until (receipt of a positive custody signal OR 
appropriate rules to address the case that no other node in the system 
implements the signaling, etc.).
 
                                --keith
 
From: "[email-masked]" <[email-masked]>
Date: Thursday, March 31, 2022 at 11:19 AM
To: Keith Scott <[email-masked]>, Felix Flentge <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, "[email-masked]" <
[email-masked]>, 'dtn' <[email-masked]>, "'Sanchez Net, 
Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, "'Chaoua, 
Rachid(GSFC-4500)'" <[email-masked]>
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
Sorry, I just realized that the ?if not? clause in #1 below could be 
ambiguous.  What I meant there was ?if the sending node does not try 
again.?
 
Scott
 
From: [email-masked] <[email-masked]> 
Sent: Thursday, March 31, 2022 8:10 AM
To: 'Dr. Keith L Scott' <[email-masked]>; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; 'dtn' <
[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <
[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' <
[email-masked]>
Subject: RE: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
I think we are converging on a few principles:
1.      If all of the CLAs (usually only one, but there could be parallel 
transmissions) report to the BPA that forwarding failed, then the sending 
node may or may not try again, possibly on a different path and/or relying 
on different CLA support; if not, it may or may not release the 
retransmission resources occupied by the bundle.  This is a matter for 
implementation decision, which might vary depending on the size, QoS, 
time-to-live, etc. of the bundle.
2.      If successful forwarding is reported from the convergence layer, 
then at that time the sending node may or may not release retransmission 
resources occupied by the bundle.  This is again a matter for 
implementation decision, which might vary depending on the size, QoS, 
time-to-live, etc. of the bundle.
3.      If, following the reporting of successful or unsuccessful 
forwarding results from the convergence layer, the sending node does not 
release the retransmission resources occupied by the bundle, then that 
release of resources may (or may not) instead occur upon reception of a 
Compressed Bundle Report (or whatever we decide to call it; effectively, a 
custody signal) from the receiving node or from some other node further 
downstream, in the event that the bundle was flagged for this behavior. 
The absence of such an acknowledgment may or may not result in either 
automatic (timeout-driven) or managed re-forwarding of the bundle.  Again 
these are implementation matters.
4.      If all else fails, expiration of the bundle?s time-to-live will 
eventually and unconditionally result in the release of the retransmission 
resources occupied by the bundle.
 
What we?ve currently got in the BPv7 specification already supports all of 
the above except #3; I think that new flagging/signaling mechanism is the 
augmentation that we need to focus on.
 
Scott
 
From: Dr. Keith L Scott <[email-masked]> 
Sent: Thursday, March 31, 2022 5:24 AM
To: [email-masked]; [email-masked]
Cc: 'Sipos, Brian J.' <[email-masked]>; [email-masked]; dtn <
[email-masked]>; 'Sanchez Net, Marc(JPL-332H)[JPL Employee]' <
[email-masked]>; 'Chaoua, Rachid(GSFC-4500)' <
[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs

I do think the kernel of the issue under discussion is whether the rover 
is willing to release resources based on a reliable CLA confirming 
receipt, or if it (the rover) should wait for some signaling from the BP 
layer.  The former implies an assumption that successful receipt by the 
(reliable) CLA à ?custody transfer? to the receiving node and that the 
receiving node will hold onto the bundle until it can be reliably 
transferred (via a reliable CLA) to the next hop.  Without the assumption 
that every node takes custody, BP is a best-effort service and I think 
that?s too loose a service for what we want in space.
 
The way I read RFC9171, a bundle node that attempts to send a bundle that 
results in one or more of the CLAs reporting failure to forward MAY 
attempt to retransmit the bundle (or, presumably, it MAY not).  [The text 
after step 5 in section 5.4].  Depending on settings, some sort of signal 
SHOULD / could be generated noting the deletion of the bundle, but it 
would be lost regardless.
 
So, even if forwarding is attempted over a reliable CLA (e.g. TCPCLv4), if 
that fails for some reason, bundles can be lost.  Even if the space 
community modifies the processing rules to mandate that bundles that fail 
CLA forwarding ARE retransmitted, such a change can only apply to the 
portion of the path that we control.  If we end up buying commercial DTN 
routers for use in the ground, they could very well stick to the MAY 
clause and drop bundles for which the initial CLA transmission failed (I 
think).
 
To signaling (or at least being able to signal) all sorts of interesting 
events per Felix?s email: yeah, we should be able to do that.

                                --keith

 
From: Felix Flentge <[email-masked]>
Date: Thursday, March 31, 2022 at 4:37 AM
To: <[email-masked]>
Cc: "'Sipos, Brian J.'" <[email-masked]>, <[email-masked]>, dtn <
[email-masked]>, Keith Scott <[email-masked]>, "'Sanchez Net, 
Marc(JPL-332H)[JPL Employee]'" <[email-masked]>, "'Chaoua, 
Rachid(GSFC-4500)'" <[email-masked]>
Subject: Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle custody 
transfer and reliable CLs
 
Well, 

After this: 

Regards, 
Felix 

From:        <[email-masked]> 
To:        "'Chaoua, Rachid\(GSFC-4500\)'" <[email-masked]>, 
"'Sanchez Net,  Marc\(JPL-332H\)[JPL Employee]'" <
[email-masked]>, <[email-masked]>, "'Dr. Keith L 
Scott'" <[email-masked]> 
Cc:        "'Sipos, Brian J.'" <[email-masked]>, [email-masked] 
Date:        30/03/2022 19:52 
Subject:        Re: [dtn] [EXTERNAL] Re: [Sis-dtn] [EXT] Re: Bundle 
custody transfer and reliable CLs 
Sent by:        "dtn" <[email-masked]> 

 
Scott
 
To: [email-masked]; Sanchez Net, Marc (JPL-332H)[JPL Employee] <

Cheers!

 
 
Scott

 

 

 
 
Scott

Regards, 
Felix 

 

 

                                --keith

 
Best
mehmet

 

 
 
Scott

Felix,

  

 

Regards, 
Felix 

 
 
//Will

 

 
 
Scott

All,