Subject: [dtn] TCP-CL and partial transfers

From: Rick Taylor <[email-masked]>
Date: Sun, 4 Feb 2018 13:28:36 +0000

Hi All,

(Chair Hat Off)

I thought I would start a new thread to discuss partial transfers with TCP-CL.  I can't find any text in the draft that clearly tells me how I should implement the receiver-side behaviour for partial transfers.  To use my (corrected) previous example, how do I complete the following pseudo-code?

onUnexpectedHangup()
{
    // 'transfer' is the current in-progress transfer
   // 'bytes_received' is the sum of the sizes of all successfully received segments

    if (bytes_received == transfer.total_length)
    {
        // The TCP connection closed before a clean SHUTDOWN occurred, but we did get all the data
        // Pass opaque data to BPA
        notifyBPA("Reception Success",transfer);
    }
    else
    {
        // What happens here?
        handlePartialTransfer(transfer);
    }
}

As far as I can work out, there are 3 ways of handling partial transfers:

handlePartialTransfer(transfer)
{
    switch (transfer.mode)
    {
    case 'Atomic':
         // Throw away the data, it will be sent again later from the start
        discard(transfer);

    case 'Reactive Fragment':
        // Tell the BPA to create a bundle fragment from 'transfer' (if it contains the primary block), and forward it.
        notifyBPA("Reactive Fragment",transfer);

    case 'Cache':
         // Add to the CL cache, ready for the rest of the data to be resumed later
        addToCache(transfer.id, now() + transfer.validity_period, transfer.data);
    }
}

There are some problems associated with this suggestion:

1) As Scott pointed out, not every implementation wants to support all modes.  So there must be some way (contact header?) to negotiate support, and politely refuse transfers with unsupported modes. 

2) The transfer mode must be set by the sender for each transfer, and this must be driven by the sending BPA.  If a bundle prohibits fragmentation (bundle control flags), then it must not be sent with a transfer mode of 'Reactive Fragment', and the receiver must somehow politely refuse the transfer if the rules are broken.  This will need a change to XFER_REFUSE.

3) If the BPA fails to create a bundle fragment from a partial transfer when requested by 'Reactive Fragment' transfer mode, there is no way for the receiver to notify the sender that it has failed as the TCP connection has already died (ignoring half-close).  Can BP administrative reports cover this use-case?

4) As Brian pointed out, with reactive fragmentation, the remaining half of the bundle may be forwarded using an entirely different CL, but I don't see a problem with this.

5) 'Cache' transfer mode should be parameterised by the length of time a partial transfer should be retained by the receiver, ready to be resumed. This will need changes to XFER_INIT.

6) When a 'Cache' transfer is resumed by the sender, there must be some mechanism for the sender to say "I am resuming a transfer I asked you to cache previously".

7) When a 'Cache' transfer is resumed by the sender, there must be some mechanism for the receiver to say "I threw it away because of local constraints/policy, start again from 0". 

8) When using 'Cache' transfer, the transaction ID becomes persistent between TCP-CL connections, and the draft specifies IDs as only being unique per session.  Or there has to be a cache/entity ID separate from the transaction ID, can the Bundle ID be used?

This leads to the following broader questions:

A) If the WG decide that 'Atomic' is the only sensible transfer mode, why do we bother with transfer segments? Sure, showing progress in a UI is nice, but is it worth the overhead?

B) 'Cache' transfer mode is another store-and-forward mechanism running beneath BP, and requires quite a few additions to TCP-CL.  Is this even desired? I can imagine it being unpopular with resource-constrained devices, but it might have value for 'fat' hub/gateway nodes.

C) The WG hasn't really discussed reactive fragmentation, although some people think it is a good idea.  Are we unnecessarily opening a can of worms here, when we are trying to get TCP-CL out of last call?

D) Have I missed other possible modes, or ways of handling partial transfers?

Sorry for the long post, but I think this is a valid discussion.

Cheers,

Rick

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Sun, 4 Feb 2018 21:01:05 +0000

Rick, just a couple of thoughts:

One could signal a different transfer mode on each transfer, but the original TCPCL didn't do that and I don't see a lot of advantage.  I'd say just agree on a transfer mode immediately upon contact header exchange and stick with it until the end of the contact.

Your item (3) is an excellent point and, in my mind, an argument against reactive fragmentation.

As I remarked earlier (I think), the "cache" transfer mode seems like a bad idea to me: a lot of additional complexity for meager benefit.

And, again, I am dubious of the utility of reactive fragmentation.  If it were up to me I'd say let's support only the "atomic" mode.

Scott

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Sunday, February 4, 2018 5:29 AM
To: [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>; Burleigh, Scott C (312B) <[email-masked]>
Subject: TCP-CL and partial transfers

Hi All,

(Chair Hat Off)

onUnexpectedHangup()
{

    case 'Cache':

Cheers,

Rick

---

From: Carlo Caini <[email-masked]>
Date: Mon, 5 Feb 2018 11:21:35 +0100

Dear all,
     reactive fragmentation is in my opinion (I usually agree with 
Scott on everything, but this is an excpetion) one of the nicest 
features of DTN. It has been introduced in RFC4838 and is implemented 
I think in all BP implementations but ION. There is a reason for 
that, a part from Scott's personal dislike for it. It is basically 
useless in deterministic environments, and interplanetary 
communications are mostly deterministic. In these environments 
proactive fragmentation is all what you need.
By contrast, in opportunistic environments, such as basically all 
other environments, and in particular in mobile environments and in 
IoT, disruption is possible at any time in an unpredictable way, due 
to time variation of the propagation channel (tunnels, obstacles, 
radio coverage, etc.). This random disruptions are nicely tackled by 
reactive fragmentation, which fragment bundles only when and if 
necessary. In most cases it just does not fragment anything, in other 
it just splits one bundle into two fragments. The use of proactive 
fragmentation to cope with random disruptions, as previously 
suggested, has a lot of contraindications, which derive from the 
simple fact that it was not conceived and designed for this purpose.

A simple example can show this. Let me assume the transfer of an 
image from a smartphone to another (I did this experiment on my Nokia 
N900 with DTN2 on board in 2009) by means of a WiFi. The image is big 
(few MB) and takes a while to be transferred, however the actual 
transfer time cannot be predicted because the Tx speed in WiFi 
changes dynamically with the channel (2 orders of magnitude!; by the 
way, the same is common in satellite communications, which are a 
space environment as well, because of ACM, Adaptive Coding and 
Modulation, which largely changes the available bit rate 
dynamically). We can have two cases: 99% of time no disruption, thus 
any fragments, or 1% of time one disruption due to the motion of the 
smartphone outside of the AP range, i.e. 2 fragments. If you wanted 
to use proactive, first, you could not select the right dimension 
because you cannot predict the Tx rate (no contact plans here!), thus 
you would be obliged to use small fragments to cope with the worst 
case. Second, you would have had in both the presence (1%) or the 
absence (99%) of disruption, to generate and handle at any node in 
the path hundred of (99% useless) fragments, which is computationally 
much more complex and demanding than doing 99% of time absolutely 
nothing, simple because not necessary. By the way, this fllooding of 
fragments would kill OCGR, if present.
Let me add this analogy, to conclude this example. We do not walk 
with an umbrella open if it does not rain. If we are prudent we open 
the umbrella in advance, if it rains (proactive), and we take the 
umbrella close, but with us if it might rain, ready to open it if and 
only if it rains (proactive).

Last, I have already remarked that I have been an happy user of 
proactive by twelve years with DTN2. This gives us an important 
information: it is perfectly possible to implement it. We know this 
with certainty because this has been done! If this is possible, I 
suppose that it should be possible also to write the standard in such 
a way to preserve this nice feature. For people, as me, that like it, 
leaving free others not to implement it, or not to use it. I fagree 
with Scott on this freedom of choice.

Yours,
     Carlo

At 22:01 04/02/2018, Burleigh, Scott C (312B) wrote:
>One could signal a different transfer mode on each transfer, but the 
>original TCPCL didn't do that and I don't see a lot of 
>advantage.  I'd say just agree on a transfer mode immediately upon 
>contact header exchange and stick with it until the end of the contact.
>
>As I remarked earlier (I think), the "cache" transfer mode seems 
>like a bad idea to me: a lot of additional complexity for meager benefit.
>
>And, again, I am dubious of the utility of reactive 
>fragmentation.  If it were up to me I'd say let's support only the 
>"atomic" mode.
>
>Scott
>
>Cc: Brian Sipos ([email-masked]) <[email-masked]>; Burleigh, 
>Scott C (312B) <[email-masked]>
>I thought I would start a new thread to discuss partial transfers 
>with TCP-CL.  I can't find any text in the draft that clearly tells 
>me how I should implement the receiver-side behaviour for partial 
>transfers.  To use my (corrected) previous example, how do I 
>complete the following pseudo-code?
>
>onUnexpectedHangup()
>{
>         // Tell the BPA to create a bundle fragment from 'transfer' 
> (if it contains the primary block), and forward it.
>1) As Scott pointed out, not every implementation wants to support 
>all modes.  So there must be some way (contact header?) to negotiate 
>support, and politely refuse transfers with unsupported modes.
>
>2) The transfer mode must be set by the sender for each transfer, 
>and this must be driven by the sending BPA.  If a bundle prohibits 
>fragmentation (bundle control flags), then it must not be sent with 
>a transfer mode of 'Reactive Fragment', and the receiver must 
>somehow politely refuse the transfer if the rules are broken.  This 
>will need a change to XFER_REFUSE.
>
>3) If the BPA fails to create a bundle fragment from a partial 
>transfer when requested by 'Reactive Fragment' transfer mode, there 
>is no way for the receiver to notify the sender that it has failed 
>as the TCP connection has already died (ignoring half-close).  Can 
>BP administrative reports cover this use-case?
>
>4) As Brian pointed out, with reactive fragmentation, the remaining 
>half of the bundle may be forwarded using an entirely different CL, 
>but I don't see a problem with this.
>
>5) 'Cache' transfer mode should be parameterised by the length of 
>time a partial transfer should be retained by the receiver, ready to 
>be resumed. This will need changes to XFER_INIT.
>
>6) When a 'Cache' transfer is resumed by the sender, there must be 
>some mechanism for the sender to say "I am resuming a transfer I 
>asked you to cache previously".
>
>7) When a 'Cache' transfer is resumed by the sender, there must be 
>some mechanism for the receiver to say "I threw it away because of 
>local constraints/policy, start again from 0".
>
>8) When using 'Cache' transfer, the transaction ID becomes 
>persistent between TCP-CL connections, and the draft specifies IDs 
>as only being unique per session.  Or there has to be a cache/entity 
>ID separate from the transaction ID, can the Bundle ID be used?
>
>A) If the WG decide that 'Atomic' is the only sensible transfer 
>mode, why do we bother with transfer segments? Sure, showing 
>progress in a UI is nice, but is it worth the overhead?
>
>B) 'Cache' transfer mode is another store-and-forward mechanism 
>running beneath BP, and requires quite a few additions to 
>TCP-CL.  Is this even desired? I can imagine it being unpopular with 
>resource-constrained devices, but it might have value for 'fat' 
>hub/gateway nodes.
>
>C) The WG hasn't really discussed reactive fragmentation, although 
>some people think it is a good idea.  Are we unnecessarily opening a 
>can of worms here, when we are trying to get TCP-CL out of last call?
>
>_______________________________________________
>dtn mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/dtn

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Mon, 5 Feb 2018 17:13:09 +0000

Well, a fair point, and it is true that my attention mostly goes to deterministic forwarding scenarios.  But it is our intent to enable ION to support opportunistic forwarding well later this year (just as it was last year), and if reactive fragmentation is a key part of that model then ION will have to be modified to implement it.

I am still not absolutely convinced that the numbers work out.  In order to do reactive fragmentation you have to impose additional overhead in both directions -- segmentation overhead and acknowledgment overhead -- at all times, whether links are interrupted or not.  If opportunistic links (discovered contacts) are formed by WiFi Direct, for example, then we might expect data rates in the range 11 Mbps to 250 Mbps.  If we are sending an image of, say, 4MB (32 megabits), then we are looking at a maximum of 3 seconds of transmission at 11 Mbps.  If on this link we proactively fragment all bundles to a maximum of 1.375 MB (1 second of transmission), then in the worst case we have wasted 1 second of transmission at the moment when the link is lost, and it has only cost us two additional bundle headers -- and the proactive fragmentation may have been a good idea anyway, as it mitigates head-of-line-blocking issues.

For extremely brief contacts and/or very low data rates, in cases where bundle headers are extremely lengthy, and assuming that the segmentation overhead and acknowledgment is very modest, I can see a strong argument for reactive fragmentation.  Maybe somebody with some spare cycles can work out a model that would provide some quantitative perspective on this?

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]] 
Sent: Monday, February 5, 2018 2:22 AM
To: Burleigh, Scott C (312B) <[email-masked]>; Rick Taylor <[email-masked]>; [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>
Subject: Re: [dtn] TCP-CL and partial transfers

Dear all,
     reactive fragmentation is in my opinion (I usually agree with Scott on everything, but this is an excpetion) one of the nicest features of DTN. It has been introduced in RFC4838 and is implemented I think in all BP implementations but ION. There is a reason for that, a part from Scott's personal dislike for it. It is basically useless in deterministic environments, and interplanetary communications are mostly deterministic. In these environments proactive fragmentation is all what you need.
By contrast, in opportunistic environments, such as basically all other environments, and in particular in mobile environments and in IoT, disruption is possible at any time in an unpredictable way, due to time variation of the propagation channel (tunnels, obstacles, radio coverage, etc.). This random disruptions are nicely tackled by reactive fragmentation, which fragment bundles only when and if necessary. In most cases it just does not fragment anything, in other it just splits one bundle into two fragments. The use of proactive fragmentation to cope with random disruptions, as previously suggested, has a lot of contraindications, which derive from the simple fact that it was not conceived and designed for this purpose.

A simple example can show this. Let me assume the transfer of an image from a smartphone to another (I did this experiment on my Nokia
N900 with DTN2 on board in 2009) by means of a WiFi. The image is big (few MB) and takes a while to be transferred, however the actual transfer time cannot be predicted because the Tx speed in WiFi changes dynamically with the channel (2 orders of magnitude!; by the way, the same is common in satellite communications, which are a space environment as well, because of ACM, Adaptive Coding and Modulation, which largely changes the available bit rate dynamically). We can have two cases: 99% of time no disruption, thus any fragments, or 1% of time one disruption due to the motion of the smartphone outside of the AP range, i.e. 2 fragments. If you wanted to use proactive, first, you could not select the right dimension because you cannot predict the Tx rate (no contact plans here!), thus you would be obliged to use small fragments to cope with the worst case. Second, you would have had in both the presence (1%) or the absence (99%) of disruption, to generate and handle at any node in the path hundred of (99% useless) fragments, which is computationally much more complex and demanding than doing 99% of time absolutely nothing, simple because not necessary. By the way, this fllooding of fragments would kill OCGR, if present.
Let me add this analogy, to conclude this example. We do not walk with an umbrella open if it does not rain. If we are prudent we open the umbrella in advance, if it rains (proactive), and we take the umbrella close, but with us if it might rain, ready to open it if and only if it rains (proactive).

Last, I have already remarked that I have been an happy user of proactive by twelve years with DTN2. This gives us an important
information: it is perfectly possible to implement it. We know this with certainty because this has been done! If this is possible, I suppose that it should be possible also to write the standard in such a way to preserve this nice feature. For people, as me, that like it, leaving free others not to implement it, or not to use it. I fagree with Scott on this freedom of choice.

Yours,
     Carlo

---

From: Lloyd Wood <[email-masked]>
Date: Mon, 5 Feb 2018 22:57:33 +0000 (UTC)

proactive vs reactive fragmentation is a good example
of how delay-tolerant networking is not disruption-tolerant

networking.

Scott doesn't care about reactive fragmentation, because his
use case of scheduled deep space links benefits far less from it.
Disruption tolerant networking is not Scott's problem.

(We described these differences in
Lloyd Wood, Peter Holliday, Daniel Floreani and Wesley M. Eddy, 
'Sharing the dream: The consensual networking hallucination offered 
by the Bundle Protocol', peer-reviewed conference paper, Workshop on 
the Emergence of Delay-/Disruption-Tolerant Networks (E-DTN), 
part of the International Conference on Ultra Modern Telecommunication 
(ICUMT), St. Petersburg, Russia, 14 October 2009.
http://dx.doi.org/10.1109/ICUMT.2009.5345655  
http://personal.ee.surrey.ac.uk/Personal/L.Wood/publications/index.html#e-dtn-position
no room for discussion of fragmentation, alas.)

The bundle protocol isn't for everybody, is it? 

Lloyd Wood
http://sat-net.com/L.Wood/dtn

________________________________
From: Carlo Caini <[email-masked]>
To: "Burleigh, Scott C (312B)" <[email-masked]>; Rick Taylor <[email-masked]>; "[email-masked]" <[email-masked]> 
Cc: "Brian Sipos ([email-masked])" <[email-masked]>
Sent: Monday, 5 February 2018, 21:21
Subject: Re: [dtn] TCP-CL and partial transfers

Dear all,

Yours,
     Carlo

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 6 Feb 2018 00:01:06 +0000

Hi Scott,

On 05/02/18 17:13, Burleigh, Scott C (312B) wrote:
> I am still not absolutely convinced that the numbers work out.  In
> order to do reactive fragmentation you have to impose additional
> overhead in both directions -- segmentation overhead and
> acknowledgment overhead -- at all times, whether links are
> interrupted or not.  If opportunistic links (discovered contacts) are
> formed by WiFi Direct, for example, then we might expect data rates
> in the range 11 Mbps to 250 Mbps.  If we are sending an image of,
> say, 4MB (32 megabits), then we are looking at a maximum of 3 seconds
> of transmission at 11 Mbps.  If on this link we proactively fragment
> all bundles to a maximum of 1.375 MB (1 second of transmission), then
> in the worst case we have wasted 1 second of transmission at the
> moment when the link is lost, and it has only cost us two additional
> bundle headers -- and the proactive fragmentation may have been a
> good idea anyway, as it mitigates head-of-line-blocking issues.

I'm not sure I buy that argument tbh. Last I tried to look (which
is a while back), it's not easy to find an API that's usable for a
userland implementer that says what kind of bandwidth is likely
available for the next 100ms on a specific wireless link that's
about to be used to Tx a kernel buffer. (If I'm wrong there, I'd be
glad to be better educated on that.) And even if you did have such
APIs on many platforms, the situation can change (truck drives past
antenna, train goes into tunnel) so one still has to show that
proactive is better for some model(s) of disruption, and I don't
recall reading such studies.

But in any case, given that such APIs would need to be easy to use
and near ubiquitous for mid-path proactive fragmentation to be at
all feasible, if that's not the case (as I believe), then I think
the conclusion is clear that mid-path proactive fragmentation is
a non-starter.

Cheers,
S.

PS: I didn't consider proactive fragmentation at the origin in the
above as the origin, in general, cannot see the link characteristics
at the point where any form of fragmentation might be useful.

PPS: At the top there, I said I wasn't sure I bought your argument.
To be more honest, I don't buy your argument here at all:-)

-- 
PGP key change time for me.
New-ID 7B172BEA; old-ID 805F8DA2 expires Jan 24 2018.
NewWithOld sigs in keyservers.
Sorry if that mucks something up;-)

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 6 Feb 2018 00:18:29 +0000

Hi, Stephen, great to hear from you.  My argument doesn't rest in the slightest on being able to know what kind of bandwidth is likely available for the next 100ms on a specific wireless link that's about to be used to Tx a kernel buffer.  Precise knowledge in this regard is of no importance.  Pick a data rate that isn't ridiculous for the link you're using and, if the bundle is larger than 1 second of transmission at that data rate then fragment it to that size.  Now the only issue is how much overhead you have added (additional bundle headers and occasional retransmission) versus how much overhead you have saved (segmentation and acknowledgment traffic at the convergence layer for every bundle).  Plug in the numbers and calculate whether you have increased or reduced bandwidth consumption.  That's the only question.  You get the benefit of head-of-line blocking mitigation anyway.

The ION API for initiating this sort of mid-path proactive fragmentation is trivially easy.  While I admit it is not ubiquitous, I don't think that's a deficiency in the concept, which is very far from being a non-starter.

Scott

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]] 
Sent: Monday, February 5, 2018 4:01 PM
To: Burleigh, Scott C (312B) <[email-masked]>; Carlo Caini <[email-masked]>; Rick Taylor <[email-masked]>; [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>
Subject: Re: [dtn] TCP-CL and partial transfers

Hi Scott,

On 05/02/18 17:13, Burleigh, Scott C (312B) wrote:
> acknowledgment overhead -- at all times, whether links are interrupted 
> or not.  If opportunistic links (discovered contacts) are formed by 
> WiFi Direct, for example, then we might expect data rates in the range 
> 11 Mbps to 250 Mbps.  If we are sending an image of, say, 4MB (32 
> megabits), then we are looking at a maximum of 3 seconds of 
> transmission at 11 Mbps.  If on this link we proactively fragment all 
> bundles to a maximum of 1.375 MB (1 second of transmission), then in 
> the worst case we have wasted 1 second of transmission at the moment 
> when the link is lost, and it has only cost us two additional bundle 
> headers -- and the proactive fragmentation may have been a good idea 
> anyway, as it mitigates head-of-line-blocking issues.

I'm not sure I buy that argument tbh. Last I tried to look (which is a while back), it's not easy to find an API that's usable for a userland implementer that says what kind of bandwidth is likely available for the next 100ms on a specific wireless link that's about to be used to Tx a kernel buffer. (If I'm wrong there, I'd be glad to be better educated on that.) And even if you did have such APIs on many platforms, the situation can change (truck drives past antenna, train goes into tunnel) so one still has to show that proactive is better for some model(s) of disruption, and I don't recall reading such studies.

But in any case, given that such APIs would need to be easy to use and near ubiquitous for mid-path proactive fragmentation to be at all feasible, if that's not the case (as I believe), then I think the conclusion is clear that mid-path proactive fragmentation is a non-starter.

Cheers,
S.

PS: I didn't consider proactive fragmentation at the origin in the above as the origin, in general, cannot see the link characteristics at the point where any form of fragmentation might be useful.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 6 Feb 2018 00:32:41 +0000

Hiya,

On 06/02/18 00:18, Burleigh, Scott C (312B) wrote:
> Hi, Stephen, great to hear from you.  My argument doesn't rest in the
> slightest on being able to know what kind of bandwidth is likely
> available for the next 100ms on a specific wireless link that's about
> to be used to Tx a kernel buffer.  Precise knowledge in this regard
> is of no importance.  Pick a data rate that isn't ridiculous for the
> link you're using and, if the bundle is larger than 1 second of
> transmission at that data rate then fragment it to that size.  Now
> the only issue is how much overhead you have added (additional bundle
> headers and occasional retransmission) versus how much overhead you
> have saved (segmentation and acknowledgment traffic at the
> convergence layer for every bundle).  Plug in the numbers and
> calculate whether you have increased or reduced bandwidth
> consumption.  That's the only question.  You get the benefit of
> head-of-line blocking mitigation anyway.

Well, if one considers that's a question, then I guess evidence
is the next step. I don't recall publications about this, but if
there are, pointers to those would help. If there aren't, I guess
we can have our opinions arm-wrestle:-)

S.

> 
> The ION API for initiating this sort of mid-path proactive
> fragmentation is trivially easy.  While I admit it is not ubiquitous,
> I don't think that's a deficiency in the concept, which is very far
> from being a non-starter.
> 
> Scott
> 
> -----Original Message----- From: Stephen Farrell
> [mailto:[email-masked]] Sent: Monday, February 5, 2018
> 4:01 PM To: Burleigh, Scott C (312B) <[email-masked]>;
> Carlo Caini <[email-masked]>; Rick Taylor
> <[email-masked]>; [email-masked] Cc: Brian Sipos
> ([email-masked]) <[email-masked]> Subject: Re: [dtn] TCP-CL
> and partial transfers
> 
> 
> Hi Scott,
> 
>> interrupted or not.  If opportunistic links (discovered contacts)
>> are formed by WiFi Direct, for example, then we might expect data
>> rates in the range 11 Mbps to 250 Mbps.  If we are sending an image
>> of, say, 4MB (32 megabits), then we are looking at a maximum of 3
>> seconds of transmission at 11 Mbps.  If on this link we proactively
>> fragment all bundles to a maximum of 1.375 MB (1 second of
>> transmission), then in the worst case we have wasted 1 second of
>> transmission at the moment when the link is lost, and it has only
>> cost us two additional bundle headers -- and the proactive
>> fragmentation may have been a good idea anyway, as it mitigates
>> head-of-line-blocking issues.
> 
> I'm not sure I buy that argument tbh. Last I tried to look (which is
> a while back), it's not easy to find an API that's usable for a
> available for the next 100ms on a specific wireless link that's about
> to be used to Tx a kernel buffer. (If I'm wrong there, I'd be glad to
> be better educated on that.) And even if you did have such APIs on
> many platforms, the situation can change (truck drives past antenna,
> train goes into tunnel) so one still has to show that proactive is
> better for some model(s) of disruption, and I don't recall reading
> such studies.
> 
> and near ubiquitous for mid-path proactive fragmentation to be at all
> feasible, if that's not the case (as I believe), then I think the
> conclusion is clear that mid-path proactive fragmentation is a
> non-starter.
> 
> Cheers, S.
> 
> -- PGP key change time for me. New-ID 7B172BEA; old-ID 805F8DA2
> expires Jan 24 2018. NewWithOld sigs in keyservers. Sorry if that
> mucks something up;-) 
> _______________________________________________ dtn mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/dtn
>

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 6 Feb 2018 00:40:01 +0000

Right, that would be why I said "For extremely brief contacts and/or very low data rates, in cases where bundle headers are extremely lengthy, and assuming that the segmentation overhead and acknowledgment is very modest, I can see a strong argument for reactive fragmentation.  Maybe somebody with some spare cycles can work out a model that would provide some quantitative perspective on this?"

Scott

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]] 
Sent: Monday, February 5, 2018 4:33 PM
To: Burleigh, Scott C (312B) <[email-masked]>; Carlo Caini <[email-masked]>; Rick Taylor <[email-masked]>; [email-masked]
Cc: Brian Sipos ([email-masked]) <[email-masked]>
Subject: Re: [dtn] TCP-CL and partial transfers

Hiya,

On 06/02/18 00:18, Burleigh, Scott C (312B) wrote:
> to be used to Tx a kernel buffer.  Precise knowledge in this regard is 
> of no importance.  Pick a data rate that isn't ridiculous for the link 
> you're using and, if the bundle is larger than 1 second of 
> transmission at that data rate then fragment it to that size.  Now the 
> only issue is how much overhead you have added (additional bundle 
> have saved (segmentation and acknowledgment traffic at the convergence 
> layer for every bundle).  Plug in the numbers and calculate whether 
> you have increased or reduced bandwidth consumption.  That's the only 
> question.  You get the benefit of head-of-line blocking mitigation 
> anyway.

Well, if one considers that's a question, then I guess evidence is the next step. I don't recall publications about this, but if there are, pointers to those would help. If there aren't, I guess we can have our opinions arm-wrestle:-)

S.

> I'm not sure I buy that argument tbh. Last I tried to look (which is a 
> while back), it's not easy to find an API that's usable for a userland 
> implementer that says what kind of bandwidth is likely available for 
> the next 100ms on a specific wireless link that's about to be used to 
> Tx a kernel buffer. (If I'm wrong there, I'd be glad to be better 
> educated on that.) And even if you did have such APIs on many 
> platforms, the situation can change (truck drives past antenna, train 
> goes into tunnel) so one still has to show that proactive is better 
> for some model(s) of disruption, and I don't recall reading such 
> studies.
> 
> But in any case, given that such APIs would need to be easy to use and 
> near ubiquitous for mid-path proactive fragmentation to be at all 
> mucks something up;-) _______________________________________________ 
> dtn mailing list [email-masked]

---

From: Rick Taylor <[email-masked]>
Date: Tue, 6 Feb 2018 08:45:13 +0000

(Chair Hat back on)

Thank you all for your input,  I can see us circling around some consensus here:

1) 'Cache' mode is not desired at the CL layer; store and forward is the role of the BPA.

2) 'Atomic' mode should be the default transmission mode, as it 'just works'.

3) Although 'Reactive' mode appears to have little utility in Delay Tolerant networks, it does appeal to people working in the Disruption Tolerant field.

Have I sampled the opinions correctly?

If I have, then I consider the next steps to be:

A) Explicitly define 'atomic' behaviour in TCP-CL, especially from the receiver side. This whole thread started because I couldn't see clear text about partial transfer behaviour.

B) Define what flags/extension points need to be defined in TCP-CL to not exclude reactive fragmentation, perhaps pushing the fine details out to another draft at a later date (those interested in helping please raise a hand).

Cheers,

Rick

> From: Burleigh, Scott C (312B) [mailto:[email-masked]]
> Sent: 06 February 2018 00:40
> To: Stephen Farrell; Carlo Caini; Rick Taylor; [email-masked]
> Cc: Brian Sipos ([email-masked])
> Subject: RE: [dtn] TCP-CL and partial transfers
> 
> Right, that would be why I said "For extremely brief contacts and/or very low
> data rates, in cases where bundle headers are extremely lengthy, and
> assuming that the segmentation overhead and acknowledgment is very
> modest, I can see a strong argument for reactive fragmentation.  Maybe
> somebody with some spare cycles can work out a model that would provide
> some quantitative perspective on this?"
> 
> Scott
> 
> To: Burleigh, Scott C (312B) <[email-masked]>; Carlo Caini
> <[email-masked]>; Rick Taylor <[email-masked]>;
> [email-masked]
> Well, if one considers that's a question, then I guess evidence is the next
> step. I don't recall publications about this, but if there are, pointers to those
> would help. If there aren't, I guess we can have our opinions arm-wrestle:-)
> 
> S.
> 
> 
> >

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Tue, 6 Feb 2018 16:04:12 +0000

I personally disagree with (2).  

I’m not sure if I agree with (3) or not – doing anything reactive across a link with high delay is tough, but there’s a philosophical question with regard to the specific line between “reactive” and “proactive” when one starts tuning future fragment sizes based on the historical statistical performance / aggregate transfer volumes of a given link.  At some point, a sufficiently intelligent proactive scheme running in such an environment is probably going to start to resemble a pretty dumb reactive one …

Additionally, I am not yet convinced that specific (explicit) modes of CL operation are necessary.  CLs are pipes with implementation-specific signaling capability: as such, the mode will be implicitly determined based on A) what the agent tries to push through said pipe, and whether or not B) the CL is able to ACK pieces of a transmitted bundle.  If B is true, then reactive fragmentation can be done at the CL layer, and would then need to be supported at the agent layer.  If B is not true, then reactive fragmentation cannot happen regardless of what support might exist in the agent.

Hypothetically speaking: if the sender cares about the pieces it managed to move through the pipe but did not receive any kind of acknowledgment for, then it doesn’t remove them from its own storage.  If the sender does care about the pieces, then maybe the sender keeps ‘em until there’s an acknowledgment of some kind later – even if the next-hop confirmation never comes, there could be confirmation that the entire bundle was received.  Maybe the only action the sender takes based on successful partial transmission involve updating heuristics that inform its own transmission and storage policies.  In the transmission case, perhaps data that has not yet passed through a CL is sent first … or in the storage case, one drops the data that has been sent through a CL before the data which has not, since there is a chance that the data which has passed through a CL might still exist elsewhere.  Maybe the node does other things with this information that aren’t discussed here, etc.  

The point is that the data may be used in many ways, but that the specific utility of the indicated data (and, for that matter, the specific implementation of reactive fragmentation) will likely vary from implementation to implementation.  This does not seem like a bad thing, either.  Given that different DTN implementations will have different intended use-cases and operating environments to support, I’d hate to see a protocol specification act to constrain DTN’s practical utility without a demonstrable need to do so.

I’ll also note that, rather than the CLs, the agents are going to be responsible for signaling one way or another.  Further, there’s not *always* a need to ACK at the BP level: some scavenger-class, opportunistic applications would not need to do so, for instance.  Further, I’ll note that as the number of available links increases (each of which has a unique ACM profile and collection of historical performance datasets per-contact), it seems far easier (and efficient) from a resource perspective to keep sending data until the link collapses in on itself than it is to try to somehow intelligently partition the data before it is sent.

-Gilbert

The views expressed in this mail reflect the opinions of the author.  They are, therefore, not intended to reflect official positions of NASA or the U.S. Government.
 
On 2/6/18, 3:45 AM, "dtn on behalf of Rick Taylor" <[email-masked] on behalf of [email-masked]> wrote:

    (Chair Hat back on)

    

    

    
    Cheers,
    
    Rick

---

From: Brian Sipos <[email-masked]>
Date: Tue, 6 Feb 2018 18:50:24 +0000

All,

Speaking from the point of view of editing the draft further: because TCPCLv4 separates parameters of transfer MRU from segment MRU, and because a transmitter is free to send any segmentation which does not exceed the segment MRU, the current workflow can accommodate both modes:

 * A network operating in atomic mode (the receiver has a segment MRU equal to the transfer MRU, the transmitter sends only a single segment per transfer), where the whole bundle gets a single XFER_ACK message and no more. This is minimal overhead.

 * A network operating in reactive mode (either the receiver chooses a segment MRU to target some ACK data/time interval, or the transmitter chooses a segment size to target its interval, or a combination of both), where the minimum of receiver and transmitter segment limit is the actual XFER_ACK interval.

This behavior, on its own, requires no additional signaling on the CL itself. It's up to configuration of session parameters on each of the CL nodes to allow each mode. Now dynamic negotiation of which mode to use by a BP agent is currently not part of the CL. Since fragmentation is a function of the transmitter, the negotiation really is a matter of "can this node receive reactively fragmented bundles". If anything, I feel like this belongs as a contact header extension item because the CL itself does not have any behavior conditioned on whether or not the node (sending the extension item) can receive reactively fragmented bundles. It's simply data to be passed up to the BP agent for its purposes.

________________________________
From: Clark, Gilbert J. (GRC-LCA0) <[email-masked]>
Sent: Tuesday, February 6, 2018 11:04:12 AM
To: Rick Taylor; Burleigh, Scott C (JPL-312B)[Jet Propulsion Laboratory]; Stephen Farrell; Carlo Caini; [email-masked]
Cc: Brian Sipos
Subject: Re: [dtn] TCP-CL and partial transfers

-Gilbert

    (Chair Hat back on)

    Cheers,

    Rick

---

From: "Clark, Gilbert J. (GRC-LCA0)" <[email-masked]>
Date: Tue, 6 Feb 2018 19:00:24 +0000

+1; I do think this makes sense.

-Gilbert

From: dtn <[email-masked]> on behalf of Brian Sipos <[email-masked]>
Date: Tuesday, February 6, 2018 at 1:50 PM
To: "[email-masked]" <[email-masked]>
Subject: Re: [dtn] TCP-CL and partial transfers

All,

-Gilbert

    (Chair Hat back on)

    Cheers,

    Rick

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 6 Feb 2018 19:02:19 +0000

The sender needs to know (a) how much of the payload of the bundle currently in transit over the CL has been received and (b) whether or not the receiver has committed to forming a fragmentary bundle from that partially received payload.  Both are signals that can only come from the receiver.  The latter could certainly be a contact header extension item if we don't want to build it into the basic contact header.

From: dtn [mailto:[email-masked]] On Behalf Of Brian Sipos
Sent: Tuesday, February 6, 2018 10:50 AM
To: [email-masked]
Subject: Re: [dtn] TCP-CL and partial transfers

All,

________________________________
From: Clark, Gilbert J. (GRC-LCA0) <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, February 6, 2018 11:04:12 AM
To: Rick Taylor; Burleigh, Scott C (JPL-312B)[Jet Propulsion Laboratory]; Stephen Farrell; Carlo Caini; [email-masked]<mailto:[email-masked]>
Cc: Brian Sipos
Subject: Re: [dtn] TCP-CL and partial transfers

I'm not sure if I agree with (3) or not - doing anything reactive across a link with high delay is tough, but there's a philosophical question with regard to the specific line between "reactive" and "proactive" when one starts tuning future fragment sizes based on the historical statistical performance / aggregate transfer volumes of a given link.  At some point, a sufficiently intelligent proactive scheme running in such an environment is probably going to start to resemble a pretty dumb reactive one ...

Hypothetically speaking: if the sender cares about the pieces it managed to move through the pipe but did not receive any kind of acknowledgment for, then it doesn't remove them from its own storage.  If the sender does care about the pieces, then maybe the sender keeps 'em until there's an acknowledgment of some kind later - even if the next-hop confirmation never comes, there could be confirmation that the entire bundle was received.  Maybe the only action the sender takes based on successful partial transmission involve updating heuristics that inform its own transmission and storage policies.  In the transmission case, perhaps data that has not yet passed through a CL is sent first ... or in the storage case, one drops the data that has been sent through a CL before the data which has not, since there is a chance that the data which has passed through a CL might still exist elsewhere.  Maybe the node does other things with this information that aren't discussed here, etc.

The point is that the data may be used in many ways, but that the specific utility of the indicated data (and, for that matter, the specific implementation of reactive fragmentation) will likely vary from implementation to implementation.  This does not seem like a bad thing, either.  Given that different DTN implementations will have different intended use-cases and operating environments to support, I'd hate to see a protocol specification act to constrain DTN's practical utility without a demonstrable need to do so.

I'll also note that, rather than the CLs, the agents are going to be responsible for signaling one way or another.  Further, there's not *always* a need to ACK at the BP level: some scavenger-class, opportunistic applications would not need to do so, for instance.  Further, I'll note that as the number of available links increases (each of which has a unique ACM profile and collection of historical performance datasets per-contact), it seems far easier (and efficient) from a resource perspective to keep sending data until the link collapses in on itself than it is to try to somehow intelligently partition the data before it is sent.

-Gilbert

On 2/6/18, 3:45 AM, "dtn on behalf of Rick Taylor" <[email-masked] on behalf of [email-masked]<mailto:[email-masked][email-masked]>> wrote:

    (Chair Hat back on)

    Cheers,

    Rick

    > To: Stephen Farrell; Carlo Caini; Rick Taylor; [email-masked]<mailto:[email-masked]>
    > Cc: Brian Sipos ([email-masked]<mailto:[email-masked]>)
    > To: Burleigh, Scott C (312B) <[email-masked]<mailto:[email-masked]>>; Carlo Caini
    > <[email-masked]<mailto:[email-masked]>>; Rick Taylor <[email-masked]<mailto:[email-masked]>>;
    > [email-masked]<mailto:[email-masked]>
    > Cc: Brian Sipos ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>>
    > > 4:01 PM To: Burleigh, Scott C (312B) <[email-masked]<mailto:[email-masked]>>;
    > > Carlo Caini <[email-masked]<mailto:[email-masked]>>; Rick Taylor
    > > <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]> Cc: Brian Sipos
    > > ([email-masked]<mailto:[email-masked]>) <[email-masked]<mailto:[email-masked]>> Subject: Re: [dtn] TCP-CL
    > > and partial transfers
    > >
    > >
    > > Hi Scott,
    > >
    > > dtn mailing list [email-masked]<mailto:[email-masked]>
    _______________________________________________
    dtn mailing list
    [email-masked]<mailto:[email-masked]>
    https://www.ietf.org/mailman/listinfo/dtn

---

From: Brian Sipos <[email-masked]>
Date: Thu, 8 Feb 2018 02:28:25 +0000

Two questions to the WG about the parameterization of reactive fragmentation:

 1. Is the capability always desired to be bidirectional? Is there an occasion where a node may want to generate reactive fragments but not want to accept them? Or vice-versa?

 2. Are there any worthwile parameters beyond "can fragment"?

________________________________
From: Burleigh, Scott C (312B) <[email-masked]>
Sent: Tuesday, February 6, 2018 2:02:19 PM
To: Brian Sipos; [email-masked]
Subject: RE: [dtn] TCP-CL and partial transfers

The sender needs to know (a) how much of the payload of the bundle currently in transit over the CL has been received and (b) whether or not the receiver has committed to forming a fragmentary bundle from that partially received payload.  Both are signals that can only come from the receiver.  The latter could certainly be a contact header extension item if we don’t want to build it into the basic contact header.

All,

-Gilbert

    (Chair Hat back on)

    Cheers,

    Rick