Subject: [tsvwg] Tsvart last call review of draft-ietf-tsvwg-le-phb-08

From: Olivier Bonaventure <[email-masked]>
Date: Sat, 09 Feb 2019 03:13:10 -0800

Reviewer: Olivier Bonaventure
Review result: Ready with Nits

This document has been reviewed as part of the transport area review team's
ongoing effort to review key IETF documents. These comments were written
primarily for the transport area directors, but are copied to the document's
authors and WG to allow them to address any issues raised and also to the IETF
discussion list for information.

When done at the time of IETF Last Call, the authors should consider this
review as part of the last-call comments they receive. Please always CC
[email-masked] if you reply to or forward this review.

The document proposes lower than best effort service and discusses its
implementation using Diffserv techniques. It is generally in good shape, but
there are two parts which could be clarified from a transport viewpoint.

First, Section 3 mentions : "Since
   LE traffic could be starved completely for a longer period of time,
   transport protocols or applications (and their related congestion
   control mechanisms) SHOULD be able to detect and react to such a
   starvation situation. "

This is an important point for such a service. Applications and/or transport
protocols that are intended to be used with this service should be capable of
supporting long losses of connectivity that may cause connections to fail. The
document should strongly recommend to only use this service with
applications/protocols that are capable of resuming an aborted data transfert.
A regular TCP connection is usually not capable of doing this and thus using
the service correctly requires more than simply tagging the packets sent by a
given TCP connection with the chose DSCP.

Later, the document states "   While it is desirable to achieve a quick
resumption of the transfer
   as soon as resources become available again, it may be difficult to
   achieve this in practice. "

I'm not personally convinced that a quick resumption of the transfer is the
best approach to deal with periods where no LE packet is forwarded by the
network. If a connection using LE fails, it does not seem to be appropriate to
try to resume it immediately. It is likely that an approach like exponential
backoff could make sense to avoid trying to restart such connections too early.

Second, there is a small discussion of ECN in section 4: "   Since congestion
control is also useful within the LE traffic class,
   Explicit Congestion Notification [RFC3168] SHOULD be used for LE
   packets, too."

Does this imply that LE packets SHOULD also be ECT capable packets, i.e. when a
transport protocol is used to provide LE service, it should also support ECN or
is this requirement weaker ?

Finally, Section 9 discusses the Multicast considerations. It mentions the
utilisation of forward error correction schemes. One risk with FEC combined
with LE is that FEC increases the amount of data that needs to be transferred
and thus consumes ressources in non-congested parts of the network for packets
that will be discarded downstream during periods of congestion. If there are
simulation or measurement results that demonstrate that combining FEC and LE
provides good results, it would be interesting to cite those results.

---

From: Dave Taht <[email-masked]>
Date: Sat, 09 Feb 2019 10:42:41 -0800

I plan to add support for this to sch_cake and sqm-scripts as soon as it
is standarized, as CS1 is not good enough.

However I plan to ignore the wishful thinking that any transport
can stand long periods of total starvation and just slam it into the
existing "background" bin of those two qos systems, which is ~5%
of bandwith. I believe in nagle's dictum that "every application
has the right to one packet in the network.

I think giving LE some sort of garunteed minimum bandwidth would give it
uptake, otherwise, it will not be very successful. Hopefully a bare
minimum will end up being the case in the actual deployment.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 10 Feb 2019 09:52:23 +1300

Hi Olivier,
> applications/protocols that are capable of resuming an aborted data transfert.

Isn't that what the SHOULD already says? There's no implication that
one would use TCP. I would expect a pragmatic UDP-based mechanism. But
I don't think this draft should cover that - its role is to define the
code point and the per-hop behaviour, not the end to end protocol.

> backoff could make sense to avoid trying to restart such connections too early.

Yes and no. Suppose that the LE class is starved on a particular path except
between 2 a.m. and 4 a.m. every morning. A naive exponential backoff would
start at 4 a.m. on Monday and by 2 a.m. on Tuesday it could have reached
a timeout of more than 2 hours. To make prompt use of the bandwidth at 2 a.m.,
it would instead need to be retrying every minute or two.

Again - we shouldn't say too much about this in the PHB definition.
 
> is this requirement weaker ?

Same comment - a good question, maybe a good M.Sc. topic, but not part of
the PHB definition.
 
> provides good results, it would be interesting to cite those results.

I agree, although LE only makes an existing problem worse. There does seem
to be some literature on FEC+diffserv+multicast, e.g. https://doi.org/10.1109/ICECS.2003.1301723

Regards
     Brian

---

From: Christian Huitema <[email-masked]>
Date: Sat, 9 Feb 2019 14:33:11 -1000

> code point and the per-hop behaviour, not the end to end protocol.

There is some experience of that with other scavenger modes, e.g. using
LEDBAT. Slowing down to a trickle is fine, but slowing down to zero is
not. In practice, many applications that are willing to use a scavenger
mode will trigger an error if they see no progress for a sustained
amount of time, and then they will restart the connection and remove the
LEDBAT or LE option -- which of course defeats the purpose.

-- Christian Huitema

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 10 Feb 2019 16:29:01 +1300

On 2019-02-10 13:33, Christian Huitema wrote:
> There is some experience of that with other scavenger modes, e.g. using LEDBAT. Slowing down to a trickle is fine, but slowing down to zero is not. In practice, many applications that are willing to use a scavenger mode will trigger an error if they see no progress for a sustained amount of time, and then they will restart the connection and remove the LEDBAT or LE option -- which of course defeats the purpose.

Well, that's not a true scavenger application. Restarting the transport connection
might be a reasonable option, but giving up on LE is effectively saying "lower
effort is not really what I wanted."

Anyway, I think that reinforces my point: it's an interesting discussion, but
out of scope for this draft.

    Brian

---

From: Christian Huitema <[email-masked]>
Date: Sat, 9 Feb 2019 22:17:01 -1000

> out of scope for this draft.

I kind of made the same point at the time, but the reasoning was that
complete lack of progress is also an indication of complete failure, so
you end up triggering the management path for failure. The typical
candidate scavenger app at Microsoft would be software update download
or telemetry upload. None of them would have accepted a scavenger mode
that completely starved them; we had to program around that.

And by the way, that's generally OK because when the apps operate at the
bottom speed they don't go in the way of VOIP, video games, etc.

But I am not going to repeat myself on the thread...

-- Christian Huitema

---

From: Christian Huitema <[email-masked]>
Date: Sat, 9 Feb 2019 22:17:01 -1000

-- Christian Huitema

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Tue, 12 Feb 2019 11:20:55 +0100

Hi Olivier,

> Review result: Ready with Nits

thanks for the review. See comments inline.

> given TCP connection with the chose DSCP.

That is correct. That's why it's stated in the draft. So one
can use TCP and hope that it works well enough with the available
residual bandwidth, but the application should be prepared to
react on a connection timeout and initiate a new TCP connection
with an application level session resumption. In other cases
one can probably use a transport protocol that is specifically
designed to deal with this kind of starvation periods. However,
that is the worst case and I would expect that there is always
some residual bandwidth for LE available. I think that it is
application dependent, whether long starvation is acceptable or a
connection timeout should indicate a failure (taking Christian
Huitema's comments into account).

> backoff could make sense to avoid trying to restart such connections too early.

You are right. I had something like an exponential backoff in mind, but
probably with some kind of upper bound (retry every minute or two).
This statement came in by a comment from Spencer who referred to the
trigtran experience. I can add a bit more text on this.
But as Brian Carpenter already mentioned:
this is a PHB specification and transport protocol issues are related
but would be a different focus.

> is this requirement weaker ?

Again, since this is a PHB specification, the focus is on forwarding.
For ECN you need both: the ECN capable transport protocol and the ECN
marking in the queue. So the recommendation is to use an ECN capable
transport (that would use ECT) for LE and to have ECN capable marking
for the LE queue. The recommendation was targeted a bit at the latter:
if the node is able to set the CE bit, then it only depends on the
transport to use ECN (which is also recommended). I'm not sure that this
should be stated more explicitly.

> provides good results, it would be interesting to cite those results.

I think the idea was to use LE for the redundancy data only as the last
sentence points out:
"The previously mentioned redundancy data traffic could nicely use the
   varying available residual bandwidth being utilized the by LE PHB,
   but only if the previously specific requirements in the internal
   implementation of the network devices are considered.
So the normal data stream uses BE, while the FEC data uses LE.
The multicast considerations were suggested by Toerless Eckert. Maybe he
can provide some pointers if there are some.

best regards
 Roland