Subject: [tsvwg] L4S vs SCE

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 19 Nov 2019 12:12:48 +0000

Hi Chairs + others

I am not convinced that all these L4S vs SCE mail threads will make us any wiser.
 
The first question is around how safe L4S is to deploy on the internet and the proof points for either option differ widely. 

The other question is if we should have another ECN experiment (SCE) at this stage, also here I see that we have vastly differing opinions. 

In both discussion points I see that the discussion is going in circles.
My personal conclusion is that on a technical level it will be real hard to come to an agreement.  

On a more non-technical matter, I have argued (rightly or wrongly) about the connection to work in 3GPP, admittedly so far only proposed work. Others have pointed out work related to WiFi and DOCSIS, how much should this weight in here ?

Are the chairs any wiser ? :-)

/Ingemar
================================
Ingemar Johansson  M.Sc. 
Master Researcher

Ericsson Research
Network Protocols & E2E Performance
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
[email-masked]
www.ericsson.com

      Is there a time for different colors?
  Different names you find it hard to spell
                    U2 - Miss Sarajevo 
=================================

---

From: Wesley Eddy <[email-masked]>
Date: Tue, 19 Nov 2019 12:24:17 -0500

> Are the chairs any wiser ? :-)

Hi Ingemar, just speaking for myself and not any other co-chairs ... 
here are a couple of thoughts:

Yes, I'm getting a few positive things out of the threads.  For 
instance, there are some points it seems like everyone might agree on, 
including:

  * work on testing and publishing results has been very helpful
  * importance of RTT fairness
  * impact of the TCP Prague (or other end-host/transport) behaviors on
    the dual-queue performance
  * maybe some other things

I think where there are big issues, confusions, and disagreements 
causing back-and-forth, seems to be around:

  * entangled evaluation of L4S with TCP Prague (I see this badly in the
    threads ... From what I can tell, the public test Prague code
    doesn't yet meet all the Prague requirements in the L4S draft, and
    that's caused some explosion of emails.)
  * maybe lack of understanding about how and where people are hoping to
    use/deploy L4S?  (like as an operator, what else would be there that
    mitigates some of the concerns people have, and that supports
    gathering experimental results, etc)
  * concern that there can only be one of L4S or SCE happening
  * maybe some other things

It probably would be good if we all focus messages on making important 
new points and observations, not picking apart each sentence in replies 
or re-iterating points several times.  That would be in the spirit of 
working together towards the best outcomes, in contrast to just holding 
a public debate.  There are 700+ subscribers on TSVWG, and posters 
should continue to keep this in mind.

---

From: "Holland, Jake" <[email-masked]>
Date: Wed, 20 Nov 2019 03:46:36 +0000

From: Wesley Eddy <[email-masked]>
Date: 2019-11-20 at 01:24
> • concern that there can only be one of L4S or SCE happening 

There was a side discussion about this I think is worth mentioning:

It might not be hard to consider the consequences of mis-classification
that could occur in the "both experiments" scenario.  If the concern
about co-existence is primarily technical, at first glance it doesn't
seem too dire:

(PS: I acknowledge that there may be legitimate non-technical concerns
with running both experiments simultaneously, but have nothing else
to add to that discussion right now--if your concerns are only non-
technical feel free to skip this section.)

1. If a SCE connection runs across a chained SCE-L4S link,
then there's a risk that the SCE node would mark ECT(1), which
would then be mis-classified as an L4S packet by the L4S box
box.  This makes the marked packet likely to be re-ordered,
plus gives it a higher probability of being marked CE, which
would cause a higher backoff than necessary for the SCE sender.

This seems like not a disaster, since at least it fails safely
with a MD backoff.  The cost is some performance penalty to
SCE, but only when both aqms were lightly congested, and the
sender responds as if at least one was heavily congested.

(In the reverse direction, ECT(0) would already be classic traffic
for the L4S queue, so there's no impact.)

2. If an L4S connection runs across a chained SCE-L4S link, the
ECT(1) packets from the L4S sender would be mis-classified in the
SCE node as already-marked SCE by upstream.

If I understand the SCE versions of CNQ and CAKE correctly, this
would have no impact--the ECT(1) packets would be marked CE when
the congestion level is high enough (and won't get any SCE marks
since they already appear to be SCE-marked), but this is no
different from the behavior of any other 3168 aqm.

Are there other scenarios that need consideration?

As far as technical concerns go, these 2 situations don't seem
like a high barrier to running both experiments simultaneously, but
of course I'd welcome comments about other considerations this
brief analysis misses.

> • maybe lack of understanding about how and where people are hoping to use/deploy L4S?  (like as an operator, what else would be there that mitigates some of the concerns people have, and that supports gathering experimental results, etc)

+1, these suggestions sound very helpful to me.  Mitigation strategies,
expectations, end conditions for the experiments, monitoring and
evaluation plans, any restrictions on scope of intended deployment;
all these would be valuable additions to the docs IMO, and it would
strongly mitigate my safety concerns if there were reasonable guard
rails described.

> • entangled evaluation of L4S with TCP Prague (I see this badly in the threads ... From what I can tell, the public test Prague code doesn't yet meet all the Prague requirements in the L4S draft, and that's caused some explosion of emails.) 

Yes, sorry about my contributions to this problem.  I wasn't sure
how to make the point about the level of confidence we can have in
the safety properties of an L4S rollout (and how it relates to the
likely impact of wg adoption decisions on external SDOs) without
bringing up the examples from the TCP Prague testing.

Maybe it would have been better just to point out that we don't yet
know whether the safety requirements of L4S can be achieved, since
we haven't yet seen running code that meets them, rather than
introducing confusion by pointing to example tests of something
that doesn't yet try to implement those requirements.

The point is well-taken, both from you and from Greg's earlier
message to that effect, and I'll try to be more clear on this
distinction in the future.

Best regards,
Jake

---

From: G Fairhurst <[email-masked]>
Date: Wed, 20 Nov 2019 12:03:49 +0800

Helpful, see in-line.

On 20/11/2019, 11:46, Holland, Jake wrote:
> From: Wesley Eddy<[email-masked]>
Since the chief aim of L4S was towards low latency, I'm also interested 
in queuing properties of the traffic, some initial questions:

Would an SCE router likely preserve or change the queueing of ECT(1) L4S 
traffic? - is this behvaiour different to any existing ECN-marking router?

And would SCE traffic arriving at the L4S queue meet L4S expectations in 
terms of responsiveness?

> strongly mitigate my safety concerns if there were reasonable guard
 From what I recall when we discuused the EXP deployment of L4S: If this 
concludes at some time, I guess we can expect ECT(0) behaviour to 
continue as-is, and we can expect use of ECT(1) for L4S to terminate, or 
be filtered to avoid L4S.

Have we thoughts on what would happen with SCE?
> distinction in the future.
Thanks for these thoughts,

Gorry

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 20 Nov 2019 12:23:54 +0800

> On 20 Nov, 2019, at 12:03 pm, G Fairhurst <[email-masked]> wrote:
> 
> Would an SCE router likely preserve or change the queueing of ECT(1) L4S traffic? - is this behvaiour different to any existing ECN-marking router?

SCE middleboxes don't queue or handle ECN(1)-marked or CE-marked traffic any differently from ECT(0) traffic.  The a-priori ECT(1) marking merely removes the option of applying an SCE mark.

CE marking on L4S traffic will be as per RFC-3168; it is already incumbent upon L4S to cope with that.

> And would SCE traffic arriving at the L4S queue meet L4S expectations in terms of responsiveness?

Yes, since the MD response to CE is actually more severe/conservative than what L4S expects.

> Have we thoughts on what would happen with SCE?

The current Linux implementations all have SCE functionality gated behind a non-default setting, so users have to explicitly turn it on.  I think this should make it relatively easy to clean up an abortive deployment, should it be required.  Specifically:

- The sch_cake, sch_fq_codel, and sch_cnq_cobalt qdiscs each require a non-default tc parameter to enable SCE marking.  Without it, they purely support RFC-3168 and will not increase the incidence of ECT(1) codepoints.

- Receiver echoing of SCE marks into the ESCE flag is controlled by the net.ipv4.tcp_sce sysctl.  This behaviour differs from that of Nonce Sum receivers but involves the same codepoint and flag bit.

- Sender responses to the ESCE flag are enabled by explicitly selecting one of the modified TCP algorithms.  If these are selected but no ESCE flags appear, they behave like ordinary RFC-3168 compliant Reno or CUBIC implementations.

Is that sufficient, or does a more detailed analysis need to be organised?

 - Jonathan Morton

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 20 Nov 2019 14:49:17 +0800

Gorry, Jonathan,

> SCE middleboxes don't queue or handle ECN(1)-marked or CE-marked traffic any differently from ECT(0) traffic.  The a-priori ECT(1) marking merely removes the option of applying an SCE mark.

Will a SCE middlebox keep a rolling update of the incoming fraction of
ECT(1) vs. the fraction it would want to (re-)mark with ECT(1)? Or is
"ships-in-the-night" behavior between two fractionally congested SCE
middleboxes sufficient with an overall remarking probability of
1-((1-p1)*(1-p2))

With regular RED, ships-in-the-night was acepted, but the typical
marking ratios for 3168-TCP would be an order of magnitude lower,
limiting the deviation compared to "correct" max(p1, p2) marking.

As SCE has a target marking probability (ect0 -> ect1) from close to 50%
under steady state, such couplings between consecutive bottlenecks would
be much more prononounced.

> Yes, since the MD response to CE is actually more severe/conservative than what L4S expects.

Well, SCE would be starved out due to the high p of CE marks in a L4S
queue, even if not sharing the Q.

[...]

> Is that sufficient, or does a more detailed analysis need to be organised?

A good analytical model of the sce cc response function would be good to
have always.

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 20 Nov 2019 09:31:56 +0100

Hi Gory,

new question below.

> On Nov 20, 2019, at 05:03, G Fairhurst <[email-masked]> wrote:
> 
> Helpful, see in-line.
> 
> Since the chief aim of L4S was towards low latency, I'm also interested in queuing properties of the traffic, some initial questions:
> 
> From what I recall when we discuused the EXP deployment of L4S: If this concludes at some time, I guess we can expect ECT(0) behaviour to continue as-is, and we can expect use of ECT(1) for L4S to terminate, or be filtered to avoid L4S.

	[SM] According to Koen:
"I don't expect many large real world service/application providers to use the basic reference Prague implementation anyway. They will have their own version ready, eager to deploy low latency applications and to test and check if they are sufficiently complying to the Prague requirements."

And according to Ingemar:
"I have argued (rightly or wrongly) about the connection to work in 3GPP, admittedly so far only proposed work. Others have pointed out work related to WiFi and DOCSIS, how much should this weight in here ?"

It seems like the industry is ready to adopt L4S rather quickly. 

	How do you expect an industry/commercial roll-out of L4S technology to behave, if the L4S experiment should terminated without adoption as a standard track RFC? Are they supposed to phase-out using ECT(1) as well, or is it understood that deployed L4S instances continue using L4S methods? 

--
	Sebastian

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 20 Nov 2019 17:02:51 +0800

NB: most of the questions below don't seem to be directly relevant to the coexistence question this thread is supposed to be exploring.

> limiting the deviation compared to "correct" max(p1, p2) marking.

This question is not important.  SCE admits the possibility of a wide variety of marking strategies, some of which may choose to take this factor into account.  The reference AQMs do not.

> be much more prononounced.

You are again looking at old, long-superseded information.  The target marking probability is typically below 50% at steady state.  It would be as high as 50% in the case of a cwnd of about 4 segments.

> queue, even if not sharing the Q.

Remember that SCE-marked packets exist due to upstream congestion at an SCE-aware bottleneck.  As soon as the affected flow reacts to CE, it will back off sufficiently to ensure the upstream queue is drained, and SCE marking will generally then cease, ensuring that the flow is now entirely ECT(0) marked.  This dynamic behaviour is important to understand when considering the system as a whole.

> A good analytical model of the sce cc response function would be good to

Sure, but that's not relevant to the coexistence question.  Let's keep it to its own thread.

 - Jonathan Morton

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 20 Nov 2019 10:04:43 +0000

Hi
Please see inline , an answer to the question that is to some extent to directed me personally 

/Ingemar

> -----Original Message-----
> From: Sebastian Moeller <[email-masked]>
> Sent: den 20 november 2019 09:32
> To: G Fairhurst <[email-masked]>
> Cc: Holland, Jake <[email-masked]>; [email-masked];
> Ingemar Johansson S <[email-masked]>; tsvwg-
> [email-masked]; De Schepper, Koen (Koen)
> <[email-masked]>; [email-masked]
> Subject: Re: [tsvwg] L4S vs SCE
> 
> Hi Gory,
> 
> new question below.
> 
> >> It might not be hard to consider the consequences of
> >> mis-classification that could occur in the "both experiments"
> >> scenario.  If the concern about co-existence is primarily technical,
> >> at first glance it doesn't seem too dire:
> >>
> >> (PS: I acknowledge that there may be legitimate non-technical
> >> concerns with running both experiments simultaneously, but have
> >> nothing else to add to that discussion right now--if your concerns
> >> are only non- technical feel free to skip this section.)
> >>
> >> 1. If a SCE connection runs across a chained SCE-L4S link, then
> >> there's a risk that the SCE node would mark ECT(1), which would then
> >> be mis-classified as an L4S packet by the L4S box box.  This makes
> >> the marked packet likely to be re-ordered, plus gives it a higher
> >> probability of being marked CE, which would cause a higher backoff
> >> than necessary for the SCE sender.
> >>
> >> This seems like not a disaster, since at least it fails safely with a
> >> MD backoff.  The cost is some performance penalty to SCE, but only
> >> when both aqms were lightly congested, and the sender responds as if
> >> at least one was heavily congested.
> >>
> >> ECT(1) packets from the L4S sender would be mis-classified in the SCE
> >> node as already-marked SCE by upstream.
> >>
> >> would have no impact--the ECT(1) packets would be marked CE when the
> >> congestion level is high enough (and won't get any SCE marks since
> >> they already appear to be SCE-marked), but this is no different from
> >> the behavior of any other 3168 aqm.
> > Since the chief aim of L4S was towards low latency, I'm also interested in
> queuing properties of the traffic, some initial questions:
> >
> >> As far as technical concerns go, these 2 situations don't seem like a
> >> high barrier to running both experiments simultaneously, but of
> >> course I'd welcome comments about other considerations this brief
> >> analysis misses.
> >>
> >>
> >>> • maybe lack of understanding about how and where people are hoping
> >>> to use/deploy L4S?  (like as an operator, what else would be there
> >>> that mitigates some of the concerns people have, and that supports
> >> +1, these suggestions sound very helpful to me.  Mitigation
> >> +strategies,
> concludes at some time, I guess we can expect ECT(0) behaviour to continue
> as-is, and we can expect use of ECT(1) for L4S to terminate, or be filtered to
> avoid L4S.
> 
> 	[SM] According to Koen:
> "I don't expect many large real world service/application providers to use the
> basic reference Prague implementation anyway. They will have their own
> version ready, eager to deploy low latency applications and to test and check
> if they are sufficiently complying to the Prague requirements."
> 
> "I have argued (rightly or wrongly) about the connection to work in 3GPP,
> admittedly so far only proposed work. Others have pointed out work related
> to WiFi and DOCSIS, how much should this weight in here ?"
> 
> 	How do you expect an industry/commercial roll-out of L4S
> technology to behave, if the L4S experiment should terminated without
> adoption as a standard track RFC? Are they supposed to phase-out using
> ECT(1) as well, or is it understood that deployed L4S instances continue using
> L4S methods?

[IJ] The premise would be that L4S is declared a failure. I doubt that anybody has a good enough crystal ball to predict what happens. First it is necessary to come to the conclusion that L4S is a failure and I would argue that we are not yet there and I don't currently see that coming. Before that possible event I don't see it meaningful to speculate.
This reminds me of a question the late Ingvar Kamprad, the founder of IKEA, got when an IKEA warehouse was opened in northern Sweden, something that he was very engaged in personally : "How long are you going to support it financially?" was the question. His answer was "Well.. let us open the warehouse first..".. I can add that the parking lot outside IKEA in Haparanda, Sweden is most often jam-packed.
And.. as regards to 3GPP adoption, it must be understood that 3GPP has not yet adopted L4S, I certainly hope that it will go that way and I try to do all that I can to make that happen, but alike IETF, 3GPP is a collection of minds with differing preferences and goals.

> >>> • entangled evaluation of L4S with TCP Prague (I see this badly in
> >>> the threads ... From what I can tell, the public test Prague code
> >> Yes, sorry about my contributions to this problem.  I wasn't sure how
> >> to make the point about the level of confidence we can have in the
> >> safety properties of an L4S rollout (and how it relates to the likely
> >> impact of wg adoption decisions on external SDOs) without bringing up
> >> the examples from the TCP Prague testing.
> >>
> >> know whether the safety requirements of L4S can be achieved, since we
> >> haven't yet seen running code that meets them, rather than
> >> introducing confusion by pointing to example tests of something that
> >> doesn't yet try to implement those requirements.
> >>

---

From: Kyle Rose <[email-masked]>
Date: Wed, 20 Nov 2019 18:13:41 +0800

On Wed, Nov 20, 2019 at 6:04 PM Ingemar Johansson S <ingemar.s.johansson=
[email-masked]> wrote:

> [IJ] The premise would be that L4S is declared a failure. I doubt that
> anybody has a good enough crystal ball to predict what happens. First it is
> necessary to come to the conclusion that L4S is a failure and I would argue
> that we are not yet there and I don't currently see that coming. Before
> that possible event I don't see it meaningful to speculate.
>

I'm going to have to go ahead and disagree with you strongly here. Given
the potential consequences of cleaning up after a failed experiment without
a plan worked out beforehand, this blithe approach is simply not acceptable.

In lots of cases, experiments are easy to terminate in an obvious way: for
example, in one typical case, a code point can simply be abandoned, or
(even better) a pollutable experimental code point returned to the
available pool after some time. If that were the case here, it would not be
difficult to enumerate a sequence of steps required to do so. It doesn't
appear that's the case, however, so all the more reason to make sure we
address this as part of the experiment setup.

A launch escape system of the necessary complexity should be a requirement
of any experimental deployment. In this case, that might circumscribe the
scope of the experiment itself.

Kyle

---

From: Greg White <[email-masked]>
Date: Wed, 20 Nov 2019 10:48:14 +0000

In the case of Low Latency DOCSIS, the network operator will need to explicitly configure L4S “on” via the cable modem configuration file or the Aggregate QoS Profile table on the CMTS, and then reboot the cable modem.  If there is a need at some future point to discontinue L4S support, an operator could do so quickly by reverting the configuration to disable L4S, then rebooting the cable modem.  Additionally, CableLabs would write an Engineering Change against the DOCSIS spec to deprecate the feature and remove the ability from the CMTS to enable the feature on cable modems that continue to support it.

From: tsvwg <[email-masked]> on behalf of Kyle Rose <[email-masked]>
Date: Wednesday, November 20, 2019 at 6:14 PM

I'm going to have to go ahead and disagree with you strongly here. Given the potential consequences of cleaning up after a failed experiment without a plan worked out beforehand, this blithe approach is simply not acceptable.

In lots of cases, experiments are easy to terminate in an obvious way: for example, in one typical case, a code point can simply be abandoned, or (even better) a pollutable experimental code point returned to the available pool after some time. If that were the case here, it would not be difficult to enumerate a sequence of steps required to do so. It doesn't appear that's the case, however, so all the more reason to make sure we address this as part of the experiment setup.

A launch escape system of the necessary complexity should be a requirement of any experimental deployment. In this case, that might circumscribe the scope of the experiment itself.

Kyle

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 20 Nov 2019 10:49:30 +0000

Hi

So given the imagined outcome that L4S fails.. two scenarios

If other SDOs or developers don’t pick up L4S then things are quite simple I guess, just declare the L4S drafts as deprecated, or is there more to do ?

But, if e.g. 3GPP somehow thinks that this is a good idea and adopts it.. Will the IETF send a message (LS?) to 3GPP with the message “please stop using L4S”, is this even a reasonable scenario?. After all, the fact that it is picked up by other SDOs, speaks against a failure ?

/Ingemar

From: Kyle Rose <[email-masked]> 
Sent: den 20 november 2019 11:14
To: Ingemar Johansson S <ingemar.s.johansson=[email-masked]>
Cc: Sebastian Moeller <[email-masked]>; G Fairhurst <[email-masked]>; [email-masked]; Ingemar Johansson S <[email-masked]>; [email-masked]; De Schepper, Koen (Koen) <[email-masked]>; [email-masked]
Subject: Re: [tsvwg] L4S vs SCE

Kyle

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 20 Nov 2019 19:21:48 +0800

> On 20 Nov, 2019, at 6:49 pm, Ingemar Johansson S <ingemar.s.johansson=[email-masked]> wrote:
> 

In this context I think it's wise to be on guard against perverse incentives and externalised costs.  There may, for example, be significant financial incentives to introduce a "fast lane" for Internet subscribers which is somehow "endorsed" by a standards body.

I think the IETF is in a good position to resist such charms, but as an industry body the 3GPP might not be.  If the latter is taking their initial cues from the IETF, we need to be sure that they will also respond appropriately to an experiment termination.

 - Jonathan Morton

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 20 Nov 2019 12:18:16 +0100

Hi Ingemar,

more in-line.

> On Nov 20, 2019, at 11:04, Ingemar Johansson S <[email-masked]> wrote:
> 
> Hi

	[SM] I want to apologize to you, I did not intend to offend with this question, nor did I want to drag you and Koen into this. I merely cited you as I assume that both of you have a better feel about your respective fields/industries interest in L4S than I do.

> [IJ] The premise would be that L4S is declared a failure.

	[SM] Well, I am trying to discuss "contingency planing" here, by necessity that includes potential retraction of the experiment, no? I am also happy to talk about the opposite contingency here unconditional success; where should I send the bottle of Champagne to for the deserved celebrations? ;)

> I doubt that anybody has a good enough crystal ball to predict what happens. First it is necessary to come to the conclusion that L4S is a failure and I would argue that we are not yet there and I don't currently see that coming. Before that possible event I don't see it meaningful to speculate.

	[SM] I duly note that 3GPP is not committed yet, but is it fair to claim that inside 3GPP thee is an interest in L4S' goals and/or techniques? My question is about the feasibility to roll-back L4S (under the pessimistic assumption that this is deemed required) in the light of potential uptake into industry standards and roll-out into a potentially high number of network elements (think base stations, CMTSs, ...). 

Best Regards
	Sebastian

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 20 Nov 2019 12:40:16 +0000

I agree with Ingemar. Related to the chances for failing: as history learned us, the only real world chance for failing would be that the technology would not be taken up in deployments. This whole codepoint conflict is mainly damaging/creating this deployment 'issue'. The other so called show stoppers are less of an issue for most people that worked on CCs. Blowing 'issues' (not yet implemented/integrated/tested code and bugs) out of proportion is not the way to go forward here.

To give an example:

As mentioned in another thread already: the L4S codepoint, where there's a new queue and new CCs, creates a clean-slate opportunity. This is an opportunity to select new desired behavior.

One of these opportunities here is to make L4S_TCP less RTT dependent. There have been many working implementations for less RTT-dependent CCs in the past. One that is widely deployed is Cubic, which is doing this for getting more throughput for longer RTTs. The only reason why it didn’t fly for lower RTTs on the current Internet is that they would hurt themselves (get lower throughput, competing with Reno).

As we are able to define a new independent behavior and the RTT dependence in L4S is taken serious (some even call it a show stopper) this is even a must do opportunity for L4S. It is all a matter of perspective: show-stopper <-> opportunity.

Let it be clear, I am not against adopting new ideas for implementation of congestions controls and AQMs, as long as they are not contesting with the code point and requirements that were selected and agreed on as part of the L4S BoF. I'm also not against the recent vibrant energy, interest and engagement of people, but I think we can better use this energy in making things work. As you noticed, we can use the help to speed things up on the open source part.

Regards,
Koen.

From: Ingemar Johansson S <[email-masked]>
Sent: Wednesday, November 20, 2019 11:50 AM
To: Kyle Rose <[email-masked]>; Ingemar Johansson S <[email-masked]>
Cc: Sebastian Moeller <[email-masked]>; G Fairhurst <[email-masked]>; [email-masked]; [email-masked]; De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>; [email-masked]; Ingemar Johansson S <[email-masked]>
Subject: RE: [tsvwg] L4S vs SCE

Hi

/Ingemar

From: Kyle Rose <[email-masked]<mailto:[email-masked]>>
Sent: den 20 november 2019 11:14
To: Ingemar Johansson S <ingemar.s.johansson=[email-masked]<mailto:ingemar.s.johansson=[email-masked]>>
Cc: Sebastian Moeller <[email-masked]<mailto:[email-masked]>>; G Fairhurst <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Ingemar Johansson S <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; De Schepper, Koen (Koen) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [tsvwg] L4S vs SCE

Kyle

---

From: Roland Bless <[email-masked]>
Date: Wed, 20 Nov 2019 21:22:54 +0800

Hi,

On 20.11.19 at 13:40 De Schepper, Koen (Nokia - BE/Antwerp) wrote:
>
> I agree with Ingemar. Related to the chances for failing: as history
> learned us, the only real world chance for failing would be that the
> technology would not be taken up in deployments. This whole codepoint
> conflict is mainly damaging/creating this deployment 'issue'. The
> other so called show stoppers are less of an issue for most people
> that worked on CCs. Blowing 'issues' (not yet
> implemented/integrated/tested code and bugs) out of proportion is not
> the way to go forward here.
>
>  
>
> To give an example:
>
> As mentioned in another thread already: the L4S codepoint, where
> there's a new queue and new CCs, creates a clean-slate opportunity.
> This is an opportunity to select new desired behavior.
>
Yes, but as I also expressed my concerns w.r.t. the L4S codepoint
earlier, at the cost of binding this to a quite fixed set of L4S
behaviors and "burning" the last ECT codepoint. Personally, I like
concepts with a little bit more potential to be useful for future
development (evolvability) of congestion controls, e.g., BBRv2 and LoLa
could also benefit from an SCE-like marking...

Therefore, a compromise could be to define ECT(1) with "some congestion
experienced" semantics that could also be used for L4S.
In order to further disambiguate and use the particular L4S semantics of
Dual Queue AQM etc. one could use an additional DSCP as
L4S classifier: I currently don't see the need to follow the argument of
providing L4S orthogonal to DiffServ (and for me L4S is specifying a PHB).
In case L4S doesn't take off one can at least use the SCE semantics and
reassign the DSCP (or in case of success make it later orthogonal
to DiffServ). Due to the newly opened pool of DSCPs one may have a
chance of finding a DSCP that has a high probability of
getting through from end-to-end.

> One of these opportunities here is to make L4S_TCP less RTT dependent.
> There have been many working implementations for less RTT-dependent
> CCs in the past. One that is widely deployed is Cubic, which is doing
> this for getting more throughput for longer RTTs. The only reason why
> it didn’t fly for lower RTTs on the current Internet is that they
> would hurt themselves (get lower throughput, competing with Reno).
>
>  
>
> As we are able to define a new independent behavior and the RTT
> dependence in L4S is taken serious (some even call it a show stopper)
> this is even a must do opportunity for L4S. It is all a matter of
> perspective: show-stopper <-> opportunity.
>
Achieving RTT fairness isn't easy, but the Fair Flow Balancing mechanism
of TCP LoLa is one
example of how one can achieve this...

>
> Let it be clear, I am not against adopting new ideas for
> implementation of congestions controls and AQMs, as long as they are
> not contesting with the code point and requirements that were selected
> and agreed on as part of the L4S BoF. I'm also not against the recent
> vibrant energy, interest and engagement of people, but I think we can
> better use this energy in making things work. As you noticed, we can
> use the help to speed things up on the open source part.
>
I didn't see the adoption of L4S work at the BOF as an agreement to
accept everything as proposed rather than
an agreement to discuss and work on the issues, and to come up with a
potentially different solution as it has been
at the time of the BOF.

Regards,
 Roland

>  
>
> Regards,
>
> Koen.
>
>  
>
> *From:* Ingemar Johansson S <[email-masked]>
> *Sent:* Wednesday, November 20, 2019 11:50 AM
> *To:* Kyle Rose <[email-masked]>; Ingemar Johansson S
> <[email-masked]>
> *Cc:* Sebastian Moeller <[email-masked]>; G Fairhurst
> <[email-masked]>; [email-masked]; [email-masked]; De
> Schepper, Koen (Nokia - BE/Antwerp)
> <[email-masked]>; [email-masked];
> Ingemar Johansson S <[email-masked]>
> *Subject:* RE: [tsvwg] L4S vs SCE
>
>  
>
> Hi
>
>  
>
> If other SDOs or developers don’t pick up L4S then things are quite
> simple I guess, just declare the L4S drafts as deprecated, or is there
> more to do ?
>
>  
>
> But, if e.g. 3GPP somehow thinks that this is a good idea and adopts
> it.. Will the IETF send a message (LS?) to 3GPP with the message
> “please stop using L4S”, is this even a reasonable scenario?. After
> all, the fact that it is picked up by other SDOs, speaks against a
> failure ?
>
>  
>
> /Ingemar
>
>  
>
> *From:* Kyle Rose <[email-masked] <mailto:[email-masked]>>
> *Sent:* den 20 november 2019 11:14
> *To:* Ingemar Johansson S
> <ingemar.s.johansson=[email-masked]
> <mailto:ingemar.s.johansson=[email-masked]>>
> *Cc:* Sebastian Moeller <[email-masked] <mailto:[email-masked]>>; G
> Fairhurst <[email-masked] <mailto:[email-masked]>>;
> [email-masked] <mailto:[email-masked]>; Ingemar Johansson S
> <[email-masked]
> <mailto:[email-masked]>>; [email-masked]
> <mailto:[email-masked]>; De Schepper, Koen (Koen)
> <[email-masked] <mailto:[email-masked]>>;
> [email-masked] <mailto:[email-masked]>
> *Subject:* Re: [tsvwg] L4S vs SCE
>
>  
>
> On Wed, Nov 20, 2019 at 6:04 PM Ingemar Johansson S
> <ingemar.s.johansson=[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     [IJ] The premise would be that L4S is declared a failure. I doubt
>     that anybody has a good enough crystal ball to predict what
>     happens. First it is necessary to come to the conclusion that L4S
>     is a failure and I would argue that we are not yet there and I
>     don't currently see that coming. Before that possible event I
>     don't see it meaningful to speculate.
>
>  
>
> I'm going to have to go ahead and disagree with you strongly here.
> Given the potential consequences of cleaning up after a failed
> experiment without a plan worked out beforehand, this blithe approach
> is simply not acceptable.
>
>  
>
> In lots of cases, experiments are easy to terminate in an obvious way:
> for example, in one typical case, a code point can simply be
> abandoned, or (even better) a pollutable experimental code point
> returned to the available pool after some time. If that were the case
> here, it would not be difficult to enumerate a sequence of steps
> required to do so. It doesn't appear that's the case, however, so all
> the more reason to make sure we address this as part of the experiment
> setup.
>
>  
>
> A launch escape system of the necessary complexity should be a
> requirement of any experimental deployment. In this case, that might
> circumscribe the scope of the experiment itself.
>
>  
>
> Kyle
>
>  
>

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 20 Nov 2019 15:18:47 +0100

Dear Koen, dear all,

> On Nov 20, 2019, at 13:40, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 

	[SM] I take offense in that characterization. It is by now an accepted fact that the dual queue AQM/scheduler does not work as intended or expected from reading either the dualq or the L4S-arch draft, That is not an issue if CC algorithm or RTT dependence, if the isolation between the two queues would be as robust and reliable as one assumes from reading the two internet drafts, all RTT unfairness would do is distribute bandwidth unequally between different RTT flows in each of the two queues. That is very much not the reason for me to observe dual queue failing miserably, the failure mode is the leakage between the two queues. I just think that undesired behavior in one component is best fixed in that specific component instead of basically declaring the issue goes away once all CC are made more RTT-invariant. If that would be easy, I assume it would not be an issue any more anyways, but I predict it will not be completely fixable (short RTTs generate tighter control loops).

> As mentioned in another thread already: the L4S codepoint, where there's a new queue and new CCs, creates a clean-slate opportunity. This is an opportunity to select new desired behavior.

	[SM] The precondition for that clean-slate is however that L4S and existing traffic are sufficiently isolated from another, and that isolation as currently implemented IMHO is not sufficient.

> One of these opportunities here is to make L4S_TCP less RTT dependent. There have been many working implementations for less RTT-dependent CCs in the past. One that is widely deployed is Cubic, whi…

	[SM] Looking at the pfifo_fast results in Høiland-Jørgensen T, Hurtig P, Brunstrom A (2015) The Good, the Bad and the WiFi: Modern AQMs in a residential setting. Computer Networks 89:90–106. For Cubic/pfifo_fast (linux former default combination), I fail to see a strong indicator that cubic is RTT invariant or getting more thoughput at longer RTTs (except for the 10ms versus 50ms "hump"). What paper/data should I be looking at instead?

> As we are able to define a new independent behavior and the RTT dependence in L4S is taken serious (some even call it a show stopper) this is even a must do opportunity for L4S. It is all a matter…

	[SM] I believe that working on more RTT independence is worth-while but no replacement for fixing the isolation system as it it is that equitable isolation from existing traffic that basically gives you the slate clean-"green field" development opportunity, no? 

>  
> Let it be clear, I am not against adopting new ideas for implementation of congestions controls and AQMs, as long as they are not contesting with the code point and requirements that were selected and agreed on as part of the L4S BoF.

	[SM] Not claiming that it applies here, but "not contesting with the code point and requirements" basically rules out all substantial changed to L4S...

> I'm also not against the recent vibrant energy, interest and engagement of people, but I think we can better use this energy in making things work. As you noticed, we can use the help to speed things up on the open source part.

	[SM]... while at the same time requesting help for implementation and dealing with the consequences of said "code point and requirements".

Best Regards
	Sebastian

> On Wed, Nov 20, 2019 at 6:04 PM Ingemar Johansson S <ingemar.s.johansson=[email-masked]> wrote:

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 20 Nov 2019 23:01:23 +0800

> On 20 Nov, 2019, at 10:18 pm, Sebastian Moeller <[email-masked]> wrote:
> 

CUBIC is more RTT-invariant than Reno is, and achieves more throughput on high-BDP paths than Reno does, which I think was the meaning here.

Examining the formulae for calculating steady-state average cwnd from marking probability, and converting to throughput by dividing cwnd by RTT, it's clear that any CC algo that depends only on marking probability for cwnd evolution will converge to an RTT-invariant cwnd and will thus have the same RTT-fairness of throughput with itself as Reno does.  This is in fact true for TCP Prague and all of SCE's response functions thus far.

But CUBIC includes an RTT term in its steady-state formula which partially compensates for the conversion to throughput, such that CUBIC's self-competitive throughput in its high-BDP regime is proportional to the reciprocal quartic root of RTT, rather than to the plain reciprocal.

So I think that to meet Prague Requirement #5 - reducing RTT dependence - one must first achieve at least as good performance as plain old CUBIC in this respect.  Is there work towards this in progress within L4S?

 - Jonathan Morton

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Wed, 20 Nov 2019 16:16:59 +0000

It should be fairly simple for an explicit and very frequent congestion signal, isn't it? This is basic stuff, where you can find a lot of information and papers on already...

Agreed that for delay based CCs this is another issue... but this is not the case here at all.

Koen.

-----Original Message-----
From: Jonathan Morton <[email-masked]> 
Sent: Wednesday, November 20, 2019 4:01 PM
To: Sebastian Moeller <[email-masked]>
Cc: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>; G Fairhurst <[email-masked]>; [email-masked]; Ingemar Johansson S <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [tsvwg] L4S vs SCE

 - Jonathan Morton

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 21 Nov 2019 02:34:44 +0800

Roland,

On 20/11/2019 21:22, Roland Bless wrote:
> development (evolvability) of congestion controls, e.g., BBRv2 and 
> LoLa could also benefit from an SCE-like marking...
>

My whole purpose in solving the problem of deploying scalable CCs over 
the Internet was to re-juvenate evolution (to widen the range of 
applications that could be supported by different transport behaviours, 
particularly for real-time with low latency and high throughput at the 
same time). One of the main things that has stopped CCs evolving so far 
is the need for friendliness with the Reno behaviour that was not 
scaling over the years.

If SCE is primarily supported in FQ AQMs, that will constrain flows to 
be capped at the rate that FQ gives them. How is that doing anything 
other than massively constraining future evolution of CCs, especially 
real-time ones? See Per-Flow Scheduling and the End-to-End Argument 
<http://bobbriscoe.net/projects/latency/per-flow_tr.pdf>. I don't need 
to tell you that the e2e argument is all about giving end systems the 
power to innovate without permission.

Anyway, what are you imagining would stop CCs evolving alongside other 
scalable CCs? In much the same way CCs have always evolved. With L4S you 
have a clean slate that seems just like a FIFO with a shallow ECN-only 
immediate AQM. And other flows are causing you hardly any delay and very 
rarely any loss. Think of all the things you can do with that. Go 
evolve, Roland!

The key thing here is the wording of the Prague requirements 
<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-08#section-4>. 
We have a session in the 'Prague' side-meeting tomorrow to review them 
(and I encourage this on this list too).

Later down the line, if the L4S experiment is successful, there will be 
an opportunity to review these requirements if a standards track doc 
replaces the experimental (it is easier to relax CC requirements than 
tighten them). So, for the expt track, the requirements are designed to 
protect competing flows from harm in a tighter way than you might find 
in RFC5033 or similar.

Nonetheless, even the 1/p isn't tightly spec'd, quoting:

    The inverse proportionality requirement above is worded
    as a 'SHOULD' rather than a 'MUST' to allow reasonable flexibility
    when defining these specifications.

As another example, "MUST reduce or eliminate RTT bias" is deliberately 
expressed with wriggle room, but with enough strength to address the 
harm issues.

These are all up for review. If you care about evolvability, this is 
where your focus should be.

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 21 Nov 2019 04:03:04 +0800

Just to give another example, but with more manageable scope:

Remember that both DCTCP and DCQCN (RoCEv2 as spec'ed by the IBTA)
effectively use parts of the L4S framework already in their enclaves.

Isolating such domains where L4S is found to be useful even after an
internet-wide experiment fails is obviously also a realistic option.

Richard

Am 20.11.2019 um 11:49 schrieb Ingemar Johansson S:
> it.. Will the IETF send a message (LS?) to 3GPP with the message “please
> stop using L4S”, is this even a reasonable scenario?. After all, the
> fact that it is picked up by other SDOs, speaks against a failure ?
>
> /Ingemar

---

From: Pete Heist <[email-masked]>
Date: Thu, 21 Nov 2019 04:20:49 +0800

> On Nov 21, 2019, at 2:34 AM, Bob Briscoe <[email-masked]> wrote:
> 
> Roland,
> 
>> Yes, but as I also expressed my concerns w.r.t. the L4S codepoint earlier, at the cost of binding this to a quite fixed set of L4S 
>> behaviors and "burning" the last ECT codepoint. Personally, I like concepts with a little bit more potential to be useful for future 
>> development (evolvability) of congestion controls, e.g., BBRv2 and LoLa could also benefit from an SCE-like marking...
> 
> My whole purpose in solving the problem of deploying scalable CCs over the Internet was to re-juvenate evolution (to widen the range of applications that could be supported by different transport behaviours, particularly for real-time with low latency and high throughput at the same time). One of the main things that has stopped CCs evolving so far is the need for friendliness with the Reno behaviour that was not scaling over the years. 
> 
> If SCE is primarily supported in FQ AQMs, that will constrain flows to be capped at the rate that FQ gives them. How is that doing anything other than massively constraining future evolution of CCs, especially real-time ones? See Per-Flow Scheduling and the End-to-End Argument. I don't need to tell you that the e2e argument is all about giving end systems the power to innovate without permission.

In my opinion, what would enable future innovation in CCs is a signaling mechanism that is completely safe by design, without the need for protection mechanisms, allowing alternatives to be explored without a risk-benefit analysis.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 21 Nov 2019 04:28:06 +0800

Hi Pete,

Am 20.11.2019 um 21:20 schrieb Pete Heist:
> In my opinion, what would enable future innovation in CCs is a signaling mechanism that is completely safe by design,

Is this a plug for AccECN? ;) (I just looked and found that we started
that work back in 2011...)

> without the need for protection mechanisms,

Are you referring to secondary signals like loss, CE or in a different
context?

> allowing alternatives to be explored without a risk-benefit analysis.

A basic risk-benefit analysis should always be done. Even benign
optimizations can change the dynamics of a CC.

Richard

---

From: Sebastian Moeller <[email-masked]>
Date: Wed, 20 Nov 2019 21:33:31 +0100

> On Nov 20, 2019, at 21:20, Pete Heist <[email-masked]> wrote:
> 
> 

	[SM] At the very least if protection mechanisms are employed, they need to to be robust and reliable, without nasty corner-cases... if the rest of the internet's safe operation is contingent on protection/isolation, it seems prudent to put time and effort to make sure isolation is as strict as technical possible...

Sebastian

---

From: Pete Heist <[email-masked]>
Date: Thu, 21 Nov 2019 04:53:00 +0800

> On Nov 21, 2019, at 4:28 AM, Scheffenegger, Richard <[email-masked]> wrote:
> 
> Hi Pete,
> 
> that work back in 2011…)
  :)
> Are you referring to secondary signals like loss, CE or in a different

CE, if thought as a protection mechanism, rather than just a stronger signal, is not one that needs much further evaluation.

It does seem that a publicly accessible short, low-latency queue would need protection, even if not mandated.

> optimizations can change the dynamics of a CC.

Good point, but the additional risks that a second congestion signal adds should only be reduced performance that doesn’t impact other flows.

:)

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 21 Nov 2019 05:35:18 +0800

> Good point, but the additional risks that a second congestion signal adds should only be reduced performance that doesn’t impact other flows.

There have been proposals over the years, to utilize the absence of a
congestion signal as another signal, to speed up with session-start.

I know that Bob has also been looking into this as a thought, to use the
absence (or lower than expected steady-state) feedback of a low-impact
congestion signal as a new mechanism to speed up slow-start or adjust
maximum burst sizes. and there are similar approaches in the literature
(but most were never brought in front of the IETF).

The addition of a secondary congestion signal also introduces the signal
"absence of secondary congestion signal" (provided you can affirm
yourself, that this secondary mechanism is actually working on the
bottleneck). Misuse of such a "1-x" signal could impact other flows.

But don't get me wrong - I am very much in favor to a new signal; in my
opinion, there are valid point in favor and against both of the two
current proposals, which can not be easily combined.

However, perhaps a more in-depth analysis would show, that linking SCE
and L4S queues behind each other (in any order, and any number of
queues, only some of which become the bottleneck at a time), is not
catastrophically problematic. I haven't run any numbers on this though.

 From a 20km view, the marking strategies between both - broken down on
an individual flow - seem to be fairly similar. Both appear to be using
instantaneous queue depth (sojorn time) without any smoothing of that
signal at the bottleneck. One stated of with a step function profile,
while the other with a slope. Form my understanding, both marking
strategies have found that some aspects of the other are beneficial
(deadband when the queue is fairly empty, and a more or less steep
slope, ramping up to 1 well ahead of a full Q). One crucial difference
is that one does lend itself to easily cope with 100G+ link speeds and
very tight time and memory budget on silicon, while the other one lends
itself ideally more towards the edge of the current internet.

The main contention - from my observation - is the question if a DualQ
can be implemented without infringing certain IPR, and if it actually
meets the cited design goals. Or if classification and bandwidth
allocation need to be an orthogonal means to prevent starvation of one
or the other type of flow. Similar questions remain on the other side -
is it possible to build an AQM that can be put on 200G / 400G switching
chipsets and maintain the properties of (FQ_)Cake - especially when the
Q never holds more than one packet of a flow at any time.

The L4 feedback signal should be the least concern when designing a new
CC mechanism (and AccECN would in principle support all the use cases of
SCE - not necessarily particularly efficient per the latest draft where
the TCP options need to be used; but we have had designs over the years
to match up with SCE needs much better, and could still tweak things
there. Remember that when we started, we did think that ECT1 may become
an additional congestion signal in its own right - but that design fell
through in the WG. But there is nothing stopping the WG to revisit this
decision now, in light of new data.

Remains the CC reaction, where I believe the base CC mechanism in L4S
has had more in-depth reviews (dctcp) than SCE over the last ~9 years.

Best regards,
    Richard

---

From: Greg White <[email-masked]>
Date: Wed, 20 Nov 2019 23:14:11 +0000

It seems that a component of this debate revolves around whether FQ is a good solution or not.

In my view, FQ provides a very nice benefit (flow isolation) which, if it were to be implemented in *every* bottleneck in the Internet, would allow for unfettered innovation around congestion control.   It also enforces fairness, which could be a blessing, or it could be a "don’t care" or it could be a curse, depending on the situation (and it has been argued that it weakens the end-to-end principle).  Finally, it introduces complexity concerns that likely prevent it from actually being implemented in *every* bottleneck in the Internet.

The question around the SCE signal vs the L4S signal, in my view, is mainly around the question:  Does the IETF want to select a signal that forces all bottlenecks to implement FQ (if they want to participate in high-fidelity congestion signaling), or does it want to select a signal that allows the bottleneck implementer to choose between TWO options (FQ or DualQ).   

I assume that FQ will nearly always give better per-flow-fairness than DualQ (just like it has always done better in that regard as compared to single queue systems).  With L4S, anyone can use FQ to get near-perfect fairness if that is what they think is most important (and they can participate in high-fidelity congestion signaling).  So, the argument that L4S is broken because DualQ doesn't provide the same precise fairness as FQ is not a convincing one.  

Where I think SCE fails is that it is not available to any link that doesn't implement FQ, whether that is by choice or by necessity.  I don't believe that the IETF should use the last IP codepoint for a signaling mechanism that can *only* work in FQ.

-Greg

On 11/21/19, 5:36 AM, "tsvwg on behalf of Scheffenegger, Richard" <[email-masked] on behalf of [email-masked]> wrote:

    

    

    

    
    Best regards,
        Richard

---

From: Roland Bless <[email-masked]>
Date: Thu, 21 Nov 2019 09:32:13 +0800

Hi Bob,

see inline.

> is the need for friendliness with the Reno behaviour that was not

FWIW, our delay-based approach has deployment problems at the
other end of the spectrum: it gets suppressed by loss-based CCs...
(and we don't want to sacrifice our low delay goal).

> be capped at the rate that FQ gives them. How is that doing anything

I was solely referring to the marking behavior of SCE, not a particular
implementation based on FQ-AQMs.

> rarely any loss. Think of all the things you can do with that. Go

I already said that the Dual Queue AQM is nice, but comes with the
problem due to its built-in dropping law. Other CCs may react differently
and BBR was one example of newer CCs that did not work in the L4S queue
without further adaptation.
>    when defining these specifications.

Yes, but something has to be implemented (in hardware) and therefore it
is fixed for some time and it should be consistent along a path, so I
don't see a viable path for evolvement of this coupling law...

Regards
 Roland

---

From: Markku Kojo <[email-masked]>
Date: Thu, 21 Nov 2019 03:43:18 +0200 (EET)

On Wed, 20 Nov 2019, Kyle Rose wrote:

> <ingemar.s.johansson=[email-masked]> wrote:
>       [IJ] The premise would be that L4S is declared a failure. I doubt that anybody has
>       a good enough crystal ball to predict what happens. First it is necessary to come
>       to the conclusion that L4S is a failure and I would argue that we are not yet
>       there and I don't currently see that coming. Before that possible event I don't
>       see it meaningful to speculate.
> 
> 
> I'm going to have to go ahead and disagree with you strongly here. Given the potential
> consequences of cleaning up after a failed experiment without a plan worked out beforehand,
> this blithe approach is simply not acceptable.
> 
> In lots of cases, experiments are easy to terminate in an obvious way: for example, in one
> typical case, a code point can simply be abandoned, or (even better) a pollutable experimental
> code point returned to the available pool after some time. If that were the case here, it
> would not be difficult to enumerate a sequence of steps required to do so. It doesn't appear
> that's the case, however, so all the more reason to make sure we address this as part of the
> experiment setup.
> 
> A launch escape system of the necessary complexity should be a requirement of any experimental
> deployment. In this case, that might circumscribe the scope of the experiment itself.
>

I do not intend to speak out for either proposal but to raise my concern 
about future of new AQMs and related CCs that would really solve the 
problems that still seem to remain unsolved, particularly the problem of 
load transients such as flow startup. There have been a number of 
alternative proposals for ECN earlier and I expect there will be more in 
the future. I find it extremely important to allow such evolution.

That said, we have IETF-wide consensus documented in BCP 124 (RFC 4774) 
exactly for this case, i.e., the issues and requirements for a safe 
coexistence when defining alternate semantics for the ECN field. Sally, 
once again, has been very careful when considering the issues and the 
document is in most part very relevant. I would expect very thorough 
consideration for any deviation from the requirements in this document 
(along what is discussed in RFC 2914 and RFC 5033).

RFC 4774 also proposes a reasonable way forward for coexistence that is 
not necessarily ideal for widespread deployment in the global Internet 
but that's not what I find us discussing here. The question is about 
allowing controlled experiments in the Internet when using alternative ECN 
codepoint semantics for EXP purpose only. We very well know DSCP is known 
to have problems (bleaching, remarking, etc) but I don't think it would 
have that significant impact on EXPERIMENTING, because we have tools 
available to determine whether such problems are present in the 
end-to-end paths that are experimented with.

Thanks,

/Markku

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 21 Nov 2019 08:32:44 +0100

Hi Greg,

More below.

On November 21, 2019 12:14:11 AM GMT+01:00, Greg White <[email-masked]> wrote:
>It seems that a component of this debate revolves around whether FQ is
>a good solution or not

         [SM] The only party that keeps bringing up fq consistently seems to be the L4S proponents. In the context of the dual queue coupled AQM this is just a sreawman argument IMHO that distracts from the core of the issue. Tht being that implicitly everyone reading the L4S drafts will expect equitable sharing between the two traffic classes L4S divides the world into. In a sense a two class fair sharing system is one solution for the issue, as would be a strict priority scheduler that limits L4S traffic to 50% of the bandwidth (look, no fq). I even had a look at dualq's pseudocode and identified two toggles that most likely would allow to improve the worst case sharing behavior, potentially into a range that seems acceptable at least for the experiential roll-out. Not even exploring these options does not strike me as productive.

>
>In my view, FQ provides a very nice benefit (flow isolation) which, if
>it were to be implemented in *every* bottleneck in the Internet, would
>allow for unfettered innovation around congestion control.  

         [SM] I guess we agree that the most likely bottleneck for most users, is going to be the internet access link, so fixing those for users interested in low latency seems to be a more modest, but reachable goal.

 It also
>enforces fairness, which could be a blessing, or it could be a "don’t
>care" or it could be a curse, depending on the situation (and it has
>been argued that it weakens the end-to-end principle).  

         [SM] Well to recap the L4S approach of anything goes suffers from the issue that without an oracle scheduler divining the relative importance of active flows will not do the optimal thing either, but at the cost of allowing way more "catastrophic" sharing behavior, as evidenced in the results of sharing over DualQ at short RTTs. And to add insult to injury, opening multiple parallel flows to "trick" an fq aqm will also glresult in a higher bandwidth share in L4S, the only difference being that L4S did not try to make any guarantees in the first place....

Finally, it
>introduces complexity concerns that likely prevent it from actually
>being implemented in *every* bottleneck in the Internet.

[SM] How about we start with good enough and aim for the most relevant bottlenecks first instead of constructing an argument of the kind that shows that perfection seems unachievable and hence rejects the idea of even trying.

>
>The question around the SCE signal vs the L4S signal, in my view, is
>mainly around the question:  Does the IETF want to select a signal that
>forces all bottlenecks to implement FQ (if they want to participate in
>high-fidelity congestion signaling), or does it want to select a signal
>that allows the bottleneck implementer to choose between TWO options
>(FQ or DualQ).   

        [SM] It seems clear that the actual marking strategy of SCE and L4S are similar enough, that a unified method should exist to satisfy both camps, the goal in the end is a 1/p Control Loop. The issue about mutual exclusiveness of the two solutions really only comes from the side-effects of trying to change the semantics of the ECN codepoints', either subtly (SCE) or massively (L4S).
IMHO it seems worth revisiting the ECT(1) as classifier decision, but I fear that I do not represent the majority view in this.

>
>I assume that FQ will nearly always give better per-flow-fairness than
>DualQ (just like it has always done better in that regard as compared
>to single queue systems.

         [SM] I agree, based on actually testing vodel versus fq_codel and cake on my Home-Link. I repeat though, that that issue really only isxrelevant for the sharing in each of L4S two queues. The sharing between these queues should be equitable independent on the guarantees inside each queue. IMHO that inter-queue sharing behavior is the sticking point.

With L4S, anyone can use FQ to get
>near-perfect fairness if that is what they think is most important (and
>they can participate in high-fidelity congestion signaling).  

[SM] Under the assumption that person is in control of the actual bottleneck, sure, but e.g. DOCSIS seems to plan to roll out L4S on the CMTS and I severely doubt all ISPs will give the user control over it's activation...

So, the
>argument that L4S is broken because DualQ doesn't provide the same
>precise fairness as FQ is not a convincing one.  

         [SM] I agree, the reason why it is broken is not that it is not FQ, but that it does not achieve the engineering goal of strictly separating the two traffic types. And in all fairness the "coupled" in dual queue coupled AQM and the L4S' team philosophical objections against fairness indicate to me that we are talking about a conscious design decision and notvsimplyvtge inability to come up with a strict and equitable sharing system.

>
>Where I think SCE fails is that it is not available to any link that
>doesn't implement FQ, whether that is by choice or by necessity.  

         [SM] SCE is a marking strategy pure and simple. I see no insurmountable obstacle in combining the backward compatible SCE marking strategy with a better classifier to give L4S queueing to selected traffic.

I
>don't believe that the IETF should use the last IP codepoint for a
>signaling mechanism that can *only* work in FQ.

        [SM] Again I agree, but luckily that is simply not true, tack on a better identifier for L4S style senders and you are off to the races....

Sebastian

P.S.: If it appears that I am repeating my arguments than I admit that to be true, but so are Greg's.

>
>-Greg
>
>
>
>On 11/21/19, 5:36 AM, "tsvwg on behalf of Scheffenegger, Richard"
><[email-masked] on behalf of [email-masked]> wrote:
>
>    
>    
>> Good point, but the additional risks that a second congestion signal
>adds should only be reduced performance that doesn’t impact other
>flows.
>    >
>    > :)
>    >
>

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 21 Nov 2019 08:58:48 +0100

Dear All,

I have a question I want to pose about interactions between the dual queue coupled AQM's sharing behavior.

We currently see an RTT dependent bandwidth bias to the advantage of the LL queue from ~5:1 at short RTTs to ~3:1 at ~10 ms RTT to mostly equitable sharing at 80ms RTT.
Some of us including me are uneasy with introducing such a sharing system in the wild, but overlall, people on this list do not seem overly concerned about this behavior, supposedly because short RTTs are not that common for the expected roll-out? (I disagree with that view, given how many important traffic sources have been moving closer and closer to the endusers, like CDNs and the talk about moving even computational services closer to the edge ("mini datacenter in the base station" not sure whether that is more than a phantasy)).
But one of L4S's goals is also to reduce the bandwidth dependence on RTT for L4S flows, so that L4S flows compete more evenly for bandwidth with each other in the LL queue in spite of differing RTTs.

So here is my question, what is going to be the effect of such decreased RTT-dependence of L4S flows in regards to sharing behavior with flows in non-LL queue at higher RTTs?
Put differently, will decreased RTT-dependence of L4S flows make the sharing between the two queues more or less unequal at short and long RTTs?

My hypothesis is, that if a long RTT L4S flow can better compete with short RTT L4S flows, it will also do so against equal RTT non-L4S flows, so I expect that unless RTT-dependence is reduced for all flows independent of class, this otherwise positive change to L4S's recommended CC behavior will cause even more interference with the existing internet. I hope I am wrong, and would like to ask what the L4S' teams assessement of that issue is, and that there might even be data demonstrating that my concerns are unfounded.

Thanks
	Sebastian

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 21 Nov 2019 10:05:41 +0200

>
>         [SM] SCE is a marking strategy pure and simple. I see no
insurmountable obstacle in combining the backward compatible SCE marking
strategy with a better classifier to give L4S queueing to selected traffic.

I prefer to describe SCE as a signalling mechanism, one which can support a
variety of marking strategies (e.g. threshold, RED, Codel) and response
functions.

That is the fundamental difference here, and I think that L4S could
reasonably switch to SCE signalling with essentially all of its advertised
goals intact.  That would be a robust way to close several of the
outstanding issues in the tracker.

I should also note that whenever there is unfairness between SCE and
non-SCE flows, it it's in favour of the non-SCE flows.  This means SCE is
safe to deploy even in a single queue; it just might not perform well.  The
reverse is true of L4S, which is why it needs protection for safe
deployment unless and until the RFC-3168 detection heuristic is shown to
work robustly.

- Jonathan Morton

---

From: Sebastian Moeller <[email-masked]>
Date: Thu, 21 Nov 2019 12:19:40 +0100

Dear tsvw community,

with great interest I had a look at the slides Bob posted earlier. I have a couple of questions I would like to discuss:

Issue #16: Fall-back to Reno-Friendly on Classic ECN bottleneck

A) The slide ends with the observation:
 "	• detection unlikely to be perfect"
I now want to ask, which traffic class is supposed to pay the price for this detection misses? The principle of least surprise would indicate that the detector should err on false positive RFC3168 detections.

QUESTION: Does everybody agree that it should be safety first and the cost of mis-classifications are carried by L4S endpoints/flows?
(In that case the logic of the detector should be assume RFC3168 until there is sufficient evidence to rule that out with a certain level of confidence)

B) "Active detection technique
if last two reordered, likely L4S ● reduce suspicion, and continue"

Or a bonded link...

QUESTION: Is it permissible for the detector to be fooled by bonded link segments (especially in the light of L4Ss push for time based retransmission like RACK based on the existence of bonded links)?

C) The alpha bug:
"All prior testing had been done using DCTCP.
Upon fixing this bug, the results again match DCTCP"

QUESTION: Is it really a good strategy to assume tests performed with dctcp carry over to other transport/CC algorithms? Or does this indicate that the published results so far using dctcp should at least be confirmed with TCP Prague?

D) "Issue #17: Conclusion
The main result of concern was due to a bug in initializing the value of TCP Prague alpha, which has been fixed and demonstrated to resolve the latency impact that was spanning multiple seconds"

COMMENT: In the light of https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-11-11T090559-r2/l4s-s6-2/batch-l4s-s6-2-prague-vs-cubic-50Mbit-80ms_fixed.png showing that L4S still mishandles CE markings in that condition (albeit with the fall-out restricted to itself) it seems that the alpha fix alone might not be sufficient (which basically makes TCP Pragues response to the first CE independent of the CE interpretation scheme)

E) "If the CoDel queue is upgraded to perform Immediate AQM on L4S flows, the latency spike can be largely avoided."

QUESTION: True, but is proposing to change millions of deployed hosts really a viable strategy for coexistence/backward compatibility? 

Best Regards
	Sebastian

---

From: Pete Heist <[email-masked]>
Date: Thu, 21 Nov 2019 20:58:05 +0800

> On Nov 21, 2019, at 7:14 AM, Greg White <[email-masked]> wrote:
> 

The thoughts are appreciated. What I think needs to be clarified is that SCE doesn’t necessarily require full FQ using the traditional many queues approach. It only requires some level of help from the network, when fairness between SCE and non-SCE flows is required. Options include:

- Changing the SCE marking ramp, trading off some of the advantages of SCE for improved fairness. This was first described at IETF 105 in Montreal.
- Using CNQ, which is implemented but we didn’t have time to present today. That provides a minimal level of improved fairness that can actually favor SCE flows early in their lifetime.
- Using LFQ, which like CNQ, is in draft form with a crude discrete time simulation, but doesn’t yet have an implementation. This provides closer to full FQ but with a lighter weight implementation.

This is still an active area of research with many options available to us, and we feel it’s a tractable problem. We just want to make clear that “SCE requires FQ” isn’t very accurate, and needs more clarification as to the current and future options available.

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 21 Nov 2019 22:44:52 +0800

Roland,

I'm not getting it....

>>> LoLa could also benefit from an SCE-like marking...
This last sentence... please really spell it out what you could possibly 
be meaning here. What has made you suddenly think this particular 
marking scheme has got magical properties that bestow evolvability? I'm 
serious, if you can't explain why you've said a sentence like this, it 
implies you are under the spell of some cult or fad.

> implementation based on FQ-AQMs.
This implies you believe all that silliness about a shallower SCE 
threshold not starving an SCE flow in a single queue, because it makes 
SCE flows not want to use the queue.

> without further adaptation.
The built in dropping law is for the old traffic ('pre-evolved' if you 
like) that doesn't respond to anything else but loss. That's what 
drop-based AQMs were for - to fool Reno etc into thinking they had hit 
the buffers, so to speak.

> don't see a viable path for evolvement of this coupling law...
Why does the coupling law need to evolve? It's a relationship between 
something invariant (scalable) and the past (which is over, by 
definition, so it's not going to change now).

If you want to evolve, you select what's best for you - probably the 
nice clean L4S ECN queue. I still don't get what sort of evolution you 
can't do? Evolvability isn't about a researcher being able to do what 
they'd like. It's about being weeded out if what you think is best turns 
out not to be.

Cheers

Bob

---

From: Steven Blake <[email-masked]>
Date: Thu, 21 Nov 2019 11:31:51 -0500

On Wed, 2019-11-20 at 21:22 +0800, Roland Bless wrote:
> > I agree with Ingemar. Related to the chances for failing: as
> > history learned us, the only real world chance for failing would be
> > that the technology would not be taken up in deployments. This
> > whole codepoint conflict is mainly damaging/creating this
> > deployment 'issue'. The other so called show stoppers are less of
> > an issue for most people that worked on CCs. Blowing 'issues' (not
> > yet implemented/integrated/tested code and bugs) out of proportion
> > is not the way to go forward here.
> >  
> > To give an example:
> Therefore, a compromise could be to define ECT(1) with "some
> congestion experienced" semantics that could also be used for L4S.
> In order to further disambiguate and use the particular L4S semantics
> of Dual Queue AQM etc. one could use an additional DSCP as
> L4S classifier: I currently don't see the need to follow the argument
> of providing L4S orthogonal to DiffServ (and for me L4S is specifying
> a PHB).
> In case L4S doesn't take off one can at least use the SCE semantics
> and reassign the DSCP (or in case of success make it later orthogonal

YES!! Dual-Q is defining PHBs. Packets are steered to PHBs via DSCP
marking. Experimental DSCPs are available. Parallel experiments can be
isolated by DSCP. Let's not break this simple architecture for an
experiment.

Regards,

// Steve

---

From: Roland Bless <[email-masked]>
Date: Fri, 22 Nov 2019 10:32:18 +0800

Hi Bob,

see inline.

> implies you are under the spell of some cult or fad.

Not sure what you are trying to indicate by your last sentence, but
sure, I can elaborate a bit on my last sentence.
I find it architecturally cleaner to have an additional ECN codepoint
used for indicating "SCE" rather than (ab)using it as classifier for the
dualQ AQM.
The SCE markings can be used independently whether you have multiple
queues or just a single queue. If one leaves the coupling aside the
algorithm for marking will probably not differ so much from what is
proposed in L4S.

> SCE flows not want to use the queue.

I do not understand what you are saying here.

> the buffers, so to speak.

Yes, I understand that. But what happens if the classic traffic
once vanishes and we only have low-delay congestion control and
want to re-use the "second" queue for other purposes, e.g., using
it for flows that do not use loss-based CC?

>>   > The key thing here is the wording of the Prague requirements

See above, probably we want to put something into queue which is
using a non-loss-based congestion control and/or we need to change
the (1/p,1/p^2) marking.

> they'd like. 

I also don't understand it in this way, but investigating
invariables and degrees of freedom prudently may enable or facilitate
deployment of new stuff. If that new stuff cannot be deployed
it never gets a chance of being weeded out later either.

Roland

---

From: "Holland, Jake" <[email-masked]>
Date: Fri, 22 Nov 2019 03:24:28 +0000

> COMMENT: In the light of https://www.heistp.net/downloads/sce-l4s-bakeoff/bakeoff-2019-11-11T090559-r2/l4s-s6-2/batch-l4s-s6-2-prague-vs-cubic-50Mbit-80ms_fixed.png showing that L4S still mishandle…

I brought this up, but Greg and Pete each explained to me that this
impacts only the L4S flow in the observed behavior during testing,
and doesn't make a giant persistent queue in the upstream dumb fifo.

While this is maybe still worth considering as a performance question
for L4S, I don't think it's the safety issue that was originally
opened (nor that it's especially severe and worth blocking on).

I think I agree this means the issue is closeable, AFAIK.

Just to clarify: my expectation is that finding a future test case
with another similar problem for competing flows could still happen,
and would justify re-opening the issue.

I'll also suggest I'd be more comfortable if we had a suite of test
situations that covered, in a more principled or structured way, a
"range of environments" (a la 5033) that happened to include this
case, among many others.

I won't try to argue the lack of such a suite is a blocker, just that
its lack remains a source of much of my skepticism.

Best,
Jake

---

From: Greg White <[email-masked]>
Date: Fri, 22 Nov 2019 04:02:05 +0000

Pete,

I've seen several statements being made to the effect that SCE can possibly work in a single queue or dual queue bottleneck, but I've not seen any data to suggest that it is true.

If the SCE team can come up with such an approach, and provide test results (preferably to a similar extent to those provided by the L4S team, not just the simple flent scenarios), it would be interesting to see.

-Greg

---

From: Pete Heist <[email-masked]>
Date: Fri, 22 Nov 2019 12:54:32 +0800

Yes, we look forward to providing more results later, in the form of flent tests. If there's is anything more detailed that needs testing, we’ll enhance flent as a first option.

The CNQ implementation is only a week or so old, and LFQ is not implemented yet. The crude discrete time simulations for both are referenced in the respective drafts.

A clarification for my earlier statement regarding CNQ is below.

> On Nov 22, 2019, at 12:02 PM, Greg White <[email-masked]> wrote:
> 
> Pete,
> 

It should more accurately say:

"That provides a minimal level of improved fairness that prevents SCE flows from being starved to minimum cwnd.  It also priorities sparse flows including initial handshakes & request-response transactions.”

Experimentally, we have seen SCE flows outcompete non-SCE flows early in their lifetime with CNQ, but that isn’t directly due to help from CNQ, but to the new marking strategy using Codel, which was prototyped in CNQ.  We’ll port this development to Cake in due course.

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 22 Nov 2019 08:14:47 +0100

Dear Greg,

On November 22, 2019 5:02:05 AM GMT+01:00, Greg White <[email-masked]> wrote:
>Pete,
>
>I've seen several statements being made to the effect that SCE can
>possibly work in a single queue or dual queue bottleneck, but I've not
>seen any data to suggest that it is true.
>
>If the SCE team can come up with such an approach, and provide test
>results (preferably to a similar extent to those provided by the L4S
>team, not just the simple flent scenarios), it would be interesting to
>see.

[SM] I do note that the simple flent scenarios, as you call them, revealed and illustrated instantaneously at least three issues with the dual queue coupled AQM and TCP Prague. IMHO this calls into question the extent of the L4S teams testing, at the very least it indicates that testing should be performed with the actual implementation one wants to standardize. Also as far as I can none of the dctcp tests over the DualQ aqm have so far been really testing realistic corner cases.
So I applaud and support the call for diligent testing, but I very include the L4S implementations into that call.

Best Regards
        Sebastian

>doesn't implement FQ, whether that is by choice or by necessity.  I
>The thoughts are appreciated. What I think needs to be clarified is
>that SCE doesn’t necessarily require full FQ using the traditional many
>queues approach. It only requires some level of help from the network,
>when fairness between SCE and non-SCE flows is required. Options
>include:
>    
>- Changing the SCE marking ramp, trading off some of the advantages of
>SCE for improved fairness. This was first described at IETF 105 in
>Montreal.
>- Using CNQ, which is implemented but we didn’t have time to present
>today. That provides a minimal level of improved fairness that can
>actually favor SCE flows early in their lifetime.
>- Using LFQ, which like CNQ, is in draft form with a crude discrete
>time simulation, but doesn’t yet have an implementation. This provides
>closer to full FQ but with a lighter weight implementation.
>    
>This is still an active area of research with many options available to
>us, and we feel it’s a tractable problem. We just want to make clear
>that “SCE requires FQ” isn’t very accurate, and needs more
>clarification as to the current and future options available.
>    
>    

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: Sebastian Moeller <[email-masked]>
Date: Fri, 22 Nov 2019 09:53:13 +0100

Hi Jake,

Thanks for your feed-back.

> On Nov 22, 2019, at 04:24, Holland, Jake <[email-masked]> wrote:
> 
> I brought this up,

	Great, the video's are still not up, so it will take a while before I cought up with the status quo.

> but Greg and Pete each explained to me that this

	Ah, yes, I should have made this clearer, in principle I have no objections against L4S if it did a) equitably share bandwidth with normal traffic and b) would default to only hurting itself (in case when the approximations fail). But unfortunately when L4S fails to meet its own requirements it mostly is at the expense of other traffic (not in this specific instance though), honi soit qui mal y pense...

> opened (nor that it's especially severe and worth blocking on).

	Yes, I agree and should have made that clearer, thanks for pointing that out.

> I think I agree this means the issue is closeable, AFAIK.

	Yes.

> and would justify re-opening the issue.

	I believe hope the chairs are open to that. I am going to try test this hypothesis soon, by trying to get dualq's isolation failure corner-cases onto the issue tracker asd the L4S team seems to ignore to properly address anything less than a tracked issue. I will try to frame the issue as an issue of "congestion collapse" for the normal queue (which I believe it to be) so I can cite RFC verses and rhymes to move this out of the realm of "oh, it is just Sebastian being a PITA, again".

> "range of environments" (a la 5033) that happened to include this

	Yes, it seems like a worthwhile goal for all CC/AQM work to have an enumerated list of known tricky issues a CC/AQM should/must solve.

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Fri, 22 Nov 2019 13:00:18 +0000

Hi Roland,

> FWIW, our delay-based approach has deployment problems at the other end of the spectrum: it gets suppressed by loss-based CCs...

Indeed, this is the problem that has constrained the use of delay based CCs. It gets pushed away by the loss/ECN based congestion controls. But this same effect is exactly the biggest disadvantage of flows responding to SCE. It also gets pushed away by flows that do not respond to this SCE signal.

That the network is able to separate the old and new flows is exactly the power of using the codepoint like L4S. We can create a delay-isolated queue but still can use the ECN signal to:
1: give an estimate what is the fair rate to use compared to other flows
2: give a target for a queue filling level independent of the path delay and possible variations on it

These 2 features are exactly answering explicitly what delay based CC's have to otherwise implicitly have to agree on or find out:
1: how can we make sure that one delay based CC is not pushing away the other delay based CC (what is the delay (variation) based protocol to use for determining the throughput conversion point)
2a: what is the delay target it should use (e.g. on top of the minimum RTT you experienced)
2b: what is delay caused by queuing, and should trigger reduction of throughput, and what is delay caused by scheduling or packet aggregation mechanisms that should not be taken into account (and is useless to reduce throughput for).

I see the immediate/accurate ECN signal as the missing explicit signal to further improve the delay based CCs on those 2 aspects, and I also see delay based CCs the technology to improve the ECN-only DCTCP-style CC.

So it is not a matter of opposition, but it is the combination that gives the power!

Regards,
Koen.

-----Original Message-----
From: Roland Bless <[email-masked]> 
Sent: Thursday, November 21, 2019 2:32 AM
To: Bob Briscoe <[email-masked]>
Cc: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>; Ingemar Johansson S <[email-masked]>; Kyle Rose <[email-masked]>; [email-masked]; [email-masked]
Subject: Re: [tsvwg] L4S vs SCE

Hi Bob,

see inline.

On 21.11.19 at 19:34 Bob Briscoe wrote:
> applications that could be supported by different transport 
> behaviours, particularly for real-time with low latency and high 
> throughput at the same time). One of the main things that has stopped 
> CCs evolving so far is the need for friendliness with the Reno 
> behaviour that was not scaling over the years.

FWIW, our delay-based approach has deployment problems at the other end of the spectrum: it gets suppressed by loss-based CCs...
(and we don't want to sacrifice our low delay goal).

> be capped at the rate that FQ gives them. How is that doing anything

I was solely referring to the marking behavior of SCE, not a particular implementation based on FQ-AQMs.

> scalable CCs? In much the same way CCs have always evolved. With L4S 
> you have a clean slate that seems just like a FIFO with a shallow 
> ECN-only immediate AQM. And other flows are causing you hardly any 
> delay and very rarely any loss. Think of all the things you can do 
> with that. Go evolve, Roland!

I already said that the Dual Queue AQM is nice, but comes with the problem due to its built-in dropping law. Other CCs may react differently and BBR was one example of newer CCs that did not work in the L4S queue without further adaptation.
> Later down the line, if the L4S experiment is successful, there will 
> be an opportunity to review these requirements if a standards track 
> doc replaces the experimental (it is easier to relax CC requirements 
> than tighten them). So, for the expt track, the requirements are 
> designed to protect competing flows from harm in a tighter way than 
> you might find in RFC5033 or similar.
> 
> 

Yes, but something has to be implemented (in hardware) and therefore it is fixed for some time and it should be consistent along a path, so I don't see a viable path for evolvement of this coupling law...

Regards
 Roland