Subject: ECN in QUIC

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 24 Jan 2018 08:01:25 +0000

Hi

A few impressions on the ECN in QUIC discussions. The audio was so-so, frequent interruptions and LARGE differences in audio level + badly implemented ears 😊, so sometimes I missed the details and possibly kick in open doors.

*  Feedback format. I understand that the proposed proposal based on ECT(0), ECT(1) and CE counters does not play well with connection migration. I believe, but I am not 100% sure that I understand the problem. I understand that there are two concerns (perhaps more?):
  1. Counters are reset, this means that there will be a potentially very large (negative?) difference in CE marked packets. I believe that this can be detected as false congestion signal and ignored by the congestion signal.
  2. Difficulties to detect ECN bleaching. It is tricky to understand if packets along the new path are bleached based on the ECT counters. This can be a more complicated case.
In any case it was suggested to indicate the ECN bits for the packets in the ACK frame. Such a format was actually outlined in earlier proposal (see page 6, Figure 3 in https://tools.ietf.org/html/draft-johansson-quic-ecn-01 ). It is Ok with me to change to a format similar to this if the group prefers that.

* Congestion control for ECT(1), L4S : My feeling is that it is for now OK to leave this as future work. That does not mean that there are absolutely no ideas on how to implement this, for short RTT environments DCTCP style congestion control is quite OK. Furthermore I and recently worked on a L4S congestion control that is based on BBR, the results look quite promising and the changes to the BBR algorithm are quite minimal. I am willing to present this at ICCRG in London if there is any interest in this.
* ECN in QUIC v1 or later : I would very much prefer that ECN support goes into v1. On the technical side it makes it possible to experiment with ECN, potentially on a larger scale. On the political side it helps to make the “chicken and egg” problem become history.

/Ingemar
==================================
Ingemar Johansson  M.Sc.
Master Researcher

Ericsson Research
Network Protocols & E2E Performance
Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
[email-masked]<mailto:[email-masked]>
www.ericsson.com

The world is full of magical things patiently
    waiting for our wits to grow sharper
               Bertrand Russell
==================================

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 24 Jan 2018 13:50:43 +0000

You are right - it was not easy to catch everything. I came with an impression that the main objection to counters was not connection migration but incompatibility with the future multi-path. Even though multi-path is not in scope for v1, people were afraid of the total incompatibility of the scheme. Am I the only one with that impression?

- Igor

-----Original Message-----
From: Ingemar Johansson S [[email-masked]]
Received: Wednesday, 24 Jan 2018, 3:01AM
To: QUIC WG [[email-masked]]
Subject: ECN in QUIC

Hi

A few impressions on the ECN in QUIC discussions. The audio was so-so, frequent interruptions and LARGE differences in audio level + badly implemented ears ?, so sometimes I missed the details and possibly kick in open doors.

*  Feedback format. I understand that the proposed proposal based on ECT(0), ECT(1) and CE counters does not play well with connection migration. I believe, but I am not 100% sure that I understand the problem. I understand that there are two concerns (perhaps more?):
  1. Counters are reset, this means that there will be a potentially very large (negative?) difference in CE marked packets. I believe that this can be detected as false congestion signal and ignored by the congestion signal.
  2. Difficulties to detect ECN bleaching. It is tricky to understand if packets along the new path are bleached based on the ECT counters. This can be a more complicated case.
In any case it was suggested to indicate the ECN bits for the packets in the ACK frame. Such a format was actually outlined in earlier proposal (see page 6, Figure 3 in https://tools.ietf.org/html/draft-johansson-quic-ecn-01<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Djohansson-2Dquic-2Decn-2D01&d=DwMGaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=xVzIA9f7WjK8CVIQ3B-v9ZNl-Uf8HCY4nYPb0fMRO-4&s=S9aqc8UIGaTDX3ZMs1VTyWiLSSG01ndFhhr_Zg01H1Y&e=> ). It is Ok with me to change to a format similar to this if the group prefers that.

* Congestion control for ECT(1), L4S : My feeling is that it is for now OK to leave this as future work. That does not mean that there are absolutely no ideas on how to implement this, for short RTT environments DCTCP style congestion control is quite OK. Furthermore I and recently worked on a L4S congestion control that is based on BBR, the results look quite promising and the changes to the BBR algorithm are quite minimal. I am willing to present this at ICCRG in London if there is any interest in this.
* ECN in QUIC v1 or later : I would very much prefer that ECN support goes into v1. On the technical side it makes it possible to experiment with ECN, potentially on a larger scale. On the political side it helps to make the "chicken and egg" problem become history.

/Ingemar

Ericsson Research
Network Protocols & E2E Performance
Labratoriegr?nd 11
971 28, Lule?, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
[email-masked]<mailto:[email-masked]>
www.ericsson.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ericsson.com&d=DwMGaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=xVzIA9f7WjK8CVIQ3B-v9ZNl-Uf8HCY4nYPb0fMRO-4&s=tySiMlgTyIT8IJmaKYS-yMYBNj0YUw_M246hX4szKZ4&e=>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 24 Jan 2018 08:52:55 -0600

FWIW,

On Wed, Jan 24, 2018 at 2:01 AM, Ingemar Johansson S <
[email-masked]> wrote:

> Hi
>
>
>
> A few impressions on the ECN in QUIC discussions. The audio was so-so,
> frequent interruptions and LARGE differences in audio level + badly
> implemented ears 😊, so sometimes I missed the details and possibly kick
> in open doors.
>
>
>
> *  Feedback format. I understand that the proposed proposal based on
> ECT(0), ECT(1) and CE counters does not play well with connection
> migration. I believe, but I am not 100% sure that I understand the problem.
> I understand that there are two concerns (perhaps more?):
>
>   1. Counters are reset, this means that there will be a potentially very
> large (negative?) difference in CE marked packets. I believe that this can
> be detected as false congestion signal and ignored by the congestion signal.
>
>   2. Difficulties to detect ECN bleaching. It is tricky to understand if
> packets along the new path are bleached based on the ECT counters. This can
> be a more complicated case.
>
> In any case it was suggested to indicate the ECN bits for the packets in
> the ACK frame. Such a format was actually outlined in earlier proposal (see
> page 6, Figure 3 in https://tools.ietf.org/html/
> draft-johansson-quic-ecn-01 ). It is Ok with me to change to a format
> similar to this if the group prefers that.
>
> * Congestion control for ECT(1), L4S : My feeling is that it is for now OK
> to leave this as future work. That does not mean that there are absolutely
> no ideas on how to implement this, for short RTT environments DCTCP style
> congestion control is quite OK. Furthermore I and recently worked on a L4S
> congestion control that is based on BBR, the results look quite promising
> and the changes to the BBR algorithm are quite minimal. I am willing to
> present this at ICCRG in London if there is any interest in this.
>

We deprecated existing use of ECT(1) to allow experiments, which is great,
but (as AD) I'd be more comfortable if QUIC didn't slam some current idea
about L4S into QUICv1, which is targeted as a proposed standard. So, if the
working group is OK leaving that as future work, I'd be thrilled.

I'm always willing to be convinced that I'm wrong, of course. Many of you
have already had the pleasure of convincing me I was wrong on other topics
:D ...

Spencer

* ECN in QUIC v1 or later : I would very much prefer that ECN support goes
> into v1. On the technical side it makes it possible to experiment with ECN,
> potentially on a larger scale. On the political side it helps to make the
> “chicken and egg” problem become history.
>
>
>
> /Ingemar
>
> SMS/MMS +46-73 078 3289 <+46%2073%20078%2032%2089>
>
> [email-masked]
>
> www.ericsson.com
>
>
>

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 24 Jan 2018 18:01:36 +0000

I believe the primary concern during the meeting yesterday was the ECN feedback format (counters).  Here is the alternative, similar to what Christian voiced.

I am making these assumptions:

  *   CE is not going to be a very common occurrence
  *   The endpoint will usually either send (almost) all packets with no ECT markings, or it will send (almost) all packets with a specific ECT (ECT[0] or ECT[1]).
None of these assumptions are required.  They are just used to identify “common cases” and save a few bytes per ACK frame.  We may discard these assumptions and the optimizations.
The worst case for this proposal is a case when ECT[0] and ECT[1] marking alternate, because that prevents acknowledging consecutive packets as a block.  My inclination is to direct the sender to just not do this.

Part 1. Primary ECT
--------------------------
During the handshake, each endpoint notifies the other of the ECT codepoint (ECT[0] or ECT[1]) that is intends to use primarily.  This is known the Primary ECT.  By default, ECT[0] is the Primary ECT.

Part 2.  Two ACK varieties
----------------------------------

In addition to ACK Frame, we introduce a new ACK-ECT frame.  The contents of the frames are identical and unchanged from the current ACK frame contents.
All packets acknowledged by the ACK Frame are packets that have no ECN bits set at all.  All packets acknowledged by the ACK-ECT Frame are packets that have Primary ECT code point.  It is expected that in the common case, all packets will be acknowledged by ACK or ACK-ECT.

Part 3.  ACK-ECN-BITMAP
----------------------------------

The new ACK-ECN-BITMAP frame is used when ACK or ACK-ECT frames cannot be used.  Its format is very similar to the format of the ACK frame, except there is a new ECN Bitmap field.

~~~
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                     Largest Acknowledged (i)                ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ACK Delay (i)                      ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       ACK Block Count (i)                   ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ECN Bitmap (*)                     ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                          ACK Blocks (*)                     ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~~~
{: #ack-ecn-bitmap format title="ACK-ECN-BITMAP Frame Format"}

The fields in the ACK-ECN-BITMAP frame are as follows:

Largest Acknowledged:  (SAME AS IN ACK FRAME)

ACK Delay:  (SAME AS IN ACK FRAME)

ACK Block Count:
SAME AS IN ACK FRAME, except packets in each ACK Block MUST have the same ECN marking.  This may cause blocks of consecutive packets to be split into multiple ACK Blocks (with Gap=0 in between), if those packets have different ECN markings.

ECN Bitmap:
A sequence of ACK Block Count pairs of bits that indicate ECN markings for the corresponding ACK Block.  The size of the ECN Bitmap field is Ceiling((1+“ACK Block Count”)/4) octets.  The unused bits should be set to 0.
For example:
~~~
0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|AA|BB|CC|DD|…
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Bits AA represent ECN bits for packets acknowledged in the First ACK Block.
Bits BB represent ECN bits for packets acknowledged in Additional ACK Block 0.
Bits CC represent ECN bits for packets acknowledged in Additional ACK Block 1
…

ACK Blocks: (SAME AS IN ACK FRAME)

  *   Igor

From: Ingemar Johansson S [mailto:[email-masked]]
Sent: Wednesday, January 24, 2018 3:01 AM
To: QUIC WG <[email-masked]>
Subject: ECN in QUIC

Hi

/Ingemar

Ericsson Research

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 24 Jan 2018 20:51:59 +0000

Hi

Comment on “CE is not going to be a very common occurrence”.
While that may be true for (almost) drop equivalent ECN (a.k.a. classic), it does unfortunately not hold at all for L4S. Typical values of ECN marking ratio with L4S is in the order of 10 to 50%. In addition the CE marks can be spread randomly (for instance with phantom queues).
This means that the ACK blocks in the suggested format below can become very short, in the worst case representing only one packet per ACK block if we have an alternating ECT-CE-ECT-CE… pattern in the received packets.

So my take is that if an ECN bit pattern is really necessary then it is better to have it as an bitmap that represents all packets that are listed in the ACK block. If one make an assumption that an ACK frame contains two ACK blocks that ACKs 5 and 7 packets then the ECN block will become (5+7)*2 = 24bits , i.e. 3 octets.

But with that said I am still not convinced that the packet counter approach is flawed (I’m am happy to be enlightened 😊) .
Taking multipath as an example : Isn’t it the case that two connections are setup, each with its own congestion control context (I omit coupled CC for the moment). If that is the case then I don’t see any problem with the packet counters.
As regards to connection migration. One possible matter is that it can become tricky to detect ECN bleaching or ECN bit access problems in OS immediately , however during the course of 1 RTT or even less, this should be possible to detect as e.g. ECN bleaching will result in that the neither on the ECT(0),ECT(1) and CE counters increase. But I am fully aware that I may miss things here.

/Ingemar

From: Lubashev, Igor [mailto:[email-masked]]
Sent: den 24 januari 2018 19:02
To: Ingemar Johansson S <[email-masked]>; QUIC WG <[email-masked]>; huitema ([email-masked]) <[email-masked]>
Subject: RE: ECN in QUIC

Part 1. Primary ECT

Part 3.  ACK-ECN-BITMAP

~~~
0                   1                   2                   3

ACK Block Count:

ECN Bitmap:

  *   Igor

From: Ingemar Johansson S [mailto:[email-masked]]
Sent: Wednesday, January 24, 2018 3:01 AM
To: QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: ECN in QUIC

Hi

/Ingemar

Ericsson Research

---

From: Christian Huitema <[email-masked]>
Date: Wed, 24 Jan 2018 11:37:34 -1000

On 1/24/2018 10:51 AM, Ingemar Johansson S wrote:
> While that may be true for (almost) drop equivalent ECN (a.k.a.
> classic), it does unfortunately not hold at all for L4S. Typical
> values of ECN marking ratio with L4S is in the order of 10 to 50%. In
> addition the CE marks can be spread randomly (for instance with
> phantom queues).
>
> This means that the ACK blocks in the suggested format below can
> become very short, in the worst case representing only one packet per
> ACK block if we have an alternating ECT-CE-ECT-CE… pattern in the
> received packets.
>
Yes.
>
>  
>
> So my take is that if an ECN bit pattern is really necessary then it
> is better to have it as an bitmap that represents all packets that are
> listed in the ACK block. If one make an assumption that an ACK frame
> contains two ACK blocks that ACKs 5 and 7 packets then the ECN block
> will become (5+7)*2 = 24bits , i.e. 3 octets.
>
Yes, we have to take that into account in the design.
>
>  
>
> But with that said I am still not convinced that the packet counter
> approach is flawed (I’m am happy to be enlightened 😊) .
>
> Taking multipath as an example : Isn’t it the case that two
> connections are setup, each with its own congestion control context (I
> omit coupled CC for the moment). If that is the case then I don’t see
> any problem with the packet counters.
>

We are heading towards a single packet sequence number space shared by
multiple paths. That means that we need precise per packet feedback. If
a packet was marked with CE, you want to know on which path, and the way
to know that is to find the "path" tag in the saved copy of the packet.
You cant do that with counters.

> As regards to connection migration. One possible matter is that it can
> become tricky to detect ECN bleaching or ECN bit access problems in OS
> immediately , however during the course of 1 RTT or even less, this
> should be possible to detect as e.g. ECN bleaching will result in that
> the neither on the ECT(0),ECT(1) and CE counters increase. But I am
> fully aware that I may miss things here.
>

Again, there is no problem if we have precise per packet feedback,
because we can find out which packets were bleached.

-- Christian Huitema

---

From: Christian Huitema <[email-masked]>
Date: Wed, 24 Jan 2018 11:37:34 -1000

Yes.

-- Christian Huitema

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 24 Jan 2018 22:53:29 +0000

I think I like a per-packet bitmap.  If you are ACKing 30 packets, it is only 8 bytes.  The advantage of per-packet bitmap is that you do not need to split ACK Blocks – savings right there.

  *   Igor

From: Christian Huitema [mailto:[email-masked]]
Sent: Wednesday, January 24, 2018 4:38 PM
To: Ingemar Johansson S <[email-masked]>; Lubashev, Igor <[email-masked]>; QUIC WG <[email-masked]>
Cc: Bob Briscoe ([email-masked]) <[email-masked]>; [email-masked]
Subject: Re: ECN in QUIC

We are heading towards a single packet sequence number space shared by multiple paths. That means that we need precise per packet feedback. If a packet was marked with CE, you want to know on which path, and the way to know that is to find the "path" tag in the saved copy of the packet. You cant do that with counters.

Again, there is no problem if we have precise per packet feedback, because we can find out which packets were bleached.

-- Christian Huitema

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Thu, 25 Jan 2018 07:03:22 +0100

> On 24 Jan 2018, at 23:53, Lubashev, Igor <[email-masked]> wrote:
> 

I'll also note that in the current Internet case (no L4S or DCTCP, sporadic deployment of CE-marking routers), there are opportunities for large savings with the bitmap approach through trivially simple compression (maybe even as simple as a special representation for "all of these packets were 0/ECT0/ECT1").

Cheers,

Brian

> 	• Igor

---

From: "Lubashev, Igor" <[email-masked]>
Date: Thu, 25 Jan 2018 06:29:53 +0000

> On Thursday, January 25, 2018 1:03 AM, Brian Trammell (IETF) [mailto:[email-masked]] wrote:
>

Completely agree.

That's exact why I had ACK and ACK-ECT frames in my proposal (see earlier in the thread) -- to make "all packets have no ECN markings" and "all packets are ECT" cases cost 0 bytes.  The common case today would cost nothing.  When some path gets "interesting", it would start costing ~2 bits per packet.

(See Parts 1&2 and ignore Part 3, which has a per-ACK_Block instead of a per-packet bitmap: https://www.ietf.org/mail-archive/web/quic/current/msg02973.html).

- Igor

---

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Thu, 25 Jan 2018 10:10:30 +0000

Hi all,

Some questions related to the remark of Christian: "We are heading towards a single packet sequence number space shared by multiple paths"
What is the rationale behind this?
Is the reason for this that there are currently no per path states/communications required? (because congestion control is currently out of scope, and it is currently the only per path processing required). In that case I understand why the current design of QUIC is optimizing itself towards application to application state, but we must make sure we don't make our live difficult if we have per connection state (when it is needed, like congestion control).

Looking at per path processing for congestion control, it seems to be a big impact and burden (processing overload) to handle bitmaps (compose at receiver and parse at sender), keep track of which packets were send on which path (in sender) and mapping the bits back to the per path counters (in the sender). Even for drop, detecting if packets are lost, on which path they were lost seems already non-trivial.

Even with single packet sequence number space, is it not possible for both the sender and the receiver to detect the multiple paths, keep ECN counter context per path, and send these counters only for acks on that path.

Is it envisioned to allow acking packets on another path than on the path they were received (other than in exceptional conditions like after path failure)? In that case we would need a path id in the per path counters feedback. Also the impact for loss based congestion control would be hard to manage (needs a view in all paths for all paths, meaning acks need to be shared over all paths). Maybe received packet counter state per path will improve loss based congestion control too to make the processing of this more feasible...

Additionally, in the example mentioned before, indeed 30 packets fit in 8 bytes, but on a 1Gbps link with a 10ms RTT, 833 packets are in flight, and you need 833 bits. Does this mean we have to sacrifice 104 bytes per ack on such a path, and filter, map and count 833 bits per ack packet received?

Regards,
Koen.

-----Original Message-----
From: Lubashev, Igor [mailto:[email-masked]] 
Sent: Thursday, January 25, 2018 7:30 AM
To: Brian Trammell (IETF) <[email-masked]>
Cc: Christian Huitema <[email-masked]>; Ingemar Johansson S <[email-masked]>; QUIC WG <[email-masked]>; Bob Briscoe ([email-masked]) <[email-masked]>; De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>
Subject: RE: ECN in QUIC

Completely agree.

- Igor

---

From: Christian Huitema <[email-masked]>
Date: Thu, 25 Jan 2018 12:11:25 -1000

> Is the reason for this that there are currently no per path states/communications required? (because congestion control is currently out of scope, and it is currently the only per path processing r…

This is related to encryption options for multipath. I explained the
design choices in
https://datatracker.ietf.org/doc/draft-huitema-quic-mpath-req/. AEAD
encryption requires that sequence numbers do not repeat for a given
encryption context. For multipath, this requirement can be met by either
having separate encryption contexts per path, or having a single
encryption context for all paths and using the same sequence number
space for every paths. This is also a requirement when multiple paths
are used for connection migration.

We had that discussion in Melbourne, and the sense of the room was
clearly that having separate encryption contexts per paths was too
complicated. Having a single sequence number space and a single
encryption context is significantly simpler. There is a privacy issue,
linkability via the sequence numbers, but that privacy issue can be
addressed by encrypting the sequence numbers, and we pretty much agreed
to do that. Hence my statement that "We are heading towards a single
packet sequence number space shared by multiple paths".

Of course we still need to maintain per path variables for RTT, PMTU and
congestion control. As long as we only support migration, that could be
finessed somewhat by resetting these variables to their initial state at
the end of the migration. But when we do true multipath, there will be a
need to associate congestion control signals with paths. If we have a
single sequence number, the way to do that is to maintain associations
between packets and path, e.g. having a path tag in the packet copy kept
for retransmission purposes. That's sufficient for signals like ACK or
losses, but it requires that ECN signals be tied to specific packets.

-- Christian Huitema

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 26 Jan 2018 07:44:13 +0000

Hi

I believe that I start to understand the issue. Still I don't believe that it is any problem with using packet counters. As Koen pointed out it can simplify processing and this was also stated when this question was raised on the tracker (see https://github.com/quicwg/base-drafts/issues/805 ).

Assuming that we tag which packet goes where it will be possible to track how many packets that gone in which direction. Also the ECT/CE counters indicate the count of ECT/CE marked packets when the packet with the highest known sequence number is received. So if we keep separate "congestion control and transmission list" context for the different path then it should work and the correct congestion control will be achieved regardless it is ECT(0) or ECT(1) based congestion control.
It will also be possible to detect ECN bleach along new paths with this. 
  
/Ingemar

> Sent: den 25 januari 2018 23:11
> To: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-
> bell-labs.com>; Lubashev, Igor <[email-masked]>; Brian Trammell
> (IETF) <[email-masked]>
> Cc: Ingemar Johansson S <[email-masked]>; QUIC WG
> <[email-masked]>; Bob Briscoe ([email-masked])
> <[email-masked]>
> Subject: Re: ECN in QUIC
> 
> > Some questions related to the remark of Christian: "We are heading
> towards a single packet sequence number space shared by multiple paths"
> > Is the reason for this that there are currently no per path
> states/communications required? (because congestion control is currently
> out of scope, and it is currently the only per path processing required). In
> that case I understand why the current design of QUIC is optimizing itself
> towards application to application state, but we must make sure we don't
> make our live difficult if we have per connection state (when it is needed,
> like congestion control).
> 
> This is related to encryption options for multipath. I explained the design
> choices in https://datatracker.ietf.org/doc/draft-huitema-quic-mpath-req/.
> AEAD encryption requires that sequence numbers do not repeat for a given
> having separate encryption contexts per path, or having a single encryption
> context for all paths and using the same sequence number space for every
> paths. This is also a requirement when multiple paths are used for
> connection migration.
> 
> We had that discussion in Melbourne, and the sense of the room was clearly
> that having separate encryption contexts per paths was too complicated.
> Having a single sequence number space and a single encryption context is
> significantly simpler. There is a privacy issue, linkability via the sequence
> numbers, but that privacy issue can be addressed by encrypting the
> sequence numbers, and we pretty much agreed to do that. Hence my
> statement that "We are heading towards a single packet sequence number
> space shared by multiple paths".
> 
> finessed somewhat by resetting these variables to their initial state at the
> end of the migration. But when we do true multipath, there will be a need to
> associate congestion control signals with paths. If we have a single sequence
> number, the way to do that is to maintain associations between packets and
> path, e.g. having a path tag in the packet copy kept for retransmission
> purposes. That's sufficient for signals like ACK or losses, but it requires that
> ECN signals be tied to specific packets.
> 
> -- Christian Huitema
> 
>

---

From: Mike Bishop <[email-masked]>
Date: Sun, 28 Jan 2018 04:33:05 +0000

I may have missed the final state where this landed, but with Martin's reworking of the key schedule that includes the Connection ID in the key generation, wouldn't you automatically have a different encryption context any time you intentionally changed Connection IDs (which is likely when changing paths) anyway?

And if you don't change Connection ID, you're already linkable because of the CID, so you don't care if you're continuing to use the same sequence numbers.  (But we agreed to encrypt them anyway.)

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Christian Huitema
Sent: Thursday, January 25, 2018 2:11 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]>; Lubashev, Igor <[email-masked]>; Brian Trammell (IETF) <[email-masked]>
Cc: Ingemar Johansson S <[email-masked]>; QUIC WG <[email-masked]>; Bob Briscoe ([email-masked]) <[email-masked]>
Subject: Re: ECN in QUIC

> Is the reason for this that there are currently no per path states/communications required? (because congestion control is currently out of scope, and it is currently the only per path processing r…

This is related to encryption options for multipath. I explained the design choices in https://datatracker.ietf.org/doc/draft-huitema-quic-mpath-req/. AEAD encryption requires that sequence numbers do not repeat for a given encryption context. For multipath, this requirement can be met by either having separate encryption contexts per path, or having a single encryption context for all paths and using the same sequence number space for every paths. This is also a requirement when multiple paths are used for connection migration.

We had that discussion in Melbourne, and the sense of the room was clearly that having separate encryption contexts per paths was too complicated. Having a single sequence number space and a single encryption context is significantly simpler. There is a privacy issue, linkability via the sequence numbers, but that privacy issue can be addressed by encrypting the sequence numbers, and we pretty much agreed to do that. Hence my statement that "We are heading towards a single packet sequence number space shared by multiple paths".

Of course we still need to maintain per path variables for RTT, PMTU and congestion control. As long as we only support migration, that could be finessed somewhat by resetting these variables to their initial state at the end of the migration. But when we do true multipath, there will be a need to associate congestion control signals with paths. If we have a single sequence number, the way to do that is to maintain associations between packets and path, e.g. having a path tag in the packet copy kept for retransmission purposes. That's sufficient for signals like ACK or losses, but it requires that ECN signals be tied to specific packets.

-- Christian Huitema

---

From: Christian Huitema <[email-masked]>
Date: Sun, 28 Jan 2018 11:34:30 -1000

> And if you don't change Connection ID, you're already linkable because of the CID, so you don't care if you're continuing to use the same sequence numbers.  (But we agreed to encrypt them anyway.)

Martin presented his connection-ID based encryption proposal in
https://github.com/quicwg/wg-materials/blob/master/interim-18-01/grease.pdf.
The proposal is described in slide 10. The minutes should note that the
WG was less than enthusiastic. The consensus was that most benefits
claimed in slide 11 of that deck could be obtained in a simpler way, by
just encrypting the packet sequence number. People then started
designing encryption methods on the fly, which was fun but probably not
the best use of our time. I think that we asked a small design team to
come out with a packet sequence number encryption method, and that we
leaved open the issue of greasing the packet type.

Of course, you still need to use new connection-ID upon migration,
nothing changed there.

-- Christian Huitema