Subject: [tcpm] I-D Action: draft-ietf-tcpm-accurate-ecn-16.txt

From: [email-masked]
Date: Thu, 03 Feb 2022 06:43:49 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the TCP Maintenance and Minor Extensions WG of the IETF.

        Title           : More Accurate ECN Feedback in TCP
        Authors         : Bob Briscoe
                          Mirja Kühlewind
                          Richard Scheffenegger
	Filename        : draft-ietf-tcpm-accurate-ecn-16.txt
	Pages           : 60
	Date            : 2022-02-03

Abstract:
   Explicit Congestion Notification (ECN) is a mechanism where network
   nodes can mark IP packets instead of dropping them to indicate
   incipient congestion to the end-points.  Receivers with an ECN-
   capable transport protocol feed back this information to the sender.
   ECN was originally specified for TCP in such a way that only one
   feedback signal can be transmitted per Round-Trip Time (RTT).  Recent
   new TCP mechanisms like Congestion Exposure (ConEx), Data Center TCP
   (DCTCP) or Low Latency Low Loss Scalable Throughput (L4S) need more
   accurate ECN feedback information whenever more than one marking is
   received in one RTT.  This document specifies a scheme to provide
   more than one feedback signal per RTT in the TCP header.  Given TCP
   header space is scarce, it allocates a reserved header bit previously
   assigned to the ECN-Nonce.  It also overloads the two existing ECN
   flags in the TCP header.  The resulting extra space is exploited to
   feed back the IP-ECN field received during the 3-way handshake as
   well.  Supplementary feedback information can optionally be provided
   in a new TCP option, which is never used on the TCP SYN.  The
   document also specifies the treatment of this updated TCP wire
   protocol by middleboxes.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/

There is also an htmlized version available at:
https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-16

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-ecn-16

Internet-Drafts are also available by rsync at rsync.ietf.org::internet-drafts

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 3 Feb 2022 15:24:26 +0000

tcpm folks,

This rev to accurate-ecn is the first of two. The second will hopefully 
follow on its heels in the next couple of days.

Diffs between this first rev (-16) and -15:

1.  switches round two fairly large sections, so I've deferred other 
changes to a second rev so the diffs won't be masked by the switch round 
of sections.
Suggested by Ilpo to match the order in which the tests in these 
sections will be executed:
* Test for mangling the IP-ECN field (now 3.2.2.3),
* Then test for zeroing the ACE field (now 3.2.2.4).

2. Ilpo suggested some clarifications in "3.2.3.2.5.  Consistency 
between AccECN Feedback Fields", which is about the receiver of feedback 
ensuring consistency between the mandatory 3-bit ACE field and the 
optional 24-bit counters. In brief (paraphrasing) it previously only 
said "MUST consider both fields", when it is now clearer that it 
actually meant "MUST reconcile both fields", so that there is always a 
consistent baseline for subsequent ACKs.

3. A minor point is added in an appendix about the details that the 
pseudocode doesn't cover.

Bob

PS. #2 & #3 were added to the XML ages ago (Jul '21), so you will have 
seen them in the HTML. However, prob due to my clumsiness, the posted 
TXT didn't include them whereas the posted XML did (ironic for a section 
about consistency). In turn, inconsistency was only possible because I 
am having to manually post the TXT for this draft, due to an unresolved 
issue with v3 RFC XML tables.

On 03/02/2022 14:43, [email-masked] wrote:
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Vidhi Goel <[email-masked]>
Date: Thu, 03 Feb 2022 23:37:42 -0800

Hello Bob and authors,

I have read the draft a few times and will send my extensive review at a later point. But right now, I want to ask some critical things that an implementation could benefit from.

The draft says,
If a TCP client has set the SYN to Not-ECT, but
   receives feedback that the IP-ECN field on the SYN arrived with a
   different codepoint, it can detect such middlebox interference and
   send Not-ECT for the rest of the connection

1. On the forward path from client to server, the client will revert to Not-ECT when it sees for example, that it sent a Non-ECT SYN but received ACE encoding other than 0 1 0. What does the client do in the last ACK of the 3WHS -
 a. does it stay in AccECN mode and still send AccECN encoding based on the IP code point of SYN-ACK (Table 4)? This would mean that client won’t participate in ECN on the sender half of the connection and only provide AccECN feedback to the server as a receiver.
 b.  does it disable AccECN mode and set ACE= 0 0 0 so that a server in AccECN mode can disable ECN based on Table4?

While writing this, I realized that the intention is probably a. Could you confirm? Also, when the sender sets Not-ECT in its data packets, it should also disable acting upon any ACE feedback as we could still receive false ACE feedback from the server if the network, lets say, changed 00 (not ECT) to 01(ECT1). If you agree, we should add some text around this. Based on the current text, the sender will always respond to the ACE feedback even if it sends Not-ECT.

TBH, this is a complicated scenario, where sender said to the network - I don’t trust you so I can’t use ECN. Feel free to drop my packets. And the network mangles the IP to ECT1 and then set CE (when congested) which would be feedback’ed from the receiver. Now, the packet wasn’t dropped which it should have been. So, is it better to just ignore this feedback because sender doesn’t trust the network or just act on it and reduce cwnd in order to reduce congestion in the network somewhere.

2. On the reverse path from server to client, if a server sends a Not-ECT SYN-ACK and receives ACE handshake encoding on last ACK other than 0 1 0, there is no text like above that says server should send Not-ECT for the rest of the connection (or at least I didn’t find it). I think the server should also do same as client as the two paths could be different. One could make it more complicated by saying, if both client and server decide to not use ECN on their corresponding sender half, then ECN should be disabled but let’s talk about that later.

3. In general, not setting the ECT (ECT0 or ECT1) code point on an outgoing packet is different from supporting AccECN right as in the host can still provide AccECN feedback on the receive path.
….

To be continued if more questions come to my mind.

Thanks,
Vidhi

> On Feb 3, 2022, at 7:24 AM, Bob Briscoe <[email-masked]> wrote:
> 
> tcpm folks,
> 
> This rev to accurate-ecn is the first of two. The second will hopefully follow on its heels in the next couple of days.
> 
> 1.  switches round two fairly large sections, so I've deferred other changes to a second rev so the diffs won't be masked by the switch round of sections.
> Suggested by Ilpo to match the order in which the tests in these sections will be executed:
> 2. Ilpo suggested some clarifications in "3.2.3.2.5.  Consistency between AccECN Feedback Fields", which is about the receiver of feedback ensuring consistency between the mandatory 3-bit ACE field and the optional 24-bit counters. In brief (paraphrasing) it previously only said "MUST consider both fields", when it is now clearer that it actually meant "MUST reconcile both fields", so that there is always a consistent baseline for subsequent ACKs.
> 
> 3. A minor point is added in an appendix about the details that the pseudocode doesn't cover.
> 
> 
> 
> Bob
> 
> PS. #2 & #3 were added to the XML ages ago (Jul '21), so you will have seen them in the HTML. However, prob due to my clumsiness, the posted TXT didn't include them whereas the posted XML did (ironic for a section about consistency). In turn, inconsistency was only possible because I am having to manually post the TXT for this draft, due to an unresolved issue with v3 RFC XML tables.
> 
>

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 4 Feb 2022 13:44:03 +0000

Vidhi,

I've increased the table of contents to tocdepth = "4" which might help 
find the relevant sections better, because a lot of the meat in this 
draft is under sections 3.2.2 (ACE) and 3.2.3 (Option).

I've responded below, but I'd prefer it if you would say these things on 
the list, so we have a better justification for changing the draft. 
Reviewing a draft that has just be re-posted shouldn't imply anything 
about whether Apple is implementing the draft or not.

(BTW, regarding the proposed change to the initial value of r.e1b, I was 
going to send that  to the list once co-authors agreed to it. But it 
would be nice if some of the edits were not self-generated.)

See [BB] inline for responses (which I'll repeat if you send to the list).

On 04/02/2022 07:37, Vidhi Goel wrote:
> Hello Bob and authors,
>
> I have read the draft a few times and will send my extensive review at 
> a later point. But right now, I want to ask some critical things that 
> an implementation could benefit from.
>
> The draft says,
> /If a TCP client has set the SYN to Not-ECT, but/
> /   receives feedback that the IP-ECN field on the SYN arrived with a/
> /   different codepoint, it can detect such middlebox interference and/
> /   send Not-ECT for the rest of the connection/
>
> 1. On the forward path from client to server, the client will revert 
> to Not-ECT when it sees for example, that it sent a Non-ECT SYN but 
> received ACE encoding other than 0 1 0. What does the client do in the 
> last ACK of the 3WHS -
>  a. does it stay in AccECN mode and still send AccECN encoding based 
> on the IP code point of SYN-ACK (Table 4)? This would mean that client 
> won’t participate in ECN on the sender half of the connection and 
> only provide AccECN feedback to the server as a receiver.
>  b.  does it disable AccECN mode and set ACE= 0 0 0 so that a server 
> in AccECN mode can disable ECN based on Table4?
>
> While writing this, I realized that the intention is probably a. Could 
> you confirm? Also, when the sender sets Not-ECT in its data packets, 
> it should also disable acting upon any ACE feedback as we could still 
> receive false ACE feedback from the server if the network, lets say, 
> changed 00 (not ECT) to 01(ECT1). If you agree, we should add some 
> text around this. Based on the current text, the sender will always 
> respond to the ACE feedback even if it sends Not-ECT.
>
> TBH, this is a complicated scenario, where sender said to the network 
> - I don’t trust you so I can’t use ECN. Feel free to drop my packets. 
> And the network mangles the IP to ECT1 and then set CE (when 
> congested) which would be feedback’ed from the receiver. Now, the 
> packet wasn’t dropped which it should have been. So, is it better to 
> just ignore this feedback because sender doesn’t trust the network or 
> just act on it and reduce cwnd in order to reduce congestion in the 
> network somewhere.

[BB] Your points are all good ones.
I'll address this last complex scenario first, because it has 
implications for the earlier questions. First I'll define some terminology:

Simple mangling scenario:

  * some network function, e.g. broken Diffserv ToS-byte remarking,
    illegally remarks a Not-ECT  SYN to CE.

Your complex mangling scenario, repeated here:

  * some network function, e.g. broken Diffserv ToS-byte remarking,
    illegally remarks a Not-ECT SYN to ECT0 or ECT1,
  * then congestion at a subsequent bottleneck is marking some packets CE

Then potentially the client could tailor its behaviour after sending a 
Not-ECT SYN, If the SYN/ACK feedback is

 1. ECT: disable sending ECT, but continue responding to ECN feedback
 2. CE, disable sending ECT, and disable response to ECN feedback

However, this is uncertain, because CE feedback on the SYN/ACK could 
indicate either the simple or the complex mangling scenario.

A simpler alternative would be to always continue responding to ECN 
feedback. Rationale:

  * Whether case #1 or #2, assume that mangling of the SYN might have
    been Not-ECT to ECT, even if the feedback off the SYN is CE.
  * Then as the connection progresses, if /all/ feedback is CE, there's
    already a recommendation to fall-back to disabling congestion response.

If we do this, I think we ought to say "SHOULD continue to respond to 
ECN feedback", not "MUST".

And we'll need to put this all to the WG.

1.a) Now back to the beginning of your point. The text you quote is from 
§2.5 which is in the non-normative "Overview and Rationale" section (§2).
You really need the normative text from:

          3.2.2.3.
          <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing
          for Mangling of the IP/ECN Field
          <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>

where it says (end of 1st para):

    " ...for the remainder of the connection the client MUST NOT send
    ECN-capable packets, but it MUST continue to feed back any ECN
    markings on arriving packets."

So, you're right, the answer is (a).
It can't be (b), because, once in AccECN mode, 000 on the ACE field just 
becomes a counter value, and no longer negotiates the feedback mode.

1.b) As per the above, let's conservatively assume complex mangling. So, 
at the end of the first para quoted above I suggest we add:

    " ...for the remainder of the connection the client MUST NOT send
    ECN-capable packets *but**it**SHOULD continue to respond to ECN
    feedback even though **it **is no longer sending ECN-capable packets
    (see reasoning below)**. T**he client**MUST remain in the same
    feedback mode and***it MUST continue to feed back any ECN markings
    on arriving packets."

I'll do the same for next para about the server.
I'll work out some text for the reasoning, but I won't give it here.

There are two other cases where it disables sending ECT, which don't say 
whether it continues to respond to congestion:

  * "Testing for Mangling" section, penultimate para,  where it's
    receiving solid CE:
    "Once a Data Sender has entered AccECN mode it SHOULD check whether
    all feedback received for the first three or four rounds indicated
    that every packet it sent was CE-marked. If so, for the remainder of
    the connection, the Data Sender SHOULD NOT send ECN-capable
    packets*and it SHOULD NOT respond to ECN feedback*, but *it MUST
    remain in the same feedback mode and *it MUST continue to feed back
    any ECN markings on arriving packets*(in its role as Data Receiver)*."
  * Next section "Zeroing of the ACE Field", 2nd para:
    "If the value of this ACE field is zero (0b000), the Data Sender
    disables sending ECN-capable packets for the remainder of the
    half-connection by setting the IP/ECN field in all subsequent
    packets to Not-ECT, *but**it**SHOULD continue to respond to ECN
    feedback even though **it **is no longer sending ECN-capable
    packets.* *It MUST also remain in the same feedback mode and it MUST
    continue to feed back any ECN markings on arriving packets (in its
    role as Data Receiver).*"

I'll also add reasoning (in the zeroing section, it already says that 
ACE=0b000 is not an unambiguous indication of mangling).

And this prompts me to edit the bullet in an earlier section about the 
obligation of a sender to respond to congestion feedback:

        3.1.5.
        <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.1.5>Implications
        of AccECN Mode
        <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-implications-of-accecn-mode>

    "It is still obliged to respond appropriately to AccECN feedback
    that indicates there were ECN marks on *ECN-capable *packets it had
    previously sent."

And I'll add a bullet:

    "*If the sender chooses not to send ECN-capable packets (e.g.
    because path traversal of the ECN field is suspect), it can ignore
    any **ECN **feedback about those packets if it is certain that it
    cannot be valid (see Section 3.2.2, which gives normative
    requirements for certain specific cases).*"

I've said 'can' rather than 'MUST' because it's hard to cover all cases, 
e.g. single packets without ECT when it's not clear whether the feedback 
covered another packet that was ECT.

How does all this sound?

> 2. On the reverse path from server to client, if a server sends a 
> Not-ECT SYN-ACK and receives ACE handshake encoding on last ACK other 
> than 0 1 0, there is no text like above that says server should send 
> Not-ECT for the rest of the connection (or at least I didn’t find it). 
> I think the server should also do same as client as the two paths 
> could be different. One could make it more complicated by saying, if 
> both client and server decide to not use ECN on their corresponding 
> sender half, then ECN should be disabled but let’s talk about that later.

[BB] It's in the second para of:

          3.2.2.3.
          <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing
          for Mangling of the IP/ECN Field
          <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>

> 3. In general, not setting the ECT (ECT0 or ECT1) code point on an 
> outgoing packet is different from supporting AccECN right as in the 
> host can still provide AccECN feedback on the receive path.

[BB] Yup.

Do you think any further explanation is needed in the above sections?

Bob

>> This rev to accurate-ecn is the first of two. The second will 
>> hopefully follow on its heels in the next couple of days.
>>
>> changes to a second rev so the diffs won't be masked by the switch 
>> round of sections.
>> between AccECN Feedback Fields", which is about the receiver of 
>> feedback ensuring consistency between the mandatory 3-bit ACE field 
>> and the optional 24-bit counters. In brief (paraphrasing) it 
>> previously only said "MUST consider both fields", when it is now 
>> clearer that it actually meant "MUST reconcile both fields", so that 
>> there is always a consistent baseline for subsequent ACKs.
>>
>> PS. #2 & #3 were added to the XML ages ago (Jul '21), so you will 
>> have seen them in the HTML. However, prob due to my clumsiness, the 
>> posted TXT didn't include them whereas the posted XML did (ironic for 
>> a section about consistency). In turn, inconsistency was only 
>> possible because I am having to manually post the TXT for this draft, 
>> due to an unresolved issue with v3 RFC XML tables.
>>
>>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/

---

From: Vidhi Goel <[email-masked]>
Date: Fri, 04 Feb 2022 17:14:39 -0800

I like your response for all the points. I can review the diff for the proposed changes, if you’d like before committing them to the draft.

Sorry, I want to add a new comment for this text,
If a TCP client has set the SYN to Not-ECT, but
   receives feedback that the IP-ECN field on the SYN arrived with a
   different codepoint, it can detect such middlebox interference and
   send Not-ECT for the rest of the connection.

This statement holds good for Not-ECT but doesn’t if lets say the SYN was ECT0 or ECT1 as those could be marked CE and still be valid. Should we add a statement for ECT marked SYN as well?

Thanks,
Vidhi

> On Feb 4, 2022, at 5:44 AM, Bob Briscoe <[email-masked]> wrote:
> 
> Vidhi,
> 
> I've increased the table of contents to tocdepth = "4" which might help find the relevant sections better, because a lot of the meat in this draft is under sections 3.2.2 (ACE) and 3.2.3 (Option).
> 
> I've responded below, but I'd prefer it if you would say these things on the list, so we have a better justification for changing the draft. Reviewing a draft that has just be re-posted shouldn't imply anything about whether Apple is implementing the draft or not.
> 
> (BTW, regarding the proposed change to the initial value of r.e1b, I was going to send that  to the list once co-authors agreed to it. But it would be nice if some of the edits were not self-generated.)
> 
> I'll address this last complex scenario first, because it has implications for the earlier questions. First I'll define some terminology:
> 
> some network function, e.g. broken Diffserv ToS-byte remarking, illegally remarks a Not-ECT  SYN to CE.
> some network function, e.g. broken Diffserv ToS-byte remarking, illegally remarks a Not-ECT SYN to ECT0 or ECT1, 
> then congestion at a subsequent bottleneck is marking some packets CE
> Then potentially the client could tailor its behaviour after sending a Not-ECT SYN, If the SYN/ACK feedback is
> ECT: disable sending ECT, but continue responding to ECN feedback
> CE, disable sending ECT, and disable response to ECN feedback
> However, this is uncertain, because CE feedback on the SYN/ACK could indicate either the simple or the complex mangling scenario.
> 
> A simpler alternative would be to always continue responding to ECN feedback. Rationale: 
> Whether case #1 or #2, assume that mangling of the SYN might have been Not-ECT to ECT, even if the feedback off the SYN is CE.
> Then as the connection progresses, if /all/ feedback is CE, there's already a recommendation to fall-back to disabling congestion response.
> If we do this, I think we ought to say "SHOULD continue to respond to ECN feedback", not "MUST".
> 
> 1.a) Now back to the beginning of your point. The text you quote is from §2.5 which is in the non-normative "Overview and Rationale" section (§2). 
> 3.2.2.3.  <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing for Mangling of the IP/ECN Field <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>where it says (end of 1st para): 
> " ...for the remainder of the connection the client MUST NOT send ECN-capable packets, but it MUST continue to feed back any ECN markings on arriving packets."
> It can't be (b), because, once in AccECN mode, 000 on the ACE field just becomes a counter value, and no longer negotiates the feedback mode.
> 
> 1.b) As per the above, let's conservatively assume complex mangling. So, at the end of the first para quoted above I suggest we add:
> " ...for the remainder of the connection the client MUST NOT send ECN-capable packets but it SHOULD continue to respond to ECN feedback even though it is no longer sending ECN-capable packets (see reasoning below). The client MUST remain in the same feedback mode and it MUST continue to feed back any ECN markings on arriving packets."
> There are two other cases where it disables sending ECT, which don't say whether it continues to respond to congestion:
> "Testing for Mangling" section, penultimate para,  where it's receiving solid CE:
> "Once a Data Sender has entered AccECN mode it SHOULD check whether all feedback received for the first three or four rounds indicated that every packet it sent was CE-marked. If so, for the remainder of the connection, the Data Sender SHOULD NOT send ECN-capable packets and it SHOULD NOT respond to ECN feedback, but it MUST remain in the same feedback mode and it MUST continue to feed back any ECN markings on arriving packets (in its role as Data Receiver)."
> Next section "Zeroing of the ACE Field", 2nd para:
> "If the value of this ACE field is zero (0b000), the Data Sender disables sending ECN-capable packets for the remainder of the half-connection by setting the IP/ECN field in all subsequent packets to Not-ECT, but it SHOULD continue to respond to ECN feedback even though it is no longer sending ECN-capable packets. It MUST also remain in the same feedback mode and it MUST continue to feed back any ECN markings on arriving packets (in its role as Data Receiver)."
> I'll also add reasoning (in the zeroing section, it already says that ACE=0b000 is not an unambiguous indication of mangling).
> 
> And this prompts me to edit the bullet in an earlier section about the obligation of a sender to respond to congestion feedback:
> 3.1.5.  <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.1.5>Implications of AccECN Mode <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-implications-of-accecn-mode>
> "It is still obliged to respond appropriately to AccECN feedback that indicates there were ECN marks on ECN-capable packets it had previously sent."
> And I'll add a bullet:
> "If the sender chooses not to send ECN-capable packets (e.g. because path traversal of the ECN field is suspect), it can ignore any ECN feedback about those packets if it is certain that it cannot be valid (see Section 3.2.2, which gives normative requirements for certain specific cases)."
> I've said 'can' rather than 'MUST' because it's hard to cover all cases, e.g. single packets without ECT when it's not clear whether the feedback covered another packet that was ECT. 
> 
> How does all this sound?
> 
> 3.2.2.3.  <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3>Testing for Mangling of the IP/ECN Field <https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#name-testing-for-mangling-of-the>3. In general, not setting the ECT (ECT0 or ECT1) code point on an outgoing packet is different from supporting AccECN right as in the host can still provide AccECN feedback on the receive path.
> 
> [BB] Yup.
> 
>>> On Feb 3, 2022, at 7:24 AM, Bob Briscoe <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>> tcpm folks,
>>> 
>>> On 03/02/2022 14:43, [email-masked] <mailto:[email-masked]> wrote:
>>>> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/ <https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/>
>>>> 
>>>> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-16 <https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-16>
>>>> 
>>>> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-ecn-16 <https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-ecn-16>
>>>> 
>>>> 
>>>> [email-masked] <mailto:[email-masked]>
>>>> https://www.ietf.org/mailman/listinfo/tcpm <https://www.ietf.org/mailman/listinfo/tcpm>
>>> 
>>> -- 
>>> Bob Briscoe                               http://bobbriscoe.net/ <http://bobbriscoe.net/>
>>> 
>>> [email-masked] <mailto:[email-masked]>
>>> https://www.ietf.org/mailman/listinfo/tcpm <https://www.ietf.org/mailman/listinfo/tcpm>
>> 
> 
> -- 
> Bob Briscoe                               http://bobbriscoe.net/ <http://bobbriscoe.net/>

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Sat, 5 Feb 2022 13:01:19 +0100

Vidhi,

Thanks for reading the draft very closely!

Well, in the non-ECT SYN case, where the SYNACK indicates an ECT (ect0,
ect1, ce) is the obvious - and fully semantically correct - use case of
that possible use.

For the set of three codepoints (ECT0, ECT1, CE) the only other possible
valid detection scenario would be if a SYN+CE is sent (deliberately,
althought one can argue that a sender should not originate CE-marked
packets), but a SYNACK without CE is returned (full or partial
bleaching). At least under the current RFC3168 definition of IP ECN
codepoints.

allowed transitions:
nonECT -> nonECT

ECT0 -> ECT0
ECT0 -> ECT1
ECT0 -> CE

ECT1 -> ECT1
ECT1 -> ECT0 (possibly)
ECT1 -> CE

CE -> CE

Thinking about this aspect more, I am now unsure, if the higher level
functions, such as detection of IP ECN codepoint mangling, or validating
across multiple RTTs, should be in this document - or not rather be in a
separate draft, in order to facilitate independent implementation and
improvement of mechanisms in that space.

However, the above table of allowed IP ECN transistions may be updated
independently of AccECN - and performing extensive checks as part of a
signal protocol ossifies this IMHO.

As a middle ground, perhaps having these descriptions in a non-normative
section, clearly stating these to be examples of what an impemented
could choose to do?

Richard

Am 05.02.2022 um 02:14 schrieb Vidhi Goel:
> I like your response for all the points. I can review the diff for the
> proposed changes, if you’d like before committing them to the draft.
>
> This statement holds good for Not-ECT but doesn’t if lets say the SYN
> was ECT0 or ECT1 as those could be marked CE and still be valid. Should
> we add a statement for ECT marked SYN as well?
>
> Thanks,
> Vidhi
>
>> On Feb 4, 2022, at 5:44 AM, Bob Briscoe <[email-masked]
>> <mailto:[email-masked]>> wrote:
>>
>> Vidhi,
>>
>> I've increased the table of contents to tocdepth = "4" which might
>> help find the relevant sections better, because a lot of the meat in
>> this draft is under sections 3.2.2 (ACE) and 3.2.3 (Option).
>>
>> I've responded below, but I'd prefer it if you would say these things
>> on the list, so we have a better justification for changing the draft.
>> (BTW, regarding the proposed change to the initial value of r.e1b, I
>> was going to send that  to the list once co-authors agreed to it. But
>> it would be nice if some of the edits were not self-generated.)
>>
>>> I have read the draft a few times and will send my extensive review
>>> at a later point. But right now, I want to ask some critical things
>>> that an implementation could benefit from.
>>>
>>> The draft says,
>>> received ACE encoding other than 0 1 0. What does the client do in
>>> the last ACK of the 3WHS -
>>> on the IP code point of SYN-ACK (Table 4)? This would mean that
>>> client won’t participate in ECN on the sender half of the connection
>>> and only provide AccECN feedback to the server as a receiver.
>>> While writing this, I realized that the intention is probably a.
>>> Could you confirm? Also, when the sender sets Not-ECT in its data
>>> packets, it should also disable acting upon any ACE feedback as we
>>> could still receive false ACE feedback from the server if the
>>> network, lets say, changed 00 (not ECT) to 01(ECT1). If you agree, we
>>> should add some text around this. Based on the current text, the
>>> sender will always respond to the ACE feedback even if it sends Not-ECT.
>>>
>>   * Then as the connection progresses, if /all/ feedback is CE,
>>     there's already a recommendation to fall-back to disabling
>>     congestion response.
>>
>> 1.a) Now back to the beginning of your point. The text you quote is
>> from §2.5 which is in the non-normative "Overview and Rationale"
>> section (§2).
>> It can't be (b), because, once in AccECN mode, 000 on the ACE field
>> just becomes a counter value, and no longer negotiates the feedback mode.
>>
>> 1.b) As per the above, let's conservatively assume complex mangling.
>> So, at the end of the first para quoted above I suggest we add:
>>
>>     packets (see reasoning below)**. T**he client**MUST remain in the
>>     same feedback mode and***it MUST continue to feed back any ECN
>> There are two other cases where it disables sending ECT, which don't
>> say whether it continues to respond to congestion:
>>
>>     "Once a Data Sender has entered AccECN mode it SHOULD check
>>     whether all feedback received for the first three or four rounds
>>     indicated that every packet it sent was CE-marked. If so, for the
>>     remainder of the connection, the Data Sender SHOULD NOT send
>>     ECN-capable packets*and it SHOULD NOT respond to ECN feedback*,
>>     but *it MUST remain in the same feedback mode and *it MUST
>>     continue to feed back any ECN markings on arriving packets*(in its
>>     role as Data Receiver)*."
>>     packets.* *It MUST also remain in the same feedback mode and it
>>     MUST continue to feed back any ECN markings on arriving packets
>>     (in its role as Data Receiver).*"
>>
>>     that indicates there were ECN marks on *ECN-capable *packets it
>>     had previously sent."
>>
>> And I'll add a bullet:
>>
>> I've said 'can' rather than 'MUST' because it's hard to cover all
>> cases, e.g. single packets without ECT when it's not clear whether the
>> feedback covered another packet that was ECT.
>>
>> How does all this sound?
>>
>>> Not-ECT for the rest of the connection (or at least I didn’t find
>>> it). I think the server should also do same as client as the two
>>> paths could be different. One could make it more complicated by
>>> saying, if both client and server decide to not use ECN on their
>>> corresponding sender half, then ECN should be disabled but let’s talk
>>> about that later.
>>
>>>> posted TXT didn't include them whereas the posted XML did (ironic
>>>> for a section about consistency). In turn, inconsistency was only
>>>> possible because I am having to manually post the TXT for this
>>>> draft, due to an unresolved issue with v3 RFC XML tables.
>>>>
>>>>
>>>>> Internet-Drafts are also available by rsync at rsync.ietf.org
>>>>> <http://rsync.ietf.org>::internet-drafts
>>>>>
>>>>>

---

From: Vidhi Goel <[email-masked]>
Date: Sat, 05 Feb 2022 14:59:31 -0800

> section, clearly stating these to be examples of what an impemented

This draft already covers some of the path mangling detection so it would be good to cover all of it. I agree with your middle ground suggestion.

Vidhi

> On Feb 5, 2022, at 4:02 AM, Scheffenegger, Richard <[email-masked]> wrote:
> 
> Vidhi,
>

---

From: Bob Briscoe <[email-masked]>
Date: Sun, 6 Feb 2022 22:27:45 +0000

Richard,

Indeed, I recently discovered I had missed an item that I added to my 
ToDo list in Mar 2021, based on a conversation between Gorry and Mirja 
where they both made the same point - that fall-back behaviour after 
discovering ECN mangling ought to be non-normative in this draft, 
because it's beyond the scope of the new feedback mechanism:
https://mailarchive.ietf.org/arch/msg/tcpm/00URm1uwptgrAzKq96T6KSh8dQ4/

We do need to say the following normatively in "Testing for Mangling of 
the IP/ECN Field" because they are relevant to feedback, the Data Sender:
* remains in AccECN mode
* MUST continue to feed back any ECN markings on arriving packets.

BTW, Vidhi, the text you quoted was from §2 on "Protocol Overview and 
Rationale". That was meant to be an example of why it is useful for the 
handshake to give feedback on exactly which IP-ECN codepoint arrived. It 
wasn't meant to be a full spec. So I'll make it clearer that it's only 
an example, rather than try to make it comprehensive, which isn't 
appropriate at that point in the doc.

Nonetheless, your points are correct, but if they are dealt with anyway, 
they should be dealt with in §3.2.2.3. "Testing for Mangling of the 
IP/ECN Field" where these scenarios are already covered (except for the 
congestion response, which was silently left out of scope):
https://www.ietf.org/archive/id/draft-ietf-tcpm-accurate-ecn-16.html#section-3.2.2.3 

We could just say that the questions of whether to set ECT for 
subsequent packets, and whether to respond to feedback of CE markings 
are beyond the scope of the draft. I think we should, but I think we 
should also give some non-normative advice on what to do. It can only be 
interim advice, not normative requirements. Both because it's out of 
scope, and because the best behaviour and the alternatives will depend 
on deployment experience. So they shouldn't be written normatively in a 
Proposed Standard yet.

We will do what the WG wants, but assuming it agrees to including 
non-normative advice, following from your (Richard's) list of cases, 
here's a full matrix. I don't propose this matrix should go in the 
AccECN draft - it's just for this email, to check every case is covered.

	Feedback in SYN/ACK of SYN's IP-ECN
IP-ECN on SYN 	Not-ECT 	ECT(0) 	ECT(1) 	CE
Not ECT 	√ 	NR 	NR 	NR
ECT(0) 	ND 	√ 	√ 	√
ECT(1) 	ND 	CU
	√ 	√
CE 	n/a 	n/a 	n/a 	n/a

Key:
√       Valid response
NR    Non-normative advice: Set *N*ot-ECT for subsequent packets, but 
continue to *R*espond to CE feedback
ND    Non-normative advice: Set *N*ot-ECT for subsequent packets, and 
*D*isable response to CE feedback
CU    Currently unused transition (not listed as invalid in the draft)
n/a    not applicable in the AccECN spec, in which there is never a case 
where the sender sets CE when sending a packet.

In the draft we don't need to mention any cases where CE is sent (but a 
draft on fall-back probably would, because the sender might set CE to 
test the path). We can cover the other cases with two groups of invalid 
transitions: those where the SYN (or SYN-ACK) arriving at the receiver 
appears to have been
i) ECN-capable
ii) Not-ECN-capable.

I propose to also say that details (like what 'subsequent packets' 
implies about how long this behaviour continues) are for future study.

I'll try to write some text and propose it to this list. Perhaps it 
should be in a new appendix?

Bob

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Mon, 7 Feb 2022 00:00:52 +0100

Hi Bob,

could it be, that the first "data" row and column in this table are
parially transposed?

In the case of sending a Non-ECT SYN, but getting an indication of a
received CE on the SYN-ACK, why would you respond to that indication of
an incorrect CE marking? (With the response being, that the cwnd would
probably never grow beyond 1 or 2 packets per RTT, or possibly diminish
to n RTTs per packet, one a stack supports such low rates).

I am thinking of the most common probable cause - a middlebox
unconditionally overriding the full TOS byte, and simply setting both
rightmost bits continously.

Such a path would be rendered utterly unusable for an AccECN session, not?

So at least that case would be a ND in your nomenclature.

In the column - if the IP ECN is bleached, why disable response to CE
(if that is unlike to come back, as long as the session traverses the
bleaching path)? A "NR" seems more logical here in case of path changes?

But again, all these ins-and-outs should be discussed in a different draft.

I fully agree, that only the normative response of the AccECN feedback
mechanism has to be fully defined - the two bullet point is exactly what
I expect to be stated.

Regards,

    Richard

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 10 Feb 2022 16:53:42 +0000

Richard,see [BB] inline.

> So at least that case would be a ND in your nomenclature.

[BB] The full story for the first row (Not-ECT to ECT/CE transitions) 
was pointed out by Vidhi, and discussed in my previous response to her:

  * Not-ECT to CE might have been two transitions
      o Not-ECT to ECT at a broken ToS remarker (prob first hop);
      o ECT to CE at a subsequent bottleneck;
  * We can only assume that a ToS remarking transition will be
    consistent. The congestion part could be transient.
      o So /initially/ all cases where Not-ECT changes to anything else
        need to be treated as a change to ECT;
      o But then there's /another/ test, which disables response to CE
        if there's solid CE marking;
      o I.e. NR, that potentially becomes ND later.

> bleaching path)? A "NR" seems more logical here in case of path changes?

[BB] The thinking was that, if an ECT SYN becomes Not-ECT, then 
subsequently the sender only sends Not-ECT data, if that becomes CE, it 
seems pretty likely that is not a real congestion marking, whether 
there's a path change or not (because Not-ECT should remain Not-ECT).

> But again, all these ins-and-outs should be discussed in a different 

[BB] Yes, deployment experience is needed before anything definitive can 
be said about fallback, because there will be cases where the 
recommendation depends on which of a number of possible odd behaviours 
is most likely.

> mechanism has to be fully defined - the two bullet point is exactly what

[BB] Great. I'll write that up.

Bob