Subject: Re: [tcpm] New Version Notification fordraft-ietf-tcpm-accurate-ecn-11.txt

From: "Scharf, Michael" <[email-masked]>
Date: Tue, 10 Mar 2020 06:38:56 +0000

Yep, if one byte was added, there would probably be remaining bits left.

RFC 5218 Section 2.2.1: „Protocols that are extensible are more likely to be wildly successful in terms of being used for purposes outside their original design.“

Michael

Von: Ilpo Järvinen<mailto:[email-masked]>
Gesendet: Montag, 9. März 2020 23:32
An: Scharf, Michael<mailto:[email-masked]>
Cc: Bob Briscoe<mailto:[email-masked]>; tcpm IETF list<mailto:[email-masked]>; Richard Scheffenegger<mailto:[email-masked]>; Mirja Kuehlewind<mailto:[email-masked]>
Betreff: Re: [tcpm] New Version Notification fordraft-ietf-tcpm-accurate-ecn-11.txt

Hi Michael,

I agree with out  that it's somewhat painful to deal with the marker bit
at the beginning for the option field order.

Another challenge I've come across is to correctly reproduce the
Accurate ECN state at the sender which would benefit from having
a few extra bits available.

It currently requires guessing at every option arrival what the receiver
is doing next (what counter it will be increasing next) which is not very
robust to ACK losses. On change-triggered ACK, two byte counters are
increasing rather than one. ACK losses will easily confuse approach that
just assumes back-and-forth transitions. Thus, I suggested sending one
extra option after the change-triggered ACK to allow sender to see
unambiguous counter delta between the change-triggered ACK and that
next ACK (it is now in -11).

To allow the sender to immediately sync with the receiver on any
AccECN option arrival, it would make sense to encode that information
into the option itself. It would need 2 extra bits into the option
so adding that one byte there would help. ...It might be possible to
combine it to field order and define that the first encoded field is
the currently increasing counter (the order of the remaining two fields
still need to be addressed somehow).

--
 i.

On Mon, 9 Mar 2020, Michael Scharf wrote:

>
> With chair hat off, I really wonder if the solution to encode the two
> different orders in the TCP Option is an example for good and robust
> protocol engineering.
>
>
>
> For instance, the current design makes it hard to decode the field in a
> monitoring tool (such as Wireshark). Also, as far as I understand, it does
> not allow to switch the encoding during a connection, which limits
> flexibility. We almost certainly do not understand *now* all future use
> cases of this Standard.
>
>
>
> Unless I miss something, there would be several other solutions:
>
>
>
> First, IMHO, we have enough TCP option codepoints left to spend two
> codepoints if there is a good reason for doing so. As compared to the
> current design proposal in -10/-11, spending two different option kinds
> would look to me like *much* better protocol engineering.
>
>
>
> Second, if the TCPM community insists in only one option kind codepoint for
> whatever reason, IMHO one could add one „sub-type“ byte to the option. The
> TCP Option field has to be multiples of 4 byte, i.e., if a segment only
> contains a 11 byte AccECN TCP option, an additional NOP TCP option is needed
> for padding, no? So, what downside have 12 bytes as compared to 11 bytes?
> For the shorter variants, the overhead of a „sub-type“ field increases, but
> it may still be within reasonable limits. What do I miss?
>
>
>
> Third, one could use different lengths for the different orders, e.g.,
> lenths 5/8/11 for type 0 and 6/9/12 for type 12. Is this not possible?
>
>
>
> In all these cases, the resulting protocol looks simpler and more robust to
> me. What prevents us from using the KISS principle?
>
>
>
> Michael
>
>
>
>
>
>
>
> Von: Bob Briscoe
> Gesendet: Freitag, 6. März 2020 04:34
> An: tcpm IETF list
> Cc: Richard Scheffenegger; Mirja Kuehlewind
> Betreff: Re: [tcpm] New Version Notification for
> draft-ietf-tcpm-accurate-ecn-11.txt
>
>
>
> tcpm,
>
> You will have seen draft-10 then draft-11 in quick succession, as already
> explained.
> The diffs from draft-09 to -10 were those that had built up since Jul'19.
> The diffs from draft-10 to -11 were solely those for the change from EXP
> track to STD track.
> Draft-10 doesn't seem to display in the list of links to each version, but
> you can manually write the URL.
>
> The main technical changes in draft-10 were numerous - many will be
> recognized from list discussion since Jul'19.
> Particular thanks to Ilpo Järvinen who identified many niggles (and their
> solutions) while writing and testing a full Linux implementation (based on
> Olivier Tilmans's, in turn based on Mirja's).
>
>  *  Allowed 2 different orders of the fields in the AccECN Option
>  *  Reflect IP-ECN field of SYN/ACK only on ACK of SYN/ACK, not also on
>     first data packet
>      +  greatly simplifies implementation, esp with TFO.
>      +  repeating on first data packet was for reliable delivery, which is
>         now achieved with ACE counter (see next bullet)
>  *  Increment the ACE counter if CE on SYN/ACK (but still not if CE on SYN)
>      +  Reliable delivery of feedback of CE on SYN/ACK
>  *  Redefine 'first packet' as first to arrive, not first in sequence in 2
>     cases:
>      +  Handshake reflection on the ACK of the SYN/ACK
>      +  In the test for zeroing of ACE
>      +  Reason: greatly simplifies implementation
>  *  if ACE could have wrapped more than once, SHOULD assume “safest likely
>     case”
>     not "conservatively assume" it did cycle
>      +  Reason: avoid unnecessary hit on performance
>  *  More robustness (with flexibility) in rules for when to include an
>     AccECN Option
>      +  Change-triggered AccECN Option as SHOULD, not MUST
>      +  SHOULD follow change-triggered AccECN Option with another (removes
>         ambiguity if ACK thinning or loss)
>      +  when same counter continues to increment, SHOULD consistently
>         include it every n ACKs
>      +  Made rule about precedence of SACK conditional (max 2 SACK blocks)
>      +  MAY exclude counters that have not changed for the whole connection
>  *  Allowed an AccECN server not to implement RFC3168 ECN (all clients still
>     have to)
>  *  Precluded mixed capability negotiation from either end
>      +  reduces freedom to choose SYN & SYN/ACK fall-back strategies
>      +  to prevent cases where each end's outcome after handshake could be
>         inconsistent (in reordering corner-cases)
>  *  Reserved the codepoint combination used by the historic nonce case
>  *  Merged in a number of points from RFC3168 that we hadn't covered
>      +  (a whole new subsection about obligations to do with ECN)
>  *  Explicit about checking "acceptable packets"
>      +  before counting their ECN markings or before counting the ECN
>         feedback they carry
>  *  Required retransmitted Fallback SYN to use same ISN
>      +  allows servers to detect ECN downgrade SYN attacks
>  *  Handled corner cases like In-window SYN during TIME-WAIT
>
>
>
> Bob
>
> On 06/03/2020 02:24, [email-masked] wrote:
>
>
>
> A new version of I-D, draft-ietf-tcpm-accurate-ecn-11.txt
>
> has been successfully submitted by Bob Briscoe and posted to the
>
> IETF repository.
>
>
>
> Name:            draft-ietf-tcpm-accurate-ecn
>
> Revision: 11
>
> Title:           More Accurate ECN Feedback in TCP
>
> Document date:   2020-03-05
>
> Group:           tcpm
>
> Pages:           58
>
> URL:            https://www.ietf.org/internet-drafts/draft-ietf-tcpm-accurat
> e-ecn-11.txt
>
> Status:         https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ec
> n/
>
> Htmlized:       https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11
>
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accura
> te-ecn
>
> Diff:           https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-e
> cn-11
>
>
>
> Abstract:
>
>    Explicit Congestion Notification (ECN) is a mechanism where network
>
>    nodes can mark IP packets instead of dropping them to indicate
>
>    incipient congestion to the end-points.  Receivers with an ECN-
>
>    capable transport protocol feed back this information to the sender.
>
>    ECN is specified for TCP in such a way that only one feedback signal
>
>    can be transmitted per Round-Trip Time (RTT).  Recent new TCP
>
>    mechanisms like Congestion Exposure (ConEx), Data Center TCP (DCTCP)
>
>    or Low Latency Low Loss Scalable Throughput (L4S) need more accurate
>
>    ECN feedback information whenever more than one marking is received
>
>    in one RTT.  This document specifies a scheme to provide more than
>
>    one feedback signal per RTT in the TCP header.  Given TCP header
>
>    space is scarce, it allocates a reserved header bit, that was
>
>    previously used for the ECN-Nonce which has now been declared
>
>    historic.  It also overloads the two existing ECN flags in the TCP
>
>    header.  The resulting extra space is exploited to feed back the IP-
>
>    ECN field received during the 3-way handshake as well.  Supplementary
>
>    feedback information can optionally be provided in a new TCP option,
>
>    which is never used on the TCP SYN.
>
>
>
>
>
>
>
>
>
>
> Please note that it may take a couple of minutes from the time of submission
>
> until the htmlized version and diff are available at tools.ietf.org.
>
>
>
> The IETF Secretariat
>
>
>
>
>
>
>
>

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 10 Mar 2020 23:37:10 +0000

Ilpo,

Can you please describe the idea more fully? I cannot understand it from 
the description, although I thought I understood it when you explained 
it to me (off list).

Here's my understanding of the problem, for the benefit of the list 
(Ilpo and I have been discussing this off list, and I don't think the 
list will understand without this):

* This is all in the context of a Receiver that is sending AccECN TCP 
Options. It doesn't have to send these options at all but, if it does, 
they complement and help disambiguate the 3-bit ACE counter in the main 
TCP header (which is always used).
* Further context: Even if a receiver is sending the AccECN Option, it 
can choose to sometimes send ACKs without an AccECN Option, and the 
sender can then assume that all the bytes ACK'd had the same ECN marking 
(perhaps wrongly if there have been ACK losses).
* The proposed rules for when to use the AccECN Option are here:
https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11#section-3.2.3.3

The first rule is:

    Change-Triggered ACKs:  If an arriving packet increments a different
       byte counter to that incremented by the previous packet, the Data
       Receiver SHOULD immediately send an ACK with an AccECN Option,
       without waiting for the next delayed ACK [...].

       Even though this bullet is stated as a "SHOULD", it is important
       for a transition to immediately trigger an ACK if at all possible,
       ...

Originally Ilpo thought this made the sender implementation too 
difficult, even in very common delayed ACK cases, e.g.:

     ECT1
ACK#1 (ECT1 counter)
     ECT1
     CE
ACK#2 (ECT1 & CE counters)
     ECT1
ACK#3 (ECT1 counter)
     ECT1
     ECT1
ACK#4 (no AccECN Option)
     ECT1
     ECT1
ACK#5 (no AccECN Option)
...

In ACK#2, both ECT1 and CE counters have incremented, so how does the 
sender know which counter triggered this ACK? it needs to know this to 
guess which counter to increment if it gets an ACK without an AccECN Option.
My answer: the sender can infer that CE triggered the second ACK shown, 
because in the previous ACK, only ECT1 was incrementing.

Ilpo was concerned that, in the case where ACK#3 is lost or thinned, the 
sender could assume from the subsequent lack of any AccECN Options that 
more and more CE was arriving, when actually more and more ECT1 was 
arriving.

I agreed, but my argument was that there are more "SHOULD" rules for the 
receiver, such as:

       *  It SHOULD always include an AccECN Option if the r.ceb counter
          is incrementing and it MAY include an AccECN Option if r.ec0b
          or r.ec1b is incrementing

So, certainly it is possible that there will be ambiguous occasions when 
there is no AccECN Option. However, the idea is not to omit an AccECN 
option at all unless there is no space (e.g. because some other option 
takes priority). So these cases where assumptions have to be made are 
not intended to be routine - the rules are only written as SHOULDs in 
order to give the receiver flexibility if it needs the space to send 
other options. it is not expected to break all the SHOULD rules all the 
time.

Nonetheless, it did make sense to add this rule, at Ilpo's suggestion:

       *  It SHOULD include a counter that has continued to increment on
          the next scheduled ACK following a change-triggered ACK;

For example, ACks #3 and #5 below are due to this rule:

     ECT1
ACK#1 (ECT1 counter)
     ECT1
     CE
ACK#2 (ECT1 & CE counters)
     CE
     CE
ACK#3 (CE counter)
     ECT1
ACK#4 (ECT1 counter)
     ECT1
     ECT1
ACK#5 (ECT1 counter)
     ECT1
     ECT1
ACK#6 (no AccECN Option)
...

But I pointed out that, of course, this doesn't fully solve the problem, 
because this second AccECN Option could be lost, just as easily as the 
first (obviously a lower probability that they are both lost, but this 
could be common with heavy ACK thinning).

It was at this point that Ilpo suggested that a 2-bit field in a 1-byte 
option could specify which was the last counter to have changed.

However, now I re-read both Ilpo's offlist and the onlist emails, I'm 
not sure I understand the rules of the scheme, whereas I thought I did.

Ilpo, can you write a really precise and full spec of your proposal, 
stating all assumptions?
Are ACKs without any AccECN Option allowed? Or are you saying this 1 
byte option MUST always be included?

Bob

On 10/03/2020 06:38, Scharf, Michael wrote:
> RFC 5218 Section 2.2.1: „Protocols that are extensible are more likely 
> to be wildly successful in terms of being used for purposes outside 
> their original design.“
>
> Michael
>
> *Von: *Ilpo Järvinen <mailto:[email-masked]>
> *Gesendet: *Montag, 9. März 2020 23:32
> *An: *Scharf, Michael <mailto:[email-masked]>
> *Cc: *Bob Briscoe <mailto:[email-masked]>; tcpm IETF list 
> <mailto:[email-masked]>; Richard Scheffenegger 
> <mailto:[email-masked]>; Mirja Kuehlewind 
> <mailto:[email-masked]>
> *Betreff: *Re: [tcpm] New Version Notification 
> fordraft-ietf-tcpm-accurate-ecn-11.txt
>
> Hi Michael,
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Ilpo Järvinen" <[email-masked]>
Date: Wed, 11 Mar 2020 14:37:52 +0200 (EET)

Hi all,

Here's the option format with the extra octet:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Kind = TBD1  |  Length = 11  | Reserved  |Cnt|   EE0B field  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | EE0B (cont'd)                 |           ECEB field          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |                  EE1B field                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Cnt indicates the counter (IP-ECN field value) from the last received
packet. The receiver incremented this counter with this ACK. The counter
value might have not increased if the payload length was zero but
a subsequent packet would increase the same counter if IP-ECN field
remains the same.

The octet containing Cnt MUST be included when any byte counter is
included. It MAY be left out otherwise. If an AccECN option without bytes 
counters is increased but Cnt value is included and changed, the sender 
switches to the indicated counter (but cannot sync counters which may be 
be somewhat off in case of ACK losses).

If the Option is sent in a change-triggered ACK, the Cnt field will
differ from the previously sent value. For other ACKs the option, Cnt is 
the same as previously. As a new AccECN options should be sent in
the next change-triggered ACK, the IP-ECN field edges are captured.

Now, even if there were any number of ACK losses prior to receiving
such an AccECN option, the sender can immediately sync with the receiver's 
state and knows what counter the receiver will be increasing, no need to 
guess that based on any byte counter delta (on the next ACK if it 
doesn't contain AccECN option).

The main point with Cnt is to help the sender to do more accurate 
estimation in the cases where ACKs without AccECN option arrive.
As such estimation needs only run on ACKs, the Cnt information does
not require sending extra ACKs at any point of time. In fact, the
change-triggered ACKs requirement could even be relaxed with it
because together the byte counters and Cnt sync the AccECN state
such that knowing the exact point of the IP-ECN field edge is no
longer as necessary as earlier.

If Cnt is zero, the receiver got a packet with non-ECT and no counter
is increasing as a result of the ACK or subsequent ACKs until next
the AccECN option arrival.

Since the byte counter indicated with Cnt field is typically increasing,
it would be possible to make the Cnt value dual purpose and use it to
indicate which counter appears first in the Option (if there are any
counters in the first place). This would partically resolve Michael's
concern about the field ordering. Other counters that have changed since 
the receiver sent last AccECN option should be included after the
first field (in some order, specifying the order of the remaining
fields is beyond this Cnt proposal).

--
 i.

On Tue, 10 Mar 2020, Bob Briscoe wrote:

> Ilpo,
> 
> Can you please describe the idea more fully? I cannot understand it from the
> description, although I thought I understood it when you explained it to me
> (off list).
> 
> Here's my understanding of the problem, for the benefit of the list (Ilpo
> and I have been discussing this off list, and I don't think the list will
> understand without this):
> 
> Options. It doesn't have to send these options at all but, if it does, they
> complement and help disambiguate the 3-bit ACE counter in the main TCP
> header (which is always used).
> * Further context: Even if a receiver is sending the AccECN Option, it can
> choose to sometimes send ACKs without an AccECN Option, and the sender can
> then assume that all the bytes ACK'd had the same ECN marking (perhaps
> wrongly if there have been ACK losses).
> Originally Ilpo thought this made the sender implementation too difficult,
> even in very common delayed ACK cases, e.g.:
> 
>     ECT1
> ACK#1 (ECT1 counter)
>     ECT1
>     CE
> In ACK#2, both ECT1 and CE counters have incremented, so how does the sender
> know which counter triggered this ACK? it needs to know this to guess which
> counter to increment if it gets an ACK without an AccECN Option.
> sender could assume from the subsequent lack of any AccECN Options that more
> and more CE was arriving, when actually more and more ECT1 was arriving.
> 
> there is no AccECN Option. However, the idea is not to omit an AccECN option
> at all unless there is no space (e.g. because some other option takes
> priority). So these cases where assumptions have to be made are not intended
> to be routine - the rules are only written as SHOULDs in order to give the
> receiver flexibility if it needs the space to send other options. it is not
> expected to break all the SHOULD rules all the time.
> 
> because this second AccECN Option could be lost, just as easily as the first
> (obviously a lower probability that they are both lost, but this could be
> common with heavy ACK thinning).
> 
> However, now I re-read both Ilpo's offlist and the onlist emails, I'm not
> sure I understand the rules of the scheme, whereas I thought I did.
> 
> 
> Ilpo, can you write a really precise and full spec of your proposal, stating
> all assumptions?
> Are ACKs without any AccECN Option allowed? Or are you saying this 1 byte
> option MUST always be included?
> 
> 
> 
> Bob
> 
>       RFC 5218 Section 2.2.1: „Protocols that are extensible are more
>       likely to be wildly successful in terms of being used for
>       purposes outside their original design.“
> 
>        
> 
>       Michael
> 
>        
> 
>       Von: Ilpo Järvinen
>       An: Scharf, Michael
>       Cc: Bob Briscoe; tcpm IETF list; Richard Scheffenegger; Mirja
>       Kuehlewind
> > URL:           
> https://www.ietf.org/internet-drafts/draft-ietf-tcpm-accurat
> 
> > e-ecn-11.txt
> 
> >
> 
> > Status:        
> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ec
> 
> > n/
> 
> >
> 
> > Htmlized:      
> https://tools.ietf.org/html/draft-ietf-tcpm-accurate-ecn-11
> 
> >
> 
> > Htmlized:      
> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accura
> 
> > te-ecn
> 
> >
> 
> > Diff:          
> https://www.ietf.org/rfcdiff?url2=draft-ietf-tcpm-accurate-e
> 
> > cn-11
> 
> >
> 
> >  
> 
> >
> 
> > Abstract:
> 
> >
>

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 12 Mar 2020 16:14:59 +0000

Ilpo,

On 11/03/2020 12:37, Ilpo Järvinen wrote:
> included.
I don't think this needs to be a MUST at all. The most important thing 
is to include the Cnt bits when there are no other counters (the case in 
the sentences below).
> It MAY be left out otherwise.
I think you are referring to an empty option on the SYN/ACK here (to 
declare that the server is sending options). I can't think of any other 
use for an empty option.

> If an AccECN option without bytes
This last sentence is the most important part that you hadn't stated before.
     (I assume there's a typo: s/increased/included/.)
But you haven't said whether it's MUST or SHOULD (except the previous 
sentence says that it MAY be left out).
It cannot be a MUST (in case there isn't any option space on a 
particular packet).
If we were to go in this direction, this sentence surely ought be a 
"SHOULD" with the same strength as the "SHOULD" for Change-Triggered ACKs.

> state
I think you only mean the receiver's state relating to which ACK counter 
is currently changing. The state of the counters can still be wrong, due 
to previous ACK losses.

> and knows what counter the receiver will be increasing, no need to
I believe you are saying that the meaning of Cnt on an acceptable ACK is 
twofold:
1) For the past:
     * If the byte counter indicated by Cnt is absent from this option, 
consider the new bytes acked by this packet to have incremented that 
counter.
     * If the byte counter indicated by Cnt is present on this option 
??? (Does it have any other use for the past?)
2) For the future:
     * If there is no AccECN Option on a subsequent acceptable ACK that 
acknowledges new data, consider the new bytes acked by that packet to 
have incremented the byte counter indicated by the last seen value of Cnt.

In the presence of ACK loss:
* Meaning#1 is much more useful.
* Meaning #2 is less useful, because if ACKs are being thinned or lost, 
the value of Cnt most recently received by the Data Sender won't 
necessarily be the one most recently sent by the Data Receiver.

Assessment:
* Meaning #2: To put it ironically, this scheme tries to solve the 
problem of losing ACKs that say which counter will change in future, by 
introducing a field on an ACK that says which counter will change in 
future. But the ACK carrying this new field is just as likely to be lost 
as the old way.

* Meaning #1: Then the question has to be, if Cnt is most useful on a 
packet without the associated counter (meaning#1), then why not include 
that counter? IOW, Cnt consumes 3B to save 5B (or sometimes to save 8B) 
(including the overhead of the option kind and length). And note that 
Cnt is still ambiguous, whereas including the actual counter is unambiguous.

I guess the idea of Cnt arose from reading all the SHOULDs about when to 
send an AccECN Option as if they might not be adhered to. So this 
additional 1B field is really only solving a problem where there is not 
space for 5B or 8B, but there is space for 3B.

> the AccECN option arrival.
More precisely, I think you mean that all the new bytes ACK'd by this 
packet were Not-ECT.
That's useful. Without Cnt this is only really possible by sending an 
option with all three counters then sending one later with the counters 
unchanged.

> fields is beyond this Cnt proposal).
As you say, we don't have to do this.
I'm yet to be convinced of the value of allowing the field order to 
change on each packet, just because we want the field order to be able 
to be changed for each connection.

Bob

---

From: "Ilpo Järvinen" <[email-masked]>
Date: Fri, 20 Mar 2020 23:44:09 +0200 (EET)

> is to include the Cnt bits when there are no other counters (the case in 

Well, the point was just to say that if byte counters are there, then the 
Cnt must be there too. Any discussion below from my side about the case
where there's just Cnt but no associated byte counter is written just
in an attempt to "spec it fully" (and should be read in that light).

> declare that the server is sending options). I can't think of any other 

You asked me to answer whether it's always sent so I did try answer that
with this. The point was to say that Cnt needs to not be sent if byte 
counters are not sent nor does Cnt require option to be always sent which 
I forgot to explicitly state.

I just tried to also collect my thoughts about the cases where only Cnt 
appear but not byte counters although that was not the main idea (just to
"fully spec it"). But while writing them down I realized that in (nearly) 
ACK lossless environments the Cnt alone could indicate the IP-ECN edges
which probably affected the text too much and just created more confusion.

>      (I assume there's a typo: s/increased/included/.)

Yes, although I'd say "received" now that you note it.

> "SHOULD" with the same strength as the "SHOULD" for Change-Triggered ACKs.

Yes, the intention was to say that Cnt MUST be included to ACKs that have 
any byte counters. And therefore "SHOULD" be included to change-triggered 
ACKs.

The sender receiving Cnt MUST use the newest Cnt value for estimating 
which byte counter is incrementing on a subsequent ACK if it contains no 
AccECN option.

> is currently changing. The state of the counters can still be wrong, due 

...If the byte counters are not included, that may occur due to ACK 
losses, yes. The delta will be correct though.

In my mind model, however, byte counters and appearance of Cnt are more 
tightly coupled than you seem to think. It is possible to see/send Cnt 
alone but that never was my intention to endorse (but as mentioned above,
I can see some use case for Cnt to appear alone).

> have incremented the byte counter indicated by the last seen value of Cnt.

I was only trying to solve case #2.

> necessarily be the one most recently sent by the Data Receiver.

I disagree this evaluation, #2 is the reason why I proposed this.
The cases related to #1 are just things that are a side-product of
not being able to fit the Cnt and byte counters always to the limited
option space.

If ACKs are lost, Cnt definately is not "less useful". With Cnt the 
situation is "equal or better". ACK+Option losses will obviously have an 
impact on any estimation heuristic. Cnt+byte counters, however, gets
immediately back on track when ACK+Option gets through at some point after 
the loss, unlike the other approaches that so far have been on the table.
I cannot see how estimates from the other approaches could be "more 
useful" than Cnt assisted approach in the ACK thinning / ACK loss cases.

> future. But the ACK carrying this new field is just as likely to be lost 

Of course but when ACK with Cnt+byte counters get through, the state is 
immediately brought up to date and the sender knows how to continue on 
the next ACK if that doesn't contain AccECN option.

> Cnt is still ambiguous, whereas including the actual counter is unambiguous.

No, from estimation point of view (on the next ACK without AccECN option) 
Cnt is unambiguous whereas counters aren't. If you bring ACK losses 
again on the table here, then you should explain how the other estimation 
approach would outperform because I cannot see how they could less 
ambiguous (other than by chance). But I guess your comment mainly stemmed 
from misconcept that I was proposing to send Cnt w/o byte counters.

> space for 5B or 8B, but there is space for 3B.

No. The idea of Cnt came from the requirement to include AccECN not only 
on Change-Triggered ACKs but the next ACK too to more robustly communicate 
which counter is increasing. You were somewhat against consuming those 
extra bytes for sending the option second time.

> option with all three counters then sending one later with the counters 

Agreed, for this case it would also provide slightly more information 
than the counters alone.

> to be changed for each connection.

I came across recently something related to this when creating a 
packetdrill tests for shrinking the option size due to other options.
The receiver couldn't send the AccECN option because all modified counters 
didn't fit. In the test was just "randomly" using ECT0, CE, ECT1 packets 
to create edges while SACK blocks indicated holes, alignment+TS+SACK 
blocks ate so much space that the last byte counter (ECT1 in this cases) 
no longer fit to the option so AccECN degrades to optionless mode until 
SACK blocks are no longer required (typically, that would be until 
recovery ends). Maybe not a big problem for use cases with only two ECN IP 
field values but perhaps still worth to consider if something needs to be 
done or not.

--
 i.

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 20 Mar 2020 23:18:14 +0000

Ilpo,

> in an attempt to "spec it fully" (and should be read in that light).

[BB] I don't think you've realized the full power of your own idea.
See later about 'past' and 'future'.

> I forgot to explicitly state.

[BB] If we're going to have this new byte, there is no justification for 
saying "Cnt needs to not be sent if byte
counters are not sent".

Certainly the reason counters are not sent might be lack of space for 6B 
or 9B, in which case there might not be space for 3B option either. But 
if there is space for 3B, including just the Cnt field helps a lot.

> which probably affected the text too much and just created more confusion.

[BB] The point is, if you have a counter in an option, you don't need to 
guess what that counter is (because it's there!). Cnt is only to give 
the algo knowledge of which counter will update in future, in case the 
Data Sender gets future ACKs without an AccECN option, or possibly with 
an AccECN option, but with only one{Note 1} counter.

{Note 1}: Strictly, I should say "with only N counters, where N is less 
than the number of counters that have changed at some point during the 
connection".

> which byte counter is incrementing on a subsequent ACK if it contains no

[BB] if we use this scheme, I don't think we will need to make it 
mandatory how to /use/ Cnt. The behaviour that interop depends on is the 
conditions for sending Cnt, not how it's used. That can just be "The 
sender receiving Cnt can use ..."

> I was only trying to solve case #2.

[BB] #1 is for when time reaches the point in the future that #2 
prepared for in the past.

> not being able to fit the Cnt and byte counters always to the limited

[BB] Yes, I know that. But that doesn't mean the side product isn't more 
useful than you first realized.

> useful" than Cnt assisted approach in the ACK thinning / ACK loss cases.

[BB]
#2 (future) is vulnerable to losing an ACK in the future that might 
state a different value of Cnt.
#1 (past) is vulnerable to having lost an ACK since the last value of 
Cnt, that might have stated a different value of Cnt.

However, both together give more certainty. If Cnt is the same on two 
subsequent packets (even if there's no associated counter on the later 
one), it reduces the chance that there were ACK loss(es) in between that 
might have changed Cnt (there would have had to be >1 ACK loss that 
changed it then changed it back again.

If Cnt is different on two subsequent packets, but the new counter 
indicated by Cnt has not been shown since that last packet, at least it 
reveals that there must have been an intermediate ACK with that counter, 
that must have been lost.

> the next ACK if that doesn't contain AccECN option.

[BB] Surely, if counters get through, the state can immediately be 
brought up to date anyway, without needing Cnt.
So, if both (or all) counters are included, Cnt is actually only useful 
for the future (which soon becomes the past, obviously).

> Cnt is unambiguous whereas counters aren't.

[BB] Eh? There's something seriously different between our 
understandings here.
If a counter is included on ACK#1, it /unambiguously/ gives the state now.

If Cnt is included on ACK#1 it /ambiguously/ gives what the state will 
be when a future ACK (#2) without AccECN options appears. That's 
ambiguous, because you cannot know whether an ACK or ACKs with a 
different Cnt value was lost between #1 & #2.

if Cnt is included on ACK#2 (even if counters are not), it significantly 
reduces that ambiguity, because you can compare it with the Cnt on Ack#1 
(as explained two responses ago).

> If you bring ACK losses
> ambiguous (other than by chance).

[BB] See previous answer.

(Also, that first sentence is odd? If ACK losses were not on the table, 
we wouldn't need to be talking about Cnt. It is only there to deal with 
ACK losses.)

> But I guess your comment mainly stemmed

[BB] I know you didn't intend it. It's not a misconcept just because you 
didn't.
I'm saying it's much more powerful if Cnt can always be included, even 
when there are no counters.

> field values but perhaps still worth to consider if something needs to be

[BB] Altho just 2 counters changing is at the top of our minds, the idea 
is to be able to work well with 3-counters changing as well. For 
instance, SCE (even tho the scetcp draft doesn't use AccECN, it might). 
And there might be other 3-counter schemes in future.

Bob