Subject: Re: [tcpm] Possible error in accurate-ecn

From: Bob Briscoe <[email-masked]>
Date: Tue, 23 Feb 2021 00:12:00 +0000

Mirja, Richard, Yoshi,

I just posted a new rev of AccECN for a number of other reasons, and 
realized we hadn't converged on an answer to the tricky problem in this 
old thread from Nov'20. For now, the text says the following (which I've 
justified below, but I don't think it's the solution yet), and I've 
asked Richard & Mirja if we (as co-authors) can act as a little design 
team to thrash out this problem in the next few days.

3.2.2.5.1.  Data Receiver Safety Procedures

    An AccECN Data Receiver:

    o  SHOULD immediately send an ACK whenever a data packet marked CE
       arrives after the previous packet was not CE.

    o  MUST immediately send an ACK once 'n' CE marks have arrived since
       the previous ACK, where 'n' SHOULD be 2 and MUST be in the range 2
       to 6 inclusive.

Justification for first bullet: The aim is to trigger an ACK on a 
transition from non-CE to CE, but only when the latest packet is a data 
packet. Cases where the previous packet was a CE-marked pure ACK and 
therefore not ACK'd ought to be picked up by the 2nd bullet, not the 
first. Similarly, if there's CE-marked packets (data or pure ACKs) that 
haven't been ACKd yet, and the next data packet is not CE-marked, I 
think the first bullet is still correct, and the second rule will pick 
up cases with enough CE marks.

The second bullet is more tricky. I've added Richard's suggesting of 
flooring 'n' at 2. But made no other changes. That's deliberate, 'cos I 
think it's sufficient to allow ACK CC to be added, but not to preclude 
it. And I think the danger of being interpreted as DupACKs shouldn't 
apply if there's no outstanding data. I know I'm wrong in those odd 
cases I mentioned when new data might cross the ACKs in the other 
direction. But I don't think that's going to cause too much harm.

However, I'm probably very wrong. Somehow I feel we haven't got a good 
model to check all the possibilities.

Bob

On 17/11/2020 08:48, Scheffenegger, Richard wrote:
>
> See my suggestion; The critical requirement for the 2nd bullet point is,
> to have n > 1 for pure, CE-marked ACKs. The exponential decay, after how
> many ACKs triggered by CE-marked ACKs these ACK-for-ACK stop, strongly
> depends on "n", so the recommendation should be to use a higher value of
> n in the case where you only observe a stream of pure ACKs (with or
> without marks; only those marked would account against "n" though).
>
> In the case of data packets mixed in, you don't want to delay
> excessively, thus "n" may be choosen lower...
>
> Richard
>
>
>
> Am 10.11.2020 um 17:38 schrieb Bob Briscoe:
>>
>> Moving on to the second bullet...
>>>> The second bullet doesn't consider the possibility that the 'n'th CE
>>>> mark might arrive on a pure ACK. Then, the wording as it stands says
>>>> the Data Receiver MUST immediately ACK a pure ACK. I know TCP never
>>>> ACKs a pure ACK, but I'm not actually sure it does any harm to do so
>>>> in this case (it cannot cause an infinite loop of ACKs). However,
>>>> given it would be unorthodox, we maybe ought to rule it out by
>>>> rewording anyway?
>>> [MK] I think we also can leave this as it is because if that ’n’
>>> packet is a pure ACK that still means that you have unacked data
>>> packets with CE marks and you should trigger an ACK for those packet
>>> now (rather than waiting for the delayed ACK timer to expire). However
>>> this case is less important. Also we should probably make sure that
>>> this doesn’t apply if there are only pure ACKs with CE marks, maybe by
>>> add “if unacknowledged data are outstanding” or something.
>>
>> [BB] By the end of your last para you start to realize that, if the n'th
>> CE mark arrives on pure ACK, it doesn't say anything about whether the
>> previous (n-1) CE marks were on data or pure ACKs.
>>
>> Consider another common scenario; a server delivering chunks of video
>> over a high BDP path (e.g. 500 packets per RTT), and going idle between
>> times (perhaps in response to an "exceeded high watermark" data packet
>> from the server). Now consider the possibility that the reverse path
>> bottleneck builds a queue during each chunk, and starts CE marking
>> towards the end of each chunk. Then, after the server stops sending a
>> chunk, it continues to see a couple of hundred CE-marked pure ACKs
>> arriving.
>>
>> However, by your proposed rule, it doesn't send any feedback, 'cos it
>> has no data to acknowledge. So the client cannot regulate its ACK
>> stream. That's just tough, 'cos the client will stop sending more pure
>> ACKs after it has received the last data, which is before any later
>> feedback from the server can reach it.
>>
>> However, if the client later sends a data packet that the server can
>> acknowledge (e.g. "a low watermark" message), the server's ACE counter
>> will have wrapped dozens of times, and the client won't be able to work
>> out how many. That doesn't matter if a few round trips have elapsed -
>> the congestion info will have become stale. But it does matter if it's
>> fairly soon.
>>
>> Instead, if we keep the text of the second bullet... if the server's 'n'
>> was 2, it would send a pure ACK for every other pure ACK it received.
>> But, what if these pure ACKs from the server were also CE-marked? Then
>> the client would feed back pure ACKs using its 'n'. This regression
>> would eventually die out, but I don't like it.
>>
>> This conversation about the second bullet presumes AccECN is being used
>> for ACK congestion control. I'm not comfortable with writing anything
>> specific about ACK congestion control into a draft on the standards
>> track. But I don't want to preclude AccECN being used for ACK CC.
>>
>>
>> I believe the second bullet needs something changed to limit the ACKing
>> of pure ACKs. However, before we put effort into wordsmithing, I'd like
>> to hear whether you agree with the general idea of not defining what to
>> do for ACK CC, but not precluding it.
>>
>> Cheers
>>
>>
>>
>>
>> Bob
>>
>>>
>>> Mirja
>>>
>>>
>>>
>>>> Thoughts anyone?
>>>>
>>>>
>>>> Bob
>>>>
>>>> -- 
>>>> ________________________________________________________________
>>>> Bob Briscoe
>>>> http://bobbriscoe.net/
>>> _______________________________________________
>>> tcpm mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 24 Feb 2021 11:02:38 +0100

Hi Bob,

unless there is ACK duplication going on, I don't see how the current
text would cause any issues.

The only aspect I would emphasize is, that in the 2nd case, "n" MUST NOT
be as low as one, as that may cause a ACK-triggering-ACK when the path
exhibits a pathological misconfiguration, where all packets get CE
marked. As long as a smaller numer of ACKs ("n" >= 2) carry a sum of CE
marks seen from received frames, any such pathological paths would
possible trigger a few rounds of ACKs, but these will eventually die
down, correct?

Do you have a different scenario in mind?

Richard

Am 23.02.2021 um 01:12 schrieb Bob Briscoe:
> Bob Briscoehttp://bobbriscoe.net/
>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 24 Feb 2021 02:03:31 -0800

Hi Bob,

On Mon, Feb 22, 2021 at 4:12 PM Bob Briscoe <[email-masked]> wrote:

> realized we hadn't converged on an answer to the tricky problem in this old
> thread from Nov'20. For now, the text says the following (which I've
> justified below, but I don't think it's the solution yet), and I've asked
> Richard & Mirja if we (as co-authors) can act as a little design team to
> thrash out this problem in the next few days.
>
> therefore not ACK'd ought to be picked up by the 2nd bullet, not the first.
> Similarly, if there's CE-marked packets (data or pure ACKs) that haven't
> been ACKd yet, and the next data packet is not CE-marked, I think the first
> bullet is still correct, and the second rule will pick up cases with enough
> CE marks.
>
> think it's sufficient to allow ACK CC to be added, but not to preclude it.
> And I think the danger of being interpreted as DupACKs shouldn't apply if
> there's no outstanding data. I know I'm wrong in those odd cases I
> mentioned when new data might cross the ACKs in the other direction. But I
> don't think that's going to cause too much harm.
>
> This might be too pathological, but I'm thinking about a certain extreme
example..
In my understanding, let's say a node sends 50 ACKs for some reasons and
all ACKs have CE marks, then its peer sends back 25 ACKs.
If these 25 ACKs have CE marks as well, it can generate another 12 ACKs,
and so on.
If this understanding is correct, I'm not sure if this is a good behavior.
I am thinking we had better do something here such as don't send an ACK if
the correspond ACKs carries accecn signals.

Also, I think it's hard to tell there's no outstanding data from the
receiver side.
Let's say a sender send some data packets after sending 10 acks, and the
receiver sends back 5 acks as the 10 ACKs have CE marks.
In this case, they look like dup acks from the sender's point of view.  I
know this is a rare case, though.

--
Yoshi

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 24 Feb 2021 19:32:52 +0100

Hi Yoshi,

A valid concern. Two ways to remediate these at least to some extent:

a) used a "n" higher than 2 (e.g. 6)
b) refrain from sending the ACK for n CE immediately, but hold off for
either the delayed ack timeout, or the n+1'th receipt of CE, or a valid
data segment by that time.

While this would not fully rule out any possibilty of an inadvertend
DupAck, it significantly reduces the time window when this may happen.

Also, if an implementation uses other clues (e.g. TS opt, SACK
negotiated but not present in the ACKs incrementing the CE.P ),
ambiguity can be further reduced.

However, I feel that these optimizations may overburden the definition
of the mechanism.

Best regards,
    Richard

Am 24.02.2021 um 11:03 schrieb Yoshifumi Nishida:
> Hi Bob,
>
>
> On Mon, Feb 22, 2021 at 4:12 PM Bob Briscoe <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Mirja, Richard, Yoshi,
>
>     realized we hadn't converged on an answer to the tricky problem in
>     this old thread from Nov'20. For now, the text says the following
>     (which I've justified below, but I don't think it's the solution
>     yet), and I've asked Richard & Mirja if we (as co-authors) can act
>     as a little design team to thrash out this problem in the next few days.
>
>     transition from non-CE to CE, but only when the latest packet is a
>     data packet. Cases where the previous packet was a CE-marked pure
>     ACK and therefore not ACK'd ought to be picked up by the 2nd bullet,
>     not the first. Similarly, if there's CE-marked packets (data or pure
>     ACKs) that haven't been ACKd yet, and the next data packet is not
>     CE-marked, I think the first bullet is still correct, and the second
>     rule will pick up cases with enough CE marks.
>
>     flooring 'n' at 2. But made no other changes. That's deliberate,
>     'cos I think it's sufficient to allow ACK CC to be added, but not to
>     preclude it. And I think the danger of being interpreted as DupACKs
>     shouldn't apply if there's no outstanding data. I know I'm wrong in
>     those odd cases I mentioned when new data might cross the ACKs in
>     the other direction. But I don't think that's going to cause too
>     much harm.
>
> I am thinking we had better do something here such as don't send an ACK
> if the correspond ACKs carries accecn signals.
>
> In this case, they look like dup acks from the sender's point of view.
> I know this is a rare case, though.
>
> --
> Yoshi
>
>
>
>>>>>     http://bobbriscoe.net/ <http://bobbriscoe.net/>
>>>>     [email-masked] <mailto:[email-masked]>
>>>>     <https://www.ietf.org/mailman/listinfo/tcpm>
>>>
>>
>>     [email-masked] <mailto:[email-masked]>
>>     <https://www.ietf.org/mailman/listinfo/tcpm>
>
>     --
>     Bob Briscoehttp://bobbriscoe.net/  <http://bobbriscoe.net/>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/tcpm>
>