Subject: [tcpm] some comments on draft-gomez-tcpm-ack-rate-request-02.

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 1 Mar 2021 13:12:37 -0800

Hi,
I've read the draft. I like the basic concept of the draft and think it's
useful in some ways. I put some comments belows.

1: In general, I would like to see some more example usages or guidances of
the features described in the draft.
    For example, one of my naive questions is how often we want to change
the frequency of ACKs.
    In 'small' or 'large' scenarios in the draft, it seems to me that it
might be sufficient to use the same ACK frequency during the connection.
    It would be good if the draft describes the needs of changing frequency.

2:  "a TARR-option-capable receiving TCP MUST modify its ACK rate to one
ACK every R full-sized received data segments from the sender, as long as
packet reordering does not occur"
    -> Do this mean if recorder happens, TARR option will be ignored?
        Let's say if you send packet 1 with TARR and packet 2 without TARR,
if packet 2 arrives earlier than packet1, the TARR option in packet1 will
be ignored?
        How about storing the highest seqno that carries TARR and compare
it when you receive packets with TARR?

3:  "If all bits of this field are set to 0, the field indicates a request
by the sender for the receiver to trigger one or more ACKs immediately. "
    -> For me, this might be read as if it can request to send back
multiple ACKs immediately. How about changing it 'without delay' or
something?

4:  "Otherwise, the field carries the binary encoding of the number of
full-sized segments"
    -> I'm not sure about the meaning of binary encoding here. does it
mean storing 1-255 values or else?

5: I have some concerns on ignoring reorder.
    In case of QUIC, even if reorder is ignored, timer-based loss detection
scheme can still find packet losses.
    However, in TCP, RACK might not be always supported, so it might take
significant amount of time to find losses if this feature is enabled.
    I think the doc needs to discuss this point and it would be good to
have some guidelines here.

6: Similar to 1:, I am a bit wondering about the use case of N field in
TARR option.
   I can imagine this would be useful if an endpoint uses Hystart++ or
similar approach.
   However, is it still useful for other cases such as plain slow-start
algorithm?
   I think it would be better to have some usage guidelines for N as well.
   Also, since N looks finite number, I'm wondering how to set R=0 for the
entire connection. Or, we shouldn't do such a thing?

Thanks,
--
Yoshi

---

From: "Carles Gomez Montenegro" <[email-masked]>
Date: Fri, 5 Mar 2021 13:34:56 +0100

Hi Yoshi,

Thanks a lot for your insightful comments!

Also, sorry for the late response.

>     It would be good if the draft describes the needs of changing

Agreed, we will add some guidance in this regard.

Yes, in many scenarios it may make sense to just set the ACK frequency
once during the connection.

We'll think further about whether there could be particular scenarios
where the ACK frequency might need to be changed during the connection
lifetime.

> it when you receive packets with TARR?

Well, yes, the phrase "as long as packet reordering does not occur" might
not be clear enough here. The intent was to say that one ACK will be sent
every R full-sized received data segments, but if there is reordering,
then a different amount/rate of ACKs may be sent for a while.

Anyway, your suggestion is very useful, since the behavior in a scenario
like the one in your example needs to be clear as well.

Therefore, we take two action points from this comment: clarifying the
text, and also incorporating your suggestion.

> multiple ACKs immediately. How about changing it 'without delay' or

Agreed, we will edit the text.

> mean storing 1-255 values or else?

Yes, that is the intended meaning (well, for 1-127 values with the new
field size).

The original intent was to be clear on how to represent the value, but is
perhaps "binary encoding" misleading?

> have some guidelines here.

Agreed!

>    I think it would be better to have some usage guidelines for N as well.

Agreed. We will add this guidance.

> entire connection. Or, we shouldn't do such a thing?

Good point!

Perhaps we can define a special value for N meaning "infinity".

There are also considerations similar to the earlier question on whether
the expected behavior would be the same for the whole connection lifetime.
Perhaps, in some cases, setting N to "infinity" might represent an
improvement (since the option would only be sent once). However, if this
can represent any potential issue, a different setting should be chosen
for N.

Let's provide guidance on this.

We will incorporate your comments in our next draft update.

Thanks!

Cheers,

Carles

> Thanks,
> --
> Yoshi
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Bob Briscoe <[email-masked]>
Date: Sun, 14 Mar 2021 21:20:42 +0000

Carles,

> where the ACK frequency might need to be changed during the connection

[BB] Here's a couple of reasons why the DelACK ratio would change during 
the lifetime of a flow.

1/ I am interested in being able to use a facility like TARR to suppress 
delACKs in order to measure the RTT of each packet, particularly during 
flow start-up, but then release that constraint once the flow has started.

A Linux receiver has a heuristic to detect slow start and suppress 
DelACKs just for that period.  But variants of slow-start at the sender 
such as hystart, hystart++ etc. alter the ending of slow-start, so they 
can confuse the heuristics at the receiver. Similarly we've been working 
with a technique called paced chirping, which confuses the heuristics. 
Or put another way, if someone invents a better approach than slow 
start, but it confuses the heuristics at the receiver, it means that the 
receiver's heuristics have ossified the existing slow-start behaviour at 
the sender. In contrast, an explicit signal between the peers like TARR 
allows for evolution.

2/ In the high cwnd scenarios, the sender wants the DelACK ratio to 
still ensure a minimum number of ACKs per RTT. Obviously cwnd is not 
instantly large - it grows from the start of the flow onwards, and when 
the flow starts, the sender doesn't know what it will reach. Also, the 
window changes as other traffic comes and goes or the link capacity 
changes. So clearly the sender will need to keep updating the DelACK ratio.

==Units==

This last example raises the question of whether the number of packets 
between ACKs is the correct unit to use for the DelACK ratio. If the 
sender is trying to ensure there are say 4 ACKs per RTT, you might 
naively think it would be better to tell the receiver that. I worked 
through this question of units here:
https://github.com/quicwg/base-drafts/issues/1978#issuecomment-436553339
pasted below

>>> Packets? bytes? time? fraction of RTT?
>>>
>>> Think of this question in two stages:
>>> Q1. what units the sender starts from
>>> Q2. what units the message is in, which the sender has to derive from Q1
>>>
>>> If the message was a fraction of 1 RTT, the receiver would have to 
>>> calulate the sender's RTT. It can do that, using ACKs of ACKs, but 
>>> that implies work and delay.
>>>
>>> In all cases that I could see in #1428 
>>> <https://github.com/quicwg/base-drafts/issues/1428>, the sender had 
>>> all the info to translate from one unit to another (it knows its own 
>>> SMSS, its window, its RTT). This leaves Q2, to which a good enough 
>>> answer is packets (which also compresses the message size).
>>>

more...

>> have some guidelines here.

[BB] It wasn't clear to me what problem was intended to be addressed by 
the 'ignore reordering' flag.
If it was that the sender is using time-based detection, so it tells the 
receiver it doesn't need ACKs of out of order packets, that would make 
sense. But perhaps you had a different use-case in mind?

> Agreed. We will add this guidance.

[BB] It seemed to me that the ability to suppress ACKs for N packets 
would be redundant. Because the sender can send a different TARR option 
after N packets to ask the receiver to change the DelACK ratio. This 
would seem more useful in 2 respects:
1) It doesn't consume any option space
2) Typically the sender doesn't know exactly when its need for quick 
ACKs will end, until it does end.

> Let's provide guidance on this.

[BB] I have some other comments on the draft, but I'll send a separate 
review email for those that don't relate directly to Yoshi's thread.

Cheers

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Jonathan Morton <[email-masked]>
Date: Mon, 15 Mar 2021 00:26:55 +0200

>>    For example, one of my naive questions is how often we want to change the frequency of ACKs.
>>    In 'small' or 'large' scenarios in the draft, it seems to me that it might be sufficient to use the same ACK frequency during the connection.

It seems to me that the ack frequency could depend mostly on the cwnd value used by the sender, which can change over the connection's lifetime for many reasons, and is generally not directly known by the receiver (though it might be inferred by estimated RTT, segment size, and observed delivery rate).

In particular, cwnd will start at a low value and grow rapidly during the slow-start phase, then settle into a reasonably consistent range for the congestion-avoidance phase - assuming the underlying BDP remains constant.  Shifts in routing, or the addition or removal of significant load from the path the flow shares, may change the underlying BDP significantly; the cwnd should be expected to change accordingly, and this may prompt a change in ack rate.

So we should expect some changes to the ack rate to be requested over the lifetime of the connection, but very frequent updates seem unlikely if the ack rate is derived from the cwnd.  This does not seem difficult to me.

Conversely, the ack frequency may be controlled by "ack congestion control", alone or in combination with a derivation from the cwnd.  That is, the sender may notice that acks it receives cover more segments than the ack rate requested, indicating ack decimation en route, and decide to reduce the ack frequency to reduce network load up to the ack decimation point.  Future updates may also permit CE marks to appear on pure acks.  This might involve more frequent ack rate updates, perhaps once an RTT, as the algorithm adjusts and probes around an operating point.

> then a different amount/rate of ACKs may be sent for a while.

Isn't this sort of thing covered by existing Delayed Ack specifications?  They are just cases in which an immediate ack is called for, overriding the Delayed Ack algorithm and resetting the counter and timer associated with it.  You should just be able to make a normative reference to that.

> Perhaps we can define a special value for N meaning "infinity".

An R value of 1 or 0 should produce a similar effect, should it not?  I think this falls into the category of choosing the ack frequency based on the sender's cwnd, which for the resource-constrained sender is a permanently small value.  I think it's important to describe with precision what the meanings of R=1 and R=0 are in any case; it's one of those pesky little corner cases.

Conversely, the N field would be most useful for advanced congestion control algorithms which have a need to probe latency statistics at high fidelity, but only for a short period at a time.  We shouldn't use the N field to duplicate functionality that can adequately be described by the R field.

 - Jonathan Morton