Subject: [tcpm] A question about PRR/loss recovery

From: Yi Huang <[email-masked]>
Date: Fri, 16 Apr 2021 19:08:28 +0000

Hi tcpm and PRR authors,

We are revisiting PRR implementation in Windows and I noticed there might be a problem in our implementation (and it is actually not specific to PRR).

PRR algorithm computes a sndcnt on each ACK during loss recovery and thus use it to accurately regulate bytes in-flight.

In the RFC (6937 or bis), I see this statement and note "local" and "response to each ACK".
   We introduce a local variable "sndcnt", which indicates exactly how
   many bytes should be sent in response to each ACK.

In Windows' TCP implementation, new data can be also sent out inline during loss recovery from app send context (app calling send()) and they are not regulated by sndcnt/PRR. Instead, we just check if CWND allows us to send in this case. So, effectively, we use two variables controlling the number of bytes in flight during loss recovery, i.e. sndcnt and CWND. It apparently seems a bad behavior because it ignores PRR's sendcnt but I just want to double check with the PRR authors and the community.

Thanks,
Yi

---

From: Neal Cardwell <[email-masked]>
Date: Fri, 16 Apr 2021 15:43:58 -0400

On Fri, Apr 16, 2021 at 3:08 PM Yi Huang <huanyi=
[email-masked]> wrote:

> Hi tcpm and PRR authors,
>
>
>
> We are revisiting PRR implementation in Windows and I noticed there might
> be a problem in our implementation (and it is actually not specific to PRR).
>
>
>
> PRR algorithm computes a sndcnt on each ACK during loss recovery and thus
> use it to accurately regulate bytes in-flight.
>
>
>
> In the RFC (6937 or bis), I see this statement and note “local” and
> “response to each ACK”.
>
> In Windows’ TCP implementation, new data can be also sent out inline
> during loss recovery from app send context (app calling send()) and they
> are not regulated by sndcnt/PRR. Instead, we just check if CWND allows us
> to send in this case. So, effectively, we use two variables controlling the
> number of bytes in flight during loss recovery, i.e. sndcnt and CWND. It
> apparently seems a bad behavior because it ignores PRR’s sendcnt but I just
> want to double check with the PRR authors and the community.
>
>
Thanks for the note.

I believe that approach runs contrary to the intent of the PRR
spec/algorithm.

Implicit in the end of the algorithm in
https://tools.ietf.org/html/rfc6937#section-3 is the following line (in
pseudocode):

  cwnd = pipe + sndcnt

That is what the Linux TCP implementation of the PRR algorithm does. That
way there is only one variable (cwnd) controlling the number of bytes in
flight during loss recovery. Effectively, this allows the PRR algorithm
decision to persist, in case there is no data available to send at the
instant the ACK arrives.

Perhaps the update to the PRR RFC can make this explicit?

best,
neal

---

From: Yuchung Cheng <[email-masked]>
Date: Mon, 19 Apr 2021 09:55:16 -0700

I think the current PRR RFC is fairly clear about cwnd computation and it
should be used for both retransmits and new data (so both implicitly use
sndcnt). Having limited-transmit (i.e. new-data) only follow the
(unmodified) cwnd would break PRR substantially -- the packet conservation
principle is not really implemented. I suppose it's easy to fix Windows
code?

---

From: Yi Huang <[email-masked]>
Date: Mon, 19 Apr 2021 18:21:11 +0000

Thanks Neal and Yuchung, the RFC is clear enough to me. Just wanted to double check. We will fix that this problem.

From: tcpm <[email-masked]> On Behalf Of Yuchung Cheng
Sent: Monday, April 19, 2021 9:55 AM
To: Neal Cardwell <[email-masked]>
Cc: [email-masked]; Matt Mathis <[email-masked]>; Yi Huang <huanyi=[email-masked]>
Subject: [EXTERNAL] Re: [tcpm] A question about PRR/loss recovery

I think the current PRR RFC is fairly clear about cwnd computation and it should be used for both retransmits and new data (so both implicitly use sndcnt). Having limited-transmit (i.e. new-data) only follow the (unmodified) cwnd would break PRR substantially -- the packet conservation principle is not really implemented. I suppose it's easy to fix Windows code?

On Fri, Apr 16, 2021 at 12:44 PM Neal Cardwell <[email-masked]<mailto:[email-masked]>> wrote:

On Fri, Apr 16, 2021 at 3:08 PM Yi Huang <huanyi=[email-masked]<mailto:[email-masked]>> wrote:
Hi tcpm and PRR authors,

Thanks for the note.

I believe that approach runs contrary to the intent of the PRR spec/algorithm.

Implicit in the end of the algorithm in https://tools.ietf.org/html/rfc6937#section-3<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc6937%23section-3&data=04%7C01%7Chuanyi%40microsoft.com%7C18ee4b0df35b42907bda08d90354073b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637544482333596954%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xs9imRi3L2B6d737Iu07Vlrg4MZNDTv7dHHdFTFRBjs%3D&reserved=0> is the following line (in pseudocode):

  cwnd = pipe + sndcnt

That is what the Linux TCP implementation of the PRR algorithm does. That way there is only one variable (cwnd) controlling the number of bytes in flight during loss recovery. Effectively, this allows the PRR algorithm decision to persist, in case there is no data available to send at the instant the ACK arrives.

best,
neal