Subject: [tcpm] Introduce RFC 6937 bis (Proportional Rate Reduction) as a tcpm work item? INPUT NEEDED

From: Matt Mathis <[email-masked]>
Date: Wed, 18 Nov 2020 19:56:00 -0800

The authors of PRR would like to update PRR to Proposed Standard status.
This entails introducing a new document as an tcpm work item.

*Please indicate (non) support and/or comment.*

For more details see the tcpm meeting materials from IETF 109
minutes:
https://datatracker.ietf.org/meeting/109/materials/minutes-109-tcpm-00
slides: https://tools.ietf.org/html/draft-mathis-tcpm-rfc6937bis-00

There were about four "I support this work" remarks at the mic (not
recorded in the minutes), and about as many in the Meetecho chat.

Abridged IETF/tcpm/PRRbis slides:
--
PRR  recap (RFC6937 experimental)
PRR is a special congestion control effective only during fast recovery

   - When inflight >= ssthresh, send at loss_beta*rate_before_loss (e.g.
   loss_beta = 0.5 for Reno (aka rate-halving), 0.7 for Cubic)
   - When inflight < ssthresh, send at the same or twice the delivery_rate
   (more later)
   - Used by all congestion control modules in Linux during fast recovery
      - Can be more dominant than the actual C.C. for lossy flows that’re
      in fast recovery constantly (e.g. video streaming through policers)

--
Current Status

   -

   PRR is widely deployed
   -

      At least three major OSs: Linux, Windows, (NetFlix) BSD
      -

      Vast majority of Web traffic for years
      -

   No changes to algorithms published in RFC 6937
   -

      PRR-CRB - Conservative Reduction Bound - strict packet conversion
      during loss recovery
      -

      PRR-SSRB - Slowstart Reduction Bound - one extra segment per ACK
      during loss recovery
      -

   2015 Heuristic to dynamically select which reduction bound
   -

      Only use PRR-SSRB when making good forward progress
      -

         ACKs that advanced snd.una and report no new losses
         -

      Resolves some pathological cases with token bucket policers
      -

         CC estimates ssthresh before it can possibly measure the token rate
         -

         The heuristic makes the best of a bad situation

--
Tentative path forward

   -

   Adopt as a tcpm work item
   -

   Update the text
   -

      Normative RFC 2119 language
      -

      Add MAY use the heuristic...
      -

      Trim redundant and obsolete language
      -

         RFC 6937 repeats itself and is much longer than necessary
         -

         Focus on what an implementer needs to know
         -

         Use non-normative references to RFC 6937 for prior measurement
         work, etc

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

We must not tolerate intolerance;
       however our response must be carefully measured:
            too strong would be hypocritical and risks spiraling out of
control;
            too weak risks being mistaken for tacit approval.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 19 Nov 2020 08:42:50 +0100

As the heuristic is the one major improvement over RFC6937, that is the
one area I am especially interested in. Currently working to have PRR in
FreeBSD, it would be the best time window to add this heuristig instead
of switching between the two variants in a static fashion.

I support this and will certainly be reviewing the documents and provide
feedback!

Richard

Am 19.11.2020 um 04:56 schrieb Matt Mathis:
> The authors of PRR would like to update PRR to Proposed Standard
> status.  This entails introducing a new document as an tcpm work item.
>
>   * When inflight >= ssthresh, send at loss_beta*rate_before_loss (e.g.
>   * When inflight < ssthresh, send at the same or twice the
>     delivery_rate (more later)
>   * Used by all congestion control modules in Linux during fast recovery
>       o Can be more dominant than the actual C.C. for lossy flows
>         that’re in fast recovery constantly (e.g. video streaming
>         through policers)
>
> --
> Current Status
>
>   *
>
>     PRR is widely deployed
>
>       o
>
>       o
>   *
>       o
>         PRR-CRB - Conservative Reduction Bound - strict packet
>         conversion during loss recovery
>
>       o
>
>   *
>       o
>           +
>       o
>           +
>           +
>   *
>   *
>       o
>       o
>       o
>           +
>           +
>           +
>             Use non-normative references to RFC 6937 for prior
>             measurement work, etc
>
> Thanks,
> --MM--
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm
>

---

From: Ian Swett <[email-masked]>
Date: Thu, 19 Nov 2020 09:20:49 -0500

I support this work.

One question for authors and the group.  This specification could be
applied to QUIC, but my memory is that there are some small changes to
adapt it correctly.  Would noting those changes be in scope for this
document, a separate document in QUIC, or an exercise left to the reader?
I'd be happy to contribute any QUIC specific text if that's helpful.

Thanks, Ian

---

From: "Scharf, Michael" <[email-masked]>
Date: Thu, 19 Nov 2020 14:35:06 +0000

The TCPM charter would allow TCPM to specify a TCP mechanism including minor adaptations needed for QUIC. The TCPM charter explicitly states:

“TCP's congestion control algorithms are the model followed by
other IETF transports (e.g., SCTP or DCCP), which are standardized in
other working groups, such as the Transport Area WG (tsvwg). In the
past, the IETF has worked on several documents about algorithms that
are specified for multiple protocols (e.g., TCP and SCTP) in the
same document. Which WG shepherds such documents will be determined
on a case-by-case basis. In any case, the TCPM WG will remain in
close contact with other relevant WGs working on these protocols to
ensure openness and stringent review from all angles.”

(This was written before QUIC standardization started.)

Having one document for TCP and QUIC would thus be allowed by the TCPM charter. In that case, the document would probably be last called also in the QUIC working group – at least this is what we have done in similar cases with TSVWG in the past.

I think the formal details could be figured out if there is consensus to have one document only. It is up to the TCPM and QUIC community to decide…

Michael

From: tcpm <[email-masked]> On Behalf Of Ian Swett
Sent: Thursday, November 19, 2020 3:21 PM
To: Scheffenegger, Richard <[email-masked]>
Cc: [email-masked] Extensions <[email-masked]>
Subject: Re: [tcpm] Introduce RFC 6937 bis (Proportional Rate Reduction) as a tcpm work item? INPUT NEEDED

I support this work.

One question for authors and the group.  This specification could be applied to QUIC, but my memory is that there are some small changes to adapt it correctly.  Would noting those changes be in scope for this document, a separate document in QUIC, or an exercise left to the reader?  I'd be happy to contribute any QUIC specific text if that's helpful.

Thanks, Ian

On Thu, Nov 19, 2020 at 2:43 AM Scheffenegger, Richard <[email-masked]<mailto:[email-masked]>> wrote:

Richard

Am 19.11.2020 um 04:56 schrieb Matt Mathis:
> [email-masked]<mailto:[email-masked]>

_______________________________________________
tcpm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tcpm

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 19 Nov 2020 14:37:27 +0000

On 19/11/2020 14:20, Ian Swett wrote:
> document, a separate document in QUIC, or an exercise left to the 
> reader?  I'd be happy to contribute any QUIC specific text if that's 
> helpful.
>
> Thanks, Ian

I'd hope the differences for QUIC ARE captured in the RFC series. If 
they are small, it might be possible to add to this draft perhaps?

Gorry

>     <https://datatracker.ietf.org/meeting/109/materials/minutes-109-tcpm-00>
>     > slides:
>     https://tools.ietf.org/html/draft-mathis-tcpm-rfc6937bis-00
>     <https://tools.ietf.org/html/draft-mathis-tcpm-rfc6937bis-00>
>     >
>     > [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/tcpm>
>     >
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/tcpm>
>
>

---

From: Matt Mathis <[email-masked]>
Date: Thu, 19 Nov 2020 08:40:42 -0800

If we can get away with a non-normative description of the heuristic, QUIC
is no problem at all.    If we need normative descriptions, then TCP and
QUIC have to be different, and we need to include normative references to
both recovery machines and recovery state variables, which are not in scope
for the current doc.

I would prefer to stay with a non-normative description of the heuristic,
and scrub the rest of the document for language that conflicts with QUIC.

Thanks,
--MM--

On Thu, Nov 19, 2020 at 6:37 AM Gorry Fairhurst <[email-masked]>
wrote:

> I'd hope the differences for QUIC ARE captured in the RFC series. If they
> are small, it might be possible to add to this draft perhaps?
>
> Gorry
>
>
> tcpm mailing [email-masked]://www.ietf.org/mailman/listinfo/tcpm
>
>

---

From: Praveen Balasubramanian <[email-masked]>
Date: Thu, 19 Nov 2020 17:36:48 +0000

I am supportive of updating PRR to PS status and will review the document.

Re. QUIC. The current loss recovery spec for QUIC already refers to PRR as a MAY and has similar other references to TCP RFCs without any additional text for QUIC specific adaptations. The current title of RFC 6937 scopes it to TCP: Proportional Rate Reduction for TCP. I also see about 38 hits for the word TCP in the text.

From: tcpm <[email-masked]> On Behalf Of Matt Mathis
Sent: Thursday, November 19, 2020 8:41 AM
To: Gorry Fairhurst <[email-masked]>; Ian Swett <ianswett=[email-masked]>
Cc: [email-masked] Extensions <[email-masked]>
Subject: [EXTERNAL] Re: [tcpm] Introduce RFC 6937 bis (Proportional Rate Reduction) as a tcpm work item? INPUT NEEDED

If we can get away with a non-normative description of the heuristic, QUIC is no problem at all.    If we need normative descriptions, then TCP and QUIC have to be different, and we need to include normative references to both recovery machines and recovery state variables, which are not in scope for the current doc.

I would prefer to stay with a non-normative description of the heuristic, and scrub the rest of the document for language that conflicts with QUIC.

Thanks,
--MM--

We must not tolerate intolerance;
       however our response must be carefully measured:
            too strong would be hypocritical and risks spiraling out of control;
            too weak risks being mistaken for tacit approval.

On Thu, Nov 19, 2020 at 6:37 AM Gorry Fairhurst <[email-masked]<mailto:[email-masked]>> wrote:
On 19/11/2020 14:20, Ian Swett wrote:
I support this work.

Thanks, Ian

I'd hope the differences for QUIC ARE captured in the RFC series. If they are small, it might be possible to add to this draft perhaps?

Gorry

Richard

Am 19.11.2020 um 04:56 schrieb Matt Mathis:
> https://datatracker.ietf.org/meeting/109/materials/minutes-109-tcpm-00<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fmeeting%2F109%2Fmaterials%2Fminutes-109-tcpm-00&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705303486%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GotM4AhjDjqbRs6okuMEeAQxQBUmmuNaFhI7grBtQu4%3D&reserved=0>
> slides: https://tools.ietf.org/html/draft-mathis-tcpm-rfc6937bis-00<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-mathis-tcpm-rfc6937bis-00&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705303486%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1I8wixEmKrng%2FqDAzYajQpv85dagIleRdvqCSupoaUQ%3D&reserved=0>
>
> https://www.ietf.org/mailman/listinfo/tcpm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>
>

_______________________________________________
tcpm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tcpm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>

tcpm mailing list

https://www.ietf.org/mailman/listinfo/tcpm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>

_______________________________________________
tcpm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tcpm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705323397%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JXlBlyxDMB2qQUlx%2BY44b7fZfgcj8HaDcEAkfAv7AAI%3D&reserved=0>

---

From: Neal Cardwell <[email-masked]>
Date: Thu, 19 Nov 2020 15:56:08 -0500

I am very supportive of updating PRR to PS status and adopting this as a
TCPM work item. I will be happy to review the document as well.

The "Tentative path forward" outlined by Matt sounds good to me.

best,
neal

On Thu, Nov 19, 2020 at 12:37 PM Praveen Balasubramanian <pravb=
[email-masked]> wrote:

> Re. QUIC. The current loss recovery spec for QUIC already refers to PRR as
> a MAY and has similar other references to TCP RFCs without any additional
> text for QUIC specific adaptations. The current title of RFC 6937 scopes it
> to TCP: Proportional Rate Reduction for TCP. I also see about 38 hits for
> the word TCP in the text.
>
>
>
> *From:* tcpm <[email-masked]> *On Behalf Of * Matt Mathis
> *Sent:* Thursday, November 19, 2020 8:41 AM
> *To:* Gorry Fairhurst <[email-masked]>; Ian Swett <ianswett=
> [email-masked]>
> *Cc:* [email-masked] Extensions <[email-masked]>
> *Subject:* [EXTERNAL] Re: [tcpm] Introduce RFC 6937 bis (Proportional
> Rate Reduction) as a tcpm work item? INPUT NEEDED
>
>
>
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fmeeting%2F109%2Fmaterials%2Fminutes-109-tcpm-00&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705303486%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GotM4AhjDjqbRs6okuMEeAQxQBUmmuNaFhI7grBtQu4%3D&reserved=0>
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-mathis-tcpm-rfc6937bis-00&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705303486%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1I8wixEmKrng%2FqDAzYajQpv85dagIleRdvqCSupoaUQ%3D&reserved=0>
> >
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>
> >
>
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>
>
>
>
> https://www.ietf.org/mailman/listinfo/tcpm <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705313440%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JZsMBF3VehUvHSJfggPYbPco34h9UiTG%2BKbyXU1Bz%2BA%3D&reserved=0>
>
>
>
> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Ftcpm&data=04%7C01%7Cpravb%40microsoft.com%7Cc95254bcba0f41c14f3608d88ca9eac5%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637414008705323397%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JXlBlyxDMB2qQUlx%2BY44b7fZfgcj8HaDcEAkfAv7AAI%3D&reserved=0>
>

---

From: Yuchung Cheng <[email-masked]>
Date: Fri, 20 Nov 2020 12:51:55 -0800

Thanks for everybody's reply and support

One additional thing we authors would like more feedback on is ECN. The
current RFC and proposed -bis do not cover ECN handling. However in Linux,
ECN handling uses the same PRR algorithm (aka "CWR" aka "cwnd reduction")
just like fast recovery. In other words, if RFC3168 or DCTCP triggers a
congestion window reduction, PRR takes over to control the transmission
until SND_UNA crosses SND.NXT upon reduction.

This arrangement was inherented from the original Linux rate-halving design
before the PRR was introduced. This paper has details
https://www.cs.helsinki.fi/research/iwtcp/papers/linuxtcp.pdf

Compared to fast recovery, this part of the implementation is far less
tested due to the lack of wide classic-ECN usage on the Internet. It's also
unclear how other PRR implementations have applied to ECN handling like
Linux does. Adding ECN handling in -bis PS makes me uneasy.

On Thu, Nov 19, 2020 at 12:57 PM Neal Cardwell <ncardwell=
[email-masked]> wrote:

>> Re. QUIC. The current loss recovery spec for QUIC already refers to PRR
>> as a MAY and has similar other references to TCP RFCs without any
>> additional text for QUIC specific adaptations. The current title of RFC
>> 6937 scopes it to TCP: Proportional Rate Reduction for TCP. I also see
>> about 38 hits for the word TCP in the text.
>>
>>
>>
>> If we can get away with a non-normative description of the heuristic,
>> QUIC is no problem at all.    If we need normative descriptions, then TCP
>> and QUIC have to be different, and we need to include normative references
>> to both recovery machines and recovery state variables, which are not in
>> scope for the current doc.
>>
>>
>>

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 30 Nov 2020 10:12:27 -0800

Hi folks,

This mail starts a WG adoption call for RFC6937bis draft.
As we've already seen positive supports for this work, we are not asking
people who already expressed their opinions this time. (but, if you want to
do it for some reason, please go ahead)
However, if you haven't done it yet, especially if you're against it, we
would like to encourage you to provide feedback.

We are thinking that the intended status of the doc will be PS based on the
feedback we've seen so far. But, if you have opinions on it, please let us
know.

Thank you so much,
--
Yoshi on behalf of tcpm co-chairs

On Wed, Nov 18, 2020 at 7:56 PM Matt Mathis <mattmathis=
[email-masked]> wrote:

>    - When inflight < ssthresh, send at the same or twice the