Subject: [Rdma-cc-interest] Data Center Congestion Control

From: Paul Congdon <[email-masked]>
Date: Mon, 3 Aug 2020 12:57:11 -0700

Hello RDMA CC list participants,

There were two interesting presentations at IETF-108 related to Data Center
Congestion Control.  A couple of discussion points came up that have
surfaced before, so I thought it might be worth highlighting them for
ongoing discussion.

The two presentations in the second session of TSVWG were:

1. HPCC++: Enhanced High Precision Congestion Control - link
<https://datatracker.ietf.org/meeting/108/materials/slides-108-tsvwg-sessb-81-hpcc-enhanced-high-precision-congestion-control>
2. PFC free low delay control protocol - link
<https://datatracker.ietf.org/meeting/108/materials/slides-108-tsvwg-sessb-82-pfc-free-low-delay-control-protocol>

Both of these presentations discussed improving congestion control for
RoCEv2 (as well as other protocols), which re-surfaced a couple of issues:

1. RoCEv2 is specified by IBTA, so the IETF should not propose 'specific'
changes for this protocol.
2. If work were to be done in the IETF, where should it begin; ICCRG or
TSVWG.

I propose the following comments on these issues:

1. Agreed that RoCEv2 and it's DCQCN are under the stewardship of IBTA.
Perhaps what is really needed from IETF is a 'data center' focused CC
approach that any UDP based protocols can use - in the 'data center'. This
was briefly discussed several IETFs ago in a side meeting.  Maybe some of
the tenants of that protocol could come from the recent presentations and
include:
   - Use of in-band signaling (i.e. IOAM) to provide more precise
congestion information to end-points in order to allow a CC to react
quickly and accurately (not overdoing it or under doing it) - motivated by
HPCC.
   - Specify how ECN thresholds can be fluid/dynamic to allow protocols to
start fast and stabilize quickly without congestion  - motivated by LDCP

2. The leadership of IETF have consistently stated that congestion control
ideas should/could start off in ICCRG and then move to TSVWG when/if a
standards track draft is needed.

Just some thoughts... Would love to hear what others have to say.

Paul

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 4 Aug 2020 07:18:57 +0000

Hi Paul,

this is a very good summary, and I have made similar statements on the tsvwg list after having read LDCP.

Also, to repeat my meeting comments regarding LDCP:

  *   At least the congestion avoidance phase appears to very closely resemble what has been called fluid-model DCTCP by the DCTCP authors (which is has only been looked at from a theoretical point of view, afaik, back when DCTCP itself was implemented). The per-packet cwnd adjustment in this phase has a good theoretical basis for its mode of operation.
  *   The gradual marking threshold increase based on current instantaneous queue occupancy has been suggested before by Bob, as offering a few nice benefits over the step-change marking described with DCTCP.
  *   The feedback signaling however has even higher overhead than DCTCP, and its unclear if LRO / TSO “supersegments” have sufficient ACK feedback for LDCP has described currently to work.

  *   Working on a protocol-agnostic, Datacenter focused CC algorithm to yield ultra-low latency is absolutely something which I would support in the IETF (or rather, IRTF for the initial ICCRG discussions)

Richard Scheffenegger

From: Rdma-cc-interest <[email-masked]> On Behalf Of Paul Congdon
Sent: Montag, 3. August 2020 21:57
To: [email-masked]
Subject: [Rdma-cc-interest] Data Center Congestion Control

NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.

There were two interesting presentations at IETF-108 related to Data Center Congestion Control.  A couple of discussion points came up that have surfaced before, so I thought it might be worth highlighting them for ongoing discussion.

1. HPCC++: Enhanced High Precision Congestion Control - link<https://datatracker.ietf.org/meeting/108/materials/slides-108-tsvwg-sessb-81-hpcc-enhanced-high-precision-congestion-control>
2. PFC free low delay control protocol - link<https://datatracker.ietf.org/meeting/108/materials/slides-108-tsvwg-sessb-82-pfc-free-low-delay-control-protocol>

Both of these presentations discussed improving congestion control for RoCEv2 (as well as other protocols), which re-surfaced a couple of issues:

1. RoCEv2 is specified by IBTA, so the IETF should not propose 'specific' changes for this protocol.
2. If work were to be done in the IETF, where should it begin; ICCRG or TSVWG.

1. Agreed that RoCEv2 and it's DCQCN are under the stewardship of IBTA.  Perhaps what is really needed from IETF is a 'data center' focused CC approach that any UDP based protocols can use - in the 'data center'. This was briefly discussed several IETFs ago in a side meeting.  Maybe some of the tenants of that protocol could come from the recent presentations and include:
   - Use of in-band signaling (i.e. IOAM) to provide more precise congestion information to end-points in order to allow a CC to react quickly and accurately (not overdoing it or under doing it) - motivated by HPCC.
   - Specify how ECN thresholds can be fluid/dynamic to allow protocols to start fast and stabilize quickly without congestion  - motivated by LDCP

2. The leadership of IETF have consistently stated that congestion control ideas should/could start off in ICCRG and then move to TSVWG when/if a standards track draft is needed.

Paul