Subject: Impact of hardware offloads on network stack performance

From: Praveen Balasubramanian <[email-masked]>
Date: Wed, 4 Apr 2018 20:15:40 +0000

Some comparative numbers from an out of box default settings Windows Server 2016 (released version) for a single connection with a microbenchmark tool:

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

This is for a fully bottlenecked CPU core -- if you run lower data rates there is still a significant difference in Cycles/byte cost. Same increased CPU cost applies for client systems going over high data rate Wifi and cellular.

This is without any crypto. Once you add crypto the numbers become much worse with crypto cost becoming dominant. Adding another crypto step further exacerbates the problem. Hence crypto offload gains in importance followed by these batch offloads.

If folks need any more numbers I'd be happy to provide them.

Thanks

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 4 Apr 2018 22:34:16 +0200

Thanks for sharing these numbers.

My guess is that these offloads deal with kernel scheduling, memory cache
issues, and interrupt scheduling.

It probably has very little to do with crypto, TCP headers, and any other
cpu sensitive processing.

This is where netmap enters: you can transparently feed the data as it
becomes available with very little sync work and you can also efficiently
pipeline so you pass data to decryptor, then to app, with as little bus
traffic as possible. No need to copy data or synchronize kernel space.

For 1K packets you would use about 1000ns (3 cycles/byte) on crypto and
this would happen in L1 cache. It would of course consume a core which a
crypto offload would not, but that can be debated because with 18 core
systems, your problem is memory and network more than CPU.

My concern with PNE is for small packets and low latency where
1) an estimated 24ns for PN encryption and decryption becomes measurable if
your network is fast enough.
2) any damage to the packet buffer cause all sorts of memory and cpu bus
traffic issues.

1) is annoying. 2) is bad, completely avoidable but not as PR 1079 is
formulated currently.

2) is also likely bad for hardware offload units as well.

As to SRV-IO - I’m not familiar with it, but obviously there is some IO
abstraction layer - the question is how you make it accessible to apps as
opposed to device drivers that does nor work with you QUIC custom stack,
and netmap is one option here.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 4 April 2018 at 22.15.52, Praveen Balasubramanian (
pravb=[email-masked]) wrote:

Some comparative numbers from an out of box default settings Windows Server
2016 (released version) for a single connection with a microbenchmark tool:

*Offloads enabled*

*TCP gbps*

*UDP gbps*

*LSO + LRO + checksum*

24

3.6

*Checksum only*

7.6

3.6

*None*

5.6

2.3

This is for a fully bottlenecked CPU core -- *if you run lower data rates
there is still a significant difference in Cycles/byte cost*. Same
increased CPU cost applies for client systems going over high data rate
Wifi and cellular.

This is without any crypto. Once you add crypto the numbers become much
worse *with crypto cost becoming dominant*. Adding another crypto step
further exacerbates the problem. Hence crypto offload gains in importance
followed by these batch offloads.

If folks need any more numbers I’d be happy to provide them.

Thanks

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 4 Apr 2018 22:34:16 +0200

Kind Regards,
Mikkel Fahnøe Jørgensen

*Offloads enabled*

*TCP gbps*

*UDP gbps*

*LSO + LRO + checksum*

24

3.6

*Checksum only*

7.6

3.6

*None*

5.6

2.3

Thanks

---

From: Mike Bishop <[email-masked]>
Date: Thu, 5 Apr 2018 21:56:40 +0000

That’s interesting data, Praveen – thanks.  There is one caveat there, which is that (IIUC, on current hardware) you can’t do packet pacing with LSO, and so my understanding is that even for TCP many providers disable this in pursuit of better egress management.  So what we’re really saying is that:

  *   On current hardware and OS setups, UDP runs less than half as much throughput as TCP; that needs some optimization, as we’ve already discussed
  *   TCP will dramatically gain performance once hardware offload catches up and allows paced LSO 😊

However, as Mikkel points out, the crypto costs are likely to overwhelm the OS throughput / scheduling issues in real deployments.  So I think the other relevant piece to understanding the cost here is this:

  *   What is the throughput and relative cost of TLS/TCP versus QUIC (i.e. how much are the smaller units of encryption hurting us versus, say, 16KB TLS records)?
     *   TLS implementations already vary here:  Some implementations choose large record sizes, some vary record sizes to reduce delay / HoLB, so this probably isn’t a single number.
  *   How much additional load does PNE add to this difference?
  *   To what extent would PNE make future crypto offloads impossible versus requires more R&D to develop?

From: QUIC [mailto:[email-masked]] On Behalf Of Mikkel Fahnøe Jørgensen
Sent: Wednesday, April 4, 2018 1:34 PM
To: Praveen Balasubramanian <pravb=[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

My guess is that these offloads deal with kernel scheduling, memory cache issues, and interrupt scheduling.

It probably has very little to do with crypto, TCP headers, and any other cpu sensitive processing.

This is where netmap enters: you can transparently feed the data as it becomes available with very little sync work and you can also efficiently pipeline so you pass data to decryptor, then to app, with as little bus traffic as possible. No need to copy data or synchronize kernel space.

For 1K packets you would use about 1000ns (3 cycles/byte) on crypto and this would happen in L1 cache. It would of course consume a core which a crypto offload would not, but that can be debated because with 18 core systems, your problem is memory and network more than CPU.

My concern with PNE is for small packets and low latency where
1) an estimated 24ns for PN encryption and decryption becomes measurable if your network is fast enough.
2) any damage to the packet buffer cause all sorts of memory and cpu bus traffic issues.

1) is annoying. 2) is bad, completely avoidable but not as PR 1079 is formulated currently.

As to SRV-IO - I’m not familiar with it, but obviously there is some IO abstraction layer - the question is how you make it accessible to apps as opposed to device drivers that does nor work with you QUIC custom stack, and netmap is one option here.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 4 April 2018 at 22.15.52, Praveen Balasubramanian (pravb=[email-masked]<mailto:pravb=[email-masked]>) wrote:
Some comparative numbers from an out of box default settings Windows Server 2016 (released version) for a single connection with a microbenchmark tool:

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Mike Bishop <[email-masked]>
Date: Thu, 5 Apr 2018 21:56:40 +0000

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Ian Swett <[email-masked]>
Date: Fri, 06 Apr 2018 02:06:01 +0000

On Thu, Apr 5, 2018 at 5:57 PM Mike Bishop <[email-masked]> wrote:

> That’s interesting data, Praveen – thanks.  There is one caveat there,
> which is that (IIUC, on current hardware) you can’t do packet pacing with
> LSO, and so my understanding is that even for TCP many providers disable
> this in pursuit of better egress management.  So what we’re really saying
> is that:
>
>    - On current hardware and OS setups, UDP runs less than half as much
>    throughput as TCP; that needs some optimization, as we’ve already discussed
>    - TCP will dramatically gain performance once hardware offload catches
>    up and allows *paced* LSO 😊
>
>
>
> However, as Mikkel points out, the crypto costs are likely to overwhelm
> the OS throughput / scheduling issues in real deployments.  So I think the
> other relevant piece to understanding the cost here is this:
>

I have a few cases(both client and server) where my UDP send costs are more
than 30%(in some cases 50%) of CPU consumption, and crypto is less than
10%.  So currently, I assume crypto is cheap(especially AES-GCM when
hardware acceleration is available) and egress is expensive.  UDP ingres is
not that expensive, but could use a bit of optimization as well.

The only 'fancy' thing our current code is doing for crypto is encrypting
in place, which was a ~2% win.  Nice, but not transformative.  See
EncryptInPlace
<https://cs.chromium.org/chromium/src/net/quic/core/quic_framer.cc?sq=package:chromium&l=1893>

I haven't done much work benchmarking Windows, so possibly the Windows UDP
stack is really fast, and so crypto seems really slow in comparison?

>
>    - What is the throughput and relative cost of TLS/TCP versus QUIC
>    (i.e. how much are the smaller units of encryption hurting us versus, say,
>    16KB TLS records)?
>       - TLS implementations already vary here:  Some implementations
>       choose large record sizes, some vary record sizes to reduce delay / HoLB,
>       so this probably isn’t a single number.
>    - How much additional load does PNE add to this difference?
>    - To what extent would PNE make future crypto offloads *impossible*
>    versus *requires more R&D to develop*?
>
>
>
> *From:* QUIC [mailto:[email-masked]] *On Behalf Of *Mikkel Fahnøe
> Jørgensen
> *Sent:* Wednesday, April 4, 2018 1:34 PM
> *To:* Praveen Balasubramanian <pravb=[email-masked]>;
> IETF QUIC WG <[email-masked]>
> *Subject:* Re: Impact of hardware offloads on network stack performance
>
>
>
> Some comparative numbers from an out of box default settings Windows
> Server 2016 (released version) for a single connection with a
> microbenchmark tool:
>
>
>
> *Offloads enabled*
>
> *TCP gbps*
>
> *UDP gbps*
>
> *LSO + LRO + checksum*
>
> 24
>
> 3.6
>
> *Checksum only*
>
> 7.6
>
> 3.6
>
> *None*
>
> 5.6
>
> 2.3
>
>
>

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 6 Apr 2018 04:45:22 +0000

To be clear - I don’t think crypto is overshadowing other issues as Mike read my post. It certainly comes at a cost, but either multiple cores or co-processors will deal with this. 1000ns is 10Gbps crypto speed on one core and it is higly parallelisable and cache friendly.

But if you have to drip your packets through a traditional send interface that is copying, buffering or blocking, and certainly sync’ing with the kernel - it is going to be tough.

For receive you risk high latency or top much scheduling in receive buffers.

________________________________
From: Ian Swett <[email-masked]>
Sent: Friday, April 6, 2018 4:06:01 AM
To: Mike Bishop
Cc: Mikkel Fahnøe Jørgensen; Praveen Balasubramanian; IETF QUIC WG
Subject: Re: Impact of hardware offloads on network stack performance

On Thu, Apr 5, 2018 at 5:57 PM Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
That’s interesting data, Praveen – thanks.  There is one caveat there, which is that (IIUC, on current hardware) you can’t do packet pacing with LSO, and so my understanding is that even for TCP many providers disable this in pursuit of better egress management.  So what we’re really saying is that:

I have a few cases(both client and server) where my UDP send costs are more than 30%(in some cases 50%) of CPU consumption, and crypto is less than 10%.  So currently, I assume crypto is cheap(especially AES-GCM when hardware acceleration is available) and egress is expensive.  UDP ingres is not that expensive, but could use a bit of optimization as well.

The only 'fancy' thing our current code is doing for crypto is encrypting in place, which was a ~2% win.  Nice, but not transformative.  See EncryptInPlace<https://cs.chromium.org/chromium/src/net/quic/core/quic_framer.cc?sq=package:chromium&l=1893>

I haven't done much work benchmarking Windows, so possibly the Windows UDP stack is really fast, and so crypto seems really slow in comparison?

From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Mikkel Fahnøe Jørgensen
Sent: Wednesday, April 4, 2018 1:34 PM
To: Praveen Balasubramanian <pravb=[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Impact of hardware offloads on network stack performance

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Mike Bishop <[email-masked]>
Date: Fri, 6 Apr 2018 20:37:37 +0000

Thanks for the clarification.  I guess what I’m really getting down to is how do we quantify the actual cost of PNE?  Are we substantially increasing the crypto costs?  If Ian is saying that crypto is comparatively cheap and the cost is that it’s harder to offload something that’s comparatively cheap, what have we lost?  I’d think we want to offload the most intensive piece we can, and it seems like we’re talking about crypto offloads....  Or are we saying instead that the crypto makes it harder to offload other things in the future, like a QUIC equivalent to LRO/LSO?

From: Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
Sent: Thursday, April 5, 2018 9:45 PM
To: Ian Swett <[email-masked]>; Mike Bishop <[email-masked]>
Cc: Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

From: Ian Swett <[email-masked]<mailto:[email-masked]>>

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Ian Swett <[email-masked]>
Date: Fri, 06 Apr 2018 20:55:21 +0000

PNE doesn't preclude UDP LRO or pacing offload.

However, in a highly optimized stack(ie: The Disk|Crypt|Net paper from
SIGCOMM 2017) not touching(ie: bringing into memory or cache) the content
to be sent until the last possible moment is critical.  PNE likely means
touching that content much earlier.

But personally I consider the actual crypto we're talking about(2 AES
instructions?) adding basically free.

On Fri, Apr 6, 2018 at 4:37 PM Mike Bishop <[email-masked]> wrote:

> Thanks for the clarification.  I guess what I’m really getting down to is
> how do we quantify the actual *cost* of PNE?  Are we substantially
> increasing the crypto costs?  If Ian is saying that crypto is comparatively
> cheap and the cost is that it’s harder to offload something that’s
> comparatively cheap, what have we lost?  I’d think we want to offload the
> most intensive piece we can, and it seems like we’re talking about crypto
> offloads....  Or are we saying instead that the crypto makes it harder to
> offload other things in the future, like a QUIC equivalent to LRO/LSO?
>
>
>
> *From:* Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
> *Sent:* Thursday, April 5, 2018 9:45 PM
> *To:* Ian Swett <[email-masked]>; Mike Bishop <[email-masked]>
> *Cc:* Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <
> [email-masked]>
> To be clear - I don’t think crypto is overshadowing other issues as Mike
> read my post. It certainly comes at a cost, but either multiple cores or
> co-processors will deal with this. 1000ns is 10Gbps crypto speed on one
> core and it is higly parallelisable and cache friendly.
>
>
>
> But if you have to drip your packets through a traditional send interface
> that is copying, buffering or blocking, and certainly sync’ing with the
> kernel - it is going to be tough.
>
>
>
> ------------------------------
>
> *From:* Ian Swett <[email-masked]>
> *Sent:* Friday, April 6, 2018 4:06:01 AM
> *To:* Mike Bishop
> *Cc:* Mikkel Fahnøe Jørgensen; Praveen Balasubramanian; IETF QUIC WG
> I have a few cases(both client and server) where my UDP send costs are
> more than 30%(in some cases 50%) of CPU consumption, and crypto is less
> than 10%.  So currently, I assume crypto is cheap(especially AES-GCM when
>    - TLS implementations already vary here:  Some implementations choose
>       large record sizes, some vary record sizes to reduce delay / HoLB, so this
>       probably isn’t a single number.
>
>
>    versus * requires more R&D to develop*?
>
>
>

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 6 Apr 2018 17:20:11 -0400

Yes, it is touching the memory that is the big problem.

24ns is measurable time in my world where a 500 byte structured message is
easily handled, but in reality it is the same latency as a 7 meter longer
ethernet cable due to c (the speed of light).

CPU cores work by cache coherency and prefetching. As long as data is only
written once, data can funnel into the different CPU cache systems but if
data is later touched the message bus between CPU’s must invalidate a lot
of assumptions and start over.

In the current form of PR 1079, the packet number is encrypted by writing
over data in the buffer, so in order to decrypt it, or just to verify it,
it is necessary to either copy the buffer - which means that all other
interested CPU’s must fetch new data, or the data is modified meaning cache
invalidation.

Fortunately this part of the problem is not so hard to solve: remove the
packet number from the AEAD protected data - there is no benefit in having
it there. Store and encrypt it separately. This is a problem because an AES
block is 16 bytes and smaller encryption schemes are hard to make secure.
By overwriting packet data PR 1079 places some of block size over existing
non-packet number data which saves space but hurts cache.

My unverified assumption is that this can be improved: the packet number
can be encrypted indirectly by first encrypting arbitrary content (say the
tag) of the existing already encrypted packet. The encrypted data is NOT
stored  but xor’ed with the packet number. The packet number is now
appended after the tag, or anywhere else, just not inside AEAD data.

So how to decrypt? When the packet is received you encrypt the same data as
before and get 16 bytes out. You locate the encrypted packet number and xor
to get the clear text of the packet number.

Now you can start to decrypt and/or verify the AEAD packet content because
the decrypted packet number is the IV used in that crypto process. Notice
that in this process we only operate with a 16-byte temporary block that we
do NOT store in he packet buffer. We get a decoded packet number that we do
NOT store in the packet buffer. We can store it anywhere else, as long as
it is in a memory area with its own cache line boundary.

This is still not ideal because the latency is not only the 24ns decryption
but also writing the decrypted packet number somewhere and communicate it
to a process that operate on packet data. For example, if we have two cores
- one for verification, and one for decryption, then one must necessarily
send data to the other, or they most both do the same packet number
decryption (which probably is the better idea, but now it costs 48ns total).

On 6 April 2018 at 22.55.33, Ian Swett ([email-masked]) wrote:

On Fri, Apr 6, 2018 at 4:37 PM Mike Bishop <[email-masked]> wrote:

> *From:* QUIC [mailto:[email-masked]] *On Behalf Of* Mikkel Fahnøe
> Jørgensen

---

From: Praveen Balasubramanian <[email-masked]>
Date: Fri, 6 Apr 2018 22:57:54 +0000

>>.  There is one caveat there, which is that (IIUC, on current hardware) you can’t do packet pacing with LSO, and so my understanding is that even for TCP many providers disable this in pursuit of better egress management
Some do some don’t. Windows hasn’t done packet pacing for TCP so far and is quite widely deployed across servers and clients. Even with pacing you can come up with proper sizing so you don’t lose the advantages of LSO. As you can see in the numbers LSO has a pretty sizeable impact on perf.

>> TCP will dramatically gain performance once hardware offload catches up and allows paced LSO
TCP as deployed today on Windows uses hardware offload (LSO, LRO, checksum) by default on all systems when available and support is almost universal except for some client NICs. There is no strict requirement for pacing to do LSO since we have been successfully deployed this way for 10+ years. TCP already has dramatically better perf than UDP on all public clouds across all VM sizes.

As to overhead due to PNE, we did an experiment for that right when the PNE discussion stated. With our current QUIC implementation over the (unoptimized) UDP stack and there was 1 to 1.25% overhead. Once the UDP and QUIC implementations are optimized, this number will increase. Happy to do the measurement again.

As to what extent hardware is impacted, hardware vendors will have to chime in. Several folks had chimed in on the original thread. Since we have a lack of participation from hardware vendors, I am trying to reach out to more folks to join the group and participate.

From: Mike Bishop [mailto:[email-masked]]
Sent: Thursday, April 5, 2018 2:57 PM
To: Mikkel Fahnøe Jørgensen <[email-masked]>; Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: RE: Impact of hardware offloads on network stack performance

  *   😊

From: QUIC [mailto:[email-masked]] On Behalf Of Mikkel Fahnøe Jørgensen
Sent: Wednesday, April 4, 2018 1:34 PM
To: Praveen Balasubramanian <pravb=[email-masked]<mailto:pravb=[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Impact of hardware offloads on network stack performance

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: "Deval, Manasi" <[email-masked]>
Date: Mon, 23 Apr 2018 18:18:49 +0000

If we do TSO / LSO with QUIC, the initial segment provided can look like –
MAC – IP – UDP – QUIC Clear – QUIC encrypted header (PN=x) – QUIC Encrypted payload.

If we just use the stack’s udp segmentation (using the patch - https://marc.info/?l=linux-netdev&m=152399530603992&w=2), the outcome of QUIC packets on the wire would be:
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment 1)
MAC – IP – UDP – QUIC encrypted part (partial segment 2)
MAC – IP – UDP – QUIC encrypted part (partial segment 3)
…

Putting these back together in a stack / hardware looks like a painful and potentially non-deterministic activity.

It would be easier to do a native QUIC offload, where the packet on wire can be identified by QUIC CiD and appropriate packet number. Therefore, original packet looks like - MAC – IP – UDP – QUIC Clear – QUIC encrypted header (PN=x) – QUIC Encrypted payload.
If we just create another version of segmentation called QUIC segmentation, the outcome of packets on the wire would be:
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment 1)
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+1, partial segment 2)
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+2, partial segment 3)
…
If this set of segments were to be re-assembled, the ability to identify CiD and PN would simplify this task. I think this is a simpler implementation. Do others see an issue with this?

Thanks,
Manasi

From: QUIC [mailto:[email-masked]] On Behalf Of Ian Swett
Sent: Friday, April 06, 2018 1:55 PM
To: Mike Bishop <[email-masked]>
Cc: Praveen Balasubramanian <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

However, in a highly optimized stack(ie: The Disk|Crypt|Net paper from SIGCOMM 2017) not touching(ie: bringing into memory or cache) the content to be sent until the last possible moment is critical.  PNE likely means touching that content much earlier.

But personally I consider the actual crypto we're talking about(2 AES instructions?) adding basically free.

On Fri, Apr 6, 2018 at 4:37 PM Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
Thanks for the clarification.  I guess what I’m really getting down to is how do we quantify the actual cost of PNE?  Are we substantially increasing the crypto costs?  If Ian is saying that crypto is comparatively cheap and the cost is that it’s harder to offload something that’s comparatively cheap, what have we lost?  I’d think we want to offload the most intensive piece we can, and it seems like we’re talking about crypto offloads....  Or are we saying instead that the crypto makes it harder to offload other things in the future, like a QUIC equivalent to LRO/LSO?

From: Mikkel Fahnøe Jørgensen [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, April 5, 2018 9:45 PM
To: Ian Swett <[email-masked]<mailto:[email-masked]>>; Mike Bishop <[email-masked]<mailto:[email-masked]>>
Cc: Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Impact of hardware offloads on network stack performance

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Mon, 23 Apr 2018 11:25:00 -0700

I’m not really familiar with this dump but what does partial segmentation
mean here?

Are you splitting a single UDP datagram into multiple fragments, or are you
processing multiple separate QUIC packets, one PN each?

If you fragment UDP datagrams, that can be a big issue because different
fragments may race along different routes, causing unexpected delays and
blocking at the other end.

On 23 April 2018 at 20.18.54, Deval, Manasi ([email-masked]) wrote:

MAC – IP – UDP – QUIC Clear – QUIC encrypted header (PN=x) – QUIC Encrypted
payload.

If we just use the stack’s udp segmentation (using the patch -
https://marc.info/?l=linux-netdev&m=152399530603992&w=2), the outcome of
QUIC packets on the wire would be:

…

Putting these back together in a stack / hardware looks like a painful and
potentially non-deterministic activity.

It would be easier to do a native QUIC offload, where the packet on wire
can be identified by QUIC CiD and appropriate packet number. Therefore,
original packet looks like - MAC – IP – UDP – QUIC Clear – QUIC encrypted
header (PN=x) – QUIC Encrypted payload.

If we just create another version of segmentation called QUIC segmentation,
the outcome of packets on the wire would be:

MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+1, partial segment
2)

MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+2, partial segment
3)

…

If this set of segments were to be re-assembled, the ability to identify
CiD and PN would simplify this task. I think this is a simpler
implementation. Do others see an issue with this?

Thanks,

Manasi

*From:* QUIC [mailto:[email-masked]] *On Behalf Of *Ian Swett
*Sent:* Friday, April 06, 2018 1:55 PM
*To:* Mike Bishop <[email-masked]>
*Cc:* Praveen Balasubramanian <[email-masked]>; Mikkel Fahnøe
Jørgensen <[email-masked]>; IETF QUIC WG <[email-masked]>
*Subject:* Re: Impact of hardware offloads on network stack performance

However, as Mikkel points out, the crypto costs are likely to overwhelm the
OS throughput / scheduling issues in real deployments.  So I think the
other relevant piece to understanding the cost here is this:

   - What is the throughput and relative cost of TLS/TCP versus QUIC (i.e.
   how much are the smaller units of encryption hurting us versus, say, 16KB
   TLS records)?

   - TLS implementations already vary here:  Some implementations choose
      large record sizes, some vary record sizes to reduce delay /
HoLB, so this
      probably isn’t a single number.

*From:* QUIC [mailto:[email-masked]] *On Behalf Of *Mikkel Fahnøe
Jørgensen
*Sent:* Wednesday, April 4, 2018 1:34 PM
*To:* Praveen Balasubramanian <pravb=[email-masked]>; IETF
QUIC WG <[email-masked]>
*Subject:* Re: Impact of hardware offloads on network stack performance

Kind Regards,

Mikkel Fahnøe Jørgensen

*Offloads enabled*

*TCP gbps*

*UDP gbps*

*LSO + LRO + checksum*

24

3.6

*Checksum only*

7.6

3.6

*None*

5.6

2.3

Thanks

---

From: "Deval, Manasi" <[email-masked]>
Date: Mon, 23 Apr 2018 18:37:53 +0000

I’m trying to lay out the options for TSO/LSO with QUIC. In my previous e-mail I compared the outcome of vanilla UDP segmentation as defined by the latest patch (https://marc.info/?l=linux-netdev&m=152399530603992&w=2), to an alternative native QUIC segmentation.

A segment is a large buffer produced in an application. The segment is often much too large to go out on the wire with just pre-pended headers. Therefore segment needs to be chopped up into appropriate size and the pre-pended headers need to be replicated to make MTU size packets for the wire.

I’m suggesting that we should consider processing a large data buffer as multiple QUIC packets each with a unique PN.

Thanks,
Manasi

From: Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
Sent: Monday, April 23, 2018 11:25 AM
To: Ian Swett <ianswett=[email-masked]>; Deval, Manasi <[email-masked]>; Mike Bishop <[email-masked]>
Cc: IETF QUIC WG <[email-masked]>; Praveen Balasubramanian <[email-masked]>
Subject: RE: Impact of hardware offloads on network stack performance

I’m not really familiar with this dump but what does partial segmentation mean here?

Are you splitting a single UDP datagram into multiple fragments, or are you processing multiple separate QUIC packets, one PN each?

If you fragment UDP datagrams, that can be a big issue because different fragments may race along different routes, causing unexpected delays and blocking at the other end.

On 23 April 2018 at 20.18.54, Deval, Manasi ([email-masked]<mailto:[email-masked]>) wrote:
If we do TSO / LSO with QUIC, the initial segment provided can look like –
MAC – IP – UDP – QUIC Clear – QUIC encrypted header (PN=x) – QUIC Encrypted payload.

If we just use the stack’s udp segmentation (using the patch - https://marc.info/?l=linux-netdev&m=152399530603992&w=2), the outcome of QUIC packets on the wire would be:
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment 1/3)
MAC – IP – UDP – QUIC encrypted part (partial segment 2/3)
MAC – IP – UDP – QUIC encrypted part (partial segment 3/3)
…

It would be easier to do a native QUIC offload, where the packet on wire can be identified by QUIC CiD and appropriate packet number. Therefore, original packet looks like - MAC – IP – UDP – QUIC Clear – QUIC encrypted header (PN=x) – QUIC Encrypted payload.
If we just create another version of segmentation called QUIC segmentation, the outcome of packets on the wire would be:
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment 1/3)
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+1, partial segment 2/3)
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x+2, partial segment 3/3)
…
If this set of segments were to be re-assembled, the ability to identify CiD and PN would simplify this task. I think this is a simpler implementation. Do others see an issue with this?

Thanks,
Manasi

From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Ian Swett
Sent: Friday, April 06, 2018 1:55 PM
To: Mike Bishop <[email-masked]<mailto:[email-masked]>>
Cc: Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>>; Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: Impact of hardware offloads on network stack performance

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Mon, 23 Apr 2018 11:45:56 -0700

Ah, I see. Thanks for explaining.

QUIC has a much reacher interface, so there could be, say 100 active
streams waiting for transmission. The QUIC implementation would use some
sort of scheduler to move the most important parts first without loosing
progress on other parts, for example using

https://en.wikipedia.org/wiki/Deficit_round_robin

A simple huge buffer segmentation would probably not work directly, but a
means to forward data pre-wrapped in QUIC frames, in reasonable package
units, but without AEAD envelope would make sense. These could be queued in
volumes, leaving it an offloader to proceed.

I’ve been looking into the netmap interface where you can feed content in
units directly to hardware, and I’m sure you have detailed knowledge of
DPDK that does something similar.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 23 April 2018 at 20.38.06, Deval, Manasi ([email-masked]) wrote:

I’m trying to lay out the options for TSO/LSO with QUIC. In my previous
e-mail I compared the outcome of vanilla UDP segmentation as defined by the
latest patch (https://marc.info/?l=linux-netdev&m=152399530603992&w=2), to
an alternative native QUIC segmentation.

A segment is a large buffer produced in an application. The segment is
often much too large to go out on the wire with just pre-pended headers.
Therefore segment needs to be chopped up into appropriate size and the
pre-pended headers need to be replicated to make MTU size packets for the
wire.

I’m suggesting that we should consider processing a large data buffer as
multiple QUIC packets each with a unique PN.

Thanks,

Manasi

*From:* Mikkel Fahnøe Jørgensen [mailto:[email-masked]]
*Sent:* Monday, April 23, 2018 11:25 AM
*To:* Ian Swett <ianswett=[email-masked]>; Deval, Manasi <
[email-masked]>; Mike Bishop <[email-masked]>
*Cc:* IETF QUIC WG <[email-masked]>; Praveen Balasubramanian <
[email-masked]>
*Subject:* RE: Impact of hardware offloads on network stack performance

MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment
1/3)

…

MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment
1/3)

2/3)

3/3)

…

Thanks,

Manasi

Kind Regards,

Mikkel Fahnøe Jørgensen

*Offloads enabled*

*TCP gbps*

*UDP gbps*

*LSO + LRO + checksum*

24

3.6

*Checksum only*

7.6

3.6

*None*

5.6

2.3

Thanks

---

From: Ian Swett <[email-masked]>
Date: Mon, 23 Apr 2018 19:19:48 +0000

To clarify my understanding, you'd have to receive all 3 segments to
decrypt the payload?  If that's what you're proposing, then I think it's
not a good design for QUIC, because it amplifies the effective loss rate
and it doesn't save that much CPU.

It's not that hard for userspace to provide 3 fully formed encrypted
packets and use an API like Willem's UDP GSO patch, and I believe that
approach is more agile as well.

If you wanted to do crypto offload, then you could provide the ability to
offload payload and PN encryption instead of doing it in userspace.  As a
bonus, support iovec style scatter-gather, so bulk payloads never need to
be brought into CPU cache before they're encrypted and copied to the NIC.

On Mon, Apr 23, 2018 at 2:46 PM Mikkel Fahnøe Jørgensen <[email-masked]>
wrote:

> Are you splitting a single UDP datagram into multiple fragments, or are
> you processing multiple separate QUIC packets, one PN each?
>
>
>
> If we just create another version of segmentation called QUIC
> segmentation, the outcome of packets on the wire would be:
>

---

From: "Lubashev, Igor" <[email-masked]>
Date: Mon, 23 Apr 2018 19:36:15 +0000

The point of HW offloads is to reduce the number of times the kernel (“transport”) stack is invoked to locate the flow, validate headers, enqueue data for delivery to sockets, etc.

If HW can associate multiple QUIC packets with a single flow, remove encryption and pass it on to the stack, it does not really need to simulate the data arriving in a single jumbo QUIC packet.

  *   Igor

From: Ian Swett [mailto:ianswett=[email-masked]]
Sent: Monday, April 23, 2018 3:20 PM
To: Mikkel Fahnøe Jørgensen <[email-masked]>
Cc: Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <[email-masked]>; Mike Bishop <[email-masked]>; Deval, Manasi <[email-masked]>; ianswett=[email-masked]
Subject: Re: Impact of hardware offloads on network stack performance

To clarify my understanding, you'd have to receive all 3 segments to decrypt the payload?  If that's what you're proposing, then I think it's not a good design for QUIC, because it amplifies the effective loss rate and it doesn't save that much CPU.

It's not that hard for userspace to provide 3 fully formed encrypted packets and use an API like Willem's UDP GSO patch, and I believe that approach is more agile as well.

If you wanted to do crypto offload, then you could provide the ability to offload payload and PN encryption instead of doing it in userspace.  As a bonus, support iovec style scatter-gather, so bulk payloads never need to be brought into CPU cache before they're encrypted and copied to the NIC.

On Mon, Apr 23, 2018 at 2:46 PM Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>> wrote:
Ah, I see. Thanks for explaining.

QUIC has a much reacher interface, so there could be, say 100 active streams waiting for transmission. The QUIC implementation would use some sort of scheduler to move the most important parts first without loosing progress on other parts, for example using

https://en.wikipedia..org/wiki/Deficit_round_robin<https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Deficit-5Fround-5Frobin&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kLn2KgKLPtGJxRvKXL-FJ5JKb-Ru0P0jELdhxpWwM8Y&s=EiFxElkH4sg30mBAg17hM6E1uSnsk1m_CcnEpbfAR4Q&e=>

A simple huge buffer segmentation would probably not work directly, but a means to forward data pre-wrapped in QUIC frames, in reasonable package units, but without AEAD envelope would make sense. These could be queued in volumes, leaving it an offloader to proceed.

I’ve been looking into the netmap interface where you can feed content in units directly to hardware, and I’m sure you have detailed knowledge of DPDK that does something similar.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 23 April 2018 at 20.38.06, Deval, Manasi ([email-masked]<mailto:[email-masked]>) wrote:
I’m trying to lay out the options for TSO/LSO with QUIC. In my previous e-mail I compared the outcome of vanilla UDP segmentation as defined by the latest patch (https://marc.info/?l=linux-netdev&m=152399530603992&w=2<https://urldefense.proofpoint.com/v2/url?u=https-3A__marc.info_-3Fl-3Dlinux-2Dnetdev-26m-3D152399530603992-26w-3D2&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kLn2KgKLPtGJxRvKXL-FJ5JKb-Ru0P0jELdhxpWwM8Y&s=hYHZfZyfxP7kInpTvNE9U02BPAhje-ZvaUgV1rNCSLI&e=>), to an alternative native QUIC segmentation.

Thanks,
Manasi

From: Mikkel Fahnøe Jørgensen [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Monday, April 23, 2018 11:25 AM
To: Ian Swett <ianswett=[email-masked]<mailto:[email-masked]>>; Deval, Manasi <[email-masked]<mailto:[email-masked]>>; Mike Bishop <[email-masked]<mailto:[email-masked]>>
Cc: IETF QUIC WG <[email-masked]<mailto:[email-masked]>>; Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>>
Subject: RE: Impact of hardware offloads on network stack performance

If we just use the stack’s udp segmentation (using the patch - https://marc.info/?l=linux-netdev&m=152399530603992&w=2<https://urldefense.proofpoint.com/v2/url?u=https-3A__marc.info_-3Fl-3Dlinux-2Dnetdev-26m-3D152399530603992-26w-3D2&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kLn2KgKLPtGJxRvKXL-FJ5JKb-Ru0P0jELdhxpWwM8Y&s=hYHZfZyfxP7kInpTvNE9U02BPAhje-ZvaUgV1rNCSLI&e=>), the outcome of QUIC packets on the wire would be:
MAC – IP – UDP – QUIC Clear – QUIC encrypted part (PN=x, partial segment 1/3)
MAC – IP – UDP – QUIC encrypted part (partial segment 2/3)
MAC – IP – UDP – QUIC encrypted part (partial segment 3/3)
…

Thanks,
Manasi

The only 'fancy' thing our current code is doing for crypto is encrypting in place, which was a ~2% win.  Nice, but not transformative.  See EncryptInPlace<https://urldefense.proofpoint.com/v2/url?u=https-3A__cs.chromium.org_chromium_src_net_quic_core_quic-5Fframer.cc-3Fsq-3Dpackage-3Achromium-26l-3D1893&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kLn2KgKLPtGJxRvKXL-FJ5JKb-Ru0P0jELdhxpWwM8Y&s=--DoALPZoa0L4Ie9TZrPKpDy1_dnb1kN-3Ga3MrXnb0&e=>

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Mike Bishop <[email-masked]>
Date: Mon, 23 Apr 2018 20:06:33 +0000

I agree – the point of offloading the crypto is not only to spare that cost on the CPU, though that can be useful, it’s to enable further offloads that can’t be done on the encrypted packet.

LRO that could simply group packets by CID and deliver a single batch of packets from a single CID in a single invocation seems like a great first step, and doesn’t even require crypto offload.  Once you have crypto offload, of course those packets per CID could be delivered already decrypted.  I’d want to see data on whether there’s value in making the received data look like one huge packet when there’s not a single bytestream, but that’s not something we need to specify here.

LSO is more interesting, and the challenge there is that the transport stack still needs to know what data was in which packet so that it can do loss recovery properly.  QUIC explicitly disallows fragmentation of the UDP datagram, for the “painful and potentially non-deterministic” reasons you referenced.

From: Lubashev, Igor <[email-masked]>
Sent: Monday, April 23, 2018 12:36 PM
To: Ian Swett <ianswett=[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>
Cc: Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <[email-masked]>; Mike Bishop <[email-masked]>; Deval, Manasi <[email-masked]>
Subject: RE: Impact of hardware offloads on network stack performance

  *   Igor

From: Ian Swett [mailto:ianswett=[email-masked]]
Sent: Monday, April 23, 2018 3:20 PM
To: Mikkel Fahnøe Jørgensen <[email-masked]<mailto:[email-masked]>>
Cc: Praveen Balasubramanian <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>; Mike Bishop <[email-masked]<mailto:[email-masked]>>; Deval, Manasi <[email-masked]<mailto:[email-masked]>>; ianswett=[email-masked]<mailto:ianswett=[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

Kind Regards,
Mikkel Fahnøe Jørgensen

Thanks,
Manasi

Thanks,
Manasi

Kind Regards,
Mikkel Fahnøe Jørgensen

Offloads enabled

TCP gbps

UDP gbps

LSO + LRO + checksum

24

3.6

Checksum only

7.6

3.6

None

5.6

2.3

Thanks

---

From: Watson Ladd <[email-masked]>
Date: Mon, 23 Apr 2018 13:55:56 -0700

On Mon, Apr 23, 2018 at 1:06 PM, Mike Bishop <[email-masked]> wrote:

> I agree – the point of offloading the crypto is not only to spare that
> cost on the CPU, though that can be useful, it’s to enable further offloads
> that can’t be done on the encrypted packet.
>
>
>
> LRO that could simply group packets by CID and deliver a single batch of
> packets from a single CID in a single invocation seems like a great first
> step, and doesn’t even require crypto offload.  Once you have crypto
> offload, of course those packets per CID could be delivered already
> decrypted.  I’d want to see data on whether there’s value in making the
> received data look like one huge packet when there’s not a single
> bytestream, but that’s not something we need to specify here.
>
>
>
> LSO is more interesting, and the challenge there is that the transport
> stack still needs to know what data was in which packet so that it can do
> loss recovery properly.  QUIC explicitly disallows fragmentation of the UDP
> datagram, for the “painful and potentially non-deterministic” reasons you
> referenced.
>

Forgive my complete ignorance, but doesn't it just need to know what packet
it sent?

More explicitly the sender would maintain a queue of sendable packets, and
mark which have been sent. Acknowledged packets are removed from the queue.
In event of loss, they simply retransmit the same packet. It's up to higher
layers to pick the right size of packet and take data from the individual
QUIC streams. If you wanted to expose the stream structure, that sounds a
lot more complicated.

Sincerely,
Watson

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 23 Apr 2018 18:16:17 -0700

I'll echo the concern that there should not be partial segments at all.

One key difference between designing TCP and QUIC segmentation will be that
QUIC packets are not as fluid as TCP, since QUIC isn't a byte stream.
You'll need to respect the packet boundaries that the encryption is done
over... I would want a segmentation offload interface for QUIC to support
variable packet sizes, since there's no guarantee that any QUIC packet
boundary will exactly match the max MTU.  (I could be wrong but I don't
think Willem's patch does this yet either.)

On Mon, Apr 23, 2018 at 1:55 PM, Watson Ladd <[email-masked]> wrote:

>> datagram, for the “painful and potentially non-deterministic” reasons
>> you referenced.
>>
>
> Forgive my complete ignorance, but doesn't it just need to know what
> packet it sent?
>

---

From: Mike Bishop <[email-masked]>
Date: Tue, 24 Apr 2018 16:28:50 +0000

Yes, that would be sufficient.  My point is that the offload probably needs to leave packetization to the software and not attempt to parcel them out as part of the offload.

From: Watson Ladd [mailto:[email-masked]]
Sent: Monday, April 23, 2018 1:56 PM
To: Mike Bishop <[email-masked]>
Cc: Lubashev, Igor <[email-masked]>; Ian Swett <ianswett=[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; Praveen Balasubramanian <[email-masked]>; IETF QUIC WG <[email-masked]>; Deval, Manasi <[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

On Mon, Apr 23, 2018 at 1:06 PM, Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
I agree – the point of offloading the crypto is not only to spare that cost on the CPU, though that can be useful, it’s to enable further offloads that can’t be done on the encrypted packet.

Forgive my complete ignorance, but doesn't it just need to know what packet it sent?

More explicitly the sender would maintain a queue of sendable packets, and mark which have been sent. Acknowledged packets are removed from the queue. In event of loss, they simply retransmit the same packet. It's up to higher layers to pick the right size of packet and take data from the individual QUIC streams. If you wanted to expose the stream structure, that sounds a lot more complicated.

Sincerely,
Watson

---

From: Roberto Peon <[email-masked]>
Date: Mon, 30 Apr 2018 20:37:11 +0000

QUIC (mostly) doesn’t do that (rexmit a packet verbatim)—you’d need a new packet# for the rexmitted packet.
-=R

From: QUIC <[email-masked]> on behalf of Watson Ladd <[email-masked]>
Date: Monday, April 23, 2018 at 1:56 PM
To: Mike Bishop <[email-masked]>
Cc: Praveen Balasubramanian <[email-masked]>, "Lubashev, Igor" <[email-masked]>, Ian Swett <ianswett=[email-masked]>, Mikkel Fahnøe Jørgensen <[email-masked]>, IETF QUIC WG <[email-masked]>, "Deval, Manasi" <[email-masked]>
Subject: Re: Impact of hardware offloads on network stack performance

Sincerely,
Watson