Subject: QUIC handshakes flow control

From: newbie quic <[email-masked]>
Date: Mon, 30 Mar 2020 17:15:32 +0200

Hello,

I have a question about the handshakes flow control.

According to
https://tools.ietf.org/html/draft-ietf-quic-transport-27#section-4 which
makes me think the handshakes flow control really exists:

   Data sent in CRYPTO frames is not flow controlled in the same way as
   stream data.  QUIC relies on the cryptographic protocol
   implementation to avoid excessive buffering of data; see [QUIC-TLS
<https://tools.ietf.org/html/draft-ietf-quic-transport-27#ref-QUIC-TLS>]

According to the same RFC here
https://tools.ietf.org/html/draft-ietf-quic-transport-27#section-7.4 :

   Implementations need to maintain a buffer of CRYPTO data received out
   of order.  Because there is no flow control of CRYPTO frames, an
   endpoint could potentially force its peer to buffer an unbounded
   amount of data.

These two parts of the RFC seem to be contradictory, or perhaps the flow
control of handshake packets must not be implemented via  a flow control of
CRYPTO frames. Have I missed something?

Furthermore, I did not find anything thing about the handshakes flow
control in QUIC-TLS.

How the handshakes flow control must be implemented please?

As an illustration, I am currently implementing a QUIC stack in a network
software, which lacks a lot of features.

Only the server part is implemented, without outgoing ACKs, but with
retransmissions. I can test this implementation with ngtcp2 client which
emits CRYPTO_BUFFER_EXCEEDED error when my server implementation sends
handshake packets with 2% as packet loss. This server has to send a big
128Kb certificate. Please also note that I did not implement the
anti-amplification feature  found here :
https://tools.ietf.org/html/draft-ietf-quic-transport-27#section-21.12.1.1.

May this latter feature fix the  "CRYPTO_BUFFER_EXCEEDED issue"?

Regards.

---

From: Nick Harper <[email-masked]>
Date: Mon, 30 Mar 2020 13:13:48 -0700

There is no flow control of CRYPTO frames: there is no mechanism for an
endpoint to signal to its peer how much data it is willing to accept in
CRYPTO frames. The only signal is if the peer exceeds that limit, the
endpoint can signal that with CRYPTO_BUFFER_EXCEEDED.

I don't think those two sections are contradictory, although they might be
misleading. It's true that data sent in CRYPTO frames is not flow
controlled the same way as stream data, because stream data has flow
control, while with CRYPTO frames we hope the peer doesn't send too much,
and if it does, the connection can be closed with CRYPTO_BUFFER_EXCEEDED.
That first section goes on to mention having the QUIC stack coordinate with
the crypto protocol to know how much data to buffer. In other words, how
much data should the QUIC stack buffer while waiting for retransmissions of
lost frames before sending CRYPTO_BUFFER_EXCEEDED. (Once QUIC has
reassembled CRYPTO frames into a buffer with no gaps, it can pass that to
the crypto handshake protocol.)

If you are going to deploy this server on the Internet, you MUST implement
anti-amplification protection. It is sufficient to, after sending 3 times
as much data as the client sent, to wait for an ACK from the client.
Whether or not you have this implemented should not be the cause of the
issue you are seeing with CRYPTO_BUFFER_EXCEEDED.

Once QUIC hands CRYPTO frame data to the TLS stack, the TLS stack generally
has some limit on how much data it will buffer when attempting to read a
message. In the stack I work with, that limit is 16kb for all messages
except Certificate, and is 100kb for the Certificate. The 128kb cert that
you're trying to send will likely not be accepted by most clients. However,
that failure would result in a connection close error in the 0x1XX range.

It might be possible to avoid a CRYPTO_BUFFER_EXCEEDED error by limiting
how much CRYPTO data you send.
https://tools.ietf.org/html/draft-ietf-quic-transport-27#section-7.4 states
that an implementation MUST support buffering at least 4kb of CRYPTO frame
data. You could limit to sending your cert in 4k chunks and wait to check
that each 4k chunk is ACKed before sending the next. Whether this will work
depends on the interplay between the QUIC stack and the TLS stack of the
peer's implementation. If the QUIC stack hands data off to the TLS stack
(thus freeing the QUIC stack's buffer) when data is available, then this
will probably work.

On Mon, Mar 30, 2020 at 8:16 AM newbie quic <[email-masked]> wrote:

>    implementation to avoid excessive buffering of data; see [QUIC-TLS <https://tools.ietf.org/html/draft-ietf-quic-transport-27#ref-QUIC-TLS>]
>

---

From: newbie quic <[email-masked]>
Date: Tue, 31 Mar 2020 09:14:30 +0200

Hello Nick,

Thank you for these clarifications.

Regards.

On Mon, Mar 30, 2020 at 10:14 PM Nick Harper <[email-masked]> wrote:

> Once QUIC hands CRYPTO frame data to the TLS stack, the TLS stack
> generally has some limit on how much data it will buffer when attempting to
> read a message. In the stack I work with, that limit is 16kb for all
> messages except Certificate, and is 100kb for the Certificate. The 128kb
> cert that you're trying to send will likely not be accepted by most
> clients. However, that failure would result in a connection close error in
> the 0x1XX range.
>