Subject: Stream0 Proposal: Explicit vs. Implicit ACK (Re: Stream0 Design Team Proposal)

From: Kazuho Oku <[email-masked]>
Date: Thu, 24 May 2018 10:07:33 +0900

2018-05-24 3:46 GMT+09:00 Jana Iyengar <[email-masked]>:

> A thought occurs to me: a CRYPTO_HS frame with the fin bit set can be used
> to indicate that the handshake is done.This could be used as a signal (aka
> HANDSHAKE_DONE) to solve Marten's problem of a client rtxing a CFIN. Of
> course, this would mean that this particular CRYPTO_HS frame would be
> encrypted with a 1-RTT key... which *I think* is fine. What do you think?

I am not sure if that is a good approach.

I would assume that we would like to keep the handshake flow on the
1-RTT key open for the lifetime of the QUIC connection, so that we
could send things related to the TLS (e.g. session tickets). Sending
FIN from the server right after receiving ClientFinished makes that
impossible.

Having said that, I do agree that the need for a dedicated frame is a
pain. It is a pain because we need to have a retransmission logic for
the frame.

I think that you are making a good point in suggesting to consider
reusing the retransmission logic that already exists for bidirectional
flows. But as said, FIN bit does not seem like an answer in this case.

IMO, ideally we should define a new TLS post-handshake message (that
does not convey any data), and send that from the server in response
to ClientFinished using 1-RTT packet. The message will act as a signal
that the server has received ClientFinished.

The benefit of this approach is that we can reuse the retransmission
logic that will exist for handshake flows. The downside is that TLS
stacks need to be aware of the new handshake message.

Considering the fact that the Design Team is already suggesting a
substantial change to the TLS stack, we might have chance in asking
for the addition of a new handshake message. But I am not sure.

-- 
Kazuho Oku

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 May 2018 14:43:49 +1000

The obvious solution for the problem of not retransmitting the client
Finished is to retransmit it until it is acknowledged.  With the proposed
design, that means that the server sends a Handshake message after the
handshake is done, which I think is fine.  If the client sends Finished
until it is acknowledged, then I think we're good.

I can see the appeal in saying that if you receive an ACK for 1-RTT data
then your Handshake packets must have been received.  I can see that
working even, but you have to be careful.  It's just a signal that you can
stop sending CRYPTO frames, not that you are done with the keys.  You might
need to send ACK again and you might still receive packets at that level.
You could drop read keys and ACK only the old packets that you did decrypt,
but I suspect that it's better to authenticate any packet and ACK
faithfully, which would be better for your peer's congestion state.

That also means that you probably need to have timers for cleaning up, to
avoid the pathological case where acknowledgments are sent indefinitely.

I think that the FIN bit idea might work to close out the use of the
"stream" for each set of keys. That is, as opposed to the crypto stream as
a whole.  But it's duplicative: the crypto handshake needs to know when it
moves from one set of keys to the next and replicating that signaling at
the QUIC layer only really adds things to check for consistency.

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 24 May 2018 14:49:12 +0900

> until it is acknowledged, then I think we're good.

I agree that that would work.

IIUC downsides compared to the implicit ACK approach are that the
server needs to keep the old keys for certain amount of time (e.g., 2
MSL) and that the additional Handshake packet will consume about 30
octets of the 3 packet window that we have for the server's first
flights.

> avoid the pathological case where acknowledgments are sent indefinitely.

I agree with your argument about congestion control.

One way of acquiring correct congestion control information in the
implicit ACK approach will be to send coalesced packets that combines
a Handshake packet with a packet protected by a higher encryption key.
That will guarantee that you would receive an ACK for the delivery of
the datagrams, and you can feed the information to the congestion
controller.

FWIW, I have implemented implicit ACK in quicly, and it only required
about 40 additional lines of code, including the retransmission logic
for Handshake_Done. The code can be found at
https://github.com/h2o/quicly/pull/8/commits/eee6eb5fc7c0359edfeb0b8f89ae9ac191ef0758.
Coalesced packets is not implemented.

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 25 May 2018 17:06:23 +0900

I think I might have found an issue in the explicit ACK approach that
we use in the PR.

That is that EOED cannot be ACKed.

Assuming that EOED is sent using 0-RTT keys, it needs to be acked by
the server using the same key, because the transmission of EOED needs
to be guaranteed for the TLS handshake to complete.

However, there is no 0-RTT send key on the server-side.

Do we have any rule for dealing with this? I searched though the text
of the PR but could not find an answer.

-- 
Kazuho Oku

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 25 May 2018 04:33:43 -0400

The overview doc seems to be clear on ACK’s in general, not sure about EOED
specficially.

https://docs.google.com/document/d/1fRsJqPinJl8N3b-bflDRV6auojfJLkxddT93j6SwHY8/edit

section 2.2

ACKs in 1-RTT packets will acknowledge the receipt of 0-RTT or 1-RTT
packets.  ACKs cannot be sent in 0-RTT packets.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 25 May 2018 at 10.06.49, Kazuho Oku ([email-masked]) wrote:

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 25 May 2018 17:57:30 +0900

> packets.  ACKs cannot be sent in 0-RTT packets.

Thank you for pointing that out.

That statement in the doc seem to contradict with the following
statement in the PR.

  - ACK frames MAY appear in packets of any encryption level, but
    MUST only acknowledge packets which appeared in that encryption
    level.

Maybe the reference to "encryption level" should be corrected to
"packet number space."

-- 
Kazuho Oku

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 25 May 2018 06:51:14 -0700

> Maybe the reference to "encryption level" should be corrected to

I agree. this is a defect.

-Ekr

> >>> avoid the pathological case where acknowledgments are sent
> indefinitely..
> >>
> >> https://github.com/h2o/quicly/pull/8/commits/
> eee6eb5fc7c0359edfeb0b8f89ae9ac191ef0758.
> >>>> > A thought occurs to me: a CRYPTO_HS frame with the fin bit set can
> be
> >>> used
> >>>> > to indicate that the handshake is done.This could be used as a
> signal
> >>> (aka