Subject: Re: New Version Notification for draft-kuhn-quic-0rtt-bdp-07.txt

From: Kazuho Oku <[email-masked]>
Date: Tue, 27 Oct 2020 06:36:12 +0900

Hello,

Thank you for the draft. I just had the opportunity to read it, and
therefore am leaving comments.

First of all, I think it is a good idea to write down how endpoints can
reuse information from previous connections when reconnecting. Many people
have been talking about the idea, it seems that there are pitfalls, and
having a compilation of good practices could help.

At the same time, I was puzzled with the following two aspects of the draft:

1. The draft requires the characteristics of paths be communicated via
session tickets. IIUC, QUIC resumes the properties of the *endpoint* using
TLS session tickets, while the properties of a path (e.g., peer's IP
address) is to be remembered by a NEW_TOKEN token. The draft's use of
session ticket goes against that principle.

2. The draft suggests that the server's properties (e.g., server's BDP) be
shared with the client. Is there any reason why they have to be shared? I
tend to think that each endpoint unilaterally remembering what they prefer
provides the most agility without the need to standardize something.

Cheers.

2020年5月18日(月) 17:17 Kuhn Nicolas <[email-masked]>:

> Hi,
>
> We have been working on an updated version of the 0-RTT draft.
>
> In short, the document describes a generic method to exchange path
> parameters
> relating to transport.  The additional transport parameters can help
> a connection that continues after an interruption or restarts by
> sharing connection properties.  They can be used to increase the
> performance for a path with large RTT.
>
> The motivation is described in the document, but here is an extract :
> "Currently each side has a proprietary solution to measure and to
>    store path characteristics.  Recalling the parameters of a previous
>    connection would allow:
>    o  Server and client to re-initialise the state from previously
>       established parameters (i.e., minRTT, MTU, bottleneck capacity,
>       etc.)
>    o  The client to prepare the right resources
>    o  The server to adapt to non-default path characteristics"
>
> Please let us know if you have any questions/comments.
>
> Kind regards,
>
> Nico on the behalf of the authors
>
>
> -----Message d'origine-----
> De : [email-masked] <[email-masked]>
> Envoyé : lundi 18 mai 2020 10:12
> À : Emile Stephan <[email-masked]>; Kuhn Nicolas <
> [email-masked]>; Gorry Fairhurst <[email-masked]>; Stephan
> Emile <[email-masked]>; Tom Jones <[email-masked]>
> Objet : New Version Notification for draft-kuhn-quic-0rtt-bdp-07.txt
>
>
> A new version of I-D, draft-kuhn-quic-0rtt-bdp-07.txt has been
> successfully submitted by Nicolas Kuhn and posted to the IETF repository.
>
> Name:           draft-kuhn-quic-0rtt-bdp
> Revision:       07
> Title:          Transport parameters for 0-RTT connections
> Document date:  2020-05-18
> Group:          Individual Submission
> Pages:          8
> URL:
> https://www.ietf.org/internet-drafts/draft-kuhn-quic-0rtt-bdp-07.txt
> Status:         https://datatracker.ietf.org/doc/draft-kuhn-quic-0rtt-bdp/
> Htmlized:       https://tools.ietf.org/html/draft-kuhn-quic-0rtt-bdp-07
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-kuhn-quic-0rtt-bdp
> Diff:
> https://www.ietf.org/rfcdiff?url2=draft-kuhn-quic-0rtt-bdp-07
>
> Abstract:
>    0-RTT mechanisms reduce the time it takes for the first bytes of
>    application data to be processed in a transport connection and can
>    greatly reduce connection latency during setup.  The 0-RTT transport
>    features described by quic-transport help clients establish secure
>    connections with a minimal number of round-trips.
>
>    This document describes a generic method to exchange path parameters
>    relating to transport.  The additional transport parameters can help
>    a connection that continues after an interruption or restarts by
>    sharing connection properties.  They can be used to increase the
>    performance for a path with large RTT.
>
>
>
>
> Please note that it may take a couple of minutes from the time of
> submission until the htmlized version and diff are available at
> tools.ietf.org.
>
> The IETF Secretariat
>
>
>

-- 
Kazuho Oku

---

From: Christian Huitema <[email-masked]>
Date: Mon, 26 Oct 2020 16:12:52 -0700

On 10/26/2020 2:36 PM, Kazuho Oku wrote:
> First of all, I think it is a good idea to write down how endpoints
> can reuse information from previous connections when reconnecting.
> Many people have been talking about the idea, it seems that there are
> pitfalls, and having a compilation of good practices could help.
>
> session tickets. IIUC, QUIC resumes the properties of the *endpoint*
> using TLS session tickets, while the properties of a path (e.g.,
> peer's IP address) is to be remembered by a NEW_TOKEN token. The
> draft's use of session ticket goes against that principle.

I also think that forcing the information in the session ticket is a bad
idea. As Kazuho says, the session ticket is used to resume sessions, not
necessarily from the same network location at which the session ticket
was acquired. Using a "token" is better from that point of view. It also
has the advantage that tokens are fully managed within the QUIC layer,
without any dependency on the TLS stack, which makes the implementation
significantly simpler. However, there is still an issue because New
Tokens are normally sent just once at the beginning of the connection,
and are used to manage the "stateless retry" process. If the server
sends several New Tokens, the client is expected to remember all of
them, and use them only once.

It might be simpler to create a new frame, very similar to the "New
Token" frame, maybe calling it "BDP_TOKEN", and a "bdp_token" transport
parameter. The frame carries a binary blob that encode server defined
data. The server sends the client whatever blob it wants. It may send it
several time, in which case the client only remembers the last one. The
client puts the blob in the bdp_token TP, or if no token is available
sends an empty blob to signal its support for the process. The server
may reply with an empty token if it does support the process.

>
> 2. The draft suggests that the server's properties (e.g., server's
> BDP) be shared with the client. Is there any reason why they have to
> be shared? I tend to think that each endpoint unilaterally remembering
> what they prefer provides the most agility without the need to
> standardize something.

I think that "the endpoint remembers" should be the starting point. For
one thing, we should also care of the scenario in which the client
pushes data to the server, in which case the client needs to remember
the BDP and RTT of the previous connection to the server. The server
could do the same thing, just keep an LRU list of recent client
connection addresses and the associated BDP and RTT.

The whole business of sending blobs is just an optimization on top of
the "endpoint remembers" strategy. We may be concerned that servers have
to remember data for too many clients, that local server storage would
not scale, or maybe that the same client hits a different server in the
farm each time its reconnect. Hence the idea of storing these parameters
in a blob, sending the blob to the client during the previous
connection, and have the client provide the blob back during the new
connection. Also, we seem concerned that the server does not trust the
client, because otherwise the client could just add a couple of TP such
as "previous RTT" and "previous Bandwidth", based on what the client
observed before. Managing blobs has some complexity, so the tradeoffs
should be explored.

-- Christian Huitema

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 27 Oct 2020 08:42:21 +0900

> may reply with an empty token if it does support the process.

I tend to agree with the high order design, and it is my understanding that
use of NEW_TOKEN tokens is fine for the purpose.

The transport draft has a paragraph stating that a server might send new
NEW_TOKEN tokens as the state of the connection changes, and that it makes
sense for the client to use the most recently received NEW_TOKEN token.
https://quicwg.org/base-drafts/draft-ietf-quic-transport.html#section-8.1.3-9

-- 
Kazuho Oku

---

From: Matt Joras <[email-masked]>
Date: Mon, 26 Oct 2020 17:14:56 -0700

What Christian describes, a sort of "BDP Token" is something we are
currently prototyping. Specifically we are using a new frame type to
proactively send information from server -> client containing metrics of
interest. The first such metric is "goodput", as measured recently by the
server, and we intend to use it as an input to the client-side ABR scheme
for the video player.

In general I thought this idea could be extended and standardized. It seems
likely that different applications will have interest in the same set of
metrics as measured by the peer (e.g. RTT and bandwidth).

Matt Joras

On Mon, Oct 26, 2020 at 4:43 PM Kazuho Oku <[email-masked]> wrote:

> I tend to agree with the high order design, and it is my understanding
> that use of NEW_TOKEN tokens is fine for the purpose.
>

---

From: Christian Huitema <[email-masked]>
Date: Mon, 26 Oct 2020 17:08:07 -0700

On 10/26/2020 4:42 PM, Kazuho Oku wrote:
>
>
> 2020年10月27日(火) 8:24 Christian Huitema <[email-masked]
> <mailto:[email-masked]>>:
>
>
>     > At the same time, I was puzzled with the following two aspects
>     of the
>     > draft:
>     >
> The transport draft has a paragraph stating that a server might send
> new NEW_TOKEN tokens as the state of the connection changes, and that
> it makes sense for the client to use the most recently received
> NEW_TOKEN token.
Yes, you are probably right. It is simpler to use a "new token" than to
try create yet another mechanism, and that's much cleaner than trying to
change the value of the TLS resume ticket.

>     connection addresses and the associated BDP and RTT.

Reading the draft again, I am concerned with statements such as "for
0-RTT data to be sent, the server must remember [various congestion
control parameters]". In QUIC, servers do not send 0-RTT data, only the
clients do. Servers MAY start sending data in response to a 0-RTT packet
sent by the client, but at that point the server has received the new
values of the client's transport parameters, so it can just use the
correct values, no remembering needed. Is the current text a typo,
writing "server" where "client" was meant?

There is also some ambiguity in the draft regarding the status of the
"bdp parameters". Why do we have identifiers associated with these
parameters? Is the intent to send these as unencrypted transport
parameters? That's certainly doable, although I do not understand well
the "max packet number" parameter. That seems redundant with the
"initial_max_data" and "initial_max_stream_data" parameters.

Of course, if these are clear text parameters, we have to be concerned
with issues of privacy and security. Can the value of the parameters be
used to track clients? Can the client send creative values and affect
server behavior?

-- Christian Huitema

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 27 Oct 2020 00:23:05 +0000

Of course, you could also put such info in an HTTP header :) [1]

[1] https://tools.ietf.org/html/draft-ohanlon-transport-info-header-01

On Tue, 27 Oct 2020, 00:15 Matt Joras, <[email-masked]> wrote:

> In general I thought this idea could be extended and standardized. It
> seems likely that different applications will have interest in the same set
> of metrics as measured by the peer (e.g. RTT and bandwidth).
>
> Matt Joras
>
>

---

From: Matt Joras <[email-masked]>
Date: Mon, 26 Oct 2020 18:12:43 -0700

Indeed, but as much as I love HTTP it's not the only protocol we have on
top of QUIC. A consistency argument can also be made for having a
connection-level metric tied to a connection-level semantic (i.e. a QUIC
frame) rather than the transactional-level semantic (HTTP header).

Matt Joras

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 27 Oct 2020 01:20:23 +0000

> frame) rather than the transactional-level semantic (HTTP header).

I agree! A QUIC-level frame could be the most appropriate thing in this
case. I think this will be an interesting space for innovation. And let's
not forget all the other datagram-oriented protocols that have preceeded -
so perhaps there will be some re-innovation.

This makes me wonder how Web Transport might approach supporting QUIC
extensions. But that is a digression.

Cheers
Lucas

---

From: Christian Huitema <[email-masked]>
Date: Mon, 26 Oct 2020 19:03:10 -0700

On 10/26/2020 6:20 PM, Lucas Pardue wrote:
>
>
> On Tue, 27 Oct 2020, 01:12 Matt Joras, <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Indeed, but as much as I love HTTP it's not the only protocol we
>     have on top of QUIC. A consistency argument can also be made for
>     having a connection-level metric tied to a connection-level
>     semantic (i.e. a QUIC frame) rather than the transactional-level
>     semantic (HTTP header).
>
>
> I agree! A QUIC-level frame could be the most appropriate thing in
> this case. I think this will be an interesting space for innovation.
> And let's not forget all the other datagram-oriented protocols that
> have preceeded - so perhaps there will be some re-innovation.

I am glad that we agree. Now, we have an issue regarding security.
Nicolas Kuhn and his coauthors have pursued a design in which the server
sends an encrypted blob to the client, and then client echoes it in the
new connection. This is largely based on concerns about potential
attacks. Suppose the client said "I received at 1Gbps last time", when
in fact it can only absorb 10 Mbps. Some bad stuff might well be
happening along the way. But then, Matt is looking at passing statistics
from server to client, so the client can debug issues, display
statistics in the app, and potentially also reuse the statistics to
inform server that the last connection had an RTT of 600 ms and a
data-rate of 100 Mbps, and maybe we should shortcut initial congestion
control in order to gain lots of time. How do we reconcile these
multiple goals?

-- Christian Huitema

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 27 Oct 2020 11:29:36 +0900

2020年10月27日(火) 11:14 Christian Huitema <[email-masked]>:

> I am glad that we agree. Now, we have an issue regarding security. Nicolas
> Kuhn and his coauthors have pursued a design in which the server sends an
> encrypted blob to the client, and then client echoes it in the new
> connection. This is largely based on concerns about potential attacks.
> Suppose the client said "I received at 1Gbps last time", when in fact it
> can only absorb 10 Mbps. Some bad stuff might well be happening along the
> way. But then, Matt is looking at passing statistics from server to client,
> so the client can debug issues, display statistics in the app, and
> potentially also reuse the statistics to inform server that the last
> connection had an RTT of 600 ms and a data-rate of 100 Mbps, and maybe we
> should shortcut initial congestion control in order to gain lots of time.
> How do we reconcile these multiple goals?
>

I think that the answer is simple.

Loss recovery and CC states are properties maintained by an endpoint. The
peer should not be given the right to change them (e.g., CWND, RTT).
Servers might offload the state to the client, but that state has to be
encrypted and integrity-protected. NEW_TOKEN tokens cover this purpose.

OTOH, servers can expose any information to the client regarding the
quality of the connection. It is totally reasonable to give the client the
bandwidth information that the server retains, so that the client could
choose a good bandwidth. There are no security concerns providing such
information, regardless of the medium being a QUIC frame or a HTTP header.
And that's because it does not affect loss recovery or CC.

-- Christian Huitema

-- 
Kazuho Oku

---

From: Ian Swett <[email-masked]>
Date: Tue, 27 Oct 2020 09:39:08 -0400

I agree with most of the points made by Kazuho and Christian.

In particular, I think relying on information the client repeats back to
the server which is not encrypted by the server is potentially quite
dangerous.  Even with that, there's real potential for abuse, particularly
when combined with 0-RTT, so if we're going to document something, I
believe we need a lot of MUST NOTs to prevent that abuse.  Using the same
token across many connections seems particularly concerning.

On Mon, Oct 26, 2020 at 10:30 PM Kazuho Oku <[email-masked]> wrote:

>> sends an encrypted blob to the client, and then client echoes it in the new