Subject: When should server-chosen connection IDs be sent and how are they indicated?

From: Ryan Hamilton <[email-masked]>
Date: Mon, 27 Feb 2017 16:30:02 -0800

Howdy All,

I just filed issue #349 <https://github.com/quicwg/base-drafts/issues/349> to
resolve how server-chosen connection IDs are to be used. To save a click,
here's what I wrote there:

--------------------
My sense is that we'd like the server to chose the "real" connection ID to
be used for a connection. Among other things, this enables servers to embed
routing/load balancing information into connection IDs.

I have heard two options for what the client should do before it has a
server-chosen connection ID.

1. Send a random connection ID
2. Send a connection ID of 0

If we go with approach 1, then we need some mechanism that load balancers
can use to detect if a packet's connection ID is server-chosen or not. It
seems likely that all packets encrypted with 1-RTT keys could use server
generated connection IDs. But other packets it's not so clear. Consider the
case of a non-0-RTT handshake. The initial client handshake packet would
include a client-chosen connection ID. When the server replies, it could
use a server-chosen connection ID. If so then when the client sends the
next handshake packet it could use the server-chosen connection ID as well.
But that means that we'd need a mechanism to disambiguate client handshake
packets with client-chosen connection IDs from client handshake packets
with server-chosen connection IDs. We could devote bits or packet types to
indicate this. I suspect the same argument might hold for server handshake
packets. We might not want the server to chose a connection ID until it is
ready to accept the handshake.

One benefit of this approach, however, is that a QUIC server could chose to
simply use the client's connection ID for the duration of the connection.
This would mean that it would not need to be able to generate a new
connection ID that is guaranteed to be sent by the load balancer back to
itself. In cases where the vendor of the load balancer and the server are
not the same, this might be valuable.

If we go with approach 2, then it's obvious that any packet with a non-zero
connection ID has a server-chosen connection ID (though the server still
may want to verify that it's a "valid" connection ID according to the
algorithm it uses).

On the other hand, this requires QUIC servers which are behind load
balancers to be able to generate connection IDs which the load balancer
will route back to them. If the server and load balancers are from
different vendors, this might be challenging.
--------------------

I look forward to your thoughts.

Cheers,

Ryan

---

From: Ted Hardie <[email-masked]>
Date: Mon, 27 Feb 2017 17:08:13 -0800

Comment in-line.

On Mon, Feb 27, 2017 at 4:30 PM, Ryan Hamilton <[email-masked]> wrote:

> One benefit of this approach, however, is that a QUIC server could chose
> to simply use the client's connection ID for the duration of the
> connection. This would mean that it would not need to be able to generate a
> new connection ID that is guaranteed to be sent by the load balancer back
> to itself. In cases where the vendor of the load balancer and the server
> are not the same, this might be valuable.
>
>
ekr has made a proposal that the server be able to supply multiple
connection IDs to a client, so that it can use a fresh connection ID when
shifting link origin or engaging in another change where continuing to use
the same connection ID would provide linkability.  I like that property a
lot, but I think that means the QUIC server still needs to be able to
generate new connection IDs that are guaranteed to be sent by the load
balancer back to itself, even if it retains this connection ID for the
duration of the unaltered connection.  (Note that sending a fresh
connection ID from the pool provided by a server is not enough to avoid
linkability; as dkg has pointed out, you'd also have to shift sequence
number and watch for other signals).

regards,

Ted

> If we go with approach 2, then it's obvious that any packet with a
> non-zero connection ID has a server-chosen connection ID (though the server
> still may want to verify that it's a "valid" connection ID according to the
> algorithm it uses).
>

---

From: Ryan Hamilton <[email-masked]>
Date: Mon, 27 Feb 2017 17:11:35 -0800

> number and watch for other signals).

​Indeed. In such a world, a connection in which the same client-chosen
connection ID is used throughout the connection would be prohibited from
performing such actions and a new connection would need to be opened
instead. This is, obviously, not the highest performance solution but it is
simple. ​

---

From: "Lubashev, Igor" <[email-masked]>
Date: Tue, 28 Feb 2017 01:16:26 +0000

Not sure whether a discussion here or on the issue page is preferable, so both. ☺

Ø  We might not want the server to chose a connection ID until it is ready to accept the handshake.

Or maybe the opposite -- the server can choose the assign a connection ID to this connection at the first opportunity even if it is not yet ready to accept the connection.

An example of this is version negotiation. During new software rollout, backend servers may be running different versions of the software and hence supporting different QUIC versions. When the server is doing version negotiation, it may be very interested in ensuring that the client comes back to this backend server using the QUIC version it requested. If ConnectionID is used by the load balancers to route to backend servers, the server will likely want to generate a new ConnectionID during version negotiation and before it is ready to accept the handshake.

Of course, there must be a way to look at a packet and know that it is using Server's Connection ID.

-          Igor

From: Ryan Hamilton [mailto:[email-masked]]
Sent: Monday, February 27, 2017 7:30 PM
To: IETF QUIC WG <[email-masked]>
Subject: When should server-chosen connection IDs be sent and how are they indicated?

Howdy All,

I just filed issue #349<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_349&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=LQPa6MGbPJiB88xQL_AINFO4HuHCTtdUsf1I5rDbuzE&s=PxupuWh-eo6D73e7KcdSeb28cnOQ90W5GnRCEAi7f24&e=> to resolve how server-chosen connection IDs are to be used. To save a click, here's what I wrote there:

--------------------
My sense is that we'd like the server to chose the "real" connection ID to be used for a connection. Among other things, this enables servers to embed routing/load balancing information into connection IDs.

I have heard two options for what the client should do before it has a server-chosen connection ID.

If we go with approach 1, then we need some mechanism that load balancers can use to detect if a packet's connection ID is server-chosen or not. It seems likely that all packets encrypted with 1-RTT keys could use server generated connection IDs. But other packets it's not so clear. Consider the case of a non-0-RTT handshake. The initial client handshake packet would include a client-chosen connection ID. When the server replies, it could use a server-chosen connection ID. If so then when the client sends the next handshake packet it could use the server-chosen connection ID as well. But that means that we'd need a mechanism to disambiguate client handshake packets with client-chosen connection IDs from client handshake packets with server-chosen connection IDs. We could devote bits or packet types to indicate this. I suspect the same argument might hold for server handshake packets. We might not want the server to chose a connection ID until it is ready to accept the handshake.

One benefit of this approach, however, is that a QUIC server could chose to simply use the client's connection ID for the duration of the connection. This would mean that it would not need to be able to generate a new connection ID that is guaranteed to be sent by the load balancer back to itself. In cases where the vendor of the load balancer and the server are not the same, this might be valuable.

If we go with approach 2, then it's obvious that any packet with a non-zero connection ID has a server-chosen connection ID (though the server still may want to verify that it's a "valid" connection ID according to the algorithm it uses).

On the other hand, this requires QUIC servers which are behind load balancers to be able to generate connection IDs which the load balancer will route back to them. If the server and load balancers are from different vendors, this might be challenging.
--------------------

Cheers,

Ryan

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 28 Feb 2017 13:00:15 +0100

Hi all,

> Am 28.02.2017 um 02:11 schrieb Ryan Hamilton <[email-masked]>:
> 
> 
> ekr has made a proposal that the server be able to supply multiple connection IDs to a client, so that it can use a fresh connection ID when shifting link origin or engaging in another change where continuing to use the same connection ID would provide linkability.  I like that property a lot, but I think that means the QUIC server still needs to be able to generate new connection IDs that are guaranteed to be sent by the load balancer back to itself, even if it retains this connection ID for the duration of the unaltered connection.  (Note that sending a fresh connection ID from the pool provided by a server is not enough to avoid linkability; as dkg has pointed out, you'd also have to shift sequence number and watch for other signals).
> 
> ​Indeed. In such a world, a connection in which the same client-chosen connection ID is used throughout the connection would be prohibited from performing such actions and a new connection would need to be opened instead. This is, obviously, not the highest performance solution but it is simple. ​
> 
I also like this proposal a lot but you can never fully avoid likability e.g. the client might not be aware that the path/5-tuple has changed. Also I think this will always be an optional feature because the server might just not provide you additional connection ids for whatever reason. In this case the client can use the old one which might make it possible to link to connections or it can start a new connection eating the penalty of having to re-request any data that was under transmission.

---

From: Ted Hardie <[email-masked]>
Date: Tue, 28 Feb 2017 08:37:12 -0800

On Tue, Feb 28, 2017 at 4:00 AM, Mirja Kühlewind <
[email-masked]> wrote:

> I also like this proposal a lot but you can never fully avoid likability
> e.g. the client might not be aware that the path/5-tuple has changed. Also
> I think this will always be an optional feature because the server might
> just not provide you additional connection ids for whatever reason. In this
> case the client can use the old one which might make it possible to link to
> connections or it can start a new connection eating the penalty of having
> to re-request any data that was under transmission.

There are two cases we have to consider: linkability due to a change in the
network not visible to the client and linkability that occurs because of a
client-initiated change.  You're correct that it cannot update the
connection ID to avoid linkage when it doesn't know of the change, but when
the client initiates the change (e.g. moves from WiFi to cellular), it
certainly could.  If this feature is optional, then you likely do have to
have a configuration frob that the client uses to say "re-use of connect
IDs is okay" vs. "not okay, please use 0RTT when transitions occur and only
one connection ID is available".

Personally, I'd be happy if it were not optional, and I think the protocol
to share out a pool of connection IDs from a load balancer/load balancer
farm is not that complicated. It's not in charter for QUIC, but if we
wanted a proposal for it, I'm guessing we could knock something together by
the Paris interim and take it to dispatch in Prague.

Ted

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 28 Feb 2017 12:11:37 -0600

On Mon, Feb 27, 2017 at 7:08 PM, Ted Hardie <[email-masked]> wrote:

>> I just filed issue #349
>> <https://github.com/quicwg/base-drafts/issues/349> to resolve how
>> server-chosen connection IDs are to be used. To save a click, here's what I
>> wrote there:
>>
>> --------------------
>> My sense is that we'd like the server to chose the "real" connection ID
>> to be used for a connection. Among other things, this enables servers to
>> embed routing/load balancing information into connection IDs.
>>

I'm sure I did mention this option at one point or another, but for the
reasons we discussed in
Tokyo, I don't think it actually works that well. Rather, I think you
probably need some mechanism
where a different conn id field appears in each message, but somehow tied
back to the connection.
As you say, you would also need to do something about the sequence number
(see previous
comments on that). I don't have any really good designs here, but there are
not-great designs :)

-Ekr

regards,

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 28 Feb 2017 10:13:37 -0800

On Tue, Feb 28, 2017 at 8:37 AM, Ted Hardie <[email-masked]> wrote:

> There are two cases we have to consider: linkability due to a change in
> the network not visible to the client and linkability that occurs because
> of a client-initiated change.  You're correct that it cannot update the

​My sense is that the working group believes that it is not acceptable for
a connection ID to be reused across client initiated network changes (WiFi
to Cellular). If this is accurate, then if a server did not present any
additional connection IDs to a client, the client would simply tear down
the connection if it changed networks, just as if it would do to TCP
connections. So we can still have this "list of connection IDs" feature
being optional without needing to empower clients to use the connection ID
across network changes, right?

I'm also not convinced that it is "not complicated" for a server to be in
possession of a set of connection IDs that will map back to itself in some
deployments, but I'd be happy to be wrong!

---

From: Ted Hardie <[email-masked]>
Date: Tue, 28 Feb 2017 11:19:59 -0800

On Tue, Feb 28, 2017 at 10:13 AM, Ryan Hamilton <[email-masked]> wrote:

>> Personally, I'd be happy if it were not optional, and I think the
>> protocol to share out a pool of connection IDs from a load balancer/load
>> balancer farm is not that complicated. It's not in charter for QUIC, but if
>> we wanted a proposal for it, I'm guessing we could knock something together
>> by the Paris interim and take it to dispatch in Prague.
>>
>
> Yes, that would be the alternate strategy.  At that point, the client is
not expecting the new server to have state, so it is explicitly giving that
up to avoid linkability.

> deployments, but I'd be happy to be wrong!

Well, there are clearly proprietary solutions in this space, as well as
fairly basic methods of range assignment within the full pool that work
with a standard fan-out of servers behind a load balancer.  They may not
cover all deployments, of course.

regards,

Ted

---

From: Mike Bishop <[email-masked]>
Date: Tue, 28 Feb 2017 19:40:23 +0000

Yes, but the problem is dealing with a situation where the load balancer and the server are independently managed, as in a hosting environment.  There would need to be something standardized.  (Of course, if the load balancer can be stateful, it doesn’t need nearly as much of this machinery.)

The other issue that was pointed out to me by one of our SLB guys is that the load balancer only sees packets in the client-to-server direction, not the server’s responses.  So a Connection ID change proposed by the server is invisible to the load balancer until it sees a future client packet with the same 4-tuple and a different connection ID.  But simply allowing packets from the same source with a different ID to inform the load balancer of an ID change seems ripe for attack by spoofing the source – we’d need to tightly scope how a load balancer can identify that safely.  It also means that our agreement that client IP/port can’t change “during the handshake” actually has to extend until the server has received packets with the new connection ID.

From: QUIC [mailto:[email-masked]] On Behalf Of Ted Hardie
Sent: Tuesday, February 28, 2017 11:20 AM
To: Ryan Hamilton <[email-masked]>
Cc: IETF QUIC WG <[email-masked]>; Mirja Kühlewind <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

On Tue, Feb 28, 2017 at 10:13 AM, Ryan Hamilton <[email-masked]<mailto:[email-masked]>> wrote:

On Tue, Feb 28, 2017 at 8:37 AM, Ted Hardie <[email-masked]<mailto:[email-masked]>> wrote:
There are two cases we have to consider: linkability due to a change in the network not visible to the client and linkability that occurs because of a client-initiated change.  You're correct that it cannot update the connection ID to avoid linkage when it doesn't know of the change, but when the client initiates the change (e.g. moves from WiFi to cellular), it certainly could.  If this feature is optional, then you likely do have to have a configuration frob that the client uses to say "re-use of connect IDs is okay" vs. "not okay, please use 0RTT when transitions occur and only one connection ID is available".
Personally, I'd be happy if it were not optional, and I think the protocol to share out a pool of connection IDs from a load balancer/load balancer farm is not that complicated. It's not in charter for QUIC, but if we wanted a proposal for it, I'm guessing we could knock something together by the Paris interim and take it to dispatch in Prague.

​My sense is that the working group believes that it is not acceptable for a connection ID to be reused across client initiated network changes (WiFi to Cellular). If this is accurate, then if a server did not present any additional connection IDs to a client, the client would simply tear down the connection if it changed networks, just as if it would do to TCP connections. So we can still have this "list of connection IDs" feature being optional without needing to empower clients to use the connection ID across network changes, right?

Yes, that would be the alternate strategy.  At that point, the client is not expecting the new server to have state, so it is explicitly giving that up to avoid linkability.

I'm also not convinced that it is "not complicated" for a server to be in possession of a set of connection IDs that will map back to itself in some deployments, but I'd be happy to be wrong!

Well, there are clearly proprietary solutions in this space, as well as fairly basic methods of range assignment within the full pool that work with a standard fan-out of servers behind a load balancer.  They may not cover all deployments, of course.
regards,
Ted

---

From: Mirja Kühlewind <[email-masked]>
Date: Tue, 28 Feb 2017 21:05:40 +0100

Even if the load balancer could see the traffic in the other direction, the new connection ids should not be exposed to the path (aka this information must be encrypted), otherwise it’s still linkable. 

If you want to avoid additional signaling (which could be done if the load balancer and the server have some kind of trust relationship), then both need to know the algorithm that is used to create the additional ids. If you don’t have any trust relationship (neither for signaling, nor for some kind of initial configuration) than you are the kind of middlebox that we don’t want to be able to link different connections together.

Mirja

> Am 28.02.2017 um 20:40 schrieb Mike Bishop <[email-masked]>:
>

---

From: Kyle Rose <[email-masked]>
Date: Tue, 28 Feb 2017 15:24:41 -0500

On Tue, Feb 28, 2017 at 3:05 PM, Mirja Kühlewind <
[email-masked]> wrote:

> Even if the load balancer could see the traffic in the other direction,
> the new connection ids should not be exposed to the path (aka this
> information must be encrypted), otherwise it’s still linkable.
>
> If you want to avoid additional signaling (which could be done if the load
> balancer and the server have some kind of trust relationship), then both
> need to know the algorithm that is used to create the additional ids. If
> you don’t have any trust relationship (neither for signaling, nor for some
> kind of initial configuration) than you are the kind of middlebox that we
> don’t want to be able to link different connections together.
>

Whenever I've thought about this problem, it always comes back to indexing:
since the whole goal is to figure out the connection context (including
encryption key, something probably completely unknown to the load balancer)
from some plaintext signal that must be different in each packet, the
system needs one of the following:

(1) A sufficient, but presumably limited, table of IDs that index to
connections (sized perhaps on the basis of # of packets in flight), plus
the ability to reuse IDs or otherwise re-synchronize (e.g., fall back to
PKC) in case of packet loss
(2) A shared ID generation algorithm that may only be computed by
endpoints, plus some recovery algorithm in case of de-synchronization
(e.g., due to packet reordering)
(3) PKC (expensive on a per-packet basis)

In practice, there may be little enough reordering that with some
reasonable window (say 2x # packets in flight worth of IDs), reasonable
heuristics around use of IDs to prevent de-synchronization, and
loss-recovery that simply doesn't care about packets that are too
out-of-order, that PKC fallback would be rare.

Kyle

---

From: "Lubashev, Igor" <[email-masked]>
Date: Tue, 28 Feb 2017 20:26:52 +0000

When lad balancers and the servers are working together, the server picks the Connection ID that encodes its identity for the load balancer.  In the degenerate case, "10 MSBs of a Connection ID is the Server ID".  It is very easy for a Server to generate a handful of such IDs that will all point to itself.

There is no direct signal that can tie these Connection IDs together other than they are all pointing at the same server.  It is only slightly more bits of information than is contained in the dest-ip of the packet.

- Igor

-----Original Message-----
From: Mirja Kühlewind [mailto:[email-masked]] 
Sent: Tuesday, February 28, 2017 3:06 PM
To: Mike Bishop <[email-masked]>
Cc: Ted Hardie <[email-masked]>; IETF QUIC WG <[email-masked]>; Ryan Hamilton <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

Mirja

---

From: "Lubashev, Igor" <[email-masked]>
Date: Tue, 28 Feb 2017 20:39:00 +0000

Kyle, so you are thinking of some Rolling Code<https://en.wikipedia.org/wiki/Rolling_code> to replace ConnectionIDs after a 1-RTT handshake?  That’s an interesting idea, though it may be difficult for Servers to tell apart completely martian packets (unknown connections) vs very delayed packets (this makes a difference in whether to RST the connection or drop the packet).

We would still want some additional immutable Server-Contributed data for load balancers, though.

-          Igor

From: Kyle Rose [mailto:[email-masked]]
Sent: Tuesday, February 28, 2017 3:25 PM
To: Mirja Kühlewind <[email-masked]>
Cc: Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; IETF QUIC WG <[email-masked]>; Ryan Hamilton <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

On Tue, Feb 28, 2017 at 3:05 PM, Mirja Kühlewind <[email-masked]<mailto:[email-masked]>> wrote:
Even if the load balancer could see the traffic in the other direction, the new connection ids should not be exposed to the path (aka this information must be encrypted), otherwise it’s still linkable.

Whenever I've thought about this problem, it always comes back to indexing: since the whole goal is to figure out the connection context (including encryption key, something probably completely unknown to the load balancer) from some plaintext signal that must be different in each packet, the system needs one of the following:

(1) A sufficient, but presumably limited, table of IDs that index to connections (sized perhaps on the basis of # of packets in flight), plus the ability to reuse IDs or otherwise re-synchronize (e.g., fall back to PKC) in case of packet loss
(2) A shared ID generation algorithm that may only be computed by endpoints, plus some recovery algorithm in case of de-synchronization (e.g., due to packet reordering)
(3) PKC (expensive on a per-packet basis)

In practice, there may be little enough reordering that with some reasonable window (say 2x # packets in flight worth of IDs), reasonable heuristics around use of IDs to prevent de-synchronization, and loss-recovery that simply doesn't care about packets that are too out-of-order, that PKC fallback would be rare.

Kyle

---

From: Kyle Rose <[email-masked]>
Date: Tue, 28 Feb 2017 15:41:21 -0500

Exactly. Good to know this has a name. (We can probably also do better than
a garage door opener.)

Kyle

On Tue, Feb 28, 2017 at 3:39 PM, Lubashev, Igor <[email-masked]> wrote:

> Kyle, so you are thinking of some Rolling Code
> <https://en.wikipedia.org/wiki/Rolling_code> to replace ConnectionIDs
> after a 1-RTT handshake?  That’s an interesting idea, though it may be
> difficult for Servers to tell apart completely martian packets (unknown
> connections) vs very delayed packets (this makes a difference in whether to
> RST the connection or drop the packet).
>
>
>
> *From:* Kyle Rose [mailto:[email-masked]]
> *Sent:* Tuesday, February 28, 2017 3:25 PM
> *To:* Mirja Kühlewind <[email-masked]>
> *Cc:* Mike Bishop <[email-masked]>; Ted Hardie <
> [email-masked]>; IETF QUIC WG <[email-masked]>; Ryan Hamilton <
> [email-masked]>
> *Subject:* Re: When should server-chosen connection IDs be sent and how
> are they indicated?
>
>
>
> Whenever I've thought about this problem, it always comes back to
> indexing: since the whole goal is to figure out the connection context
> (including encryption key, something probably completely unknown to the
> load balancer) from some plaintext signal that must be different in each
> packet, the system needs one of the following:
>

---

From: Mike Bishop <[email-masked]>
Date: Tue, 28 Feb 2017 21:15:06 +0000

I don't think there's a desire to avoid linkability between the client's proposed connection ID and the connection ID being used for the remainder of the connection on the same path.  If there is, that changes some things, but I'd be curious to know the motivation.

I agree that we might want to avoid linkability between the connection ID of one connection/path and the connection ID the client will use on a different path.

-----Original Message-----
From: Mirja Kühlewind [mailto:[email-masked]] 
Sent: Tuesday, February 28, 2017 12:06 PM
To: Mike Bishop <[email-masked]>
Cc: Ted Hardie <[email-masked]>; Ryan Hamilton <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

Mirja

---

From: Ted Hardie <[email-masked]>
Date: Tue, 28 Feb 2017 13:56:51 -0800

On Tue, Feb 28, 2017 at 1:15 PM, Mike Bishop <[email-masked]>
wrote:

> I don't think there's a desire to avoid linkability between the client's
> proposed connection ID and the connection ID being used for the remainder
> of the connection on the same path.  If there is, that changes some things,
> but I'd be curious to know the motivation.
>
> I don't want that, as I think there is a likelihood that on-path devices
maintaining NAT state might start using connection ID as an input to
determining the safety of rebinding.  Even if that is safe from a server
endpoint perspective (since it will use connection ID), it could trigger
differential treatment by network elements using equal-cost multipath or
similar mechanisms to spread load.  Nat re-binding can, in those cases,
cause  out-of-order packet arrival, and we don't want to help that along.

regards,

Ted

> I agree that we might want to avoid linkability between the connection ID
> of one connection/path and the connection ID the client will use on a
> different path.
>
> > Am 28.02.2017 um 20:40 schrieb Mike Bishop <[email-masked]
> >:
> >
> > Yes, but the problem is dealing with a situation where the load balancer
> and the server are independently managed, as in a hosting environment.
> There would need to be something standardized.  (Of course, if the load
> balancer can be stateful, it doesn’t need nearly as much of this machinery.)
> >
> >
> >
> > The other issue that was pointed out to me by one of our SLB guys is
> that the load balancer only sees packets in the client-to-server direction,
> not the server’s responses.  So a Connection ID change proposed by the
> server is invisible to the load balancer until it sees a future client
> packet with the same 4-tuple and a different connection ID.  But simply
> allowing packets from the same source with a different ID to inform the
> load balancer of an ID change seems ripe for attack by spoofing the source
> – we’d need to tightly scope how a load balancer can identify that safely.
> It also means that our agreement that client IP/port can’t change “during
> the handshake” actually has to extend until the server has received packets
> with the new connection ID.
> >
> >
> >
> > ​My sense is that the working group believes that it is not acceptable
> for a connection ID to be reused across client initiated network changes
> (WiFi to Cellular). If this is accurate, then if a server did not present
> any additional connection IDs to a client, the client would simply tear
> down the connection if it changed networks, just as if it would do to TCP
> > I'm also not convinced that it is "not complicated" for a server to be
> in possession of a set of connection IDs that will map back to itself in
> some deployments, but I'd be happy to be wrong!
> >
> >
> >

---

From: Mike Bishop <[email-masked]>
Date: Tue, 28 Feb 2017 22:14:55 +0000

Okay – it’s good to clarify that there’s a discussion point here where I didn’t think we had one.  If QUIC supports rebinding via the Connection ID, isn’t the simple presence of a Connection ID in a QUIC packet sufficient to disclose that our connection is “safer” to rebind than other UDP traffic?  And I don’t think you can avoid disclosing the Connection ID, since the server needs it to find the keys to decrypt the non-disclosed stuff.  This feels like a foregone conclusion.

We can make non-linkability of those IDs a function of the protocol, but it costs us the ability to have less coordination between load balancers and servers and I don’t see that it really buys us anything in exchange for that.

From: Ted Hardie [mailto:[email-masked]]
Sent: Tuesday, February 28, 2017 1:57 PM
To: Mike Bishop <[email-masked]>
Cc: Mirja Kühlewind <[email-masked]>; Ryan Hamilton <[email-masked]>; IETF QUIC WG <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

On Tue, Feb 28, 2017 at 1:15 PM, Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
I don't think there's a desire to avoid linkability between the client's proposed connection ID and the connection ID being used for the remainder of the connection on the same path.  If there is, that changes some things, but I'd be curious to know the motivation.
I don't want that, as I think there is a likelihood that on-path devices maintaining NAT state might start using connection ID as an input to determining the safety of rebinding.  Even if that is safe from a server endpoint perspective (since it will use connection ID), it could trigger differential treatment by network elements using equal-cost multipath or similar mechanisms to spread load.  Nat re-binding can, in those cases, cause  out-of-order packet arrival, and we don't want to help that along.
regards,
Ted

-----Original Message-----
From: Mirja Kühlewind [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Tuesday, February 28, 2017 12:06 PM
To: Mike Bishop <[email-masked]<mailto:[email-masked]>>
Cc: Ted Hardie <[email-masked]<mailto:[email-masked]>>; Ryan Hamilton <[email-masked]<mailto:[email-masked]>>; IETF QUIC WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

Mirja

> Am 28.02.2017 um 20:40 schrieb Mike Bishop <[email-masked]<mailto:[email-masked]>>:
>
> From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Ted Hardie
> To: Ryan Hamilton <[email-masked]<mailto:[email-masked]>>
> Cc: IETF QUIC WG <[email-masked]<mailto:[email-masked]>>; Mirja Kühlewind <[email-masked]<mailto:[email-masked]>>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 1 Mar 2017 10:16:51 +1100

On 28 February 2017 at 23:00, Mirja Kühlewind
<[email-masked]> wrote:
> I also like this proposal a lot but you can never fully avoid likability e.g. the client might not be aware that the path/5-tuple has changed.

You can avoid linkability if you don't include a connection ID.  It
just makes recovery from a path change essentially impossible.

---

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 1 Mar 2017 00:18:18 +0100

...and no packet number and in the best case you don’t expose anything…

> Am 01.03.2017 um 00:16 schrieb Martin Thomson <[email-masked]>:
>

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 1 Mar 2017 09:12:02 +0900

2017-02-28 9:30 GMT+09:00 Ryan Hamilton <[email-masked]>:
> Howdy All,
>
> I just filed issue #349 to resolve how server-chosen connection IDs are to
> be used. To save a click, here's what I wrote there:
>
> --------------------
> include a client-chosen connection ID. When the server replies, it could use
> a server-chosen connection ID. If so then when the client sends the next
> handshake packet it could use the server-chosen connection ID as well. But
> that means that we'd need a mechanism to disambiguate client handshake
> packets with client-chosen connection IDs from client handshake packets with
> server-chosen connection IDs. We could devote bits or packet types to

>From the standpoint of a server developer without having the control
of the load balancer that would be used together, I'd be more
concerned if the server could be used together with a traditional load
balancer that only does balancing based on the 5-tuple.

Interoperability with such load balancers is inevitable (if not
preferable), and I would rather not want to support another kind of
load balancer that routes the packets using the connection ID,
especially in case it cannot route the packets accurately (i.e. when
5-tuple changes or when client starts using a new connection ID due to
a noticeable IP address change).

For servers running behind a load balancer using 5-tuple, it would be
a requirement to route packets that arrive at the wrong server to the
correct server (in terms of a QUIC connection). I do not think that
requirement (i.e. communication between the QUIC servers behind a load
balancer) would be an issue, considering the fact that some (if not
many) of TLS decryptors (e.g. haproxy) are doing that already for
sharing the TLS session cache.

> connection ID has a server-chosen connection ID (though the server still may
> want to verify that it's a "valid" connection ID according to the algorithm
> it uses).
>
> balancers to be able to generate connection IDs which the load balancer will
> route back to them. If the server and load balancers are from different
> vendors, this might be challenging.
> --------------------
>

-- 
Kazuho Oku

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 28 Feb 2017 16:33:27 -0800

> balancer that only does balancing based on the 5-tuple.

​Keep in mind that ​NAT boxes will occasionally re-bind the user's source
port in the middle of a QUIC connection. So if the load balancer only does
5 tuple hashing, that load balancer will not be able to handle NAT
rebinding. Since it's (effectively) not possible to prevent these from
happening, it's not clear to me that 5-tuple only hashing works for QUIC.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 1 Mar 2017 11:44:28 +1100

On 1 March 2017 at 07:39, Lubashev, Igor <[email-masked]> wrote:
> Kyle, so you are thinking of some Rolling Code to replace ConnectionIDs

I considered using a PRP (pseudo-random function) for generating new
connection IDs.  It doesn't work.  The need for it to be known to the
client means that you need to have a different PRP (or a different PRP
key) for every client - otherwise clients could guess the next value
in the sequence for any other client (linkability is broken).  With
many clients, avoiding collisions in that case would be foolish.

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 1 Mar 2017 09:45:59 +0900

2017-03-01 9:33 GMT+09:00 Ryan Hamilton <[email-masked]>:
> Keep in mind that NAT boxes will occasionally re-bind the user's source port
> in the middle of a QUIC connection. So if the load balancer only does 5
> tuple hashing, that load balancer will not be able to handle NAT rebinding.
> Since it's (effectively) not possible to prevent these from happening, it's
> not clear to me that 5-tuple only hashing works for QUIC.

My understanding is that when a NAT rebinding occurs it is likely that
load balancers relying on 5-tuple would start directing packets to the
wrong server.

In such case, a QUIC server running behind the load balancer would be
required to route such packets (arriving at the wrong server) to the
correct server. The correct server can send the responding packets
directly to the client.

This might sound inefficient, but I think it would not be a huge issue
assuming that the possibility of a NAT rebinding happening
mid-connection is not high. Also, for HTTP over QUIC at least, a
server can gracefully tear down the connection (i.e. stop accepting
new requests and close the connection after sending all the responses
in-flight) so that the client would eventually switch to a newly
established connection.

-- 
Kazuho Oku

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 1 Mar 2017 00:48:56 +0000

On Tue, Feb 28, 2017 at 4:12 PM, Kazuho Oku <[email-masked]<mailto:[email-masked]>> wrote:

For servers running behind a load balancer using 5-tuple, it would be a requirement to route packets that arrive at the wrong server to the correct server (in terms of a QUIC connection). I do not think that requirement (i.e. communication between the QUIC servers behind a load balancer) would be an issue, considering the fact that some (if not many) of TLS decryptors (e.g. haproxy) are doing that already for sharing the TLS session cache.

Not all load balancers are fronting backend servers in a single pop. We have load balancers that are fronting servers distributed across the continent. It would not be feasible to sync state of all concurrent connections across geographically dispersed pops. In fact, even within a pop I would prefer backend servers to continue NOT syncing states of connections (even if some popular load balancer implementations do it).

As it was said earlier, if you have a middle box (load balancer) that is not cooperating with the server, then either:

a.       You cannot survive NAT rebinding / network migration, or

b.       Your middle box is QUIC-aware, but to survive NAT rebinding / network migration, your QIUC implementation must leak ConnectionId onto the new path

-          Igor

---

From: Ryan Hamilton <[email-masked]>
Date: Tue, 28 Feb 2017 16:53:59 -0800

> correct server. The correct server can send the responding packets

​How would a server know which server to redirect this packet to?​

---

From: Martin Thomson <[email-masked]>
Date: Wed, 1 Mar 2017 12:04:28 +1100

On 1 March 2017 at 11:53, Ryan Hamilton <[email-masked]> wrote:
> How would a server know which server to redirect this packet to?

I assume that the server would have more state than the load balancer.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 1 Mar 2017 01:05:07 +0000

To elaborate on Kyle's idea, the initial ConnectionID would be selected as described, then client and server would negotiate some secure connection nonce, and the subsequent ConnectionID would be hash(nonce, packet#).  Nonce is a per-connection secret, so hash(nonce, x) cannot be predicted.

- Igor 

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Tuesday, February 28, 2017 7:44 PM
To: Lubashev, Igor <[email-masked]>
Cc: Kyle Rose <[email-masked]>; Mirja Kühlewind <[email-masked]>; Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; IETF QUIC WG <[email-masked]>; Ryan Hamilton <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

On 1 March 2017 at 07:39, Lubashev, Igor <[email-masked]> wrote:
> Kyle, so you are thinking of some Rolling Code to replace 
> ConnectionIDs after a 1-RTT handshake?  That’s an interesting idea, 
> though it may be difficult for Servers to tell apart completely 
> martian packets (unknown
> connections) vs very delayed packets (this makes a difference in 
> whether to RST the connection or drop the packet).

I considered using a PRP (pseudo-random function) for generating new connection IDs.  It doesn't work.  The need for it to be known to the client means that you need to have a different PRP (or a different PRP
key) for every client - otherwise clients could guess the next value in the sequence for any other client (linkability is broken).  With many clients, avoiding collisions in that case would be foolish.

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 1 Mar 2017 10:07:27 +0900

> How would a server know which server to redirect this packet to?

Either by proactively sharing the mapping between client-generated
connection IDs and the internal address of the server that is handling
the connection, or by including the internal address of the server in
the connection ID generated by the server. And I think that we kind of
agree that the latter is a requirement for multi-POP deployments.

Sorry if I am causing confusion; I think that the issue you raised on
this thread is important; my intention is to point out that we might
not need to worry to much about load balancers that rely on connection
ID but that cannot be controlled well from the QUIC servers running
behind, when we consider the choice between a client sending a random
connection ID or zero.

-- 
Kazuho Oku

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 1 Mar 2017 01:07:38 +0000

-----Original Message-----
> I assume that the server would have more state than the load balancer.

Unfortunately, this means the server would need not only its own state but also the state of all of its peers behind that load balancer.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 1 Mar 2017 12:13:21 +1100

> To elaborate on Kyle's idea, the initial ConnectionID would be selected as described, then client and server would negotiate some secure connection nonce, and the subsequent ConnectionID would be h…

Yes, that would work, if you don't care about collisions, which you
would get a LOT of.  Also, your load balancer is completely in the
dark unless you keep it constantly updated with the nonces that every
connection (or maybe server) uses.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 1 Mar 2017 01:17:39 +0000

Ø  From: Kazuho Oku [mailto:[email-masked]]

Ø

Ø  Either by proactively sharing the mapping between client-generated connection IDs and the internal address of the server that is handling the connection, or by including the internal address of the server in the connection ID generated by the server. And I think that we kind of agree that the latter is a requirement for multi-POP deployments.

Sorry, I thought you were arguing against mapping by ConnectionID.  For me, forwarding packets to the correct backend server is load balancer’s job. If a server includes elements of the “load balancer”, that’s fine. In the end of the day, that’s still the “server” part (the one that generates ConnectionID) of the server cooperating with the “load balancer” part.  The key is that the distributed “load balancer” part is still stateless.

-          Igor

---

From: "Lubashev, Igor" <[email-masked]>
Date: Wed, 1 Mar 2017 01:29:05 +0000

Right.  That scheme would reduce entropy by log(#connIDs_accepted).  Could be a lot.  And the load balancer would still require some additional unchanged bits in the clear (but few enough bits that it would make tracking hard).  And it is sacrificing some robustness and adds extra compute to the server.

All in all, I am not in love with this approach either, but it is worth a consideration.

- Igor

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Tuesday, February 28, 2017 8:13 PM
To: Lubashev, Igor <[email-masked]>
Cc: Kyle Rose <[email-masked]>; Mirja Kühlewind <[email-masked]>; Mike Bishop <[email-masked]>; Ted Hardie <[email-masked]>; IETF QUIC WG <[email-masked]>; Ryan Hamilton <[email-masked]>
Subject: Re: When should server-chosen connection IDs be sent and how are they indicated?

> To elaborate on Kyle's idea, the initial ConnectionID would be selected as described, then client and server would negotiate some secure connection nonce, and the subsequent ConnectionID would be h…

Yes, that would work, if you don't care about collisions, which you would get a LOT of.  Also, your load balancer is completely in the dark unless you keep it constantly updated with the nonces that every connection (or maybe server) uses.