Subject: Middlebox introspection/self-describing packets

From: Eric Rescorla <[email-masked]>
Date: Wed, 8 Mar 2017 13:38:58 -0800

There's recently been some back and forth on the headers PR about the
value of having packets be self-describing so that middleboxes can parse
them.

To take a concrete example, consider the connection_id. In general, it seems
that mostly packets in a given direction either need connection_ids or don't
(assuming that the point of connection_id is partly to handle unexpected
NAT rebinding.), so this seems like a prime thing to negotiate. Jana argued
in the PR that the reason to have a bit is to allow middleboxes to know how
to parse the packet.

It seems like it would be good to come to agreement about our attitude
towards
middlebox inspection. I have been generally taking the position that we
should
conceal as much as possible from middleboxes and only expose what is needed
to make the protocol work (i.e., if possible I would encrypt conn_id and
packet
number). It seems like perhaps not everyone shares that objective, so I
thought
I would ask what principles other people thought they were adhering to.

Jana? MT? Others?

-Ekr

---

From: Brian Trammell (IETF) <[email-masked]>
Date: Wed, 8 Mar 2017 23:45:20 +0100

hi ekr,

The point in this space I'd use is “expose nothing without necessity, or clear benefit with mutual cooperation of the endpoints, but ensure exposed information is efficiently usable.”

> On 8 Mar 2017, at 22:38, Eric Rescorla <[email-masked]> wrote:
> 

Taking the case of Connection ID, specifically, it does seem to be a candidate for moving to negotiation, saving one header bit pretty much everywhere. Connection ID is designed for load balancers, and one could define negotiation to be mandatory on the client side: if the server gives you a connection ID, then the client MUST return it. Otherwise, any device trying to observe the Connection ID would have to keep the has-a-Connection-ID state observed during the negotiation, and it wouldn't have access to that state when the Connection ID is used during rebinding.

In this case, saving the bit costs you the possibility of client-side choice on whether to not return the Connection ID, which seems to me to violate the mutual cooperation principle.

Cheers,

Brian

> 
> It seems like it would be good to come to agreement about our attitude towards
> middlebox inspection. I have been generally taking the position that we should
> to make the protocol work (i.e., if possible I would encrypt conn_id and packet
> number). It seems like perhaps not everyone shares that objective, so I thought

---

From: Martin Duke <[email-masked]>
Date: Wed, 8 Mar 2017 15:43:41 -0800

I have a substantially different position on middlebox inspection. First is
the load balancing point, which many people have already made.

Second, service providers have an interest in (1) measuring the quality of
service they provide to customers, (2) enforcing bandwidth limitations,
etc. on users, and (3) identifying application types to provide QoS where
it matters. I think (1) and (2) can be done without a meaningful loss of
privacy, although (3) is highly problematic. To that end, there are a
number of open issues looking to expose some transport state to middleboxes:
- Packet number echo to allow RTT measurements (#269)
- ECN to allow a non-loss rate-limiting signal (replacing TCP zero-window)
(#68)
- Public flag bits to indicate flow-control and loss-detection causes of
throttling, to aid in troubleshooting (#279) - I believe it was someone
from Orange in Tokyo who asked "how am I going to troubleshoot this?"
- Replacing CONNECTION_CLOSE to an authenticated Public Reset to clear
connection state on middleboxes. (#353)

No one has yet demonstrated that any of these are meaningful losses of
privacy. They also require no modifications to the packet itself.

For all the above reasons I am somewhat concerned about all of the
surreptitious connection-ID changes, but I think all of these features are
valuable regardless of how those discussions turn out.

On Wed, Mar 8, 2017 at 2:45 PM, Brian Trammell <[email-masked]> wrote:

> hi ekr,
>
> The point in this space I'd use is “expose nothing without necessity, or
> clear benefit with mutual cooperation of the endpoints, but ensure exposed
> information is efficiently usable.”
>
> Taking the case of Connection ID, specifically, it does seem to be a
> candidate for moving to negotiation, saving one header bit pretty much
> everywhere. Connection ID is designed for load balancers, and one could
> define negotiation to be mandatory on the client side: if the server gives
> you a connection ID, then the client MUST return it. Otherwise, any device
> trying to observe the Connection ID would have to keep the
> has-a-Connection-ID state observed during the negotiation, and it wouldn't
> have access to that state when the Connection ID is used during rebinding.
>
> In this case, saving the bit costs you the possibility of client-side
> choice on whether to not return the Connection ID, which seems to me to
> violate the mutual cooperation principle.
>
> Cheers,
>
> Brian
>
>
> >

---

From: Martin Thomson <[email-masked]>
Date: Thu, 9 Mar 2017 11:03:16 +1100

On 9 March 2017 at 10:43, Martin Duke <[email-masked]> wrote:
> throttling, to aid in troubleshooting (#279) -

[...]

> privacy.

Let's try the above as an example.  We know very well that flow
control can be used to observe the size of encrypted data.  Flow
control in QUIC ignores any padding, so being able to observe flow
control in action defeats a traffic analysis technique.

(I apologize, but I can't find the paper on attack specifics right
now.  I remember seeing a paper copy on mnot's desk when I was there
last, and it had a twee name and everything.)

---

From: Martin Duke <[email-masked]>
Date: Wed, 8 Mar 2017 16:09:51 -0800

The proposal in #279 is nothing like fully-baked, but it's a bit indicating
that the receiver is advertising zero-window, not indicating how big the
window ever was. It might even be ambiguous about stream vs. connection
flow control.

I can't remember if we're sending an initial window in the clear or not,
and this is worth thinking about, but it's not clear to me that's an issue.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 9 Mar 2017 11:13:06 +1100

On 9 March 2017 at 11:09, Martin Duke <[email-masked]> wrote:
> window ever was. It might even be ambiguous about stream vs. connection flow
> control.
>
> I can't remember if we're sending an initial window in the clear or not, and
> this is worth thinking about, but it's not clear to me that's an issue.

It's very definitely an issue.  Being able to observe when the flow
control window hits zero was used in the attack (the one I can't
find).

---

From: Martin Thomson <[email-masked]>
Date: Thu, 9 Mar 2017 11:23:40 +1100

Found it!

https://www.blackhat.com/docs/us-16/materials/us-16-VanGoethem-HEIST-HTTP-Encrypted-Information-Can-Be-Stolen-Through-TCP-Windows-wp.pdf

I don't know what is worse, that these have naff names, or that I
forget them so quickly.  Added to #279.

---

From: Dave Dolson <[email-masked]>
Date: Thu, 9 Mar 2017 01:45:55 +0000

Within the context of PLUS, we have attempted to assemble a list of benefits provided by on-path middleboxes.
Please see https://tools.ietf.org/html/draft-dolson-plus-middlebox-benefits-02

>From the abstract:

   A primary goal of this document is to provide information to working
   groups developing new transport protocols, in particular the PLUS and
   QUIC working groups, to aid understanding of what might be gained or
   lost by design decisions that may affect (or be affected by)
   middlebox operation.

The authors hope this will provide context for discussion about what sorts of middlebox functions are worth supporting.
We believe many important diagnostic and network protection mechanisms require some visibility into the transport connection, without violating privacy.

Please take a look.

-Dave

________________________________
From: QUIC [[email-masked]] on behalf of Martin Duke [[email-masked]]
Sent: Wednesday, March 08, 2017 6:43 PM
To: Brian Trammell
Cc: Eric Rescorla; IETF QUIC WG
Subject: Re: Middlebox introspection/self-describing packets

I have a substantially different position on middlebox inspection. First is the load balancing point, which many people have already made.

Second, service providers have an interest in (1) measuring the quality of service they provide to customers, (2) enforcing bandwidth limitations, etc. on users, and (3) identifying application types to provide QoS where it matters. I think (1) and (2) can be done without a meaningful loss of privacy, although (3) is highly problematic. To that end, there are a number of open issues looking to expose some transport state to middleboxes:
- Packet number echo to allow RTT measurements (#269)
- ECN to allow a non-loss rate-limiting signal (replacing TCP zero-window) (#68)
- Public flag bits to indicate flow-control and loss-detection causes of throttling, to aid in troubleshooting (#279) - I believe it was someone from Orange in Tokyo who asked "how am I going to troubleshoot this?"
- Replacing CONNECTION_CLOSE to an authenticated Public Reset to clear connection state on middleboxes. (#353)

No one has yet demonstrated that any of these are meaningful losses of privacy. They also require no modifications to the packet itself.

For all the above reasons I am somewhat concerned about all of the surreptitious connection-ID changes, but I think all of these features are valuable regardless of how those discussions turn out.

On Wed, Mar 8, 2017 at 2:45 PM, Brian Trammell <[email-masked]<mailto:[email-masked]>> wrote:
hi ekr,

> On 8 Mar 2017, at 22:38, Eric Rescorla <[email-masked]<mailto:[email-masked]>> wrote:
>

Cheers,

Brian

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 8 Mar 2017 17:46:14 -0800

> have access to that state when the Connection ID is used during rebinding.

To the extent to which the connection ID is (a) unique (b) long and (c) in
a predictable place when it is
present, it seems like it shouldn't be hard to look for it in that place
whether the bit is there or not.
In any case, what's the persuasive benefit for middleboxes to have this
information.

> violate the mutual cooperation principle.

Well, with negotiation, someone has to decide. The bit just moves that to
the client.
Except that if the load balancer *needs* it, then the client has to conform.

-Ekr

---

From: Roland Zink <[email-masked]>
Date: Thu, 9 Mar 2017 11:22:51 +0100

Hi Martin,

The main thing which seems to be used in this attack seems the TCP 
congestion control algorithm and the initial window. Assuming QUIC will 
use the same it will be attackable as well with or without exposing this 
to the network. The attack even does run within the endpoint and does 
not require access to the network at all.

According to this paper it doesn't seem wise to use HPACK in HTTP2, at 
least without padding. Should header compression be done in QUIC?

Btw. middleboxes which are able to influence the TCP congestion control 
algorithm and the initial window (like a PEP) actually may make this 
attack more difficult.

Regards,

Roland

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Thu, 9 Mar 2017 11:43:29 +0100

> On 09 Mar 2017, at 02:46, Eric Rescorla <[email-masked]> wrote:
> 

<snip>

> To the extent to which the connection ID is (a) unique (b) long and (c) in a predictable place when it is
> present, it seems like it shouldn't be hard to look for it in that place whether the bit is there or not.
> In any case, what's the persuasive benefit for middleboxes to have this information.

Here "middleboxes" means "middleboxes other than load balancers" here, correct? You could certainly encrypt this information, at the cost of requiring the load balancer to be the server-side party to the TLS session. To understate, I'm not sure that architecture is universally applicable.

> Well, with negotiation, someone has to decide. The bit just moves that to the client.

The load balancer needs it to balance load efficiently. I presume that a client that was very interested in not providing tracking information could refuse to make Connection ID available, at the cost of degraded performance. (f that's not the case, if failure to send connection ID leads to connection failure, then "negotiation" is a euphemism: the server informs the client whether it must send a connection ID, probably encrypted, and nobody needs any flags, unless the presence of connection ID changes the offsets of fields we *do* want to explicitly expose, such as packet number echo (https://github.com/quicwg/base-drafts/issues/269) or troubleshooting flags (https://github.com/quicwg/base-drafts/issues/279).

Cheers,

Brian

---

From: Brian Trammell (IETF) <[email-masked]>
Date: Thu, 9 Mar 2017 12:22:00 +0100

hi Martin,

There is a long and storied history of work on traffic analysis that squeezes bits of entropy from wherever they leak.  While reducing information exposure makes analyses of this class harder, at the end of the day any defense bumps up against size (bandwidth efficiency) and timing (latency). In a world where latency and bandwidth efficiency are important -- and I do think we still live in that world, otherwise 0-RTT goes away and draft-ietf-quic-tls gets way simpler -- protecting against traffic analysis is always more difficult than finding another way to fingerprint.

Given this history -- my favorite timing/sizing analysis paper [1], which recovers content identification from NetFlow data with anonymized addresses, is ten years old now -- I'm somewhat skeptical of claims about the power of a particular form of redaction against generalized traffic analysis. Obviously, there is a place for allowing users to choose to trade off bandwidth efficiency and latency for privacy -- Tor exists, and it works, for example -- but I'm not sure we want to go down the rabbithole of trying to add analysis resistance to QUIC in the general case.

Cheers,

Brian

[1]: https://www.usenix.org/legacy/event/sec07/tech/full_papers/coull/coull_html/main.html

> On 09 Mar 2017, at 01:23, Martin Thomson <[email-masked]> wrote:
> 
> Found it!
>

---

From: Erik Kline <[email-masked]>
Date: Thu, 9 Mar 2017 20:22:53 +0900

My 2 yen.

First, I would personally classify middleboxes as any element on the path
between two communicating endpoints that are not administered by an entity
that administers one of the endpoints.

To take the case with which I am most familiar: my mobile phone is one
endpoint, the mobile operator's devices are middleboxes as are all others
at least until my packets enter the network of the endpoint with which I'm
communicating.  In the remote network, if I'm talking to, say, another
mobile device then the remote mobile network operator's devices are still
middleboxes.  If on the other hand I'm talking to a monolithic service
composed of several elements, then those firewalls, load balancers, and so
forth I would not classify as middleboxes.  I would offer up a better term
for this latter category of devices, but I do not yet have one (they are
kind of auxiliary/colleague/cooperating boxes).

I think the distinction is important because middleboxes as I've tried to
define them cannot be presumed to be acting in the best interest of the
communicating parties.  All other boxes might be "fixed" to work well with
QUIC.  Whether the "fixing" necessitates exposing protocol information to
middleboxes as defined above is, I think, something to be considered on a
function-by-function basis.

Second, I feel very strongly that whatever is exposed to middleboxes should
not be easily abused (even by accident) in such a way as to prevent
seamless connection migration.  With a new transport like QUIC there's a
wonderful opportunity to get this right, and if we fail we won't get to try
again for many many years I fear.

---

From: Roland Zink <[email-masked]>
Date: Thu, 9 Mar 2017 13:28:01 +0100

My 2 cents.

Neither the middleboxes nor the auxiliary boxes especially not the other 
endpoint(s) can be trusted to work in the end users interest. Even my 
own endpoints can't be trusted as vault 7 seems to proof 
(https://wikileaks.org/ciav7p1/). Do I now need to intercept all 
connections from my TV?

QUIC has the potential to spam the network from user space code. The 
network should be able to detect misuse as fast as possible.

Roland

Am 09.03.2017 um 12:22 schrieb Erik Kline:
> My 2 yen.
>
> First, I would personally classify middleboxes as any element on the 
> path between two communicating endpoints that are not administered by 
> an entity that administers one of the endpoints.
>
> endpoint, the mobile operator's devices are middleboxes as are all 
> others at least until my packets enter the network of the endpoint 
> with which I'm communicating.  In the remote network, if I'm talking 
> to, say, another mobile device then the remote mobile network 
> operator's devices are still middleboxes.  If on the other hand I'm 
> talking to a monolithic service composed of several elements, then 
> those firewalls, load balancers, and so forth I would not classify as 
> middleboxes.  I would offer up a better term for this latter category 
> of devices, but I do not yet have one (they are kind of 
> auxiliary/colleague/cooperating boxes).
>
> I think the distinction is important because middleboxes as I've tried 
> to define them cannot be presumed to be acting in the best interest of 
> the communicating parties.  All other boxes might be "fixed" to work 
> well with QUIC.  Whether the "fixing" necessitates exposing protocol 
> information to middleboxes as defined above is, I think, something to 
> be considered on a function-by-function basis.
>
> Second, I feel very strongly that whatever is exposed to middleboxes 
> should not be easily abused (even by accident) in such a way as to 
> prevent seamless connection migration.  With a new transport like QUIC 
> there's a wonderful opportunity to get this right, and if we fail we 
> won't get to try again for many many years I fear.

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 06:02:09 -0800

On Thu, Mar 9, 2017 at 2:43 AM, Brian Trammell (IETF) <[email-masked]>
wrote:

> Here "middleboxes" means "middleboxes other than load balancers" here,
> correct?

Yes. Load balancers are attached to an endpoint.

> You could certainly encrypt this information, at the cost of requiring the
> load balancer to be the server-side party to the TLS session. To
> understate, I'm not sure that architecture is universally applicable.
>

You could encrypt this information without that, by having a separate key
that's shared
with the load balancer. You don't need the server side to be a party to the
TLS connection.

> The load balancer needs it to balance load efficiently.

That's not entirely clear. Note that in the original QUIC design, the
client supplied
the connection ID, so the server side merely had to load balance based on
some combination
of a random value and the client's IP.

I presume that a client that was very interested in not providing tracking
> information could refuse to make Connection ID available, at the cost of
> degraded performance. (f that's not the case, if failure to send connection
> ID leads to connection failure, then "negotiation" is a euphemism: the
> server informs the client whether it must send a connection ID, probably
> encrypted, and nobody needs any flags, unless the presence of connection ID
> changes the offsets of fields we *do* want to explicitly expose, such as
> packet number echo (https://github.com/quicwg/base-drafts/issues/269) or
> troubleshooting flags (https://github.com/quicwg/base-drafts/issues/279).
>

Not to put too fine a point on it, but there's far from consensus that we
want to expose
either of these.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 06:36:44 -0800

On Thu, Mar 9, 2017 at 4:28 AM, Roland Zink <[email-masked]> wrote:

> Do I now need to intercept all connections from my TV?
>

Uh.... yes?

More generally, E2E encryption between closed source devices and some
network
mother ship clearly presents a risk to user privacy and security even in
cases where
the device isn't compromised. For instance, how much of your recorded voice
is
being sent back to Amazon via Alexa? It would be nice not to have to trust
Amazon
about this. We don't really have a good solution for this (I think MITM
boxes clearly
aren't it), though there's some interesting work coming out of Stanford [0]
that
tries to address this.

-Ekr

[0] https://forum.stanford.edu/events/2016/slides/iot/Judson.pdf

>> should not be easily abused (even by accident) in such a way as to prevent

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 9 Mar 2017 14:56:25 +0000

> [0] https://forum.stanford.edu/events/2016/slides/iot/Judson.pdf

It's very unclear to me that it's possible to usefully
take a two party protocol like TLS and change it into a
multiparty protocol without fatal side-effects. As the
slide deck [0] notes for example, those "auditors" get
access to all cookies/passwords etc.

But I hope we can agree that while the problem is real,
this is not the WG to try solve that particular problem.

S.

---

From: Ian Swett <[email-masked]>
Date: Thu, 9 Mar 2017 09:59:30 -0500

On Wed, Mar 8, 2017 at 6:43 PM, Martin Duke <[email-masked]> wrote:

> I have a substantially different position on middlebox inspection. First
> is the load balancing point, which many people have already made.
>
The first three are the reasons I have in mind for using flags to indicate
negotiating connection ID presence and packet number length.   If we want
to expose any subset of these to pieces of the network besides cooperating
load balancers, then we need packet number for #269 and for that we need to
know it's size and location.

If we don't need any of the first three proposals, then there's no need to
have a middlebox parse anything and we should likely negotiate everything.

The question about how we're going to troubleshoot QUIC is real for me,
since I don't want to be debugging other people's networks for the rest of
time.  But I'm not completely sure what is and isn't necessary yet.

---

From: Kyle Rose <[email-masked]>
Date: Thu, 9 Mar 2017 10:07:25 -0500

> of a random value and the client's IP.

That too-tightly constrains load balancing. Without connection tracking
(which means shared state across multiple machines in a farm of load
balancers) or a server-chosen cookie, you're limited to algorithms that are
a function of connection ID and 5-tuple. In particular, you can't choose a
target server based on its current load. Random assignment may be good
enough for some application profiles, but not all.

Kyle

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 07:07:04 -0800

> access to all cookies/passwords etc.

Sorry, I should have been clearer that I wasn't endorsing this strategy.
That's
what the juxtaposition of "don't really have a good solution" with
"interesting
work" was intended to convey, but I guess I could have been more explicit.
I think there are a number of technical challenges here:

- TLS is a fairly blunt tool that protects nothing or everything
- The Web has tended to do a really bad job of separating authentication
   from confidentiality (and in general supplying authentication via
confidentiality
   through various kinds of bearer tokens)

I do think it's important that people keep thinking about it, because the
problem of devices that are nominally under your control being hard to trust
is also serious. That doesn't mean that we currently have a good strategy
for solving it.

> this is not the WG to try solve that particular problem.

Yes.

-Ekr

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 07:08:07 -0800

> enough for some application profiles, but not all.

Well, that may be true, but I'm merely observing that that's the present
design of gQUIC.

-Ekr

---

From: Brian Trammell (IETF) <[email-masked]>
Date: Thu, 9 Mar 2017 17:09:11 +0100

hi Ekr, all,

> That's not entirely clear. Note that in the original QUIC design, the client supplied
> the connection ID, so the server side merely had to load balance based on some combination

Right; should have said that "the assertion behind server-suggested Connection ID proposal is that it's needed for efficient load balancing". I'm taking that assertion at face value now, possibly because I'm convinced of the utility of one of its side-effects: providing some additional grade of assurance to a device on path that a given packet was not injected off-path, which is useful in in-network DoS mitigation (as in section 3.3 of the just-submitted manageability draft).

>> I presume that a client that was very interested in not providing tracking information could refuse to make Connection ID available, at the cost of degraded performance.

(I would like to reiterate that I do think that it's necessary to allow clients to veto server-proposed connection ID exposure.)

>> (f that's not the case, if failure to send connection ID leads to connection failure, then "negotiation" is a euphemism: the server informs the client whether it must send a connection ID, probably encrypted, and nobody needs any flags, unless the presence of connection ID changes the offsets of fields we *do* want to explicitly expose, such as packet number echo (https://github.com/quicwg/base-drafts/issues/269) or troubleshooting flags (https://github.com/quicwg/base-drafts/issues/279).
> 
> Not to put too fine a point on it, but there's far from consensus that we want to expose
> either of these.

That's not too fine a point at all; I'm not presupposing consensus on either of these. All I'm saying here is, should consensus emerge that the benefits outweigh the costs (both in complexity and in risk) for certain kinds of exposure to the path -- which I think may be the case for some of the things under discussion in 279, and I'm obviously convinced of the utility of packet number echo as in 269 or I wouldn't have submitted two PRs on it -- we need to be clear that other choices we make about the layout of the header doesn't make that harder than it needs to be. Sticking variable-length/optional fields whose presence is dependent on endpoint-shared state after the exposed fields is probably sufficient for this.

Cheers,

Brian

---

From: Kyle Rose <[email-masked]>
Date: Thu, 9 Mar 2017 11:58:32 -0500

> Well, that may be true, but I'm merely observing that that's the present

Understood. I just didn't want the gQUIC approach to be considered
universally applicable on the basis of that statement.

I'd still rather find a way to separate minimal load balancer state from
session state, and to protect the latter, but I'm not sure if there's a way
to do it that provides real value (i.e., isn't just advisory).

Kyle

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 9 Mar 2017 09:08:00 -0800

A thoughts as I catch up on this thread (and in response to ekr's original
question):

- As ekr points out, server-side middleboxes can always rely on connection
IDs by policy. The value of the connection ID bit, in my mind, is for
middleboxes not controlled by the server. Such middleboxes are likely to
use the connection ID to identify connections where it exists. Without an
explicit per-packet bit, a middlebox that wants to use connection ID will
have to dig into the handshake to determine where it's presence negotiated.
This leads to ossification of bits inside the cleartext handshake messages,
which is highly undesirable. I'd rather that the connection ID bit be
ossified.

- As it stands, connection ID negotiation means that each side tells the
peer what it wants the peer to do. In a world with (i) no connection ID bit
and (ii) clients vetoing the server's request, policy decisions are hard
for load balancers. In this world, there's no good way for server-side load
balancers to know per connection whether it has a connection ID or not: to
find connection state, the load balancer needs a key, which would be the
connection ID, which may or may not be present for that connection. The
load balancer could use the 4-tuple to key the connection, but that
entirely defeats the point of using the connection ID.

- The privacy implications of sharing connection ID is a separate
conversation from the one this thread was started on. Though since some
points were raised here and since they speak to the question of what
information to share with middleboxes, I'll share my thoughts.

We need to separate the use of connection ID use across NAT rebindings vs
the use of connection ID use across client network changes.

We'd all probably agree that NAT rebindings for UDP are a real concern,
since NATs kill these bindings faster than they do TCP bindings. A TCP
connection would have its connection identifier (the 4-tuple) be stable for
the lifetime of a TCP connection. A QUIC connection would be similar, with
the difference that the connection identifier here would be the connection
ID. The connection ID does not expose any more state than a TCP connection
would in this case.

We'd all agree that connection ID use across client network changes is a
new beast. To this end, I think we've generally agreed (informally anyways)
that the client must use a new connection ID when it triggers connection
migration to a different network.

In sum, my argument is that having connection IDs stable across NAT
rebindings is a necessity for QUIC to work well. As a result, middleboxes
at various points in the network, specifically those past a NAT, are
expected to use it to track connection state. Explicitly noting its
presence in the packet anticipates this and eliminates this one reason for
ossification of other bits deep in handshake messages needed otherwise for
its inference.

- jana

On Thu, Mar 9, 2017 at 8:09 AM, Brian Trammell <[email-masked]> wrote:

> Right; should have said that "the assertion behind server-suggested
> Connection ID proposal is that it's needed for efficient load balancing".
> I'm taking that assertion at face value now, possibly because I'm convinced
> of the utility of one of its side-effects: providing some additional grade
> of assurance to a device on path that a given packet was not injected
> off-path, which is useful in in-network DoS mitigation (as in section 3.3
> of the just-submitted manageability draft).
>
> >> I presume that a client that was very interested in not providing
> tracking information could refuse to make Connection ID available, at the
> cost of degraded performance.
>
> (I would like to reiterate that I do think that it's necessary to allow
> clients to veto server-proposed connection ID exposure.)
>
> >> (f that's not the case, if failure to send connection ID leads to
> connection failure, then "negotiation" is a euphemism: the server informs
> the client whether it must send a connection ID, probably encrypted, and
> nobody needs any flags, unless the presence of connection ID changes the
> offsets of fields we *do* want to explicitly expose, such as packet number
> echo (https://github.com/quicwg/base-drafts/issues/269) or
> That's not too fine a point at all; I'm not presupposing consensus on
> either of these. All I'm saying here is, should consensus emerge that the
> benefits outweigh the costs (both in complexity and in risk) for certain
> kinds of exposure to the path -- which I think may be the case for some of
> the things under discussion in 279, and I'm obviously convinced of the
> utility of packet number echo as in 269 or I wouldn't have submitted two
> PRs on it -- we need to be clear that other choices we make about the
> layout of the header doesn't make that harder than it needs to be. Sticking
> variable-length/optional fields whose presence is dependent on
> endpoint-shared state after the exposed fields is probably sufficient for
> this.
>
>
> Cheers,
>
> Brian
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 09:32:55 -0800

> which is highly undesirable. I'd rather that the connection ID bit be

Sure, but this starts from the position that middleboxes should be using
connection
IDs. It's not clear to me that we want to encourage that.

> entirely defeats the point of using the connection ID.

Hmm... This seems rather more to be the result of letting the client veto.
In the design
you propose, the server does:

- If connection id bit is 1 look up connection ID
- Otherwise, look up the host/port

If you don't have a bit, you can

- Look up the host/port
- If it's missing assume that there is a connection ID and look for it
- If that's missing, drop the packet

> since NATs kill these bindings faster than they do TCP bindings.
>

Actually, no, I'm not persuaded by this as yet. I agree with the statement
that NATs kill these bindings
faster, but it's not yet clear to me that connection IDs ameliorate these
problems sufficiently to
improve the situation. I'd be happy to see any data you have that supports
this point.

> migration to a different network.

The difficulty is that we do not presently know how to arrange that you
always use a new conn id
on migration without using a new conn id on each packet.

> ossification of other bits deep in handshake messages needed otherwise for

I don't see you arguing that this is desirable so much as inevitable. So,
if we had a design that
was reasonably efficient that used a different connection ID per packet,
then you would be
fine with that and wouldn't feel the need to tell middleboxes where it was?

-Ekr

---

From: Roland Zink <[email-masked]>
Date: Thu, 9 Mar 2017 19:00:00 +0100

For TCP connections the middlebox (NAT) gets a signal when a connection 
ends. The NAT need to use the timeout only for connections where this 
doesn't happen. When there is no identificable connection and end signal 
then the NAT need to use the timeout for all such mappings. With a 
limited table size the timeout for such mappings needs to be shorter. 
With a identificable connection and an end signal NATs can increase the 
timeout for QUIC.

Roland

Am 09.03.2017 um 18:32 schrieb Eric Rescorla:
>
>
> On Thu, Mar 9, 2017 at 9:08 AM, Jana Iyengar <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     A thoughts as I catch up on this thread (and in response to ekr's
>     original question):
>
>     - As ekr points out, server-side middleboxes can always rely on
>     connection IDs by policy. The value of the connection ID bit, in
>     my mind, is for middleboxes not controlled by the server. Such
>     middleboxes are likely to use the connection ID to identify
>     connections where it exists. Without an explicit per-packet bit, a
>     middlebox that wants to use connection ID will have to dig into
>     the handshake to determine where it's presence negotiated. This
>     leads to ossification of bits inside the cleartext handshake
>     messages, which is highly undesirable. I'd rather that the
>     connection ID bit be ossified.
>
>
> Sure, but this starts from the position that middleboxes should be 
> using connection
>     - As it stands, connection ID negotiation means that each side
>     tells the peer what it wants the peer to do. In a world with (i)
>     no connection ID bit and (ii) clients vetoing the server's
>     request, policy decisions are hard for load balancers. In this
>     world, there's no good way for server-side load balancers to know
>     per connection whether it has a connection ID or not: to find
>     connection state, the load balancer needs a key, which would be
>     the connection ID, which may or may not be present for that
>     connection. The load balancer could use the 4-tuple to key the
>     connection, but that entirely defeats the point of using the
>     connection ID.
>
>
> Hmm... This seems rather more to be the result of letting the client 
> veto. In the design
>     conversation from the one this thread was started on. Though since
>     some points were raised here and since they speak to the question
>     of what information to share with middleboxes, I'll share my
>     thoughts.
>
>     We need to separate the use of connection ID use across NAT
>     rebindings vs the use of connection ID use across client network
>     changes.
>
>     We'd all probably agree that NAT rebindings for UDP are a real
>     concern, since NATs kill these bindings faster than they do TCP
>     bindings.
>
>
> Actually, no, I'm not persuaded by this as yet. I agree with the 
> statement that NATs kill these bindings
> faster, but it's not yet clear to me that connection IDs ameliorate 
> these problems sufficiently to
> improve the situation. I'd be happy to see any data you have that 
> supports this point.
>
>
>
>     We'd all agree that connection ID use across client network
>     changes is a new beast. To this end, I think we've generally
>     agreed (informally anyways) that the client must use a new
>     connection ID when it triggers connection migration to a different
>     network.
>
>
> The difficulty is that we do not presently know how to arrange that 
> you always use a new conn id
>     In sum, my argument is that having connection IDs stable across
>     NAT rebindings is a necessity for QUIC to work well. As a result,
>     middleboxes at various points in the network, specifically those
>     past a NAT, are expected to use it to track connection state.
>     Explicitly noting its presence in the packet anticipates this and
>     eliminates this one reason for ossification of other bits deep in
>     handshake messages needed otherwise for its inference.
>
>
> I don't see you arguing that this is desirable so much as inevitable. 
> So, if we had a design that
> was reasonably efficient that used a different connection ID per 
> packet, then you would be
>     On Thu, Mar 9, 2017 at 8:09 AM, Brian Trammell <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>         hi Ekr, all,
>
>         Right; should have said that "the assertion behind
>         server-suggested Connection ID proposal is that it's needed
>         for efficient load balancing". I'm taking that assertion at
>         face value now, possibly because I'm convinced of the utility
>         of one of its side-effects: providing some additional grade of
>         assurance to a device on path that a given packet was not
>         injected off-path, which is useful in in-network DoS
>         mitigation (as in section 3.3 of the just-submitted
>         manageability draft).
>
>         >> I presume that a client that was very interested in not
>         providing tracking information could refuse to make Connection
>         ID available, at the cost of degraded performance.
>
>         (I would like to reiterate that I do think that it's necessary
>         to allow clients to veto server-proposed connection ID exposure.)
>
>         >> (f that's not the case, if failure to send connection ID
>         leads to connection failure, then "negotiation" is a
>         euphemism: the server informs the client whether it must send
>         a connection ID, probably encrypted, and nobody needs any
>         flags, unless the presence of connection ID changes the
>         offsets of fields we *do* want to explicitly expose, such as
>         packet number echo
>         (https://github.com/quicwg/base-drafts/issues/269
>         <https://github.com/quicwg/base-drafts/issues/269>) or
>         troubleshooting flags
>         (https://github.com/quicwg/base-drafts/issues/279
>         <https://github.com/quicwg/base-drafts/issues/279>).
>         >
>         That's not too fine a point at all; I'm not presupposing
>         consensus on either of these. All I'm saying here is, should
>         consensus emerge that the benefits outweigh the costs (both in
>         complexity and in risk) for certain kinds of exposure to the
>         path -- which I think may be the case for some of the things
>         under discussion in 279, and I'm obviously convinced of the
>         utility of packet number echo as in 269 or I wouldn't have
>         submitted two PRs on it -- we need to be clear that other
>         choices we make about the layout of the header doesn't make
>         that harder than it needs to be. Sticking
>         endpoint-shared state after the exposed fields is probably
>         sufficient for this.
>
>
>         Cheers,
>
>         Brian
>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 10:07:45 -0800

This seems like an argument for a visible close, not for a connection ID.

-Ekr

On Thu, Mar 9, 2017 at 10:00 AM, Roland Zink <[email-masked]> wrote:

> then the NAT need to use the timeout for all such mappings. With a limited
> table size the timeout for such mappings needs to be shorter. With a
> identificable connection and an end signal NATs can increase the timeout
> for QUIC.
> Roland
>
>
>
>> connection IDs by policy. The value of the connection ID bit, in my mind,
>> is for middleboxes not controlled by the server. Such middleboxes are
>> likely to use the connection ID to identify connections where it exists.
>> Without an explicit per-packet bit, a middlebox that wants to use
>> connection ID will have to dig into the handshake to determine where it's
>> presence negotiated. This leads to ossification of bits inside the
>> cleartext handshake messages, which is highly undesirable. I'd rather that
>> the connection ID bit be ossified.
>>
>

---

From: Ian Swett <[email-masked]>
Date: Thu, 9 Mar 2017 13:12:19 -0500

> improve the situation. I'd be happy to see any data you have that supports
This happens often, particularly after 1 minute, and connection ID makes it
work seamlessly, so I'm pretty sure it works sufficiently well.  It is
critical to longer(ie: 5 minute) idle timeouts.

> fine with that and wouldn't feel the need to tell middleboxes where it was?

I don't think having the same connection ID used after a NAT rebind is a
problem.  The attack of interest is linking one or more actions in one
network with one or more on another network.  If you're on the client side
of the NAT nothing changes, and if you're on the server side you already
know the set of ports(or ports and IPs) that a given NAT uses publicly, so
they're all equivalent from the observer's perspective.  The NAT is a
physical entity and there are some fairly tight real-world constraints on
where you end up during a rebind.

Is there a different threat model that I'm not aware of?

-Ekr

---

From: Roland Zink <[email-masked]>
Date: Thu, 9 Mar 2017 19:12:48 +0100

Yes, it is only necessary to identify the connection. If it is done 
through IP addresses and ports or connection id shouldn't matter in this 
respect.

Roland

Am 09.03.2017 um 19:07 schrieb Eric Rescorla:
> On Thu, Mar 9, 2017 at 10:00 AM, Roland Zink <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     For TCP connections the middlebox (NAT) gets a signal when a
>     connection ends. The NAT need to use the timeout only for
>     connections where this doesn't happen. When there is no
>     identificable connection and end signal then the NAT need to use
>     the timeout for all such mappings. With a limited table size the
>     timeout for such mappings needs to be shorter. With a
>     identificable connection and an end signal NATs can increase the
>     timeout for QUIC.
>
>     Roland
>
>
>
>>         A thoughts as I catch up on this thread (and in response to
>>         ekr's original question):
>>
>>         - As ekr points out, server-side middleboxes can always rely
>>         on connection IDs by policy. The value of the connection ID
>>         bit, in my mind, is for middleboxes not controlled by the
>>         server. Such middleboxes are likely to use the connection ID
>>         to identify connections where it exists. Without an explicit
>>         per-packet bit, a middlebox that wants to use connection ID
>>         will have to dig into the handshake to determine where it's
>>         presence negotiated. This leads to ossification of bits
>>         inside the cleartext handshake messages, which is highly
>>         undesirable. I'd rather that the connection ID bit be ossified.
>>
>>
>>     Sure, but this starts from the position that middleboxes should
>>     be using connection
>>         - As it stands, connection ID negotiation means that each
>>         side tells the peer what it wants the peer to do. In a world
>>         with (i) no connection ID bit and (ii) clients vetoing the
>>         server's request, policy decisions are hard for load
>>         balancers. In this world, there's no good way for server-side
>>         load balancers to know per connection whether it has a
>>         connection ID or not: to find connection state, the load
>>         balancer needs a key, which would be the connection ID, which
>>         may or may not be present for that connection. The load
>>         balancer could use the 4-tuple to key the connection, but
>>         that entirely defeats the point of using the connection ID.
>>
>>
>>     Hmm... This seems rather more to be the result of letting the
>>     client veto. In the design
>>         - The privacy implications of sharing connection ID is a
>>         separate conversation from the one this thread was started
>>         on. Though since some points were raised here and since they
>>         speak to the question of what information to share with
>>         middleboxes, I'll share my thoughts.
>>
>>         rebindings vs the use of connection ID use across client
>>         network changes.
>>
>>         We'd all probably agree that NAT rebindings for UDP are a
>>         real concern, since NATs kill these bindings faster than they
>>         do TCP bindings.
>>
>>
>>     faster, but it's not yet clear to me that connection IDs
>>     ameliorate these problems sufficiently to
>>         connection ID when it triggers connection migration to a
>>         different network.
>>
>>
>>     The difficulty is that we do not presently know how to arrange
>>     that you always use a new conn id
>>         In sum, my argument is that having connection IDs stable
>>         across NAT rebindings is a necessity for QUIC to work well.
>>         As a result, middleboxes at various points in the network,
>>         specifically those past a NAT, are expected to use it to
>>         track connection state. Explicitly noting its presence in the
>>         packet anticipates this and eliminates this one reason for
>>         ossification of other bits deep in handshake messages needed
>>         otherwise for its inference.
>>
>>
>>     I don't see you arguing that this is desirable so much as
>>     inevitable. So, if we had a design that
>>         On Thu, Mar 9, 2017 at 8:09 AM, Brian Trammell
>>         <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>             hi Ekr, all,
>>
>>             server-suggested Connection ID proposal is that it's
>>             needed for efficient load balancing". I'm taking that
>>             assertion at face value now, possibly because I'm
>>             convinced of the utility of one of its side-effects:
>>             providing some additional grade of assurance to a device
>>             on path that a given packet was not injected off-path,
>>             which is useful in in-network DoS mitigation (as in
>>             section 3.3 of the just-submitted manageability draft).
>>
>>             >> I presume that a client that was very interested in
>>             not providing tracking information could refuse to make
>>             Connection ID available, at the cost of degraded performance.
>>
>>             (I would like to reiterate that I do think that it's
>>             necessary to allow clients to veto server-proposed
>>             connection ID exposure.)
>>
>>             >> (f that's not the case, if failure to send connection
>>             ID leads to connection failure, then "negotiation" is a
>>             euphemism: the server informs the client whether it must
>>             send a connection ID, probably encrypted, and nobody
>>             needs any flags, unless the presence of connection ID
>>             changes the offsets of fields we *do* want to explicitly
>>             expose, such as packet number echo
>>             consensus on either of these. All I'm saying here is,
>>             should consensus emerge that the benefits outweigh the
>>             costs (both in complexity and in risk) for certain kinds
>>             of exposure to the path -- which I think may be the case
>>             for some of the things under discussion in 279, and I'm
>>             obviously convinced of the utility of packet number echo
>>             as in 269 or I wouldn't have submitted two PRs on it --
>>             we need to be clear that other choices we make about the
>>             layout of the header doesn't make that harder than it
>>             needs to be. Sticking variable-length/optional fields
>>             whose presence is dependent on endpoint-shared state
>>             after the exposed fields is probably sufficient for this.
>>
>>
>>             Cheers,
>>
>>             Brian
>>
>>
>>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 10:17:43 -0800

On Thu, Mar 9, 2017 at 10:12 AM, Ian Swett <[email-masked]> wrote:

>>> We need to separate the use of connection ID use across NAT rebindings
>>> vs the use of connection ID use across client network changes.
>>>
> This happens often, particularly after 1 minute, and connection ID makes
> it work seamlessly, so I'm pretty sure it works sufficiently well.  It is

I am aware of this argument, but I was looking for something that looked a
little more like
data :)

> problem.

It actually is a small problem, because often many people are behind the
same NAT.
But I agree that the primary problem is network transitions.

  The attack of interest is linking one or more actions in one network with
> one or more on another network.  If you're on the client side of the NAT
> nothing changes, and if you're on the server side you already know the set
> of ports(or ports and IPs) that a given NAT uses publicly, so they're all
> equivalent from the observer's perspective.  The NAT is a physical entity
> and there are some fairly tight real-world constraints on where you end up
> during a rebind.
>

See above.

-Ekr

---

From: Ian Swett <[email-masked]>
Date: Thu, 9 Mar 2017 13:28:17 -0500

On Thu, Mar 9, 2017 at 1:17 PM, Eric Rescorla <[email-masked]> wrote:

>>>> - As it stands, connection ID negotiation means that each side tells
>>>> the peer what it wants the peer to do. In a world with (i) no connection ID
>>>> bit and (ii) clients vetoing the server's request, policy decisions are
>>>> hard for load balancers. In this world, there's no good way for server-side
>>>> load balancers to know per connection whether it has a connection ID or
>>>> not: to find connection state, the load balancer needs a key, which would
>>>> be the connection ID, which may or may not be present for that connection.
>>>> The load balancer could use the 4-tuple to key the connection, but that

What data would you like?  To my knowledge, it works 100% of the time where
connection ID routing is implemented for our infrastructure, and we
configure conservative 30 second timeouts anywhere we can't rely on
connection ID.

> It actually is a small problem, because often many people are behind the

It clearly links the post-rebind portion of the connection with the
pre-rebind portion of the connection, but I don't think it does anything
else.  A TCP connection presumably wouldn't have rebound in a similar
circumstance, so you could trivially connect the two parts of the
connection.

That being said, I'd be interested in seeing a design for a fully encrypted
header design if it was at all practical.

>   The attack of interest is linking one or more actions in one network
>> with one or more on another network.  If you're on the client side of the
>> NAT nothing changes, and if you're on the server side you already know the
>> set of ports(or ports and IPs) that a given NAT uses publicly, so they're
>> all equivalent from the observer's perspective.  The NAT is a physical
>> entity and there are some fairly tight real-world constraints on where you
>> end up during a rebind.
>>
>>>>> (I would like to reiterate that I do think that it's necessary to
>>>>> allow clients to veto server-proposed connection ID exposure.)
>>>>>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 10:39:12 -0800

On Thu, Mar 9, 2017 at 10:28 AM, Ian Swett <[email-masked]> wrote:

>>>>> concern, since NATs kill these bindings faster than they do TCP bindings.
>>>>>
>>>>
>> I am aware of this argument, but I was looking for something that looked
>> a little more like
>> data :)
>>
>
> What data would you like?  To my knowledge, it works 100% of the time
> where connection ID routing is implemented for our infrastructure, and we

I'll think about how to phrase the question precisely and either send e-mail
or grab you in ORD.

> circumstance, so you could trivially connect the two parts of the

I don't disagree with this statement. And to which our target is no worse
than TCP, then
mission accomplished here. But it's still part of the threat model :)

-Ekr

>
> That being said, I'd be interested in seeing a design for a fully
> encrypted header design if it was at all practical.
>
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 9 Mar 2017 14:03:07 -0500

> to do it that provides real value (i.e., isn't just advisory).

​So I just had an idea that maybe is relevant and maybe is not. Built into
QUIC is the idea that it is a conversation between one machine at the
client (initiator) end and one machine at the server (responder) end.

Is that real?

Lets say that Alice is surfin' the Web and goes to a page that has
streaming media, static content and such all from the same service but
being served up via different physical machines.

What happens today is that we either demultiplex the streams in the HTML by
giving them all URIs to be resolved separately or we have a Tier 1 system
on the service end that is acting as a front end to all those tier 2
resources.

Both approaches have consequences. In the first case we end up with two
machines in the same data center, often the same rack communicating through
HTTP cookies bouncing off the client.  Not just technically bad but
needless EU tracking cookie complications. In the second, we have to have
pipe huge amounts of data across our machine room when it could just be
shipped directly out the door.

What if we could change the model here? Instead of thinking of QUIC as
being a Transport layer being implemented as an application layer kludge,
maybe we are developing something a little bit higher level (presentation
?).

Assume we have some internal protocol to hand off handling of a stream to a
different host. We could do some interesting stuff.

I would like to get rid of content IDs, stream IDs, everything. Every
stream that is created is assigned a opaque unique identifier by the party
that is going to receive the packets. The allowed length is long enough to
allow whatever state needs to be packed therein.

Standards are all about making decisions that don't matter while not making
decisions that matter a lot to other people.

​

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 9 Mar 2017 11:26:04 -0800

> IDs. It's not clear to me that we want to encourage that.

Why not? It's a connection identifier, like the TCP 4-tuple. What do you
gain from middleboxes not using the connection ID to identify connections?

> - If that's missing, drop the packet

Yes, I was responding to Brian's email -- apologies for crossing the wires
a bit. This method is plausible, but it increases the work per packet at
load balancers by ~2x.

> improve the situation. I'd be happy to see any data you have that supports

I don't follow. I think you're agreeing with what I said above, that NAT
rebindings happen faster for UDP.  A QUIC packet post-rebinding for the
same connection carries the same connection ID, which leads it back to the
same server even if a NAT rebinding happened. Why do you think connection
IDs don't ameliorate this problem (of NAT rebindings happen faster for UDP)
sufficiently? In other words, what piece of this problem does connection ID
not solve?

> on migration without using a new conn id on each packet.

That's not true. We can arrange for an active client-driven migration to
use a new connection ID. It's the NAT rebinding case that we can't handle
without a new connection ID per packet. The reason I separated the
connection ID question into NAT rebinding vs client migration was to also
separate the privacy considerations for these two cases. While there's a
real concern of privacy leakage across active client migrations, privacy
leakage due to a stable connection ID across NAT rebindings is not
different than that due to a stable TCP connection for the same duration.

> fine with that and wouldn't feel the need to tell middleboxes where it was?

I wouldn't because I think middleboxes need to see this to identify
connections. I understand that server-side middleboxes can be included in
such designs, but not all, and especially not operator ones which are
neither client -controlled nor server-controlled. Operators have and
continue to use passive monitoring techniques (note, I'm using this as a
measurement term, not as a privacy leakage term, and yes, I appreciate that
there's overlap) for monitoring their network. They've used the entire TCP
header so far for doing this leading to ossification, and so I strongly
agree that every bit should be considered. That said, I don't understand
what problem denying them connection ID solves.

- jana

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 9 Mar 2017 11:35:09 -0800

> gain from middleboxes not using the connection ID to identify connections?

Yeah, I wish they wouldn't use the 4-tuple either. In fact much of this
conversation is driven by them doing nonsense based on the 4-tuple.

> a bit. This method is plausible, but it increases the work per packet at

Not really, because you can check in either order, so unless it's 50/50 you
usually get the fast-path.

> sufficiently? In other words, what piece of this problem does connection ID

1. Anything where it's the server speaking first after the rebind.
2. Situations where the server or client would have GCed the state anyway.

I'm looking for data that indicates that the remaining cases are common
enough to be
important enough to take the privacy hit. As I said in my response to Ian,
I'll try to
figure out a more precise framing of the data that I would find persuasive.

> different than that due to a stable TCP connection for the same duration.

There are plenty of times when the client does a network change but doesn't
know it
has. Tethering is one example. In such cases, continuing to use the same
connection
ID presents a privacy threat.

-Ekr

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 9 Mar 2017 11:39:16 -0800

Apologies -- I didn't see the flurry of emails over the past hour before
writing my response.

> mission accomplished here. But it's still part of the threat model :)

There's a fundamental tension between network manageability and the problem
of linkability at the packet-level: the former relies on visible "flows"
and the latter pushes against it. This is a much bigger conversation.

---

From: Jana Iyengar <[email-masked]>
Date: Thu, 9 Mar 2017 11:51:45 -0800

> conversation is driven by them doing nonsense based on the 4-tuple.

I understand now. FWIW, the stuff they do, using 4-tuples, includes AQM
(see FQ-CoDel)...  some of these things are useful. I'm not saying that's
the only way to do it, and I've not thought (or know) about all the things
that would break if the notion of flow was obliterated from public view.

>> Yes, I was responding to Brian's email -- apologies for crossing the
>> wires a bit. This method is plausible, but it increases the work per packet
>> at load balancers by ~2x.
>>
>
> Not really, because you can check in either order, so unless it's 50/50
> you usually get the fast-path.
>

> figure out a more precise framing of the data that I would find persuasive.

That'd be helpful.

> There are plenty of times when the client does a network change but
> doesn't know it

The tethering case is real, but it's a corner case; connection migrations
are dominated by untethered clients.

- jana

---

From: Dave Dolson <[email-masked]>
Date: Thu, 9 Mar 2017 21:22:07 +0000

> I've not thought (or know) about all the things that would break if the notion of flow was obliterated from public view.

Can we discuss it in the context of this draft?
https://tools.ietf.org/id/draft-dolson-plus-middlebox-benefits-01.txt

We have been thinking about the things that might break if various transport-layer info is not provided.

Here’s part of the table of contents:

   2.  Measurements  . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Packet Loss . . . . . . . . . . . . . . . . . . . . . . .   4
     2.2.  Round Trip Times  . . . . . . . . . . . . . . . . . . . .   4
     2.3.  Measuring Packet Reordering . . . . . . . . . . . . . . .   5
     2.4.  Throughput and Bottleneck Identification  . . . . . . . .   5
     2.5.  DDoS Detection  . . . . . . . . . . . . . . . . . . . . .   5
     2.6.  Packet Corruption . . . . . . . . . . . . . . . . . . . .   6
     2.7.  Application-Layer Measurements  . . . . . . . . . . . . .   6
   3.  Actions . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.1.  NAT . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.2.  Firewall  . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.3.  DDoS Scrubbing  . . . . . . . . . . . . . . . . . . . . .   7
     3.4.  Performance-Enhancing Proxies . . . . . . . . . . . . . .   8
     3.5.  Bandwidth Aggregation . . . . . . . . . . . . . . . . . .   9
     3.6.  Prioritization  . . . . . . . . . . . . . . . . . . . . .   9
     3.7.  Measurement-Based Shaping . . . . . . . . . . . . . . . .   9

-Dave

From: QUIC [mailto:[email-masked]] On Behalf Of Jana Iyengar
Sent: Thursday, March 09, 2017 2:52 PM
To: Eric Rescorla
Cc: Brian Trammell; IETF QUIC WG
Subject: Re: Middlebox introspection/self-describing packets

Why not? It's a connection identifier, like the TCP 4-tuple. What do you gain from middleboxes not using the connection ID to identify connections?

Yeah, I wish they wouldn't use the 4-tuple either. In fact much of this conversation is driven by them doing nonsense based on the 4-tuple.

I understand now. FWIW, the stuff they do, using 4-tuples, includes AQM (see FQ-CoDel)...  some of these things are useful. I'm not saying that's the only way to do it, and I've not thought (or know) about all the things that would break if the notion of flow was obliterated from public view.

- Look up the host/port

Yes, I was responding to Brian's email -- apologies for crossing the wires a bit. This method is plausible, but it increases the work per packet at load balancers by ~2x.

Not really, because you can check in either order, so unless it's 50/50 you usually get the fast-path.

Actually, no, I'm not persuaded by this as yet. I agree with the statement that NATs kill these bindings
faster, but it's not yet clear to me that connection IDs ameliorate these problems sufficiently to
improve the situation. I'd be happy to see any data you have that supports this point.

I don't follow. I think you're agreeing with what I said above, that NAT rebindings happen faster for UDP.  A QUIC packet post-rebinding for the same connection carries the same connection ID, which leads it back to the same server even if a NAT rebinding happened. Why do you think connection IDs don't ameliorate this problem (of NAT rebindings happen faster for UDP) sufficiently? In other words, what piece of this problem does connection ID not solve?

1. Anything where it's the server speaking first after the rebind.
2. Situations where the server or client would have GCed the state anyway.
I'm looking for data that indicates that the remaining cases are common enough to be
important enough to take the privacy hit. As I said in my response to Ian, I'll try to
figure out a more precise framing of the data that I would find persuasive.

That'd be helpful.

We'd all agree that connection ID use across client network changes is a new beast. To this end, I think we've generally agreed (informally anyways) that the client must use a new connection ID when it triggers connection migration to a different network.

The difficulty is that we do not presently know how to arrange that you always use a new conn id
on migration without using a new conn id on each packet.

That's not true. We can arrange for an active client-driven migration to use a new connection ID. It's the NAT rebinding case that we can't handle without a new connection ID per packet. The reason I separated the connection ID question into NAT rebinding vs client migration was to also separate the privacy considerations for these two cases. While there's a real concern of privacy leakage across active client migrations, privacy leakage due to a stable connection ID across NAT rebindings is not different than that due to a stable TCP connection for the same duration.

There are plenty of times when the client does a network change but doesn't know it
has. Tethering is one example. In such cases, continuing to use the same connection
ID presents a privacy threat.

The tethering case is real, but it's a corner case; connection migrations are dominated by untethered clients.

- jana

-Ekr

On Thu, Mar 9, 2017 at 8:09 AM, Brian Trammell <[email-masked]<mailto:[email-masked]>> wrote:
hi Ekr, all,

Cheers,

Brian

---

From: Flemming Andreasen <[email-masked]>
Date: Thu, 9 Mar 2017 17:01:00 -0500

On 3/9/17 10:07 AM, Eric Rescorla wrote:
>
> On Thu, Mar 9, 2017 at 6:56 AM, Stephen Farrell 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>
>
>     <https://forum.stanford.edu/events/2016/slides/iot/Judson.pdf>
>     >
>
> Sorry, I should have been clearer that I wasn't endorsing this 
> strategy. That's

And even devices that could be "trusted" initially often end up being 
compromised (or affected otherwise, e.g. DoS'ed), often because they are 
not patched to remedy new/known vulnerabilities, and in many cases they 
never will be patched. If you accept that premise, then you find a role 
for some middleboxes (e.g FWs and IDS/IPS) to help here, and those boxes 
track "connections" (whether TCP or UDP today) and connection phases so 
they don't have to perform the same set of inspections and (slowpath) 
lookups on every single packet flowing through them. Having a readily 
available connection id would be helpful here.

Thanks

-- Flemming

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 10 Mar 2017 05:39:06 +0000

> The tethering case is real, but it's a corner case;
> connection migrations are dominated by
> untethered clients.

I would not be banking on tethering remaining a fringe activity. This may be true today, but it is rapidly changing with proliferation of connected cars, buses, trains, and other mobile hotspots. And I will not even venture a guess about technologies available in 10-15 years.

- Igor

-----Original Message-----
From: Jana Iyengar [[email-masked]]
Received: Thursday, 09 Mar 2017, 2:51PM
To: Eric Rescorla [[email-masked]]
CC: Brian Trammell [[email-masked]]; IETF QUIC WG [[email-masked]]
Subject: Re: Middlebox introspection/self-describing packets

- Look up the host/port

That'd be helpful.

- jana

-Ekr

>> (f that's not the case, if failure to send connection ID leads to connection failure, then "negotiation" is a euphemism: the server informs the client whether it must send a connection ID, probably encrypted, and nobody needs any flags, unless the presence of connection ID changes the offsets of fields we *do* want to explicitly expose, such as packet number echo (https://github.com/quicwg/base-drafts/issues/269<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_269&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kHUB4DKMT1q9uR4OYOSDtqV4df0XzWtAKovbagg8_SE&s=ORbrwfTj0Dg91UHcIHRM4csikT0wOAADL1x3d6gkn6g&e=>) or troubleshooting flags (https://github.com/quicwg/base-drafts/issues/279<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_279&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=kHUB4DKMT1q9uR4OYOSDtqV4df0XzWtAKovbagg8_SE&s=-Yx8Uf0Y6W0tH128I5Q4JWoT58D1r9TSeWzJ2i21cZM&e=>).
>

Cheers,

Brian

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2017 07:24:51 +0100

On 3/10/17 6:39 AM, Lubashev, Igor wrote:
> I would not be banking on tethering remaining a fringe activity. This
> may be true today, but it is rapidly changing with proliferation of
> connected cars, buses, trains, and other mobile hotspots. And I will
> not even venture a guess about technologies available in 10-15 years.

But you can't engineer around such vagaries.
 
Eliot

---

From: Kyle Rose <[email-masked]>
Date: Fri, 10 Mar 2017 07:33:57 -0500

On Fri, Mar 10, 2017 at 1:24 AM, Eliot Lear <[email-masked]> wrote:

> I would not be banking on tethering remaining a fringe activity. This may
> be true today, but it is rapidly changing with proliferation of connected
> cars, buses, trains, and other mobile hotspots. And I will not even venture
> a guess about technologies available in 10-15 years.
>
>

I don't think you can assert that for all cases.

I think you may be able to conclude some variant of that statement for
classes of problems requiring the client to know where it is on the
network. E.g., basing the answer to the statement "should my podcast player
download a 100MB podcast?" on whether the device is connected to wifi can't
take into account whether wifi is really cell tethering without extra
telemetry.

But you can certainly deal with linkability of clients obliviously changing
networks if you're willing to accept some variant of never sending the same
connection ID twice. The remaining questions are around the costs and
tradeoffs of mechanisms to achieve this, including the perfectly legitimate
question of whether it's worth any added complexity at all.

Kyle

---

From: Lucas Clemente <[email-masked]>
Date: Fri, 10 Mar 2017 12:33:41 +0000

I'm a bit late to this party, but I'd like to chime in from an
implementor's perspective.

I think we should be careful about adding even more complexity to the
public header (and the protocol in general). Some of the proposals here
would require information to be passed between otherwise cleanly separated
areas of the protocol (e.g. for adding infos about received packets into
the header), or make parsing the header even more complicated. This could
increase the probability of bugs in implementations, or – even worse – in
middleboxes, which could fail to understand uncommon public header
configurations.

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2017 14:29:54 +0100

On 3/10/17 1:33 PM, Kyle Rose wrote:
> On Fri, Mar 10, 2017 at 1:24 AM, Eliot Lear <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>>     I would not be banking on tethering remaining a fringe activity.
>>     This may be true today, but it is rapidly changing with
>>     proliferation of connected cars, buses, trains, and other mobile
>>     hotspots. And I will not even venture a guess about technologies
>>     available in 10-15 years.
>
> network. E.g., basing the answer to the statement "should my podcast
> player download a 100MB podcast?" on whether the device is connected
> to wifi can't take into account whether wifi is really cell tethering
> without extra telemetry.
> But you can certainly deal with linkability of clients obliviously
> changing networks if you're willing to accept some variant of never
> sending the same connection ID twice. The remaining questions are
> around the costs and tradeoffs of mechanisms to achieve this,
> including the perfectly legitimate question of whether it's worth any
> added complexity at all.

Igor did not assert how the above technology will operate.  For 3G
networks, this probably matters very little in practice.  On the other
side of the spectrum (though not literally) there will be ad hoc
networks that route only quite locally.  And then there is 6lopan. 
State your assumptions.  Then design.

Eliot

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 10 Mar 2017 07:04:15 -0800

> But you can't engineer around such vagaries.

Huh? The point here is that we shouldn't bake in assumptions that may not
be true in the
future and will have significantly negative consequences if they become
untrue.

-Ekr

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2017 16:22:48 +0100

On 3/10/17 4:04 PM, Eric Rescorla wrote:
>
> Huh? The point here is that we shouldn't bake in assumptions that may
> not be true in the
> future and will have significantly negative consequences if they
> become untrue.
>
The reverse is also true.

Eliot

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 10 Mar 2017 16:33:56 +0000

Connected cars was just the first thing that came to mind.  I do not see why these could not jump to WiFi when available (cars are not always zooming along at highway speeds).

With some morning coffee, I can also think of phones tethering (over Wifi, Bluetooth/6LoWPAN, etc), a myriad of connected wearable devices (watches, VR glasses, fitness monitors, portable AV players/recorders, pacemakers—bad idea!).

Basically, the point is that few assumptions can be made about the relative prevalence of technologies in the long term.  From experience, people generally get this very wrong.

-          Igor

From: Eliot Lear [mailto:[email-masked]]
Sent: Friday, March 10, 2017 8:30 AM
To: Kyle Rose <[email-masked]>
Cc: Lubashev, Igor <[email-masked]>; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: Middlebox introspection/self-describing packets

On 3/10/17 1:33 PM, Kyle Rose wrote:
On Fri, Mar 10, 2017 at 1:24 AM, Eliot Lear <[email-masked]<mailto:[email-masked]>> wrote:
On 3/10/17 6:39 AM, Lubashev, Igor wrote:
I would not be banking on tethering remaining a fringe activity. This may be true today, but it is rapidly changing with proliferation of connected cars, buses, trains, and other mobile hotspots. And I will not even venture a guess about technologies available in 10-15 years.

I think you may be able to conclude some variant of that statement for classes of problems requiring the client to know where it is on the network. E.g., basing the answer to the statement "should my podcast player download a 100MB podcast?" on whether the device is connected to wifi can't take into account whether wifi is really cell tethering without extra telemetry.
But you can certainly deal with linkability of clients obliviously changing networks if you're willing to accept some variant of never sending the same connection ID twice. The remaining questions are around the costs and tradeoffs of mechanisms to achieve this, including the perfectly legitimate question of whether it's worth any added complexity at all.

Igor did not assert how the above technology will operate.  For 3G networks, this probably matters very little in practice.  On the other side of the spectrum (though not literally) there will be ad hoc networks that route only quite locally.  And then there is 6lopan.  State your assumptions.  Then design.

Eliot

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2017 17:54:10 +0100

On 3/10/17 5:33 PM, Lubashev, Igor wrote:
>
> Connected cars was just the first thing that came to mind.  I do not
> see why these could not jump to WiFi when available (cars are not
> always zooming along at highway speeds).
>
>  
>
They don't do that today (most that have connectivity have 3G).  They
MIGHT do that in the future (who knows?) but they might do something
else (again, who knows?).

> With some morning coffee, I can also think of phones tethering (over
> Wifi, Bluetooth/6LoWPAN, etc), a myriad of connected wearable devices
> (watches, VR glasses, fitness monitors, portable AV players/recorders,
> pacemakers—bad idea!).
>

All of that is *possible*, I would grant you, and some if it will surely
happen.  The question it is necessary to engineer for that here, or
whether some form of recovery is best handled at the application layer. 
It's not like we haven't taken multiple swings at this exact same
problem over the years (MIP, SCTP, etc), and yet nobody has wanted the
complexity at layers below.

>  
>
> Basically, the point is that few assumptions can be made about the
> relative prevalence of technologies in the long term.  From
> experience, people generally get this very wrong.
>

And my point is that generality comes at a price.  Future proofing too
much can prevent you from getting past the present.  That's why
engineering towards solutions succeeds where architectural approaches
have tended to fail.  In this very organization.

Eliot

---

From: Kyle Rose <[email-masked]>
Date: Fri, 10 Mar 2017 12:08:44 -0500

> State your assumptions.  Then design.

I think the key assumption from my perspective is that the protocol should
expose to passive observers as little metadata that can be used to track
the movements of users as possible while still keeping the network
operable. So, for instance, is there a legitimate traffic management reason
for a non-endpoint observer with tap points in different networks to be
able to correlate different flows for the same connection? If the answer is
"no" (and I would argue that it is), then the protocol would ideally not
enable that.

As I've argued many times, there is a limit to what we can do on the public
internet as traffic analysis is very powerful; but I haven't yet seen the
reduction that shows no possible benefit over simply putting a static
connection ID in the clear in every packet.

Kyle

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2017 18:12:27 +0100

Kyle:

On 3/10/17 6:08 PM, Kyle Rose wrote:
> On Fri, Mar 10, 2017 at 8:29 AM, Eliot Lear <[email-masked]
> I think the key assumption from my perspective is that the protocol
> should expose to passive observers as little metadata that can be used
> to track the movements of users as possible while still keeping the
> network operable.

That's a goal, not an assumption.  The assumptions have to do with what
environment this protocol will operate in over some period of time.  I'm
not arguing the goal mind you, and I'm not landing a view as to what the
right answer here is to EKR's Q.  But see my other message to Igor.  You
can engineer for worlds that will never exist or corner cases that will
make your protocol complex to operate in.  Or you could too narrowly
engineer.  Or maybe something in between.  Choose wisely.

Eliot

---

From: joel jaeggli <[email-masked]>
Date: Fri, 10 Mar 2017 09:41:42 -0800

On 3/10/17 9:08 AM, Kyle Rose wrote:
> network operable. So, for instance, is there a legitimate traffic
> management reason for a non-endpoint observer with tap points in
> different networks to be able to correlate different flows for the
> same connection? If the answer is "no" (and I would argue that it is),
> then the protocol would ideally not enable that.
Stable identifiers over the life  of a "connection" as today with a
"flow" can facilitate hashing, this could be / is simply stateless
hashing across links or stateless distribution into stateful devices.
with server assigned connection-id's pinning connections to particular
endpoint in an otherwise stateless load-balanced service could be
agnostic to changes in the 4 tuple.

> As I've argued many times, there is a limit to what we can do on the
> public internet as traffic analysis is very powerful; but I haven't
> yet seen the reduction that shows no possible benefit over simply
> putting a static connection ID in the clear in every packet.
>
> Kyle
>

---

From: Kyle Rose <[email-masked]>
Date: Fri, 10 Mar 2017 13:36:43 -0500

On Fri, Mar 10, 2017 at 12:12 PM, Eliot Lear <[email-masked]> wrote:

> That's a goal, not an assumption.
>

The assumption is that this is a goal for all new protocols. RFC 7258 and
all that.

> The assumptions have to do with what environment this protocol will
> operate in over some period of time.  I'm not arguing the goal mind you,
> and I'm not landing a view as to what the right answer here is to EKR's Q.
> But see my other message to Igor.  You can engineer for worlds that will
> never exist or corner cases that will make your protocol complex to operate
> in.  Or you could too narrowly engineer.  Or maybe something in between.
> Choose wisely.
>

I have repeatedly, in this very thread, questioned whether the benefits are
worth the added complexity. I'm asking because I feel strongly that we need
to have the conversation on this mailing list in the open, with the
tradeoffs broadly understood. Given the contentious nature of the PLUS BoF
in Berlin, I'm surprised more people haven't taken issue with designs that
assume that a static connection ID for clients oblivious to network changes
is okay.

Kyle

---

From: Christian Huitema <[email-masked]>
Date: Fri, 10 Mar 2017 11:05:15 -0800

On 3/10/2017 10:36 AM, Kyle Rose wrote:
> On Fri, Mar 10, 2017 at 12:12 PM, Eliot Lear <[email-masked]
> The assumption is that this is a goal for all new protocols. RFC 7258
> and all that.
>  
>
>     The assumptions have to do with what environment this protocol
>     will operate in over some period of time.  I'm not arguing the
>     goal mind you, and I'm not landing a view as to what the right
>     answer here is to EKR's Q.  But see my other message to Igor.  You
>     can engineer for worlds that will never exist or corner cases that
>     will make your protocol complex to operate in.  Or you could too
>     narrowly engineer.  Or maybe something in between.  Choose wisely.
>
>
> I have repeatedly, in this very thread, questioned whether the
> benefits are worth the added complexity. I'm asking because I feel
> strongly that we need to have the conversation on this mailing list in
> the open, with the tradeoffs broadly understood. Given the contentious
> nature of the PLUS BoF in Berlin, I'm surprised more people haven't
> taken issue with designs that assume that a static connection ID for
> clients oblivious to network changes is okay.

Let me second that. We don't want to create new long term static
identifiers and yet another way to track devices as they move in the
Internet. Yes, we also don't want to mandate something unduly complex.
But then, the TLS working group was faced with essentially the same
problem for "session resume", and came out with a pretty simple
solution. I don't see why we could not have something similar in QUIC.

-- Christian Huitema

---

From: Kyle Rose <[email-masked]>
Date: Fri, 10 Mar 2017 15:23:45 -0500

On Fri, Mar 10, 2017 at 2:05 PM, Christian Huitema <[email-masked]>
wrote:

> Internet. Yes, we also don't want to mandate something unduly complex. But
> then, the TLS working group was faced with essentially the same problem for
> "session resume", and came out with a pretty simple solution. I don't see
> why we could not have something similar in QUIC.
>

The problem is slightly more complex, in that TLS needs a unique identifier
only when a new TCP connection is opened (where, MPTCP notwithstanding, TCP
does not support client mobility, whereas a client can change network
address in the middle of a QUIC connection, something we explicitly want to
support.

So the mitigation strategy would be different, and in some designs would
involve greater overhead (e.g., doing the TLS-like thing and sending enough
connection IDs in the first ACK to cover expected packets in flight,
followed by more in every ACK thereafter) or different overhead (e.g.,
0-RTT resume after discover of an alien packet leading to a public reset).
But I think it's doable.

Kyle

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 10 Mar 2017 12:51:41 -0800

On Fri, Mar 10, 2017 at 12:23 PM, Kyle Rose <[email-masked]> wrote:

> The problem is slightly more complex, in that TLS needs a unique
> identifier only when a new TCP connection is opened (where, MPTCP
> notwithstanding, TCP does not support client mobility, whereas a client can
> change network address in the middle of a QUIC connection, something we
> explicitly want to support.
>
> followed by more in every ACK thereafter)
>

FWIW, I don't think that you would ever need to send connection_ids for
each packet, b/c you can generate them
deterministically from a shared secret. That's not to say that this is a
great design, but it is better than sending
a huge pile of IDs

-Ekr

> or different overhead (e.g., 0-RTT resume after discover of an alien
> packet leading to a public reset). But I think it's doable.
>
> Kyle
>
>

---

From: Kyle Rose <[email-masked]>
Date: Fri, 10 Mar 2017 18:59:24 -0500

> great design, but it is better than sending

Right, this is the rolling code stuff we were discussing in the other
thread. One disadvantage of that approach is the need for a separate
plaintext server cookie for load balancing state since the rolling code is
essentially random, but that cookie can probably be smaller than a
connection ID and should satisfy a different set of requirements, such as
that the number of values used at any one time be proportional to the
number of servers rather than to the number of clients. It's not great
because this is simply guidance and is therefore subject to abuse, but I
don't see any way out of the stateless load balancing pickle without it.

Kyle

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 10 Mar 2017 16:03:29 -0800

Right. Just for my records: if the server hands out a token for each
packet, then it can
encrypt that token with a key known just to itself. Of course, as you note,
this is sucky
in other ways.

-Ekr