Subject: RE: Version Ossification: Intended Scope for QUICv1

From: Mike Bishop <[email-masked]>
Date: Fri, 1 Nov 2019 16:13:09 +0000

The fundamental problem in all this is what happens when a server is no longer in possession of the mapping it sent to the client.  One option is to say that servers simply MUST persist this, and if they don’t, handshakes will fail – we tried that with HSTS, and we found that servers don’t like things that can leave the site broken.  I don’t think that’s a good path.  Either it will break things, or servers won’t use this mechanism, or more likely both in sequence.

The next obvious solution, sending a VN packet telling the client to come back to v1 and the spec-defined salt, contains a simple version downgrade attack by a middlebox.  There are possible mitigations – the client could continue presenting the token, meaning the server would be able to tell whether a downgrade “actually happened” or not.  (I.e. whether it could still have parsed the version associated with the token)  If the downgrade was illegitimate, that could be communicated.  However, that likely means figuring out QUIC’s own version negotiation story and downgrade mitigation and just using that.

We could also try to build something like ESNI’s recovery path, where the information includes what has to be demonstrated for a recovery value.  The trouble is that, while TLS can transfer a certificate and rely on PKI as that recovery path, we’re in a little bit more uncharted territory.  The only ready solution I see in this space is some kind of longer-lived keypair that the server is somehow “less likely” to lose, but if you can persist that key, you can probably also persist the original configuration.  (Plus, you don’t want to be doing asymmetric crypto before you’ve even decided whether to process the handshake.)

I remain a little uncomfortable that we entirely punted our version negotiation story from v1, and I think the cleanest solution is to bring it back in support of #3166.  However, that has both schedule and security implications – we don’t want to get this part wrong.

From: QUIC <[email-masked]> On Behalf Of Kazuho Oku
Sent: Thursday, October 31, 2019 5:34 PM
To: David Schinazi <[email-masked]>
Cc: QUIC <[email-masked]>
Subject: Re: Version Ossification: Intended Scope for QUICv1

Hi David,

Thank you for summarizing the high-level options. I think it's very useful to have this discussion.

Please see my comments below.

2019年11月1日(金) 6:13 David Schinazi <[email-masked]<mailto:[email-masked]>>:
Hi folks,

The conversation around version ossification has been really interesting since Cupertino. But at least for me it's reached a level of complexity that warrants taking a step back and looking at what we're trying to solve. Here's my understanding of where we are:

1) In Cupertino there was consensus in the room to do something to mitigate ossification of QUICv1.

2) The solution that folks liked was to have a concept of alternative versions and alternative salts that act like QUICv1 but are encoded differently over the wire.

3) A concrete proposal for (2) was for servers to have their own set of (alternative_version, alternative_salt) pairs, then transmit a random pair from that set to each client (i.e. via transport parameters). The server can then reconstruct the alternative_salt just by seeing the alternative_version.

4) Kazuho proposed a solution (#3166<https://github.com/quicwg/base-drafts/pull/3166>) that was both more powerful and more complex: tie the (alternative_version, alternative_salt) pair to NEW_TOKEN frames, to allow servers to generate a unique alternative_salt per client and encode it in the token.

I dispute the argument that #3166 is more complex. My view is that #3166 is more powerful and simpler than (3) (the PR being #2573).

Reasons:
* For a server associating one initial salt with one alternative version, #3166 and #2573 are identical with the exception being how the information is carryied (i.e. TP vs. NEW_TOKEN).
* For a client, bundling the alternative seeds in a NEW_TOKEN makes things simpler, as clients are not required to have a new state that has to be retained across multiple connections.

5) After some interesting back and forth on that PR, it became clear that this is a hard problem to solve: this could cause failures if servers forget their token keys, and mitigations for that problem can become downgrade attack vectors

The proposal for 3 has this problem. It leads to a failure when a server forgets the alternative initial salts.

6) While there was clear WG appetite for (1), I'm not sure we have consensus to build a more powerful solution like (4)

I fear that solving (5) could take some time, which would delay QUICv1. Instead I'd like to propose the following plan:
a) we take the simpler solution (3) and add that to the spec
b) we take the more powerful/complex solution (5) a turn it into a QUIC extension

That would allow us to ship QUICv1 with some level of ossification mitigations without delay.

As stated, the proposed solution for (4) is simpler than (3), can be used to implement (3), and also provides a stronger protection for servers certain that they can retain the token unprotection keys.

I agree that solving (5) for all parties might be hard, and that we might want to punt that to v2. But, I think we should adopt (4) than (3) due to the aforementioned reasons.

Thoughts?

David

--
Kazuho Oku

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 04 Nov 2019 08:45:41 +1100

Why can't servers just bake the "keys" into binaries?  I mean, that's not something that is likely to get lost.  And it isn't really the case that keys need to be secret, they just have to change often enough to frustrate attempts to fixate on them.

On Sat, Nov 2, 2019, at 03:13, Mike Bishop wrote:
>  
> The fundamental problem in all this is what happens when a server is no 
> longer in possession of the mapping it sent to the client. One option 
> is to say that servers simply MUST persist this, and if they don’t, 
> handshakes will fail – we tried that with HSTS, and we found that 
> servers don’t like things that can leave the site broken. I don’t think 
> that’s a good path. Either it will break things, or servers won’t use 
> this mechanism, or more likely both in sequence.
> 
> 
> The next obvious solution, sending a VN packet telling the client to 
> come back to v1 and the spec-defined salt, contains a simple version 
> downgrade attack by a middlebox. There are possible mitigations – the 
> client could continue presenting the token, meaning the server would be 
> able to tell whether a downgrade “actually happened” or not. (I.e. 
> whether it could still have parsed the version associated with the 
> token) If the downgrade was illegitimate, that could be communicated. 
> However, that likely means figuring out QUIC’s own version negotiation 
> story and downgrade mitigation and just using that.
> 
> 
> We could also try to build something like ESNI’s recovery path, where 
> the information includes what has to be demonstrated for a recovery 
> value. The trouble is that, while TLS can transfer a certificate and 
> rely on PKI as that recovery path, we’re in a little bit more uncharted 
> territory. The only ready solution I see in this space is some kind of 
> longer-lived keypair that the server is somehow “less likely” to lose, 
> but if you can persist that key, you can probably also persist the 
> original configuration. (Plus, you don’t want to be doing asymmetric 
> crypto before you’ve even decided whether to process the handshake.)
> 
> 
> I remain a little uncomfortable that we entirely punted our version 
> negotiation story from v1, and I think the cleanest solution is to 
> bring it back in support of #3166. However, that has both schedule and 
> security implications – we don’t want to get this part wrong.
> 
> 
> *From:* QUIC <[email-masked]> *On Behalf Of * Kazuho Oku
> *Sent:* Thursday, October 31, 2019 5:34 PM
> *To:* David Schinazi <[email-masked]>
> *Cc:* QUIC <[email-masked]>
> *Subject:* Re: Version Ossification: Intended Scope for QUICv1
> 
> 
> Hi David,
> 
> 
> 2019年11月1日(金) 6:13 David Schinazi <[email-masked]>:
> 
> > Hi folks,
> 
> 
> > 4) Kazuho proposed a solution (#3166 <https://github.com/quicwg/base-drafts/pull/3166>) that was both more powerful and more complex: tie the (alternative_version, alternative_salt) pair to NEW_TOKEN frames, to allow servers to generate a unique alternative_salt per client and encode it in the token.
> 
> 
> I dispute the argument that #3166 is more complex. My view is that 
> #3166 is more powerful and simpler than (3) (the PR being #2573).
> 
> 
> Reasons:
> 
> * For a server associating one initial salt with one alternative 
> version, #3166 and #2573 are identical with the exception being how the 
> information is carryied (i.e. TP vs. NEW_TOKEN).
> 
> * For a client, bundling the alternative seeds in a NEW_TOKEN makes 
> things simpler, as clients are not required to have a new state that 
> has to be retained across multiple connections.
> 
> 
> 
> The proposal for 3 has this problem. It leads to a failure when a 
> server forgets the alternative initial salts.
> 
> 
> 
> As stated, the proposed solution for (4) is simpler than (3), can be 
> used to implement (3), and also provides a stronger protection for 
> servers certain that they can retain the token unprotection keys.
> 
> 
> I agree that solving (5) for all parties might be hard, and that we 
> might want to punt that to v2. But, I think we should adopt (4) than 
> (3) due to the aforementioned reasons.
> 
> 
> 
> > Thoughts?
> 
> 
> > David
> 
> 
> 
> 
> 
> 
> -- 
> 
> Kazuho Oku
>

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 4 Nov 2019 17:23:11 -0800

On Sun, Nov 3, 2019 at 1:46 PM Martin Thomson <[email-masked]> wrote:

> Why can't servers just bake the "keys" into binaries?  I mean, that's not
> something that is likely to get lost.  And it isn't really the case that
> keys need to be secret, they just have to change often enough to frustrate
> attempts to fixate on them.
>

This is a fine idea, but how does it change anything? Whenever the keys are
rotated at the server, you would still have to deal with old versions
appearing at the server, no?

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 05 Nov 2019 12:58:08 +1100

On Tue, Nov 5, 2019, at 12:23, Jana Iyengar wrote:
> This is a fine idea, but how does it change anything? Whenever the keys 
> are rotated at the server, you would still have to deal with old 
> versions appearing at the server, no?

I'm assuming that you can a) limit the time over which a token could appear, b) use a key identifier so that keys can be changed, and c) keep old keys in the code until the associated tokens are all expired.

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 4 Nov 2019 18:22:04 -0800

On Mon, Nov 4, 2019 at 5:58 PM Martin Thomson <[email-masked]> wrote:

> > > Why can't servers just bake the "keys" into binaries? I mean, that's
> not something that is likely to get lost. And it isn't really the case that
> I'm assuming that you can a) limit the time over which a token could
> appear, b) use a key identifier so that keys can be changed, and c) keep
> old keys in the code until the associated tokens are all expired.
>

Right, that makes sense. I think your point about not requiring this to be
rotated frequently is a good one, and we might even want to suggest your
idea of baking the key into the binary.

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Tue, 5 Nov 2019 09:49:07 +0100

I really don’t think the draft should recommend key deployments like that.
It may be that baked into the binary works, but sound deployment advise is
to use an environment variable for such things, then you can bake it into
you deployment script if you wish. Point being, the WG should not be too
specific here.

On 5 November 2019 at 03.22.32, Jana Iyengar ([email-masked]) wrote:

---

From: "Lubashev, Igor" <[email-masked]>
Date: Tue, 5 Nov 2019 15:41:57 +0000

I think the intent was clear – since the keys’ purpose is not secrecy but occasional perturbation of bits, it should be easy to make “forgetting” a key impossible.  All deployments have some sort of critical configuration and hence configuration management system.  Committing such keys to that configuration management system should do the trick of the keys becoming unforgettable. :)

  *   Igor

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Sent: Tuesday, November 5, 2019 3:49 AM
To: Jana Iyengar <[email-masked]>; Martin Thomson <[email-masked]>
Cc: QUIC WG <[email-masked]>
Subject: Re: Version Ossification: Intended Scope for QUICv1

I really don’t think the draft should recommend key deployments like that.
It may be that baked into the binary works, but sound deployment advise is to use an environment variable for such things, then you can bake it into you deployment script if you wish. Point being, the WG should not be too specific here.

On 5 November 2019 at 03.22.32, Jana Iyengar ([email-masked]<mailto:[email-masked]>) wrote:
On Mon, Nov 4, 2019 at 5:58 PM Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:
On Tue, Nov 5, 2019, at 12:23, Jana Iyengar wrote:
> On Sun, Nov 3, 2019 at 1:46 PM Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

Right, that makes sense. I think your point about not requiring this to be rotated frequently is a good one, and we might even want to suggest your idea of baking the key into the binary.

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 10 Nov 2019 14:53:26 -0800

I would actually like to suggest something more aggressive (or,
conservative, if you prefer), namely that we punt version ossification
entirely to the future, whether to an extension or QUIVc2. In support
of this proposal, I would make three points:

- We are actually going to be running concurrent versions for quite
  some time. Even if everyone were to aggressively (say 6-12 months)
  deprecate all the draft versions upon shipping the RFC, we're
  looking at something that's 18-24 months out. And of course,
  I expect people to do some simple version greasing. So, that
  probably buys us some time before real bad ossification sets
  in.

- It's turning out to be surprisingly hard to design a mechanism that
  works, as evidenced by David's mechanism. And to that, I would add
  that anything that involves having a server hand out some new set of
  keys/salts is going to run into the same kind of multi-CDN problems
  we had with ESNI (i.e., failures when you get switched between
  CDNs). And then recovering from that cleanly either seems to mean
  some ESNI-style fallback or allowing a middlebox to forge VN to
  force you back to the published version/salt.

- It seems like this would all be a lot easier if we had ESNI, because
  you could tie the use of the new salt/version to the ESNIConfig
  (either as a public or symmetric key, depending on taste).

So what I would propose is that we put this problem on ice for now and
then revisit it as an extension. All of the proposals that we have
seen seem like they could be done as an extension, and then we can (1)
get QUICv1 done and (2) hopefully leverage ESNI once we have some
experience.

-Ekr

On Thu, Oct 31, 2019 at 2:13 PM David Schinazi <[email-masked]>
wrote:

> Hi folks,
>
> The conversation around version ossification has been really interesting
> since Cupertino. But at least for me it's reached a level of complexity
> that warrants taking a step back and looking at what we're trying to solve.
> Here's my understanding of where we are:
>
> 2) The solution that folks liked was to have a concept of alternative
> versions and alternative salts that act like QUICv1 but are encoded
> differently over the wire.
>
> 3) A concrete proposal for (2) was for servers to have their own set of
> (alternative_version, alternative_salt) pairs, then transmit a random pair
> from that set to each client (i.e. via transport parameters). The server
> can then reconstruct the alternative_salt just by seeing
> the alternative_version.
>
> 4) Kazuho proposed a solution (#3166
> <https://github.com/quicwg/base-drafts/pull/3166>) that was both more
> powerful and more complex: tie the (alternative_version, alternative_salt)
> pair to NEW_TOKEN frames, to allow servers to generate a
> unique alternative_salt per client and encode it in the token.
>
> 5) After some interesting back and forth on that PR, it became clear that
> this is a hard problem to solve: this could cause failures if servers
> forget their token keys, and mitigations for that problem can become
> downgrade attack vectors
>
> 6) While there was clear WG appetite for (1), I'm not sure we have
> consensus to build a more powerful solution like (4)
>
> I fear that solving (5) could take some time, which would delay QUICv1.
> Instead I'd like to propose the following plan:

---

From: Christian Huitema <[email-masked]>
Date: Sun, 10 Nov 2019 16:48:14 -0800

> get QUICv1 done and (2) hopefully leverage ESNI once we have some

+1

-- Christian Huitema

---

From: Ian Swett <[email-masked]>
Date: Mon, 11 Nov 2019 11:57:40 -0500

+1 from me as well.

I think this is useful work and I support the QUIC WG working on it, but I
don't believe it needs to be included in v1 and I'd like to get some
deployment experience with it before standardizing it.

---

From: Roberto Peon <[email-masked]>
Date: Mon, 11 Nov 2019 22:39:16 +0000

I would love to see a v2 that intends to address this chartered before agreeing that this shouldn’t be included.
-=R

From: QUIC <[email-masked]> on behalf of Ian Swett <ianswett=[email-masked]>
Date: Monday, November 11, 2019 at 8:58 AM
To: Christian Huitema <[email-masked]>
Cc: David Schinazi <[email-masked]>, Eric Rescorla <[email-masked]>, QUIC <[email-masked]>
Subject: Re: Version Ossification: Intended Scope for QUICv1

+1 from me as well.

I think this is useful work and I support the QUIC WG working on it, but I don't believe it needs to be included in v1 and I'd like to get some deployment experience with it before standardizing it.

On Sun, Nov 10, 2019 at 7:48 PM Christian Huitema <[email-masked]<mailto:[email-masked]>> wrote:

+1

-- Christian Huitema

---

From: Marten Seemann <[email-masked]>
Date: Tue, 12 Nov 2019 09:01:06 +0700

> entirely to the future, whether to an extension or QUIVc2.

I'm not sure if I can agree with this assessment. While we will have
multiple QUIC versions deployed at the same time, the salts used to encrypt
the Initial packets are all public. Middlebox vendors can easily build a
device that is able to decrypt any of these versions. The fact that draft
versions + QUIC v1 will (hopefully) work tells us little about how
middleboxes will handle unknown version, i.e. versions that they don't know
how to decrypt. This suggests to me that building a greasing mechanisms
which ensures that packets with unknown version numbers are sent over the
wire is essential for QUIC v1.

On Tue, Nov 12, 2019 at 5:39 AM Roberto Peon <[email-masked]> wrote:

> I would love to see a v2 that intends to address this chartered before
> agreeing that this shouldn’t be included.
>
> -=R
>
>
>
> *From: *QUIC <[email-masked]> on behalf of Ian Swett <ianswett=
> [email-masked]>
> *Date: *Monday, November 11, 2019 at 8:58 AM
> *To: *Christian Huitema <[email-masked]>
> *Cc: *David Schinazi <[email-masked]>, Eric Rescorla <
> [email-masked]>, QUIC <[email-masked]>
> *Subject: *Re: Version Ossification: Intended Scope for QUICv1
>
>
>
> +1 from me as well.
>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 11 Nov 2019 21:11:02 -0800

On Mon, Nov 11, 2019 at 6:01 PM Marten Seemann <[email-masked]>
wrote:

> device that is able to decrypt any of these versions.
>

ISTM that this is the scenario of interest for QUICv2 as well, namely that
the protocol will be public but vendors will have to adapt. If they do that
by reading specs and updating prompt, that seems not ideal but acceptable.

-Ekr

The fact that draft versions + QUIC v1 will (hopefully) work tells us
> little about how middleboxes will handle unknown version, i.e. versions
> that they don't know how to decrypt. This suggests to me that building a
> greasing mechanisms which ensures that packets with unknown version numbers
> are sent over the wire is essential for QUIC v1.
>
>> I think this is useful work and I support the QUIC WG working on it, but
>> I don't believe it needs to be included in v1 and I'd like to get some

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 12 Nov 2019 16:44:00 +1100

On Tue, Nov 12, 2019, at 16:11, Eric Rescorla wrote:
> ISTM that this is the scenario of interest for QUICv2 as well, namely 
> that the protocol will be public but vendors will have to adapt. If 
> they do that by reading specs and updating prompt, that seems not ideal 
> but acceptable.

Yes, that would be tolerable.  The concern is that they stop at v1 and we can't properly rely on v2 working.

---

From: Ian Swett <[email-masked]>
Date: Tue, 12 Nov 2019 10:59:58 -0500

Past experience indicates if we don't use joints, they will not work, both
internal(ie: SETTINGS) and external ones.
It also indicates that middleboxes will want to identify QUIC.  I'm happy
to sacrifice the invariants and QUIC bit for that purpose.

I have seen no evidence that if we continue using multiple versions in
production, future version numbers with the same invariants won't work.
Google is starting to deploy the 4th public header format in the last 2
years(current IETF invariants).  These public header changes require more
outreach, but even they are still possible.