Subject: [TLS] New TLS version negotiation mechanism, take 2

From: Dave Garrett <[email-masked]>
Date: Thu, 26 Mar 2015 03:30:42 -0400

Based on feedback in the previous discussion, I think I've cobbled together a framework that might be good to use.

* A new extension that contains a single ProtocolVersion, with its value named max_version
* Rename ClientHello.client_version to min_version
* Rename ServerHello.server_version to negotiated_version
* Convert ProtocolVersion to a 16-bit enum, with equivalent values to existing struct format
* The values are as follows:

1) record layer version frozen to TLS_version_1_0(0x0301)  (PR #107)
2) ClientHello.min_version set to TLS_version_1_2(0x0303)
    (for clients supporting TLS 1.2, this is a constant;
     clients not supporting TLS 1.x set this to their higher min version)
3) Both endpoints set their max_version (in ext) to their max supported version
    (all clients send; all servers reply if clients send; these values are constant based on config)
4) ServerHello.negotiated_version is picked in a manner similar to now, but from the endpoints'
    max_version fields. If the new extension is not supported, it transparently falls back to
    TLS 1.2, and old servers will fall back to TLS 1.0/1.1 if needed.

Servers now have a min_version from client to not offer to fall back too far at all. Version IDs 0x0304 through 0x03FF can be assigned as IDs for upcoming draft versions. The final version will be assigned 0x0400. As there's no longer an attempt to do something vaguely akin to major.minor, the values are arbitrary and can be assigned for draft/experimental versions as needed. Negotiation logic specifies that servers picking a lower version must pick a stable version if the client did not explicitly offer a supported experimental version.

This seems to get pretty much what we want here:
+ TLS 1.3+ version intolerance is not an issue, unless the client refuses to negotiate TLS 1.x, in which case it doesn't matter. (it won't work regardless of the server responding ideally or not)
+ Only one extension with one fixed length value
+ Both stable and draft/experimental versions can use the same negotiation mechanism (and get tested)
+ It's more or less as simple as can be, given that we're hacking in a new mechanism on top of the old. Yeah, lots of version fields, but almost all are constants.
+ All the various version fields are now named so as to indicate what they're actually for

Ditching the version field from the encrypted records is a very good idea to go along with this, as it's useless and yet another version field to confuse things.

I've created a branch with a very WIP set of changes. I have not updated the backwards compatibility section for this new system fully yet. (just a bunch of TODOs listed there) The other appendix bits are not added either, and there's a few other spots that need some extra explanation. This is just to get the main idea of this proposal for discussion. This is based on PR #107:

https://github.com/davegarrett/tls13-spec/compare/patch-3...davegarrett:NVN

Dave

---

From: Henrik Grubbström <[email-masked]>
Date: Fri, 27 Mar 2015 14:49:45 +0100

> framework that might be good to use.

Please state the problem that you attempt to fix.

> Servers now have a min_version from client to not offer to fall back too far at all.

... And prior TLS servers will still fall back further...

I don't see any benefit at all with this scheme.

  1) Any server that implements it will by definition support TLS 1.3 or later,
     and will with the current version negotiation scheme presumably select
     min(ClientHello.version, max_supported_version) as protocol version anyway,
     so what's the point with min_version?
  2) In case the server selects a version that the client doesn't like, this
     will be noticed by the client in the first packet that it
receives, at which
     point it can terminate the connection, but the same is true for your
     scheme, as the server is free to select TLS 1.2 (or any earlier, as
     it may pretend not to understand your extension).

> Version IDs 0x0304 through 0x03FF can be assigned as IDs for upcoming draft
> versions. The final version will be assigned 0x0400. As there's no longer an attempt
> to do something vaguely akin to major.minor, the values are arbitrary and can be
> assigned for draft/experimental versions as needed. Negotiation logic specifies that
> servers picking a lower version must pick a stable version if the client did not
> explicitly offer a supported experimental version.

Might the above be the problem you want to fix? Ie you want to support
negotiation of
(incompatible) draft versions of TLS 1.3?

> + TLS 1.3+ version intolerance is not an issue, unless the client refuses to negotiate
>   TLS 1.x, in which case it doesn't matter. (it won't work regardless of the server
>   responding ideally or not)

Please remind me, but wasn't the observed TLS 1.3 version intolerance mostly
limited to the version in the packet layer?

I also expect such intolerances to quickly disappear as soon as TLS
1.3 clients become available.

> + Only one extension with one fixed length value

But here you lose the benefit, as the client can't inform the server
that it supports multiple TLS 1.3 drafts if the field is of fixed
length.

> + It's more or less as simple as can be, given that we're hacking in a new
>   mechanism on top of the old. Yeah, lots of version fields, but almost all are
>   constants.

As the current version negotiation scheme is quite simple, and people
still get it wrong, I expect your scheme to be handled even worse.

I do see a potential need for an extension for informing about which
draft version(s) that the client (and server) support.

> + All the various version fields are now named so as to indicate what they're actually for

Except for when they aren't. Eg ClientHello.min_version will be
handled as if it were ClientHello.max_version by TLS 1.2 and earlier
servers.

> Ditching the version field from the encrypted records is a very good idea to go
> along with this, as it's useless and yet another version field to confuse things.

This I can agree with.

/grubba

-- 
Henrik Grubbström                                       [email-masked]
Roxen Internet Software AB                              [email-masked]

---

From: Dave Garrett <[email-masked]>
Date: Fri, 27 Mar 2015 11:53:34 -0400

> Please state the problem that you attempt to fix.

Well, it's been discussed in the previous discussion. Brian's comments on first proposal:
https://www.ietf.org/mail-archive/web/tls/current/msg15470.html

The problem is that 1.5% of the Internet is TLS 1.3 version intolerant, even with a record layer fix. Attempting to adopt even a draft version of TLS 1.3 would require insecure fallback, which we are very much trying to get rid of. Releasing a new major version of TLS that cannot interop with 1.6% (incl. 0.1% TLS 1.2 intolerant) of the Internet would seriously hamper adoption, or more likely solidify insecure fallback.

The second problem is that there is not currently a way to negotiate the usage of any draft versions. Attempting to do so via a separate extension risks using a scheme separate from the final version, meaning the actual negotiation mechanism wouldn't properly get tested until release.

> ... And prior TLS servers will still fall back further...

Yes, that's the idea. For servers supporting then new method, the server is capable of always picking a version within the range supported by the client. The range could be non-continuous, but my initial proposal to deal with this more generically was considered to complex. Old servers will of course fall back to the old mechanism, but they are at least guaranteed to fall back rather than choke on a version number they can't properly handle. It shouldn't be an issue, but it is.

> limited to the version in the packet layer?

Freezing the record layer version (PR #107) fixes about 5.3%. Freezing the ClientHello version fixes another 1.5%.

https://www.ietf.org/mail-archive/web/tls/current/msg15486.html
citing the survey from
https://www.ietf.org/mail-archive/web/tls/current/msg15141.html

> 1.3 clients become available.

I do not.

> that it supports multiple TLS 1.3 drafts if the field is of fixed

Yes. If we'd like to go back to the list idea, I'm not wholly against it.

In this mechanism, the client may support only one draft, but the server may support multiple just fine.

> still get it wrong, I expect your scheme to be handled even worse.

TLS version intolerance mainly started because sloppy implementations did not account for the fact that there would be multiple versions. There are multiple versions now. I expect the situation to be much better, and I intend to put a few simple test parameters in an appendix to cover current and future cases.

> handled as if it were ClientHello.max_version by TLS 1.2 and earlier

Within the context of TLS 1.3+, all the various version fields are now for what they're actually for. An old server will be able to parse it as TLS 1.2 compatible, and reply in a manner the client can reject if needed. (in the same way that an SSL3 server can still parse a TLS 1.2 hello and reply coherently) There is no way to make every server know everything here. This scheme provides a hello that both old and new can fully understand and handle properly based on the version negotiation mechanisms they are capable of. That's the goal here.

---

From: Xiaoyin Liu <[email-masked]>
Date: Fri, 27 Mar 2015 15:46:05 -0400

I support this version negotiation proposal, but I prefer the old one, i.e. listing all supported TLS versions. See some inline comments:
 
> From: [email-masked]
> To: [email-masked]
> Date: Fri, 27 Mar 2015 11:53:34 -0400
> CC: [email-masked]
> Subject: Re: [TLS] New TLS version negotiation mechanism, take 2
> 
> The second problem is that there is not currently a way to negotiate the usage of any draft versions. Attempting to do so via a separate extension risks using a scheme separate from the final version, meaning the actual negotiation mechanism wouldn't properly get tested until release. I agree.
> I do not. I do not either. Even TLS 1.2 intolerant servers haven't disappeared yet.

> In this mechanism, the client may support only one draft, but the server may support multiple just fine. I support the list idea. Besides being able to support draft versions, there is another benefit. If, for instance, after TLS 1.5 is released, someone finds a weakness that only exists in TLS 1.4, which makes TLS 1.4 insecure. But TLS 1.3 and TLS 1.5 are still secure to use, and TLS 1.5 is better than TLS 1.3. Then with this version negotiation mechanism, clients can very easily mitigate the TLS 1.4 vulnerability by removing TLS 1.4 from the supported version list. But with the current version negotiation, clients would have to either set highest client version to TLS 1.3 or set to TLS 1.5 but fallback to TLS 1.3 if servers try to negotiate TLS 1.4. I know this is quite unlikely to happen, but I don't think we can safely assume this will never happen. Also this mechanism enables implementations to skip versions for whatever reasons, such as that some versions are not widely negotiated, like TLS 1.1 today.
I don't think this negotiation mechanism adds too much complexity. This mechanism is similar to listing supported cipher suites, elliptic curves, signature algorithms, etc. If implementations can handle those extensions right, I don't worry they will handle this version negotiation wrong.
Best,Xiaoyin

---

From: Dave Garrett <[email-masked]>
Date: Fri, 27 Mar 2015 17:53:17 -0400

On Friday, March 27, 2015 03:46:05 pm Xiaoyin Liu wrote:
> I support the list idea. Besides being able to support draft versions,
> there is another benefit. If, for instance, after TLS 1.5 is released,
> someone finds a weakness that only exists in TLS 1.4, which makes TLS 1.4
> insecure. But TLS 1.3 and TLS 1.5 are still secure to use, and TLS 1.5 is
> better than TLS 1.3. Then with this version negotiation mechanism,
> clients can very easily mitigate the TLS 1.4 vulnerability by removing
> TLS 1.4 from the supported version list. But with the current version
> negotiation, clients would have to either set highest client version to
> TLS 1.3 or set to TLS 1.5 but fallback to TLS 1.3 if servers try to
> negotiate TLS 1.4.
>
>I know this is quite unlikely to happen, but I don't think we can safely
>assume this will never happen.
>
>Also this mechanism enables implementations to skip versions for whatever
>reasons, such as that some versions are not widely negotiated, like TLS
>1.1 today.

This was my exact reasoning for my first proposal:
https://www.ietf.org/mail-archive/web/tls/current/msg15441.html

Brian Smith posted a detailed review here:
https://www.ietf.org/mail-archive/web/tls/current/msg15470.html

His argument against the list method is that:
1) The client should not be announcing support for old versions that may be weaker than its highest supported version, and that this constitutes an unnecessary security risk.
2) The added complexity of negotiating with potentially non-continuous version support lists is not worth the risk of implementors botching it.

Personally, every time the general argument that announcing specific versions of something gives an attacker too much information, I'm not really convinced. That said, I don't really oppose limiting announced information, either.

I don't think the added complexity would make things significantly more likely to break, but I don't dispute that it's a valid argument.

The complexity differences between the current proposal and a list based proposal are that:
a) The ClientHello version field would be frozen and meaningless in a list negotiation, whereas the current proposal recycles it in a manner that avoids clients offering TLS 1.2 without supporting it.
b) List negotiation would vary the size of the extension. An arbitrary list size limit would not be a bad idea. It would nonetheless increase bandwidth usage slightly.
c) Clients will undoubtedly not all send the lists ordered properly and servers will have to sort through it to hunt for a best match to negotiate.
d) On the plus side, a list based negotiation would not be checking any version ranges anymore. It would be only looking for the highest overlapping value between the client and server's lists.

I would also add the argument that TLS should not make it more than a few major versions into the future. (DTLS is another story, and there may not even be a need for a new version negotiation mechanism there) TCP needs to be replaced with something that is reliable, fast, and secure. Something QUIC based will probably be the start, eventually. Based on that logic, super future-proofing TLS is counter-productive.

Honestly, I think the list based route could be done cleaner, though. Yeah, we might not really need things generalized this much, and non-continuous version support may not in demand, but it might work better. I'm not really opposed to the single max version route either.

That said, as I think about and work on the current WIP changeset, I am noting pitfalls that need mentioning that I think can be side-stepped with a list based solution (which might end up with its own, of course :p ).

Dave

---

From: Dave Garrett <[email-masked]>
Date: Fri, 27 Mar 2015 22:53:30 -0400

On Sunday, March 08, 2015 01:39:47 pm Dave Garrett wrote:
> A basic proposal for a new TLS version negotiation mechanism for TLS 1.3+. It'll be needed to handle draft implementations, if desired, and I think it might just be wanted for all negotiation. TLS 1.3 version intolerance is currently surveyed to be in the ~1.5% range [1]. The simplest method is to do so via a new mandatory extension, as has been suggested by people before.

On Tuesday, March 10, 2015 10:08:27 am Brian Smith wrote:
> I agree that that seems like the best way forward.
[...]
> Anyway, I support your proposal, if it is simplified.

On Friday, March 27, 2015 03:46:05 pm Xiaoyin Liu wrote:
> I support this version negotiation proposal, but I prefer the old one, i.e. listing all supported TLS versions.

Ok, I've written out the bulk of main changes that would be needed to do both potential new methods:
#1: an extension providing full lists of version support which the server find the highest common value of
#2: an extension using basically the same method as previously, but bumping the value to the extension

For others on the mailing list, the reasons for this are the irritatingly high rate of TLS 1.3+ version intolerance that would seriously hamper adoption of a new version, as well as the desire to negotiate draft versions to get widespread testing (similar to how HTTP/2 did things).

My initial idea was for method #1, however upon scrutiny by Brian and myself, I decided to work on fleshing out #2. The potential complexity did sound like a larger risk. After doing so, I think that whilst #2 ~sounds~ simpler because it's very similar to the current negotiation method, in reality, #1 is far more straightforward and has fewer potential pitfalls in implementing.

I have two WIP branches, one for each method:
#1: https://github.com/davegarrett/tls13-spec/compare/patch-3...davegarrett:EVN
#2: https://github.com/davegarrett/tls13-spec/compare/patch-3...davegarrett:NVN

(compare links because patch-3/PR#107 is approved, but not yet merged into trunk)

Comparing between the two branches, the main difference is that whilst the list method has more info for the server to deal with, it's simpler to handle and explain in fewer words more clearly. With the simple proposed value, the server has to pick the highest version it supports less than the proposed version. When throwing draft versions into the mix, a server needs to make sure not to offer an draft version if the client did not offer support for one. (alternatively we could have another extension for drafts, but that's more complicated overall and I really want a single method that gets properly tested prior to release) In enumerating the pitfalls I see, the errors that need to checked for in the list method can be explained more simply. I can lay out the entire list based algorithm in a handful of lines of pseudocode, whereas for the simple proposed value method it's a sprinkling of "MUST" instructions throughout the spec. This completely new method also ensures that nobody just uses whatever old algorithm they had with potentially problematic changes to handle the new. Note, that while I say "completely new", I mean new to TLS versioning. It's similar to ALPN in concept.

Note that in the current version of #2 (NVN branch) I'm recycling the hello version to use as a min_version to add the ability for servers to never even offer to downgrade further than the client wants. This could be dropped to simplify things, though I don't think it's worth it. The draft of #1 (EVN branch) does not need anything like that to have the server pick a viable version every time (if one exists), and can also handle non-continuous supported version lists easily. (e.g. support 1.2 and 1.4 but not 1.3, and have a 1.4 server negotiate 1.2 instead of offer 1.3 and fail) In that case, the hello version is frozen. Both branches rename the ServerHello version to negotiated_version to make it clear what it actually means. (with PR #107, the record layer version is also frozen)

So, now I think #1 is the way forward, after giving #2 a shot. Still a WIP, but any feedback would be greatly appreciated.

Dave

---

From: Dave Garrett <[email-masked]>
Date: Sun, 29 Mar 2015 02:04:49 -0400

After rewriting the list based approach, I think it really is the best way forward. I've updated my WIP branch. It's simpler and far more thorough than the prior drafts.

https://github.com/davegarrett/tls13-spec/compare/patch-3...davegarrett:EVN

This simplifies the handling of backward capable version negotiation and does everything through the extension. I think it's overall easier to follow in comparison to the other approach, or the prior spec. I think that by requiring implementations to switch over to a general list comparison approach and drop their old logic, this can create a far more reliable process. (the basic list comparison mechanism is done for plenty of other areas in TLS)

Note that the little pseudocode bit is really only there to point out the simplicity. It's probably not needed.

Dave