Subject: Re: [TLS] Update spec to match current practices for certificate chain order

From: Dave Garrett <[email-masked]>
Date: Fri, 8 May 2015 19:35:13 -0400

A PR with language based on this discussion:
https://github.com/tlswg/tls13-spec/pull/169/files

The text is mostly the same or similar, with the second "MUST" changed to a "SHOULD", and the term "trust anchor" replacing references to root keys/CA.

Is this changeset acceptable?

Dave

---

From: [email-masked] (Martin Rex)
Date: Sat, 9 May 2015 05:20:35 +0200 (CEST)

Dave Garrett wrote:
> The text is mostly the same or similar, with the second "MUST" changed
> to a "SHOULD", and the term "trust anchor" replacing references to root
> keys/CA.
> 

I am violently opposed to this change.

It creates new ambiguities, makes bogus, backwards-incompatible
and interoperability-impairing behaviour appear less objectionable,
while providing _zero_ benefit.

The "original" requirements apply to the server's Certificate
handshake message, i.e. which certificates and in what order the
server is required to encode into the Server Certificate handshake
message.

The purpose of these requirements for the server is to facilitate
the the client's job to verify the server's certificate, improve
interoperability and create a predictable&deterministic client
behaviour that should be independent from the particular client
implementation as much as possible.

Clients that are willing to perform unsafe path discovery on
a Server's certificate handshake message will *NOT* be inconvenienced
if the server puts the certificates in correct order and optionally
omits only the self-signed rootCA certificate at the end.

Clients that do not perform any fancy unsafe path discovery, on the
other hand, may easily be non-interoperable with sloppy servers that
do not properly compose their Server Certificate handshake message,
and these Client are _not_ to blame for any resulting interop failure,
because it is clearly the server TLS implementation that goofed
when it produces a Server Certificate PDU that violates the MUSTs.

-Martin

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 9 May 2015 03:43:47 +0000

I'm inclined to agree with Martin.

---

From: "Ryan Sleevi" <[email-masked]>
Date: Fri, 8 May 2015 20:51:49 -0700

>  implementation as much as possible.

Of course, none of this has happened because the X.500 directory doesn't
exist, each client's job also has to maintain an appropriate trust store,
the server cannot know apriori what that trust store is unless it knows
the client, and if it does know the client, it can know whether or not it
supports a given behaviour.

>  omits only the self-signed rootCA certificate at the end.

I know nothing I say will dissuade you from calling it "unsafe path
discovery", even though I rather vehemently disagree with you. I will
simply point out, however, that were it not for this "unsafe path
discovery", we would not finally be moving to a place where we might be
able to turn off 1024-bit RSA keys used in intermediate and root
certificates, because the PKI is hard and the Internet involves multiple
stakeholders.

>  when it produces a Server Certificate PDU that violates the MUSTs.

If you're going to write a client that works on the Internet at large and
is capable of moving at the speed of the Internet, the best I can say is
"Cheer up and deal with it". Clients doing what you propose, rather than
the thing you hate, hold Internet security back, so I'm not terribly
sympathetic (e.g. https://access.redhat.com/articles/1413643 for the most
recent example)

While I realize it's just my opinion, in my role, any client that does
what you state above I consider bugged and recommend against deploying.
Because it either won't work on the Internet or, far more commonly, it
will hold the people trying to secure it back, and in the process, hold
the whole Internet hostage to their broken security.

But c'est la vie.

---

From: Dave Garrett <[email-masked]>
Date: Sat, 9 May 2015 00:49:16 -0400

> and interoperability-impairing behaviour appear less objectionable,

Yep, but it does convert interoperability-impairing behavior into interoperable behavior.

> while providing _zero_ benefit.

It's not zero.

Advantages:
improved interoperability / implementation parity (like it or not)
allows multiple intermediates for transitional capabilities

Disadvantages:
minor validation complexity increase (that already seems common)
it's Wrong™

> Clients that do not perform any fancy unsafe path discovery, on the

Fancy, maybe; unsafe, no. We're debating whether or not paperwork should be accepted if page 3 was stapled between pages 1 and 2. In an ideal world, no, but in a world after two decades of SSL/TLS... I don't care. The on-the-wire ordering is a minor technicality and there are legitimate use-cases that need an update here, in addition to handling shoddy server implementations. (Some server implementations seem to act like this because they leave the specifics of ordering up to configuration by an admin that probably doesn't know the spec.)

See also Ryan's reply that cites the transitional issue in detail:
https://www.ietf.org/mail-archive/web/tls/current/msg16226.html
TL;DR: sanely phasing out 1024-bit RSA keys is dependent on this

Also, you've attached the adjective "unsafe" to the terminology here without providing a justification for doing so. Please explain a practical reason why this would be unsafe and we might agree with you. If you can actually show that this would be unsafe, and that mandating all clients to stop doing this and throw a fatal error is the best course of action, and can actually get a consensus to this effect on this list, then I'd probably concur. As I said in a prior email in this thread, I'm generally in favor of strict error enforcement when actually useful and properly coordinated. Arguing for keeping the text as-is, however, is just ignoring the problem.

{In case of possible misunderstanding: I do not ask rhetorical questions. I'm not being sarcastic and am genuinely curious if there is sufficient justification to consider this practice to be unsafe. I could see how it might be risky for an already sloppy implementation to add support for, but I don't see how this feature itself makes this unsafe.}

> when it produces a Server Certificate PDU that violates the MUSTs.

Yes, very much so.
*That is one of the primary reasons for this change.*
Client implementers need to be given a heads-up that this is something they need to account for, as this is already current practice. The proposed spec change is merely a documentation of this.

Dave

---

From: Viktor Dukhovni <[email-masked]>
Date: Sat, 9 May 2015 05:34:02 +0000

> Yep, but it does convert interoperability-impairing behavior into interoperable behavior.

On the contrary, the currently required server behaviour is
interoperable, and relaxing the rules is likely to reduce that
interoperability if more servers violate the rules.

The fact that many clients don't enforce the rules is not surprising,
there is no Internet protocol police, and the spec does not require
clients to enforce the rules.

We must read the spec exclusively as a server requirement to order
the chain.  Client obligations in particular TLS applications may
vary from application to application.  Browsers may well jump
through various hoops, and likely need to do so.  Other applications
do not.  The world-wide-web is not the entire Internet, and browsers
are not the only TLS-using applications.

For example, Postfix (when configured to validate peer certificates
via a set of local trust-anchors) does not tolerate missing
intermediate certificates, the trust store it uses for this typically
contains only self-signed trust-anchor certs.

Yes, I know that the specific change of language is only about
certificate order, and does not remove the requirement to send all
the requisite certs.  And in fact Postfix does tolerate out-of-order
intermediates (because OpenSSL does).  So my example is but an
analogy, and does not *specifically* warrant retaining the order
constraint, but other clients may be even more constrained, and
that's OK for those particular applications.

Nothing in the current server requirement precludes any client
accomodations for servers that don't send the requisite chain.
When those are appropriate, they can be employed.

-- 
	Viktor.

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 12 May 2015 15:22:54 +0300

On Fri, May 08, 2015 at 07:35:13PM -0400, Dave Garrett wrote:
> to a "SHOULD", and the term "trust anchor" replacing references to
> root keys/CA.
> 

What's the point of this change? To allow server to send multiple
certificate chains (all starting from one EE cert) at once?

If it is that, why isn't topological ordering still required to
be correct (certificates signed preceed certificates signing)?

And does this proposed change allow server to send incomplete
certificate chains? According to SSL Pulse (May 2015), about
5% of servers have incomplete chains, and it is marked as a
problem.

Also, regarding "chain-building", one has to be careful with its
interaction with HPKP, which is currently as-deployed the _only_
mechanism that can be usefully used to harden a site against
misissued certificates (too bad it is qute a footgun, especially
due to chain rewriting).

-Ilari

---

From: [email-masked] (Martin Rex)
Date: Tue, 12 May 2015 18:05:56 +0200 (CEST)

> sympathetic (e.g. https://access.redhat.com/articles/1413643 for the most

New 2048-bit rootCA certs have been around for several years, and the
transition from old 1024-bit rootCAs to 2048-bit rootCAs is a non-problem.

It definitely is *NOT* a problem for the existing requirements on
the Certificate handshake message.

Let's use the Web-Server above (https://access.redhat.com/) as example.

  (1) Server cert: (2048-bit RSA)
         Subject:  CN=access.redhat.com
         Issuer:   CN=VeriSign Akamai SureServer CA G14-SHA1
  (2) ServerCA cert: (2048-bit RSA)
         Subject:  CN=VeriSign Akamai SureServer CA G14-SHA1
         Issuer:   CN=Baltimore CyberTrust Root
  (3) CrossCA cert (2048-bit RSA, sha1WithRsaEncryption signature):
         Subject:  CN=Baltimore CyberTrust Root
         Issuer:   CN=GTE CyberTrust Global Root

  (4) (old) RootCA cert (1024-bit RSA)
         Subject:  CN=GTE CyberTrust Global Root
         Issuer:   CN=GTE CyberTrust Global Root

The server https://access.redhat.com sends (1)+(2)+(3) from the above
certificate path (and in the correct order), and is omitting only the
self-signed 1024-bit RootCA cert (4) from the end of the path in the
TLS Certificate Handshake message.

FireFox 34.0.5 is a little weird, because it verifies the above path
against the old 1024-bit RootCA, although it is in possession of the
new Baltimore 2048-bit rootCA Certificate:

  (3b) new Baltimore RootCA cert (2048-bit RSA)
         Subject:  CN=Baltimore CyberTrust Root
         Issuer:  CN=Baltimore CyberTrust Root

FireFox 37.0.1 seems to have been improved and properly recognize
that it only needs (1)+(2) from the Servers certificate handshake
message to verify the servers certificate against its (3b) trust anchor.

Would the server at https://access.redhat.com/ only be sending (1)+(2)
in the Certificate Handshake message, then old clients, which have the
old 1024-bit GTE CyberTrust Gobal Root in their trust anchor store
but not the new self-signed 2048-bit Baltimore Cybertrust Root, would
not be able to verify the server certificate (because of the lack
of that CrossCA cert).

New clients, which have the new self-signed 2048-bit Baltimore CyberTrust
Root should be able to recognize that they only need (1)+(2) from the
Server Certificate to chain against a known trust anchor.  FireFox 34.0.5
seems a little dense.  Given the above path, when the client has the
new 2048-bit Baltimore rootCA cert, then the client should always perform
the verification against the new RootCA when the server Certificate
handshake message starts with certificate (1)+(2), and independent of
whether the certificate handshake message futher includes (3) or (3)+(4)
or (3b).

Valid alternative forward paths for the server are:

  (a)  (1)+(2)
  (b)  (1)+(2)+(3)
  (c)  (1)+(2)+(3)+(4)
  (d)  (1)+(2)+(3b)

and (b) or (c) will have a higher likelyhood of interop that (a) or (d)
because of the legacy clients in the installed base.

Now in case that a client has (2) configured as a trust anchor,
it would only be using (1) from the Server certificate handshake message
(and ignore the other included certificates), and in case that a client
as (1) configured as a trust anchor, it would also only be looking
at certificate (1), determining that it has this certificate configured
as trust anchor, and perform all checks based on that local trust anchor.

-Martin

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 12 May 2015 16:24:05 +0000

> certificates, because the PKI is hard and the Internet involves multiple

FWIW, OpenSSL does no path discovery.  Whatever certificates the
server sends, there is a linearly ordered subset[1] of these that
achieves the same validation result as the original heap of certs
for all OpenSSL-based clients.  So sending something other than a
linearly ordered list has no value.

With OpenSSL 1.0.2 recently tweaked to do trust-store look-asides
for nodes in the middle of the provided linear list, as also noted
by Martin cross-signing with SHA256 or RSA 2048, ... works just
fine.

-- 
	Viktor.

[1] If we assign ordinals to the heap elements in order of appearance,
the equivalent linear list is the chain (successor == issuer of
predecessor) that sorts lexically first.  That is for each chain
element, the issuer with the smallest heap ordinal is chosen and
no other issuers from the heap are considered.

This is augmented with look-aside at each chain layer into the
trust store, to identify a locally trusted alternative issuer.

---

From: Watson Ladd <[email-masked]>
Date: Tue, 12 May 2015 09:31:15 -0700

> by Martin cross-signing with SHA256 or RSA 2048, ... works just

So the actual requirement is for a linearization of the DAG of certs,
and this will produce the same results as considering all chains in
the DAG? Or am I misunderstanding you?

Sincerely,
Watson Ladd

> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls

-- 
"Man is born free, but everywhere he is in chains".
--Rousseau.

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 12 May 2015 16:44:46 +0000

> the DAG? Or am I misunderstanding you?

OpenSSL only constructs one candidate path through the supplied
heap (with lookaside into a trust store at each link).  Therefore,
with respect to OpenSSL clients, supplying anything that is not
the lexically first chain is a waste of bandwidth.

If that lexically first linear chain does not work (possibly
truncated if an intermediate element is signed by a local trust
anchor), validation fails.

-- 
	Viktor.

---

From: "Ryan Sleevi" <[email-masked]>
Date: Tue, 12 May 2015 10:07:50 -0700

>  the Certificate handshake message.

I'm sorry, but I'm going to have to disagree with you again, because while
this is the way the world looks on paper, it's not the way the world works
in practice.

Your example (access.redhat.com) is one that we had to work around on OS X
due to quirks of the client not supporting trust anchors (e.g.
https://code.google.com/p/chromium/issues/detail?id=236112 ).

That is, if you supply your (b) or (c), for example, you'll break any
application using Security.framework for certificate verification
(including all those that use CFURL & friends). It seems that Safari has
special logic not present in the OS to work around that issue.

So servers supply (a) or (d).

Now you can rail about clients all you want, and I might be inclined to
agree with you, but it highlights that the world is not as pretty as you
want.

Now, with respect to cross-certificates, it's a regular occurrence to see
situations like

(1) Leaf-signed-by-(2)
(2) Intermediate-signed-by-(3)
(3a) Intermediate-signed-by-(4)
(3b) Intermediate-signed-by-(5)
(4) Root 1 [an existing root]
(5) Root 2 [a new root applying for inclusion]

In this case, valid paths are
[for old clients]
(1)+(2)+(3a)+(4)

[for new clients]
(1)+(2)+(3b)+(5)

As much as you might wish to yell that they shouldn't have done (3a)/(3b)
like that, and instead (5) should have been certified by (4), there are a
large number of reasons why that doesn't happen, and the above is actually
quite common (I know of at least three different organizations who will
have to go through the above flow right now to avoid issues)

The maximum compatibility with clients is
(1)+(2)+(3a)+(4)+(3b)+(5)

As gross and as ugly as that may seem, when working with various clients'
libraries, that config is what gets the most working. That's why I support
the language relaxation, in its simplest form.

Now, if clients did AIA fetching ubiquitously, you might supply
(1)+(2)+(3a) and have (2)'s AIA point to (3b). Clients that trust (4) will
work "out of the box", while clients that trust (5) will do an AIA chase
off (2) to get (3b). Over time, you might determine that more clients have
(5) than (4), so then you invert it to (1)+(2')+(3b), where (2') is a
version of 2 whose AIA points to (3a).

This is the PKI we live in. It's not going away - it's here and has been
for the past decades.

If you fully control your client, server, and PKI, the language is
entirely reasonable. But few people do.

---

From: [email-masked] (Martin Rex)
Date: Tue, 12 May 2015 20:14:30 +0200 (CEST)

> special logic not present in the OS to work around that issue.

If that particular client really goofs the certificate path validation
in presence of an unneeded crossCA (plus optional untrusted rootCA)
while succeeding certificate path validation when neither of these
additional certificates is present, then it is obviously defective
and ought to be fixed-- and this silly bug should have been found
and fixed a few years ago -- because several public CAs have been doing
this for years.

The straight forward workaround for such a defective client would have
been to import the crossCA certificate incarnation of the new Baltimore
RootCA certificate as additional trust anchor (since *ANY* certificate
can be declared a trust anchor, it does not need to be self-signed).

> the language relaxation, in its simplest form.

Myself, I have not seen any such Certificate handshake messages so far
(like in a customer support incident).  Do you have an example (URL)
of a server with such a configuration?  (I actually do not know how
the TLS client, for which I'm still doing maintenance, will handle this).

At least the above would be properly ordered up to and including
a self-signed rootCA certificate.  So clients that trust the old root
might be able to see the match to a known trust anchor and simply ignore
the trailing junk.

But for simple clients that trust only the newer root, this certificate
path is probably invalid (and in violation of the current requirements),
and there is no easy approach to make such a certificate_list comprehensible
to such a client (that only trusts the new root).

> version of 2 whose AIA points to (3a).

"AIA chasing" is obviously insecure and a huge security problem.
If you look at how the certificate path validation is designed in
PKIX (rfc5280) section 6.1+, it works from the trust anchor down
towards the end-entity certificate, and the algorithm will not use
any information from an untrusted certificate _unless_ an already trusted
certificate can be used to verify the signature on the still untrusted
certificate, and that verification has succeeded.

AIA chasing walks the chain in the opposite direction, and would use
information (a download-URL) from a still untrusted certificate to
perform a download.  As I previously said, this might look normal
for a browser, because browser do stupid things all the time,
but it would be definitely irresponsible for a PKIX implementation
to do something like this, and perform a download of an object with
an URL from an untrusted certificate provided by the communication peer.

The other problem here is inefficiency.  In a number of environments
performing network data exchanges during certificate path validation
is a complete non-starter.

To at least fix the security side of the problem, the publication of
additional certs would have to work strictly top down.  I.e. the trust anchor
cert will have to indicate the location where to obtain additional crossCA
certificates, and the serverCA cert would need to have additional
attributes added, enumerating the rootCAs under which the server certificate
will be alternatively verifiable.

-Martin

---

From: [email-masked] (Martin Rex)
Date: Tue, 12 May 2015 21:04:52 +0200 (CEST)

Now there would be a different *CORRECT* approach to the below problem,
rather than the dirty hack you described, and that would be fully
backwards compatible:

> (1)+(2)+(3a)+(4)+(3b)+(5)

The *CORRECT* approach would be to issue a cross-CA certificate
for (5) Root 2 signed by (4) Root 1, lets call it (5b), and
the correct path to send for servers would be

(1)+(2)+(3b)+(5b)+(4)

that would be fully conforming with the existing requirements,
fully comprehensible for clients that feed the certificate_list
from the Certificate handshake message into "Basic Path Validation".

-Martin

---

From: "Ryan Sleevi" <[email-masked]>
Date: Tue, 12 May 2015 12:07:16 -0700

>  from the Certificate handshake message into "Basic Path Validation".

How did I know you would suggest this?

Ah, because I addressed it in the original email, when I said

"As much as you might wish to yell that they shouldn't have done (3a)/(3b)
like that, and instead (5) should have been certified by (4), there are a
large number of reasons why that doesn't happen, and the above is actually
quite common (I know of at least three different organizations who will
have to go through the above flow right now to avoid issues)"

So, moving on :)

---

From: [email-masked] (Martin Rex)
Date: Tue, 12 May 2015 21:14:45 +0200 (CEST)

> have to go through the above flow right now to avoid issues)"

there is no "shouldn't have done (3a)/(3b)".  They can simple create
that crossCA cert *NOW* and start distributing it, and things will
work just fine.

The problem with your original proposal is, that no *CORRECT* TLS
implementation will allow you to send such junk in a TLS Certificate
handshake message, so this will simply not be an option.

Right now, we still have a stick to make that CA create such a crossCA
cert.

But you're not only asking to make client tolerate junk, you seem to
be asking to actively break correct TLS implementations to start sending
junk and create new interoperability problems.

-Martin

---

From: "Ryan Sleevi" <[email-masked]>
Date: Tue, 12 May 2015 12:22:38 -0700

>  that crossCA cert *NOW* and start distributing it, and things will

Again, that's an assumption about the possibilities of change that are not
bourne out by the realities of the PKI.

In a variety of circumstances, there are actively reasons why creating a
crossCA is *not* valid or will *not* resolve issues for clients. In
particular, issuing such a cross-CA may violate appropriate WebTrust or
ETSI guidelines that CAs are required to adhere to, or violate the CA's
own CP/CPS. I'm sure that's an invitation for more decrying, but it's the
reality we live in, and compared to the alternatives, it's actually a
*good* thing, even if it makes spec purity hard.

>  handshake message, so this will simply not be an option.

Not an option conforming with the TLS spec, I fully acknowledge.

But it's an area where a number of clients never implemented (again,
layering - it's more work and more attack surface to actually enforce that
requirement), and it's an area where a number of servers actively ignore
because "working for clients" is more important than theoretical spec
purity.

Which is sort of why we're here discussing this in the first place -
whether or not we want to acknowledge reality or not.

But I don't think we're making productive progress here, so it's probably
worth me bowing out. It does seem like we're at a fundamental and
ideological impasse with regards to path discovery, and while I can try to
convince you that RFC 4158-like behaviours are good for all clients, I
suspect neither this list nor this issue is the right medium for it.

I leave it up to the editors & chairs to evaluate consensus, the relevant
arguments for and against, and the behaviours of clients (both for and
against).

---

From: [email-masked] (Martin Rex)
Date: Wed, 13 May 2015 02:16:36 +0200 (CEST)

> special logic not present in the OS to work around that issue.

When looking at the above bug report, I agree with the particular
statement of [email-masked]'s bug report:

  This allows the 'proper' chain to be built (eg: through AIA fetching).
  However, such behaviour is not desirable for both performance reasons
  and because it may mask real configuration errors (eg: Issue 122500 ) 

Has adding the crossCA variant of the new Baltimore rootCA cert ever
been considered/tried?  I mean dropping the old 1024-bit rootCA should
become irrelevant when the the 2048-bit crossCA variant is added
as trust anchor.  There is no practical difference between the
self-signed variant and the crossCA variant of that trust-anchor.
And if the PKIX implemenation is a little dense and does not recognize
the equivalence all by itself, then configuring both variants as
trust anchors should do the necessary magic.

Did the supplier of the relevant PKIX implementation fix this implementation
shortcoming?

-Martin

---

From: "Ryan Sleevi" <[email-masked]>
Date: Tue, 12 May 2015 17:21:06 -0700

On Tue, May 12, 2015 5:16 pm, Martin Rex wrote:
>  been considered/tried?

Yes. It doesn't work.

> I mean dropping the old 1024-bit rootCA should

In a world different than our own, anything is possible.
In the world we have, we deal with an imperfect reality and make it work.

>  Did the supplier of the relevant PKIX implementation fix this

Nope.

Considering it is not required of an RFC 5280-compliant implementation to
support that, any claims that the relevant supplier should would largely
be inconsistent with your past messages. Even if you'd be correct in
arguing they should.