Subject: [Emu] My review ... was RE: I-D Action: draft-ietf-emu-eaptlscert-02.txt

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 24 Mar 2020 08:00:49 +0000

Hi Michael, Hi draft authors,

> I was surprised to get to the end of the document without any suggestions about sending certificates by reference rather than value.

Having seen this statement from Michael I have reviewed the document. Two generic observations about the draft:

1) Many statements are made about deployments and no references are provided. To improve quality of the write-up I would strongly suggest to add such references, as you would have to do with an academic publication as well.

2) A few techniques are missing (Certificate URLs and cTLS) and TLS Cached Info was wrongly interpreted.  They actually relate to the remark from Michael.

> TLS certificates are often relatively large, and the certificate
>   chains are often long.

I think this statement is in general incorrect. You could say that in deployment X certificates have size X and the chain is Y long.

> Also,
>  from deployment experience, EAP peers typically have longer
>   certificate chains than servers.

I would like a reference to be included here. Theoretically, it makes no sense to
have a certificate chain for an EAP peer to have a longer certificate chain than a server
given a EAP peer talks to one EAP server.

In network access authentication it is not only about authentication but the most important part is authorization. Hence, you pretty much always have a one-to-one relationship between the EAP peer and the EAP server. There are no good reasons to have complex CA hierarchies here.

> This is because EAP peers often
>  follow organizational hierarchies and tend to have many intermediate
>   certificates.  Thus, EAP-TLS authentication usually involve
>   significantly more octets than when TLS is used as part of HTTPS.

I think it would make sense to explain this organizational hierarchy aspect in a bit
more detail.

> The EAP fragment size
>  in typical deployments is just 1020 - 1500 octets.

Reference for the 1500 octets.

> For example, many EAP authenticator (access point)
>  implementations will drop an EAP session if it has not finished after
>   40 - 50 round-trips.

Is there a reference that could be included?

> However, deployment
>   experience has shown that these jumbo frames are not always
>  implemented correctly.

Add a reference here.

> RADIUS can generally transport only about 4000
>  octets of EAP in a single message.

How was this number constructed?

>   A certificate chain (called a certification path in [RFC5280]) can
>   have 2 - 6 intermediate certificates between the end-entity
>   certificate and the trust anchor [RFC5280].

The PKIX reference here is misleading. I think you included it to refer to the terms but it sounds like the statement that
the chain can have 2-6 intermediate CA certificates.

I would add the terms to the terminology section and remove the PKIX reference here.

I am also wondering what you are trying to say here with this sentence. Are you saying that you have seen deployments
for network access authentication that have up to 6 intermediate CA certificates in the chain?

> 3.  Experience with Deployments

>   Most common access point implementations drop EAP sessions that do
>   not complete within 50 round-trips.

This sentence is a repetition.

> This means that if the chain is
>  larger than ~ 60 kB, EAP-TLS authentication cannot complete
>  successfully in most deployments.

Regarding the 60 kB certificate chain: Should this be an indication to someone deploying
the technology for access network authentication that something has gone wrong?

Is this someone you have observed or is this just a theoretical statement? I hope it is the latter.

4.2.1.  Pre-distributing and Omitting CA Certificates

> When using TLS 1.3, all certificates that
> specify a trust anchor known by the other endpoint may be omitted
> (see Section 4.4.2 of [RFC8446]).  When using TLS 1.2 or earlier,
> only the self-signed certificate that specifies the root certificate
> authority may be omitted (see Section 7.4.2 of [RFC5246]

These sentences sound strange.

In TLS (and probably other protocols) it makes no sense to distribute the
trust anchors in the protocol itself (because then they wouldn't serve as an
anchor for trust).

It is my understanding that the TLS 1.3 spec does not require you
to send intermediate CA certificates if they have been provisioned to the peer
out-of-band already. The text in the TLS 1.2 spec is a bit fuzzy and calls out that the
self-signed root certificate MAY be omitted but it does not say anything about omitting
the other certs.

>4.2.2.  Caching Certificates

There seems to be the misconception that TLS Cached Info requires a full exchange to work.
It is the other way around:  If you pre-distribute certificates upfront then there is no need
to exchange the certs again. You could just send the fingerprints right away.

A spec, however, has to also cover the bootstrapping problem.

> 4.2.5.  Using Fewer Intermediate Certificates

>   The EAP peer certificate chain does not have to mirror the
>   organizational hierarchy.  For successful EAP-TLS authentication,
>   certificate chains should not contain more than 2-4 intermediate
>   certificates.

I would make a stronger statement here. There is absolutely no reason for the certificate
chain to mirror the organizational structure.

I also don't understand why you would need a hierarchy of 4 intermediate CAs.

Finally, at the end: Why did you omit
- Client Certificate URLs - RFC 6066), which is also referenced in RFC 7925 and
- cTLS (which also has a certificate compression scheme included).

Theoretically, you could even list the ability to use alternative certificate format here. Then, you could list raw public keys or the CWT extension.

Ciao
Hannes

PS: It is good to see John's name on a document that talks about TLS.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 24 Mar 2020 10:01:54 +0100

Good morning Hannes

> In network access authentication it is not only about authentication but the most important part is authorization. Hence, you pretty much always have a one-to-one relationship between the EAP peer…

Not sure I entirely understand you.  A few places are promoting new roots for manufacturers.  And so the initial chain for a peer might look like CA-Root->CA-Intermediate->Mfg  Root->Mfg Intermediate->Device.  I think it may be possible to remove one of the intermediates, but probably not both.  It seems to me that this is an onboarding problem, and the best way to reduce the cert chain size on a day to day basis would be to swap out for an operational cert where the trust anchor is within the enterprise.  That means you get one of those Big Fat transactions and then things settle down, and that transaction may be handled specially anyway.

One comment I have in the draft relates to this text:

   o  Extensions are necessary to comply with [RFC5280], but the vast
      majority are optional.  Include only those that are necessary to
      operate.

This statement is just a little too general.  It very much depends WHICH certificate we are discussing.  If it is a manufacturer certificate, as long as you can roll to an enterprise cert, then who cares?  If it is an enterprise certificate, then we have to look more closely.

So for instance, as Hannes mentions, authorization is a big issue.  Some of that can be done outside of the certificate using ACE or similar, but some may need to be present in the cert.  What we would rather not have is people working the SAN to introduce authorization attributes.

Eliot

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 24 Mar 2020 09:08:23 +0000

Hi Eliot

You bring up a good point, namely the deployment environment. Are we are talking about an IoT, an enterprise deployment environment or something else? Clearly there will be differences. Reading through the text my impression was that this is about an enterprise (or university) deployment environment. I might, however, be on the wrong track here.

Having the references to where deployment problems with large certificates/certificate chains occur would shine light on this aspect.

Ciao
Hannes

From: Eliot Lear <[email-masked]>
Sent: Tuesday, March 24, 2020 10:02 AM
To: Hannes Tschofenig <[email-masked]>
Cc: Michael Richardson <[email-masked]>; [email-masked]
Subject: Re: [Emu] My review ... was RE: I-D Action: draft-ietf-emu-eaptlscert-02.txt

Good morning Hannes

Also,

Eliot

---

From: Eliot Lear <[email-masked]>
Date: Tue, 24 Mar 2020 10:24:11 +0100

Hi Hannes

> On 24 Mar 2020, at 10:08, Hannes Tschofenig <[email-masked]> wrote:
> 
> Hi Eliot
>  

Since we’re talking about EAP, I can think of a few cases:

Classic enterprise/university case, IoT or not.  I was ASSUMING IoT because my brain goes to IOT, but I don’t think it has to be so.
There is also a wifi roaming device model, where EAP might be used in various service provider or federated environments a’la Eduroam or similar.  Today this is NOT a big IoT space, but soon?

The one place this is not a big deal at the moment is in the home, as WPA-Personal/PAE is the norm.

One additional question is how big the impact is between wired and wireless.  Obviously with the former you worry less about interference and drops.

Eliot

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 24 Mar 2020 09:30:00 +0000

Hi Eliot,

I consider the enterprise and the university case as a roaming model. From an EAP method point of view there is IMHO little difference between the roaming and the non-roaming case: the EAP exchange always runs between the EAP peer on the device and the EAP server.

The IoT case is different because it falls more in the category of  home WiFi usage. This doesn’t really use EAP in my understanding.

Ciao
Hannes

Sent: Tuesday, March 24, 2020 10:24 AM

Hi Hannes

On 24 Mar 2020, at 10:08, Hannes Tschofenig <[email-masked]<mailto:[email-masked]>> wrote:

Hi Eliot

  *   Classic enterprise/university case, IoT or not.  I was ASSUMING IoT because my brain goes to IOT, but I don’t think it has to be so.
  *   There is also a wifi roaming device model, where EAP might be used in various service provider or federated environments a’la Eduroam or similar.  Today this is NOT a big IoT space, but soon?

Eliot

Ciao
Hannes

From: Eliot Lear <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, March 24, 2020 10:02 AM
To: Hannes Tschofenig <[email-masked]<mailto:[email-masked]>>
Cc: Michael Richardson <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Emu] My review ... was RE: I-D Action: draft-ietf-emu-eaptlscert-02.txt

Good morning Hannes

Also,

Eliot

---

From: Eliot Lear <[email-masked]>
Date: Tue, 24 Mar 2020 10:42:51 +0100

> On 24 Mar 2020, at 10:30, Hannes Tschofenig <[email-masked]> wrote:
> 
> Hi Eliot, 
>  

For wifi?  The number of IoT devices using wifi is minuscule compared to what I believe we will see over time.  And so it is hard to judge.  The onboarding mechanisms need to mature a bit more.

Eliot

---

From: Alan DeKok <[email-masked]>
Date: Tue, 24 Mar 2020 10:08:06 -0400

> 1) Many statements are made about deployments and no references are provided. To improve quality of the write-up I would strongly suggest to add such references, as you would have to do with an aca…

  The issue is that deployment issues are usually discussed privately.  One public link is this:

http://freeradius.1045715.n5.nabble.com/Free-Radius-problem-with-sending-large-certificate-chains-using-EAP-TLS-td2780319.html

> I think this statement is in general incorrect. You could say that in deployment X certificates have size X and the chain is Y long.

  Maybe "can be large" and "can be long".  It's difficult to get quantitative numbers here.  I didn't instrument my software to send statistics back to a central collector. :(

> given a EAP peer talks to one EAP server.

  The peer often has a client certificate.  So it's chain can be one longer than the server in some cases.

> In network access authentication it is not only about authentication but the most important part is authorization. Hence, you pretty much always have a one-to-one relationship between the EAP peer…

  Please tell that to corporations.  :(

  I can't count how many times I've had to tell people "the technology doesn't support that", when they explain their business requirements.  It is often difficult to get people to understand that their preferred process / methods are simply impossible to implement in the real world.

> I think it would make sense to explain this organizational hierarchy aspect in a bit

  I'm not sure what else could be said here.

> Reference for the 1500 octets.

  Ethernet maximum packet size...

> Is there a reference that could be included?

  References to hostap source code.

>>  experience has shown that these jumbo frames are not always

  Private communication.  :(

  i.e. I've mediated calls between multiple large companies all pointing the finger at each other when things don't work.  The solution was to point out that one particular networking box was simply dropping jumbo EAPoL frames.  Names will not be used here.

> How was this number constructed?

  4K RADIUS maximum packet size.  Minus 20 bytes for the RADIUS header overhead, 18 for Message-Authenticator, and <hand waving> for whatever other random things the AP vendor includes in packets.

> for network access authentication that have up to 6 intermediate CA certificates in the chain?

  Yes.

> the technology for access network authentication that something has gone wrong?

  An indication that *what* has gone wrong?  Nothing in any of the current specs indicates that a 60KB certificate chain would be a problem.  I suspect few AP / OS vendors discuss that in their documentation, too.

> Is this someone you have observed or is this just a theoretical statement? I hope it is the latter.

  It's observed.

> A spec, however, has to also cover the bootstrapping problem.

  Sure.  This should be explained.

> chain to mirror the organizational structure.

  Tell that to the people who love hierarchies.

> I also don't understand why you would need a hierarchy of 4 intermediate CAs.

  "My boss says it's our corporate process.  Why don't the specs support it?"

  Alan DeKok.

---

From: Hannes Tschofenig <[email-masked]>
Date: Wed, 25 Mar 2020 07:30:37 +0000

Hi Alan

Thanks a lot for your comments. I guess you understand that I am always a bit nervous when the results of non-public conversations dictate the problem space. I have seen it often enough that people have made their measurements wrong, had wrong configuration, or had simply misunderstood concepts.

It sounds like we need a "myth-busting" document. Of course, it isn't certain whether the decision makers will indeed read RFCs but it would be worthwhile a try.

Also it appears that the authors could do something really actionable here, namely to update the hostap code to update the roundtrip limit.

Ciao
Hannes

PS: Why aren't you a co-author on this document? You know more about this than anyone else.

-----Original Message-----
From: Alan DeKok <[email-masked]>
Sent: Tuesday, March 24, 2020 3:08 PM
To: Hannes Tschofenig <[email-masked]>
Cc: Michael Richardson <[email-masked]>; [email-masked]
Subject: Re: [Emu] My review ... was RE: I-D Action: draft-ietf-emu-eaptlscert-02.txt

> I would like a reference to be included here. Theoretically, it makes
> no sense to have a certificate chain for an EAP peer to have a longer
> certificate chain than a server given a EAP peer talks to one EAP server.

> I think it would make sense to explain this organizational hierarchy
> aspect in a bit more detail.

>> For example, many EAP authenticator (access point) implementations
>> will drop an EAP session if it has not finished after
>>  40 - 50 round-trips.
>

>> RADIUS can generally transport only about 4000 octets of EAP in a
>> single message.
>

> I am also wondering what you are trying to say here with this
> sentence. Are you saying that you have seen deployments for network access authentication that have up to 6 intermediate CA certificates in the chain?

  Yes.

> Regarding the 60 kB certificate chain: Should this be an indication to
> someone deploying the technology for access network authentication that something has gone wrong?

  It's observed.

> It is the other way around:  If you pre-distribute certificates
> upfront then there is no need to exchange the certs again. You could just send the fingerprints right away.
>

> I would make a stronger statement here. There is absolutely no reason
> for the certificate chain to mirror the organizational structure.

  Alan DeKok.

---

From: Alan DeKok <[email-masked]>
Date: Wed, 25 Mar 2020 08:45:51 -0400

> Thanks a lot for your comments. I guess you understand that I am always a bit nervous when the results of non-public conversations dictate the problem space. I have seen it often enough that people…

  Sure.

  My $0.02 here is that even in the absence of quantitative evidence, we know that the recommendations in the document aren't wrong.

  i.e. there is little need to have a certificate chain 6 layers deep.  There is little need to have each certificate be 16K in size.

  We may not be *exactly* sure why those things happen.  But we can make recommendations for what *should* happen.  And, explain why certain (guessed) practices are likely to be wrong.

> It sounds like we need a "myth-busting" document. Of course, it isn't certain whether the decision makers will indeed read RFCs but it would be worthwhile a try.

  I think this is it, for the most part.

> Also it appears that the authors could do something really actionable here, namely to update the hostap code to update the roundtrip limit.

  Hostap supports 50 round trips for TLS ACKs, and 100 if it's exchanging data.  This seems reasonable.

> PS: Why aren't you a co-author on this document? You know more about this than anyone else.

  I'm one of the few willing to *talk* about it.  Most everyone else who has this data its buried 6 levels deep in a large organization.

  Alan DeKok.

---

From: Jouni Malinen <[email-masked]>
Date: Sat, 28 Mar 2020 10:25:36 +0200

>   References to hostap source code.

hostapd has a limit in the EAP server role on the number of round trips (and
wpa_supplicant in the EAP peer role). However, there is no such limit in
the EAP authenticator role, i.e., hostapd as the access point forwarding
EAP to an external RADIUS authentication server does not place such a
constraint on the exchange.

-- 
Jouni Malinen                                            PGP id EFC895FA

---

From: Hannes Tschofenig <[email-masked]>
Date: Sat, 28 Mar 2020 16:22:55 +0000

Thanks, Jouni. That's a good clarification.

-----Original Message-----
From: Jouni Malinen <[email-masked]>
Sent: Saturday, March 28, 2020 9:26 AM
To: Alan DeKok <[email-masked]>
Cc: Hannes Tschofenig <[email-masked]>; [email-masked]
Subject: Re: [Emu] My review ... was RE: I-D Action: draft-ietf-emu-eaptlscert-02.txt

>   References to hostap source code.

hostapd has a limit in the EAP server role on the number of round trips (and wpa_supplicant in the EAP peer role). However, there is no such limit in the EAP authenticator role, i.e., hostapd as the access point forwarding EAP to an external RADIUS authentication server does not place such a constraint on the exchange.