Subject: [Acme] dns-01 challenge limitations

From: Simon Ser <[email-masked]>
Date: Fri, 11 Sep 2020 13:08:33 +0000

Hi all,

I've been working on an ACME client acting as a TLS termination proxy. In order
to retrieve wildcard certificates from the Let's Encrypt ACME servers, support
for the dns-01 challenge is required.

dns-01 requires the ACME client to complete the challenge by updating a DNS
record. This is bothersome because this often requires interacting with the
DNS registry operator. This is typically done via vendor-specific APIs, with
access control handled via vendor-specific means (tokens, public keys, etc).

I understand that it's difficult for ACME clients to prove that they are
authorized to obtain wildcard certificates. However, have other alternatives
been considered?

For instance, it would be possible to require users to add a short public key
in a DNS TXT record, then ask the ACME client to sign challenges with that key.
Something like this would significantly ease the development of ACME clients.

Are there specific reasons why dns-01 requires updating a DNS record?

Thanks,

Simon Ser

(CC mholt, I figured you might be interested in this for Caddy too)

---

From: Philipp Junghannß <[email-masked]>
Date: Fri, 11 Sep 2020 15:13:55 +0200

I have asked that question in the LE forum iirc the problem is that someone
could place that record once and as long as someone doesnt look at it all
the time one can easily miss the fact that someone can create wildcards and
stuff for that domain, so the point is to prove that dns access is given at
the time of issuance.
you could maybe use a different DNS Server which has a better API, and
potentially even can be used by ACME.

Regards

Am Fr., 11. Sept. 2020 um 15:09 Uhr schrieb Simon Ser <[email-masked]>:

> _______________________________________________
> Acme mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/acme
>

---

From: Felipe Gasper <[email-masked]>
Date: Fri, 11 Sep 2020 09:17:08 -0400

> On Sep 11, 2020, at 9:08 AM, Simon Ser <[email-masked]> wrote:
> 

This would seem to introduce a new vector--key compromise--for being able to impersonate the domain, wouldn’t it?

Such an authz method would be proving not access to the domain itself, but access to the key, and would be vulnerable to local misconfigurations. It seems thus not dissimilar to the erstwhile problem with tls-sni-01/02.

-F

---

From: Simon Ser <[email-masked]>
Date: Fri, 11 Sep 2020 13:21:17 +0000

Hi,

On Friday, September 11, 2020 3:13 PM, Philipp Junghannß <[email-masked]> wrote:

> I have asked that question in the LE forum iirc the problem is that
> someone could place that record once and as long as someone doesnt
> look at it all the time one can easily miss the fact that someone can
> create wildcards and stuff for that domain, so the point is to prove
> that dns access is given at the time of issuance.

If someone has once write access to the DNS, they can set an
acme-challenge record, redirect all requests, and issue wildcard certs.
That would be easy to miss, too.

> you could maybe use a different DNS Server which has a better API,
> and potentially even can be used by ACME.

The issue at hand isn't that a particular DNS registry operator isn't
supported by a particular ACME client. What I want to fix is the need
for all ACME clients to support all DNS registry operators.

Thanks,

Simon

---

From: Simon Ser <[email-masked]>
Date: Fri, 11 Sep 2020 13:25:02 +0000

Hi,

On Friday, September 11, 2020 3:17 PM, Felipe Gasper <[email-masked]> wrote:

> > On Sep 11, 2020, at 9:08 AM, Simon Ser [email-masked] wrote:
> This would seem to introduce a new vector--key compromise--for being
> able to impersonate the domain, wouldn’t it?
>
> Such an authz method would be proving not access to the domain
> itself, but access to the key, and would be vulnerable to local
> misconfigurations. It seems thus not dissimilar to the erstwhile
> problem with tls-sni-01/02.

Right now ACME clients need vendor-specific authorizations, like API
tokens. If the DNS registry operator's token is leaked, much worse
things can happen than just being able to issue wildcard certificates
(since the token provides write access to DNS records).

Thanks,

Simon

---

From: Philipp Junghannß <[email-masked]>
Date: Fri, 11 Sep 2020 15:27:35 +0200

well Certificate transparency is one something should maybe keep
notifications for.

Also I can understand the problem, but I have not decided the outcome, I
merely stated what I got as an answer back then.

problem is obviously also the CA/Browser Forum has certain requirements,
and I guess having access to some kind of direct verification at the time
of issue might be probably one of these.

---

From: Patrik Wallström <[email-masked]>
Date: Fri, 11 Sep 2020 15:41:08 +0200

> (since the token provides write access to DNS records).

The missing piece of this puzzle is a standardized API for registrars
(or DNS operators), where changes can be made for a zone at a registrar.
Much like registry changes coming from registrars to a registry using
EPP. Many attempts has been made for this, but for some reason,
registrars like their lock-in models.

Perhaps some day there will be an attempt at both creating a really good
open source zone editor that will be adopted by registrars and other DNS
opreators, that also implements an API that is generally accepted. Then
perhaps this API could become a standard for interacting at least with
DNS operators for changing the content of a zone. (No, and I don't think
RFC 2136 is good enough for this.)

For now, this is for many ACME clients a manual step. If you run your
authoritative DNS service locally in your network, perhaps you could
look into any options for automatically update the zone content.

---

From: Ryan Sleevi <[email-masked]>
Date: Fri, 11 Sep 2020 10:26:03 -0400

> of issue might be probably one of these.
This is the correct answer.

While the IETF can certainly explore developing voluntary standards for
other methods, the ability for CAs to adopt these methods is limited by CAs
customers: the browsers and operating systems that include and use CAs to
validate domain names on their behalf.

The explicit goal by several browser/OS vendors is to obtain a fresh proof
of control over a domain, and reduce/eliminate any caching or reuse.
Delegation (by keys or persistent records) is definitely an area of
expressed concern.

I think the suggest of more uniform APIs for managing DNS is very much in
line with those goals, and would help far more than ACME.

---

From: Ilari Liusvaara <[email-masked]>
Date: Fri, 11 Sep 2020 19:07:22 +0300

> RFC 2136 is good enough for this.)

One another problem is that even if one has programmatic API, the
DNS service has many servers (due to being intended for high-reliability
slow-update serving, not low-reliability fast-update serving), with
potentially painfully slow propagation times.

> look into any options for automatically update the zone content.

I think the current best way is to have _acme-challenge be a CNAME
pointing to zone served by single master with no slaves that accepts
DNS UPDATE with TSIG HMAC-SHA256 authentication for ACME client to
update the records.

The single master is more than reliable enough for the purpose (as
there should be donzens of retries spread over time for renewal before
the certificate expires) and eliminates the propagation times.

-Ilari

---

From: Michael Richardson <[email-masked]>
Date: Fri, 11 Sep 2020 13:06:41 -0400

> access control handled via vendor-specific means (tokens, public keys,

I guess if you've hosted your zone with the registrar, then that might be
true.  my opinion: Don't do that.

Host your own zone, and/or use Dynamic DNS update (RFC3007), which is mature technology.
There are some annoyances with TSIG until you realize that the key name
really matters.

    > Something like this would significantly ease the development of ACME

So, this would be be a client key challenge.
This would not be dns-01.  It could certainly work, but it would be a new effort.
Maybe we could use SIG(0), I'm not sure.
The question would be whether or not it would get implemented.

    > Are there specific reasons why dns-01 requires updating a DNS record?

Yes, because it proves you control the zone.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: Michael Richardson <[email-masked]>
Date: Fri, 11 Sep 2020 15:45:28 -0400

> DNS UPDATE with TSIG HMAC-SHA256 authentication for ACME client to

That's precisely what I do.
I do it because bind9 does not do well when zones are managed by updates as
well as manual edits.   So I CNAME to a single (sub)zone where it is all
updates.

    > the certificate expires) and eliminates the propagation times.

I do have multiple masters, and I mean to program a query to all NS to see if
the update has propogated, but for now, I "sleep(30)", which is definitely
sub-optimal in the best cases, and a failure if there are problems.
So far, it works great.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Simon Ser <[email-masked]>
Date: Sun, 13 Sep 2020 08:44:20 +0000

On Friday, September 11, 2020 3:41 PM, Patrik Wallström <[email-masked]> wrote:

> > On Friday, September 11, 2020 3:17 PM, Felipe Gasper [email-masked] wrote:
> >

I agree a standardized API for DNS operators would be nice, but it's a
pretty massive task. I don't see this happening anytime soon, no matter
how hard I try.

For this reason, I think a different approach would be desirable.

---

From: Simon Ser <[email-masked]>
Date: Sun, 13 Sep 2020 09:13:14 +0000

> On Friday, September 11, 2020 4:26 PM, Ryan Sleevi <[email-masked]> wrote:
>
> > On Fri, Sep 11, 2020 at 9:28 AM Philipp Junghannß <[email-masked]> wrote:
> >

My take is that in theory it's an understandable goal, but that in practice,
this detoriorates security.

In practice, ACME clients:

1. Have a static, long-term token stored in their configuration file
2. The token is powerful and can update any DNS record in the zone

How come browser/OS vendors are fine with this, but not with a different
approach involving an ACME-specific key?

Sure, since this happens behind the ACME/DNS protocols and is an implementation
detail, this isn't ACME's responsibility anymore. However, because browsers/OS
vendors have this requirement of not allowing delegated proofs, we end up with
a worse situation than necessary.

Ultimately, ACME clients need a way to update DNS records to solve the dns-01
challenge. Ignoring and pushing the problem down to the DNS operators does not
fix the root cause.

If an ACME client needs to prove that they have authority over a DNS zone, they
will need some kind of authorization/key/token or similar, be it
vendor-specific or not. Why not acknowlege this fact and come up with a
reasonable standard?

> > line with those goals, and would help far more than ACME.

Yes, no matter what ACME requires, a standardized API to update DNS records
would be nice. Michael Richardson suggested that such an API (or a subset of)
already exists (via secure DDNS), but isn't supported by most DNS operators
(even if supported by some DNS daemons).

Establishing a new standard involves talking to existing DNS operators, and ask
them to implement the new standard. For them, the new standard wouldn't have a
high enough return on investment: ACME clients already volunteer to implement
each and every proprietary API. Even if a good standard ticking the checkboxes
of RFC 5218 existed, I don't think it would be successful (no "Positive Net
Value" for DNS operators).

---

From: Philipp Junghannß <[email-masked]>
Date: Sun, 13 Sep 2020 11:17:40 +0200

maybe one could make it so an email specific to the domain that is
verified could be used instead to just screw the entire DNS thing? I mean
CAs have used e-Mail based issuance over the address in the whois (no
longer  practical due to increase of whois privacy by default) or the
standardized hostmaster etc addresses. that way any given acme client would
only need access to the inbox of that specific mail which probably is a
whole lot easier than DNS stuff.

---

From: Simon Ser <[email-masked]>
Date: Sun, 13 Sep 2020 09:24:57 +0000

> There are some annoyances with TSIG until you realize that the key name

That sounds like the most reasonable way to solve the dns-01 challenge indeed.
The self-hosted zome can even be limited to just _acme-challenge.

I'm still wondering whether dns-01 is an absolutely necessary evil (see other
replies).

> Maybe we could use SIG(0), I'm not sure.

Yes, this wouldn't be dns-01 or dns-02, it would be a completely separate
thing.

> The question would be whether or not it would get implemented.

Yes, this is why I'm writing to this mailing list. Maybe I should've CC'ed some
Let's Encrypt specific mailing list as well.

> Yes, because it proves you control the zone.

Right, but there could be other ways to prove this as well.

---

From: Kas <[email-masked]>
Date: Sun, 13 Sep 2020 12:42:02 +0300

> challenge. Ignoring and pushing the problem down to the DNS operators does not
I can't agree more, so what about going after dns-02 challenge instead 
of trying to fix this ?

(And this is up for discussion and as food for thought.)
Lets say dns-02 challenge will be little different where the DNS record 
will hold a public key for specific(or per) domain/subdomain and ACME 
client will pass the server challenge by using the correspondent private 
key for that public key to pass the challenge from the server, where 
server will verify against the DNS published public key, means no DNS 
access from client side at the time of the challenge itself.

This might solve the need for DNS API or even accessing the DNS from the 
client altogether, this will solve and simplify the challenge while do 
not compromise the security of the challenge itself and will not 
compromise the DNS records for the DNS administrators piece in mind, on 
contrary it will give the ability to control DNS handling on the client 
side in more secure way, and yet if a client have the ability to change 
the public key then this will revert to something very similar to 
dns-01, so in that case ( when it can access and modify the DNS records) 
client can choose between dns-01 and dns-02, which i don't see need any 
discussing more that dns-01 itself.

Is such challenge viable and secure ? did i missed something obvious 
with such suggestion ?

---

From: Philipp Junghannß <[email-masked]>
Date: Sun, 13 Sep 2020 14:09:45 +0200

> Right, but there could be other ways to prove this as well.

care to share? what other methods are there to prove that you have access
to the DNS zone RIGHT NOW.

---

From: Jesper Kristensen <[email-masked]>
Date: Sun, 13 Sep 2020 14:32:32 +0200

> to the DNS zone RIGHT NOW.

dns-01 does not prove that today, because it allows delegation via cname.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 13 Sep 2020 16:36:37 -0400

> For this reason, I think a different approach would be desirable.

So, "DNS operators" are not the same thing as "DNS Registrars".
Many places do both, and many there is a generation of people who don't know
that they are different things.  Who think that the only way to manage zones
is through a web interface?
I think we have an education problem, not a standards problem.

DNS AXFR and IXFR has been around for a long long time.
RFC3007 is also widely implemented.
They are widely used by the majority of DNS operators, at very significant scale.

---

From: Ryan Sleevi <[email-masked]>
Date: Sun, 13 Sep 2020 17:58:32 -0400

This is the least secure option and most likely to be deprecated going
forward. Sending an email, even to the IANA-defined boundaries, is simply
not a substitute for proof of domain control.

On Sun, Sep 13, 2020 at 5:17 AM Philipp Junghannß <[email-masked]>
wrote:

> Am So., 13. Sept. 2020 um 11:13 Uhr schrieb Simon Ser <[email-masked]
> >:
>

---

From: Ryan Sleevi <[email-masked]>
Date: Sun, 13 Sep 2020 18:01:12 -0400

> Let's Encrypt specific mailing list as well.

It's certainly possible, but to be clear: any option for implementing a
validation method needs to be clear enough that browser/OS vendors will
permit it as acceptable. The gating function here is whether product
vendors, for their products, find it sufficiently secure, not whether CAs
do. CAs can often act as intermediaries, proposing ideas to browser/OS
vendors. Ultimately, as (sometimes contracted, always delegated) suppliers,
acting on behalf of the browser/OS vendor, your ultimate gate isn't
demonstrating whether there's consensus to write an I-D, or consensus with
a CA, but that there's consensus with the browser/OS vendors that would
implement it within their root programs.

---

From: Jacob Hoffman-Andrews <[email-masked]>
Date: Tue, 15 Sep 2020 09:59:20 -0700

Here are a couple of other useful resources on addressing this problem on
the client side. Essentially, you can run your own nameserver dedicated to
answering challenges, and delegate to it with CNAMEs.

https://github.com/joohoi/acme-dns
https://www.eff.org/deeplinks/2018/02/technical-deep-dive-securing-automation-acme-dns-challenge-validation

To the question of whether Let's Encrypt would implement a new DNS-related
challenge: We're potentially interested, but as Ryan Sleevi mentioned, it
would have to be specified in detail, either in an I-D or elsewhere. It
would then need to be accepted by root programs, possibly via the
CA/Browser Forum, since that's a convenient place to get agreement among
root programs.

That's a fair amount of work, and when I last proposed a similar change in
2018 it looked like there was significant opposition:

https://mailarchive.ietf.org/arch/msg/acme/6_j3fecaxIgwNTpJ3693U_n0Kec/
https://mailarchive.ietf.org/arch/msg/acme/rIV6jrETVXO2EmoG_tmRitDL0tA/

So, right now Let's Encrypt isn't prioritizing the work to advocate for
changes here, but hearing from subscribers that have trouble with the
current DNS challenges definitely helps inform our thinking.