Subject: [Acme] Trust and security in DNS challenge validation automation

From: Joona Hoikkala <[email-masked]>
Date: Tue, 16 Jan 2018 12:12:00 +0200

While ACME tries to promote automation of the challenge validation, the
landscape of said automation looks rather grim. DNS server software and
service providers rarely provide means to limit the privileges of
credentials used to update DNS zones.

This leads to users being forced to use and save their credentials often
equipped with inflated privileges to every machine that needs to acquire
certificates using DNS challenges, effectively meaning that when one of
such machines gets compromised so does the whole DNS zone, or multiple
zones in some cases.

The issue can be somewhat alleviated by using a separate validation zone
that the user can point CNAME records towards to for the CA to follow.
The problem however stays the same, if the credentials get compromised
the malicious user is again able to issue certificates for all the
domains pointing CNAMEs towards the validation zone in question. This
information is also easily acquired by going through CT logs and trying
to resolve the validation subdomains for each CN and SAN domain.

The only cloud DNS service provider I know that actually allows creation
of subdomains with separate credentials is Microsoft Azure. Some
providers make it possible to create new credentials and to assign a
subdelegate zone for these credentials, and pointing the CNAME there.
These solutions are a bit better, but still largely outside of the reach
of a typical user.

To fix these issues I wrote a small piece of server software (
https://github.com/joohoi/acme-dns/ ) that can be used to fix the issues
described above, but using it in large scale would raise new questions
about trust in general and the role of the service provider in the whole
chain. After all, this software isn't designed to be deployed by every
user of DNS challenge.

Would a reasonable solution be to deploy something similar to the
ACME-DNS software closer to the CA?

--
Joona Hoikkala

---

From: Jörn Heissler <[email-masked]>
Date: Tue, 16 Jan 2018 12:13:25 +0100

> ACME-DNS software closer to the CA?

Hello!

I had some thoughts about the issue too.
Your solution (CNAME + seperate DNS server) sounds like a good solution
to me.

But others are available too:

With bind9 it should be possible with something like this:

zone "example.net" {
    ...
    update-policy {
        grant somekey name _acme-challenge.www.example.net TXT;
    };
};

AWS Route53 *could* be improved using their IAM system, e.g. add a
condition that record names have a specific pattern.

One possible workaround for AWS Route53 is to create a lambda function
which updates route53. The lambda function could then check the caller's
permission.

Or in general one could run a separate server with full access to DNS
but only HTTP-API access to those servers that need certificates.

I'm currently working on such a (python based) server which implements
ACME client protocol, holds the ACME account key, has access to the DNS
zone somehow (may be full access) and provides an HTTP API where clients
can send their CSRs and receive Certificates. There will be some kind of
ACL "This client may request that certificate".
Client implementation may be as easy as "openssl req -new ...; curl ...;
mv key/cert to webserver conf; restart webserver".

The best solution however might be if all DNS providers / servers would
offer API access with limited access.

I hope that when there are enough securite solutions available, users will
pick one of them instead of giving their webservers full access to their
DNS zones :-)

Perhaps there should be some guide on how to do automated DNS challenges
securely with links to implementations.

Cheers
Jörn Heissler

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 16 Jan 2018 17:59:25 +0200

> such machines gets compromised so does the whole DNS zone, or multiple

It should be noted that currently compromise of the automation itself
is catastrophic. Even if you do not run the automation as root nor
give it read access to TLS keys, which prevents trivially stealing your
TLS keys. And even if you do not give it unneeded validation authority.

This is because the authority of automation necressarily includes
obtaining certificates for arbitrary keys, so if attacker can not
extract your key, they can just substitute their own.

HPKP could address attacks like this. Unfortunately, it is being
deprecated, with "replacement" being Certificate Transparency.
Unfortunately, CT does not help against attacks like this, because
after the certificate has been issued, it is too late as revocation
does not work.

I earlier had idea of Public Key Pinning with CAA records. It
would be much safer than HPKP (because if keys get lost, they
can be rather quickly changed) and could actually help against the
issue (as CAA is proactive, not reactive like CT). I should post a
draft about it...

-Ilari

---

From: Joona Hoikkala <[email-masked]>
Date: Tue, 16 Jan 2018 18:05:55 +0200

> issue (as CAA is proactive, not reactive like CT). I should post a
This actually already exists, check out:
https://tools.ietf.org/html/draft-ietf-acme-caa-03

--
Joona Hoikkala

---

From: "Sebastian Nielsen" <[email-masked]>
Date: Tue, 16 Jan 2018 17:14:52 +0100

I don't see a reason to do a ready-baked solution for privilege management, as every enviroment is different. Let people handle security for themselves.

If you want to have more security over your DNS, you could have it that you have a static private key on the web server/application server, and then the same key to create certificates from your DNS server.
Then you have a script that publishes the certificate. You could ACTUALLY publish the certificate in the DNS zone as a TXT.
Each TXT record may only contain up to 255 characters, but you could easily split it up to multiple records.

And from your web server/application server, you download the certificate periodically.
Note that certificates are already public per definition, so its no danger to make certificates public.

Thus if your web server/application server gets compromised, your DNS zone isn't compromised.

-----Ursprungligt meddelande-----
Från: Acme [mailto:[email-masked]] För Joona Hoikkala
Skickat: den 16 januari 2018 11:12
Till: [email-masked]
Ämne: [Acme] Trust and security in DNS challenge validation automation

--
Joona Hoikkala

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 16 Jan 2018 18:20:56 +0200

> https://tools.ietf.org/html/draft-ietf-acme-caa-03

I mean pinning the TLS key, not the account key. Even if the
compromised automation can not outright dump the ACME key (and
most probably it can), it can still misuse the key.

-Ilari

---

From: Jim Reid <[email-masked]>
Date: Tue, 16 Jan 2018 16:57:46 +0000

> On 16 Jan 2018, at 16:14, Sebastian Nielsen <[email-masked]> wrote:
> 

That would be a *remarkably* bad thing to do. More so when there already are RRtypes for storing certificates in the DNS (TLSA and CERT). TXT records get (mis)used for all sorts of things and it doesn't make sense to heap even more on to that existing babble. Or figuring out how to separate certificate favoured TXT records from any other TXT records that are floating about.

> Each TXT record may only contain up to 255 characters, but you could easily split it up to multiple records.

Nope. A TXT record can hold up to 65535 bytes. The name of the TXT record is limited to 255 characters.

Splitting a certificate into multiple TXT records is another very bad idea. How would something know which TXT records need to be sorted/merged to reassemble the original string? Now suppose one of those TXT records got dropped from the Additional Section of a response. How would a client (know how to) recover from that? These are rhetorical questions BTW.

---

From: "Sebastian Nielsen" <[email-masked]>
Date: Tue, 16 Jan 2018 18:06:25 +0100

Splitting certificates in DNS could be done for example:
0.certificate.mydomain.com = first part of cert
1.certificate.mydomain.com = second part of cert
and so on.

Or even a separate zonefile:
0.certificate.invalid = first part
1.certificate.invalid = second part
as your application/web servers does directly know the IP of the DNS server
and can do a direct query.

But if the DNS client of the servers support very long records, you could
aswell put the whole certificate there.

My tought was that since its only used internally to distribute certificates
out of you DNS server to your web/application servers without incurring
additional server software (and additional security risks), it does not need
to be the "correct" RRTypes (as some DNS clients does not support TLSA or
CERT records).

You could even hide these records from the public and only allow the IPs of
the web/application servers in question to lookup them, if you don't want
lots of "unneccessary" TXT records confusing the public.

-----Ursprungligt meddelande-----
Från: Jim Reid [mailto:[email-masked]] 
Skickat: den 16 januari 2018 17:58
Till: Sebastian Nielsen <[email-masked]>
Kopia: [email-masked]
Ämne: TXT records for storing certificates in the DNS [invalid signature!]

> Then you have a script that publishes the certificate. You could ACTUALLY
publish the certificate in the DNS zone as a TXT.

That would be a *remarkably* bad thing to do. More so when there already are
RRtypes for storing certificates in the DNS (TLSA and CERT). TXT records get
(mis)used for all sorts of things and it doesn't make sense to heap even
more on to that existing babble. Or figuring out how to separate certificate
favoured TXT records from any other TXT records that are floating about.

> Each TXT record may only contain up to 255 characters, but you could
easily split it up to multiple records.

Nope. A TXT record can hold up to 65535 bytes. The name of the TXT record is
limited to 255 characters.

Splitting a certificate into multiple TXT records is another very bad idea.
How would something know which TXT records need to be sorted/merged to
reassemble the original string? Now suppose one of those TXT records got
dropped from the Additional Section of a response. How would a client (know
how to) recover from that? These are rhetorical questions BTW.