Subject: [ietf-dkim] Issue 1535 - clarify need for domain existence check in the decision tree (step 2)

From: Dave Crocker <[email-masked]>
Date: Tue, 11 Mar 2008 14:16:28 -0400

Again, to repeat what I said at the mic:

The current, 3-step procedure is certainly an improvement, however I do not 
understand the need for the second step, in terms of ASP functionality.

In any early discussion of this, I believe Jim said he thought it was a 
carry-over from an earlier version of the spec where the need was more clear.

In any event, I think the current question is:  What is it about ASP -- as 
opposed to concerns outside of ASP's scope -- that requires checking for domain 
existence?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Douglas Otis <[email-masked]>
Date: Tue, 11 Mar 2008 11:32:38 -0700

> opposed to concerns outside of ASP's scope -- that requires checking  

Avoiding domain tree walking compatible with wildcards w/o depending  
on wildcards.
  a) MX mandate in conjunction with DKIM Policy
     - an empty TXT records can disavow DKIM/SMTP.
     - empty TXT records compatible with wildcard TXT records used by  
other protocols.
     - empty wildcard TXT records compatible with TXT records used by  
other protocols.
     - consumes smallest amount of DNS cache.
     - improves positive caching rates.
     - provides much stronger domain protection.
     - becomes much simpler when MX is required by SMTP.

  b) domain tree walk-up
     - results indeterminate when wildcards are in use.
     - exposes parent domains to a high volume of transactions  
dependent upon negative
       caching.
     - imposes expectation of policy to override possible parent  
domain assertions.

-Doug

---

From: Steve Atkins <[email-masked]>
Date: Tue, 11 Mar 2008 11:38:08 -0700

> opposed to concerns outside of ASP's scope -- that requires checking  

Without that check, an unsigned mail from [email-masked] will be  
considered to comply with ASP unless there is an ASP record for  
_asp._domainkey.bar.baz.ebay.com or for _asp._domainkey.baz.ebay.com

It's difficult to publish a wildcard ASP record with standard DNS  
servers. So there is no easy way to publish an ASP assertion for "my  
domain and all subdomains of it". It is only possible to publish an  
ASP assertion for a finite list of hostnames.

The domain existence check means that only a defined number of ASP  
records need to be published (the number of hostnames you publish  
would be an upper bound unless you're using wildcards anywhere else in  
your DNS, in which case all bets are off).

Removing the check removes the ability for a domain owner to make an  
ASP assertion about all possible subdomains of that domain. It seems  
within scope for ASP.

Cheers,
   Steve

---

From: Jim Fenton <[email-masked]>
Date: Wed, 12 Mar 2008 08:15:06 -0700

> ASP assertion about all possible subdomains of that domain. It seems  

Steve, thank you for refreshing my memory on this.  I would state it a 
little differently now since SSP doesn't really have a "comply", that an 
unsigned message from the domain bar.baz.ebay.com will be considered to 
have an "Unknown" ASP unless...

So yes, it is important that we keep this.

-Jim

---

From: Barry Leiba <[email-masked]>
Date: Tue, 18 Mar 2008 08:42:17 -0400

6 days with no follow-up, so....

> So yes, it is important that we keep this.

Dave, does this satisfy you?  Does anyone else have an issue with 
retaining step 2?  I'm looking for consensus to close issue 1535 and 
leave the spec as it is, with step 2 in there....

Barry (as WG chair)

---

From: Dave Crocker <[email-masked]>
Date: Tue, 18 Mar 2008 09:09:25 -0700

> _asp._domainkey.bar.baz.ebay.com or for _asp._domainkey.baz.ebay.com
...
> your DNS, in which case all bets are off).

(Thanks for Barry for reminding me to review this.)

Steve,

Many apologies, but I am simply not understanding this.

Just to make sure we are on the same page about the hierarchy  trick in the spec:

    The one-level-up hack might be useful for saving some administration, but it 
does not provide meaningful "protection", since all an attacker has to do is use 
a level down.

With respect to an A record, its presence does tell you that the name is valid, 
but it does not tell you anything about ADSP support.  Initially there will be 
virtually no adoption of ADSP.  So what does finding an A record, but no _adsp 
record, tell you?

I think what this is uncovering is that adoption of ADSP requires ensuring ADSP 
  query results for all valid names.  In that context, I guess I can see the 
benefit of having an A record serve to define what names are valid.

Mumble.  This is still feeling a bit squishy to me, although at least I'm 
starting to see the possibility of its being useful.  (I think the doc at least 
is going to have to be much more clear about its role.)

d/
-- 

   Dave Crocker

---

From: Douglas Otis <[email-masked]>
Date: Tue, 18 Mar 2008 10:26:22 -0700

On Mar 18, 2008, at 9:09 AM, Dave Crocker wrote:
> Steve Atkins wrote:
>> Without that check, an unsigned mail from [email-masked] will  
>> be considered to comply with ASP unless there is an ASP record for  
>> would be an upper bound unless you're using wildcards anywhere else  
>> in your DNS, in which case all bets are off).
>

Assuming the domain does not use wildcards for some purpose, then  
finding the non-existence of the domain offers a means to reject the  
message.  The check could depend upon the presence of SMTP discovery  
records, MX or A.

>    The one-level-up hack might be useful for saving some  
> administration, but it does not provide meaningful "protection",  
> since all an attacker has to do is use a level down.

This assumes an existence check is being made _and_ the non-use of  
wildcards.  Policy only needs to be published at nodes where A or MX  
records are being published, as not having these records should  
disqualify the domain being valid for SMTP.

> With respect to an A record, its presence does tell you that the  
> name is valid, but it does not tell you anything about ADSP  
> support.  Initially there will be virtually no adoption of ADSP.  So  
> what does finding an A record, but no _adsp
> record, tell you?

Either an A or MX record would mean the domain potentially supports  
SMTP.

> I think what this is uncovering is that adoption of ADSP requires  
> ensuring ADSP query results for all valid names.  In that context, I  
> guess I can see the benefit of having an A record serve to define  
> what names are valid.
>
> Mumble.  This is still feeling a bit squishy to me, although at  
> least I'm starting to see the possibility of its being useful.  (I  
> think the doc at least is going to have to be much more clear about  
> its role.)

Good.  A feature offering even stronger refutation than invalid  
signatures and strict policy would be to require MX records be  
published in conjunction with DKIM/SMTP policy.  In this way, when a  
DKIM/SMTP policy record is published (even an empty record) without an  
MX record, then both DKIM and SMTP are indicated as not being  
supported at the domain.

-Doug

---

From: Steve Atkins <[email-masked]>
Date: Tue, 18 Mar 2008 10:16:53 -0700

On Mar 18, 2008, at 9:09 AM, Dave Crocker wrote:

>> be  considered to comply with ASP unless there is an ASP record  
>> for  _asp._domainkey.bar.baz.ebay.com or for  
>> _asp._domainkey.baz.ebay.com
> ...

Yup. It adds no new functionality (actually it removes some), but does  
make deployment for the record publisher simpler, at the cost of  
making deployment for the record publisher less flexible and the  
checker more complex and slower.

It doesn't provide for protection of all possible hostnames in a  
domain, and using one a level down is one example of that.

> what does finding an A record, but no _adsp record, tell you?

It tells you two things. It tells you that the domain owner is aware  
of that hostname, and that they did not choose to publish an _adsp  
record that covers it.

If we consider a two step process, where in order for an _adsp[1]  
record to match a query it must either be the same, or one level up  
the domain hierarchy from the hostname (this is what we have now if we  
exclude the existence test - www.example.com will be covered only by a  
TXT record for_adsp._domainkey.www.example.com or  
_adsp._domainkey.example.com).

A domain owner cannot publish a single _adsp record that covers an  
entire domain (example.com and any hostname having .example.com as a  
suffix).

Similarly, they cannot publish a finite list of _adsp records that  
cover an entire domain. This is true whether or not you consider the  
"up one level" hack.

A domain owner cannot conveniently publish an infinite[2] list of  
_adsp records.

If a desired functionality is for a domain owner to be able to assert  
policy over all hostnames within their domain by publishing a finite  
number of _adsp records, then you need an additional step in the  
process.

One possibility for that would be to extend the "up one level" hack to  
chase up the tree until it reaches the root.

Another possibility, the one in the current draft, is to assert that  
the domain owner has a finite number of hostnames in use within their  
domain and so can publish a finite number of _adsp records to cover  
those hostnames in use (trivially, by a 1:1 correspondence between  
_adsp records and hostnames, at worst).

As there will never be a legitimate use of a hostname that may be  
checked for an _adsp record that doesn't have any DNS record  
corresponding to it[3], asserting an ADSP fail for any case where  
there is not a corresponding record in DNS will not cause any  
unintended failures, and allows the domain owner to assert policy  
across all possible hostnames within their domain. This fails in the  
case where the domain owner is using wildcard records for any use  
within their domain.

> ensuring ADSP  query results for all valid names.  In that context,  
> I guess I can see the benefit of having an A record serve to define  
> what names are valid.

An A record is not adequate for proof of validity, as there will be  
quite legitimate cases where the hostname has an MX record but not an  
A record, and we can't assert an ADSP fail there just because there's  
no A record. "A or MX" is likely adequate, but "any record" is  
certainly adequate, and likely free to check in most cases.

> think the doc at least is going to have to be much more clear about  

It's all very squishy. (And part of the squishy is that "but if  
there's no A or MX record for the email address then the mail will be  
rejected before checking ADSP" is clearly true whether or not an ADSP  
spec says so, so adding that step to the spec doesn't actually change  
what will be done operationally, but it's needed to make the spec self- 
contained and to document the thought process.)

But if there is a desire for a domain owner to be able to assert  
policy across all possible hostnames within their domain then you  
cannot remove the domain existence check without replacing it with  
something else[4]. The approach in the current draft is inexpensive to  
check, but does fail spectacularly if there is any use of wildcard  
records within the domain. We should document that, probably without  
using the phrase "fail spectacularly".

Cheers,
   Steve

[1] I'm just following the flow of conversation by using _adsp, don't  
consider it a show of support for that phrasing as I'm really agnostic  
about the name.

[2] For values of infinite that are in excess of 38^189, if not truly  
without bound. Well, you can pretty easily, but not using standard  
configurations of bind, powerdns etc, as they don't support  
generalized wildcard records.

[3] As the hostname being checked is on the RHS of an email address  
there must be an A record or MX record for it, in order for it to  
be... operationally relevant, anyway.

[4] Given that the stated purpose of ADSP is to protect email  
addresses I believe that replacing both the one level up hack and the  
existence check with an explicit check for an MX record for the given  
hostname, and returning an ADSP fail for any email sent with a RHS  
that has no MX record for it would be a Good Thing, but it would  
probably step on too many toes to actually consider as a standard.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 18 Mar 2008 11:20:32 -0700

> of that hostname, and that they did not choose to publish an _adsp  

The latter assertion is incorrect.  The word "choose" is active.

During the likely very long adoption curve, there is no way to know whether they 
"chose" not to publish adsp or whether they didn't know about it.  These have 
very different semantics, I think.

And this gets to the nub of the matter, I think:

As with DKIM, ADSP tells you something when it is there, but tells you nothing 
when it isn't.  I think the A record check is trying to pretend that you can 
learn something when ADSP isn't explicitly present for that domain.

But that's only possible if you know that the organization supports ADSP, and 
you can't.

So, when the _adsp TXT is present, you know everything you need to know.

When it isn't, you do not know anything about the organization's practices, 
including not knowing whether it has any.

Really.

> number of _adsp records, then you need an additional step in the  

The one-level hierarchy trick is the best you can do.

This effort to use the A record is overloading its semantics and you can't tell 
whether the domain owner intends the second meaning.

(BTW, I am being sloppy about referring to A, since I mean A, MX, or anything 
other than an _adsp TXT.)

> unintended failures, 

My point is that the A, MX, whatever record doesn't add any ADSP-related 
information.

It is an extra DNS query that provides no ADSP information.

d/

ps. I'm using 'ADSP' since it looks like it has rough consensus, not because I'm 
part of that consensus, which I am...

-- 

   Dave Crocker

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 18 Mar 2008 19:31:15 +0100

Steve Atkins wrote:

> As the hostname being checked is on the RHS of an email address  

JFTR, if all goes well 2821bis will explicitly add AAAA.  This does
not mean that "AAAA only" is an operationally good idea today, but
the "MX required" faction was hopelessly outnumbered for 2821bis.

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Tue, 18 Mar 2008 12:02:45 -0700

> the "MX required" faction was hopelessly outnumbered for 2821bis.

What justifications were given for including AAAA records as SMTP  
discovery records in the face of today's level of abuse?   
Nevertheless, this outcome does not prevent a requirement for MX  
records when publishing DKIM policy.  Only MX records are required to  
be at the email-address domain and are used specifically by SMTP.   
Having the MX record to qualifyuse of email policy can reduce the  
number of transactions required for policy discovery, and offers a  
practical means to disavow use of DKIM/SMTP.

-Doug

---

From: Jim Fenton <[email-masked]>
Date: Wed, 19 Mar 2008 10:46:50 -0700

> does not provide meaningful "protection", since all an attacker has to do is use 

I'm not entirely clear on what you mean by "a level down", but let me 
try to take a stab at it.

Suppose example.com has "all" signing practices that it wants to extend 
to itself and everything under it (at multiple levels).

Suppose a message comes with an author address of [email-masked] 
(hostname ns.example.com exists).  The one-level-up query will apply the 
ADSP for example.com.

Suppose a message comes with an author address of [email-masked] 
(hostname foo.example.com does not exist).  The step-2 query will result 
in an NXDOMAIN, so the ADSP result will be "the domain does not exist".  
"Domain" in this context means the right-hand-side of the address as the 
term is used in RFC 2821.

So now the attacker uses a level down...

Suppose a message comes with an author address of 
[email-masked] (hostname foo.bar.example.com does not exist).  
Again, the step-2 query results in an NXDOMAIN, and ADSP says "the 
domain does not exist".

Suppose a message comes with an author address of 
[email-masked] (hostname www.eng.example.com exists).  In this 
case, the one-level-up query does not reach the example.com ADSP 
record.  For full coverage, there MUST be an ADSP record for 
eng.example.com, as required by section 4.2 paragraph 2.

The point of the one-level-up query was not to relieve domains from 
publishing ADSP records for subdomains, but rather for hostnames and 
other terminal nodes ("terminal node" as used in RFC 2136).  This 
greatly reduces the burden of publication required to get complete ADSP 
coverage.  As I mentioned in another message, we do need to sharpen the 
language to not use the term "subdomain", which might imply multi-level 
coverage.

-Jim

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Mar 2008 12:23:45 -0700

Jim,

> to itself and everything under it (at multiple levels).

I thought we had agreed that the one-level-up hack (or one-level-down, depending 
on your spec) specified in the document was intended as an administrative 
convenience, rather than as a discrete security mechanism.

> (hostname ns.example.com exists).  The one-level-up query will apply the 

Right.  But it won't cover the cases in which the site has a multi-level domain 
naming substructure, as many organizations do.  This is why the one-level 
mechanism really is only about saving the administrator from creating a set of 
records.

(Has there been a discussion about the trade-off between saving some 
administrators this effort, versus the transactional overhead of having every 
receiving host do extra queries, so see whether there is a record one-level up?)

> term is used in RFC 2821.

So the check-for NXDomain is intended to cover those cases where the site has 
set -all and has indeed provided ADSP records (directly or implicitly) for all 
domain names it controls?

> other terminal nodes ("terminal node" as used in RFC 2136).  

I do not understand this sentence.  It parses as a sentence, but I not see the 
semantic sense in it.  Please clarify.

The first part of the sentence seems to contradict statements you made earlier 
about the mechanism, when pressed on its relevance.

This
> greatly reduces the burden of publication required to get complete ADSP 

And I do not see how your sentence, here, is different from "relieving domains 
from publishing [some] ADSP records for sub-domains.  That's what I mean by 
administrative convenience.

d/
-- 

   Dave Crocker

---

From: Jim Fenton <[email-masked]>
Date: Thu, 20 Mar 2008 14:02:55 -0700

Dave Crocker wrote:
>> Suppose example.com has "all" signing practices that it wants to 
>> extend to itself and everything under it (at multiple levels).
>
> I thought we had agreed that the one-level-up hack (or one-level-down, 
> depending on your spec) specified in the document was intended as an 
> administrative convenience, rather than as a discrete security mechanism.

At this point in the discussion I'm describing what the domain 
administrator wants to accomplish, not how they're accomplishing it.  
This is a requirements statement.

>> (hostname ns.example.com exists).  The one-level-up query will apply 
>> the ADSP for example.com.
>
> Right.  But it won't cover the cases in which the site has a 
> multi-level domain naming substructure, as many organizations do.  
> This is why the one-level mechanism really is only about saving the 
> administrator from creating a set of records.

Please read on, the discussion of multi-level domains comes later...
> administrators this effort, versus the transactional overhead of 
> having every receiving host do extra queries, so see whether there is 
> a record one-level up?)

I don't know, but it seems like an apples-and-oranges comparison to me.  
The former is either human effort or the ability of existing tools to 
support publication of records.  The latter is protocol overhead.
>> [email-masked] (hostname foo.example.com does not exist).  The 
>> step-2 query will result in an NXDOMAIN, so the ADSP result will be 
>> "the domain does not exist".  "Domain" in this context means the 
>> right-hand-side of the address as the term is used in RFC 2821.
>
> So the check-for NXDomain is intended to cover those cases where the 
> site has set -all and has indeed provided ADSP records (directly or 
> implicitly) for all domain names it controls?

Not sure I understand the question.  The NXDOMAIN check does not cover 
subdomains that exist.  These need explicit publication of ADSP records 
as required by section 4.2.  The NXDOMAIN check covers names (subdomains 
and terminal nodes) that do not exist (i.e., there is no entry in the 
DNS, under any RR type, corresponding to that name).
>> [email-masked] (hostname foo.bar.example.com does not 
>> exist).  Again, the step-2 query results in an NXDOMAIN, and ADSP 
>> says "the domain does not exist".
>>
>> [email-masked] (hostname www.eng.example.com exists).  In 
>> this case, the one-level-up query does not reach the example.com ADSP 
> I do not understand this sentence.  It parses as a sentence, but I not 
> see the semantic sense in it.  Please clarify.

Subdomains:  If www.eng.example.com exists, then eng.example.com is a 
subdomain.

Terminal nodes:  ns.example.com, www.example.com (typically, but not 
exclusively nostnames) are terminal nodes.

The one-level-up query relieves the domain from the need to publish ADSP 
records for ns.example.com and www.example.com, but it does not relieve 
the domain from the need to publish an ADSP record for eng.example.com.
>
> The first part of the sentence seems to contradict statements you made 
> earlier about the mechanism, when pressed on its relevance.

Could be.  I acknowledge that my thinking has evolved over time.
>
>
> This
>> greatly reduces the burden of publication required to get complete 
>> ADSP coverage. 
>
> And I do not see how your sentence, here, is different from "relieving 
> domains from publishing [some] ADSP records for sub-domains.  That's 
> what I mean by administrative convenience.

"Administrative convenience" is largely accurate.  The one-level-up 
query helps with ADSP deployment by relieving the domain administrator 
of the need to publish an ADSP record along with every A record in the 
domain.  This is both a convenience (in the case of manually-managed 
domains) and a tooling issue (for domains with large numbers of 
hostnames, including those that may be autopopulated from DHCP address 
pools and such).

Conversely, if it is sufficiently inconvenient or impractical for a 
domain to publish these records (absent the one-level-up query), domains 
won't do it, and it becomes a security issue (to the extent that ADSP is 
a security mechanism at all) because it will be possible for attackers 
to use those hostnames to avoid whatever recipients do with unsigned 
mail when ADSP stronger than "unknown" is published.

-Jim