Subject: Re: [ietf-dkim] forward movement, please?

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 1 May 2008 11:15:09 +0200

Arvel Hathcock wrote:

> I propose that the side advocating maintaining the NXDOMAIN
> check as an actual algorithmic step agree to remove this
> from the algorithm description in favor of placement 
> somewhere else.

I favour to swap steps 1 and 2:  
Looking for _adsp._domainkey.nxdomain.example. is a waste of
time when nxdomain.example. does not exist.  I'd favour to
keep it in the spec., it is needed for result nxdomain.  If
you nevertheless remove both (the step and the result) make
sure to tell Murray about it.

 Frank

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 01 May 2008 10:45:04 +0100

On Wed, 30 Apr 2008 21:02:21 +0100, Arvel Hathcock  
<[email-masked]> wrote:

> Now suppose that the attacker decides to send the mail from
> doesnt-exist.example.com.  Note that this sub-domain is still part of
> example.com yet since "doesnt-exist" doesn't exist there is no way the
> administrator at example.com could have deployed an ADSP record for it
> and where there is no ADSP record, the algorithm requires a result of "I
> do not sign everything."  Thus is ADSP defeated and we become, in my
> view, a laughing stock for overlooking an obvious hole in our algorithm.
>
> Enter the NXDOMAIN check.  If, as part of the ADSP algorithm, an
> NXDOMAIN check is performed, the algorithm can quickly detect that the
> domain doesn't exist and that _this_ might be the reason there is no
> ADSP record.  This added insight closes the hole and can be used by
> filtering agents.

Or alternatively, enter the Treewalk, which also cures the problem (at  
least to the level the treewalk is taken to).
>
> The debate has shifted from outright hostility to any NXDOMAIN check at
> all (complete elimination of it in it's entirely) to just removing it
> from a required algorithmic step and instead referencing it
> non-normatively with some version of "this is a good idea that you might
> want to think about if you're not already doing it."

I am not so sure the denate has shifted to the extent you imagine. And the  
trouble with non-normative text is that it is non-normative. You cannot be  
sure that people will take the hint. Currently some sites already do it,  
but many don't.

So my preference is still for treewalking, at least to the depth of 2 as  
advocated in the present draft. And by all means add that non-normative  
remark as well, just to encourage good practice. The advantage of the  
treewalking is that it IS within our remit to include normative mention of  
it.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Eliot Lear <[email-masked]>
Date: Thu, 01 May 2008 14:16:05 +0200

Steve,
> The discussion is not over what recipient behavior in that
> case will actually be (reject or discard it because it's
> claiming to come from someone who doesn't exist), rather
> it's over whether the ADSP spec needs to prescribe that
> behavior.
>
>   

Then we are (once again) counting angels on the head of a pin.  Many 
people will read this specification and attempt to discern behavior for, 
if nothing else, diagnostic purposes.  Nobody has answered the basic 
question of what harm would be caused with the text being left in.  And 
everyone agrees it's the right thing to do.

Eliot

---

From: Eliot Lear <[email-masked]>
Date: Thu, 01 May 2008 14:20:20 +0200

Dave Crocker wrote:
>
> Having the ADSP specification include normative text that calls for validating 
> the From field domain name does two things:
>
> 1. Couples an entirely separate and more generally useful mechanism (checking 
> domain name validity) to one that is considerably more limited (ADSP).
>   

I could easily put this another way: we are making use of a DNS 
facility.  "Coupling" usually implies some sort of bidirectional 
dependency.  There is none here.
> 2. Modifies SMTP.  (Yes, really.)
>
> Having non-normative text that describes it serves to promote the idea but not 
> couple it with the fate of ADSP.
>   

Perhaps so, but since everyone has agreed this is a good idea, what is 
the harm?

Eliot

---

From: Tony Finch <[email-masked]>
Date: Thu, 1 May 2008 12:53:44 +0100

> ADSP record.  This added insight closes the hole and can be used by

NXDOMAIN is the wrong check. A domain is not a valid mail domain if it has
neither MX nor A nor AAAA records. If it has a TXT record then a lookup
will not return NXDOMAIN even though it is not a valid mail domain.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: SOUTHWEST 5 BACKING SOUTH 4. ROUGH BECOMING MODERATE. SHOWERS. GOOD.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Steve Atkins <[email-masked]>
Date: Thu, 1 May 2008 08:09:53 -0700

On May 1, 2008, at 5:16 AM, Eliot Lear wrote:

> Then we are (once again) counting angels on the head of a pin.

Yup. I was summarizing where I think the group is at rather than
expressing much of an opinion.

> Many people will read this specification and attempt to discern  
> behavior for, if nothing else, diagnostic purposes.

> Nobody has answered the basic question of what harm would be caused  
> with the text being left in.

It makes the spec significantly less clean, as it stops being about  
anything DKIM related and starts redefining how SMTP works. For a BCP  
that wouldn't be a problem (as the practices it's suggesting are, IMO,  
"best" in at least that specific case) but stomping on the toes of  
unrelated standards is an issue for a standard.

>  And everyone agrees it's the right thing to do.

If you mean "reject mail from people who don't exists', yes, I think  
they do. If you mean "include that requirement in a standards spec"  
it's a bit more grey. I wouldn't be satisfied if it wasn't mentioned  
at all, but I'd be leary of anything as strong as a MUST in there too.

I think the NXDOMAIN (or "check for A or MX", if you prefer) issue is  
being caught in the crossfire of the treewalking proposal, which has  
more serious issues.

Cheers,
   Steve

---

From: <[email-masked]>
Date: Thu, 1 May 2008 17:01:00 +0000

> Date: Wed, 30 Apr 2008 15:12:34 -0700
> From: [email-masked]
> To: [email-masked]
> CC: [email-masked]
> Subject: Re: [ietf-dkim] forward movement, please? (was RE:  Are lookalike domains like parent domains?)
> 
> 
> 
> Arvel Hathcock wrote:
> > I propose that the side advocating removal of the NXDOMAIN check agree 
> > to language which makes this step AT LEAST a SHOULD and preferably a MUST.
> 
> 

For reasons I have shared before I have to disagree. The check for domain existence is not unrelated. In short the check for domain existence tells you whether something is a D about which it is valid to search for a P. The fact that people may already be doing this elsewhere and that it is useful for other reasons doesn't have any impact on the fact that this check gives you a hard algorithmic boundary for what entitiies ADSP is intended to apply to. We have a specification that is currently intended to cover a narrow and well defined set of entities, that is domains, and a check to tell whether a given entity is such a thing. 

> 2. Modifies SMTP.  (Yes, really.)

As I think has been pointed out before here is the definition of the domain portion of addr-spec from rfc2822.
"The domain portion identifies the point to which the mail is   delivered. In the dot-atom form, this is interpreted as an Internet   domain name (either a host name or a mail exchanger name) as   described in [STD3, STD13, STD14]."  Wouldn't something have to exist to meet this specification? Or does the term Internet domain include things that don't exist but maybe could?

> 
> d/
> -- 
> 
>    Dave Crocker
>    Brandenburg InternetWorking
>    bbiw.net

_________________________________________________________________
Express yourself wherever you are. Mobilize!
http://www.gowindowslive.com/Mobile/Landing/Messenger/Default.aspx?Locale=en-US?ocid=TAG_APRIL

---

From: <[email-masked]>
Date: Thu, 1 May 2008 17:06:12 +0000

> To: [email-masked]
> From: [email-masked]
> Date: Thu, 1 May 2008 11:15:09 +0200
> Subject: Re: [ietf-dkim] forward movement, please?
> 
> Arvel Hathcock wrote:
> 

+1 for reasons I have shared before.

_________________________________________________________________
Back to work after baby–how do you know when you’re ready?
http://lifestyle.msn.com/familyandparenting/articleNW.aspx?cp-documentid=5797498&ocid=T067MSN40A0701A

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Thu, 1 May 2008 15:27:28 -0400

I sensed my name invoked and was compelled to join the melee.

> -----Original Message-----
> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Al Iverson
> Sent: Wednesday, April 30, 2008 8:15 PM
> To: DKIM List
> Subject: Re: [ietf-dkim] end-users vs filtering engines
> 
> On Wed, Apr 30, 2008 at 7:02 PM, Dave Crocker <[email-masked]>
wrote:
> 
> >  While perhaps it closes off some particular names, it does not
close
> off the
> >  class of attack at all.
> >
> >  It is one thing to have a mechanisms that makes it incrementally
more
> >  difficult for an attacker to succeed. It is quite another to make
it no
> harder
> >  at all.  If all the attacker has to do is register a new name and
use a
> >  string-replacement on their previous attack, we do not have any
> meaningful
> >  added protections.
> 
> Dave, this actually reads as though you suggest we throw out ADSP all
> together. I don't see how this limit doesn't apply to ADSP regardless
> of tree walking functionality.
> 
> >  >> So the question is what sort of mechanism is going to benefit
from
> >  >> locking sub-domains, but not cousin domains?  How is the benefit
> >  >> meaningful?
> >  >
> >  > I don't understand the question but I suspect it's a variant of
> what's
> >  > already been asked and answered.  Is there something new here?
> >
> >  Asked, yes.  Answered, I don't think so.
> 
> Well, I certainly proposed one potential scenario where sub domain
> locking would be useful (to me, arguably not to you). Archives suggest
> Michael Hammer would prefer sub domain locking, as have Jim Fenton's
> comments. Perhaps they could theorize an example or two of where and
> how this would be useful to them.
> 

Cousin domains are orthogonal to the ADSP issue. 

Focusing on subdomains, I believe it may be useful for both senders and
checking receivers if a domain were to be able to assert whether it's
policy applies to all of it's subdomains. Given that we don't know how
receivers or reputation services might utilize such an assertion, I
would avoid must or should for a check at this stage.

My reasons for stating this is as follows:

1) In my estimation, ADSP is particularly useful for both senders and
receivers if it asserts that all mail is signed and/or discardable.
There is certainly some value if limited to only a specific
domain/subdomain but potentially greater value if an assertion can be
made that covers part or all of a tree. This allows a domain owner to
make a broader statement about it's practices.

2) The ability to make a policy assertion across the board from a base
domain may empower receivers and reputation services in their efforts to
identify "good" - as in conforms to signing policy - vs "bad" as in does
not match the domain owners stated policies through the mechanisms they
are empowered to express them through. ADSP is (or should be) a public
mechanism to extend and replace the private one-on-one
agreements/relationships that a handful of senders and receivers have
engaged in to fight (forged) spam and phishing prior to having a public
standard based option.

3) If such a policy assertion is included in ADSP then I have abiding
faith and confidence that there are those legitimate receivers and
reputation services that will take advantage of such an assertion. I
wouldn't even mandate any sort of tree walking, MX checks, NXDOMAIN
checks, etc on the receiver side with regard to such a policy assertion.
The assertion could be something as simple as a=y where "a" is all
subdomains sign and y is yes.

I want to emphasize that I am not currently at the point where the
domains I work with could make such a policy assertion but I am close
(maybe one exception per domain tree) and would strive to get there if I
were empowered through ADSP to make such an assertion.

What I would like to hear from software providers, receivers and
reputation folks is whether they would see a benefit from or take
advantage of such assertions by (particularly) large heavily phished
domains and other domains in general?

Ultimately, I'll implement whatever I can get from this ADSP process
whether narrowly scoped or more broadly scoped. As I see it we are
incrementally closing off specific spaces from specific types of abuse.
Nothing more and nothing less. For our website (brand) domains) we have
intentionally restricted the subdomains that we send email from. The
ability to assert signing for all subdomains in a tree makes it clear to
receivers that any subdomain in that tree should have a valid
signature....even subdomains that exist but are not necessarily used for
email currently. If need be we will publish an ADSP record for every
domain we use. 

Mike

---

From: [email-masked] (Wietse Venema)
Date: Thu, 1 May 2008 17:06:18 -0400 (EDT)

> would avoid must or should for a check at this stage.

How would a receiver discover the top-level domain given example.com,
example.ac.uk, example.org.au, etc.?

	Wietse

---

From: "J D Falk" <[email-masked]>
Date: Thu, 1 May 2008 17:24:38 -0400

> example.ac.uk, example.org.au, etc.?

The same way we do now: annoying, manually maintained case statements.

--
J.D. Falk
Receiver Products
Return Path 

work with me!
http://www.returnpath.net/careers/

---

From: Steve Atkins <[email-masked]>
Date: Thu, 1 May 2008 14:31:55 -0700

> example.ac.uk, example.org.au, etc.?

The same way they do for example.uk.com or example.dyndns.org.

Cheers,
   Steve

---

From: Dave Crocker <[email-masked]>
Date: Thu, 01 May 2008 15:01:24 -0700

Jim Fenton wrote:
> Dave Crocker wrote:
>> Having non-normative text that describes it serves to promote the idea 
>> but not couple it with the fate of ADSP.
> 
> Having the ADSP result depend on non-normative language in this case 
> does not meet the bar of interoperability that we need to achieve.  
> Making it non-normative means that two spec-compliant implementations of 
> ADSP would return completely different results for non-existent domains.

Sorry, but I can't let this one go by without asking:

I completely do not understand the claim of non-"interoperability" here.

Since the record(s) in question are not created with ADSP in mind, then the 
domain owner cannot be said to be participating in ADSP, with respect to this 
check.

SO how is inter-operation hurt or hindered by this specification's making the 
check normative vs. non-normative?

d/

-- 

   Dave Crocker

---

From: Steve Atkins <[email-masked]>
Date: Thu, 1 May 2008 15:01:44 -0700

On May 1, 2008, at 10:42 AM, Jim Fenton wrote:

>
>
>> People whose work environments consist solely of three level
>> hostnames see it as a magic bullet. Those who don't, see it as
>> a hack that adds complexity for recipients without buying the
>> senders any functionality they didn't already have via less
>> intrusive methods.
>>
>
> Intrusive?  How does this intrude?

It adds an additional network query for every email received by a  
typical receiver. That's very intrusive.

How many levels of hostnames does cisco use, as a real world example?

Cheers,
   Steve

---

From: Dave Crocker <[email-masked]>
Date: Thu, 01 May 2008 15:02:41 -0700

> The same way we do now: annoying, manually maintained case statements.

This relies on a resource that is not specified in the document, is not 
publicly standardized, and changes.

Not such a good thing.

d/
-- 

   Dave Crocker

---

From: Steve Atkins <[email-masked]>
Date: Thu, 1 May 2008 15:05:53 -0700

On May 1, 2008, at 10:37 AM, Jim Fenton wrote:

> Steve Atkins wrote:
>> On Apr 30, 2008, at 11:40 AM, Arvel Hathcock wrote:
>>
>>
>>> So, from the receiver side, I don't have a problem with the
>>> "treewalking" - whether it stays or goes.  I don't see it's   
>>> existence as
>>> the huge problem that others do but if it disappeared tomorrow this
>>> would not destroy the essence of ADSP (it would just increase it's
>>> deployment complexity).
>>>
>>
>> It would increase deployment complexity for senders.
>>
>> It would decrease deployment complexity for receivers.
>>
>
> That's true, but you also need to consider the nature and degree of  
> deployment complexity required.
>
> Senders who want full coverage would need to either add new ADSP  
> records for all their hosts or deploy new (and, as far as I know,  
> currently nonexistent) DNS tools to automatically publish ADSP  
> records for all of their hosts.

Not at all. It's a purely internal matter. It has no impact outside  
the site of someone who has chosen to use ADSP.

It's less important than it being purely a voluntary effort for ADSP  
users, but there's also no need for them to deploy or develop new  
tools to do so. If they can publish an A record, they can publish a  
TXT record with little extra effort.

>
>
> Otherwise, eceivers would need to deploy an ADSP implementation that  
> queries the parent domain, as currently described in SSP-03.  Other  
> than that, they don't need to do anything special.

Which would add a network session for every inbound email, whether  
that email was sent by an ADSP user or not. Increasing the latency of  
inbound mail processing is a significant cost.

Cheers,
   Steve

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 17:16:44 -0500

>> I propose that the side advocating maintaining the NXDOMAIN check as 
>> an actual algorithmic step agree to remove this from the algorithm 
>> description in favor of placement somewhere else.
>>   
> 
> I'd be happy with this if I knew where the "somewhere else" is.  If 
> there was a domain existence check somewhere else that we could 
> reference, that's worth discussing.  But I know of no such reference.

Right, sorry.  I'm talking about some other place within the document. 
Definitely not in some separate document.

Text which states that ADSP is to be applied only to Author Domains 
which _actually exist in DNS_ would suffice since there's only one way 
to determine whether that condition is meet.

> The other question is what the existence check should consist of:  check 
> for an NXDOMAIN response or check for MX/A/AAAA which more precisely 
> defines mail domains?

Needs to be NXDOMAIN I suspect to avoid the cry "You're retasking 
records for what they weren't meant to do."

Arvel

---

From: Dave Crocker <[email-masked]>
Date: Thu, 01 May 2008 15:18:16 -0700

Jim Fenton wrote:
> Hinestly, I struggled a bit with whether to use the term 
> interoperability there.
> 
> The point I'm trying to make is that the normative text should be 
> sufficient for different implementations to return the same result under 
> the same test conditions.  If we were to have an ADSP "bake-off", one of 
> the test conditions might be to report the result when given a 
> non-existent domain.
> 
> I think that this is a valid test case, and that implementations should 
> produce the same result.

So your issue is with wanting to ensure that different implementations use the 
result of the test in the same way.

Assuming I now do understand your point, I think my original question is fully 
answered.  Thanks.

However...

I still find myself not understanding how any of this makes a difference to 
the real-world utility of ADSP.

For example, let's say that a receiver chooses either not to do the NXDomain 
test or chooses to process the result differently than the document specificies.

Exactly what terrible outcome does this produce?

d/

-- 

   Dave Crocker

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 11:47:11 -0700

> 2. Modifies SMTP.  (Yes, really.)

This is a reason that we shouldn't reference a domain existence check in 
a separate document (if one indeed exists).  A separate description is 
likely to be interpreted in a way that modifies SMTP, while the 
description in ADSP simply returns the ADSP result, "the domain does not 
exist."  The specification is carefully non-normative what to do in this 
case, and therefore does not modify SMTP.

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 10:31:02 -0700

Al Iverson wrote:
>
> So, a potential way to address this without any sort of "tree walking"
> functionality would be:
> - As a sender, publish ADSP records for all domains/zones/fqdns you know about
> - Recommend that receivers reject mail from non-existing FQDNs used in
> PRA or MFROM (or somesuch).
>
> This seems workable. Others who prefer treewalking functionality, why
> does this not work for you? Where does this specifically fall down?
>   

Considering that "domains you know about" includes everything with an A 
record, the "publish ADSP records for all" part may not be practical for 
domains with tens of thousands of hosts without new DNS tools.

There's also the question of what constitutes a non-existing FQDN 
(NXDOMAIN?  MX/A/AAAA all fail?) and whether that check should be 
included in ADSP but that's covered in another thread.

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 10:37:30 -0700

Senders who want full coverage would need to either add new ADSP records 
for all their hosts or deploy new (and, as far as I know, currently 
nonexistent) DNS tools to automatically publish ADSP records for all of 
their hosts.

Otherwise, eceivers would need to deploy an ADSP implementation that 
queries the parent domain, as currently described in SSP-03.  Other than 
that, they don't need to do anything special.

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 11:38:14 -0700

Arvel Hathcock wrote:
>>> This is where we are at present on the NXDOMAIN issue I believe but
>>> others might have a different view.
>>>       
>> That's my impression, as well.
>>
>> What's the path towards settling this?
>>     
>
> I propose that the side advocating maintaining the NXDOMAIN check as an 
> actual algorithmic step agree to remove this from the algorithm 

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 10:42:56 -0700

Steve Atkins wrote:
>
>> It seems like the treewalking would help to address stuff like  
>> this....?
>>     
>
> Not in general. It will help only in the case where there are only one
> level of hostnames below the top level domain. It won't help in
> the cases where that's deeper.
>   

It still helps, it just doesn't make it possible to cover the entire 
multilevel domain with a single ADSP record; you need to have a record 
for each subdomain (one level up from the leaf nodes in the tree).  But 
are still substantially fewer records that need to be published in this 
case.

-Jim

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 17:31:20 -0500

> to determine whether that condition is meet.

Sorry, to clarify, the "one way" I'm talking about is: doing a DNS 
lookup.  I realize there are many different types of lookups (MX/A, etc) 
that _could_ be done.  All we care is that one is performed so we have 
NXDOMAIN (or not).

Arvel

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 15:34:27 -0700

> typical receiver. That's very intrusive.

I was focusing on the definition of "intrusive" as "intrude into one's 
private life".

It's a single, frequently-cached lookup that's performed for every 
message lacking an author signature, where the author domain does not 
have a published ADSP.

It seems a bit strange that this lookup is considered intrusive, while 
the many other queries performed in the normal course of an SMTP 
transaction (reverse lookup on client's IP address, forward lookup on 
HELO hostname, etc.) seem to be OK with everyone, maybe because they're 
just common practice and not written down everywhere.
> How many levels of hostnames does cisco use, as a real world example?

With very few exceptions, 1.  Cisco's namespace is almost entirely flat.

-Jim

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 17:42:55 -0500

> inbound mail processing is a significant cost.

It's true that the cost is non-zero and that unneeded lookups should be 
avoided however I'm wondering whether people understand the stupid 
number of queries already being done FOR EACH MESSAGE that comes in. 
It's ridiculous.  My last count was around a dozen in MDaemon (if you 
turn on all the bells and whistles).  Less than half of which are 
standardized but ALL are common amongst mail servers.  My point is that 
the time to worry overly much about whether mail servers are doing too 
many queries was about 7 years ago.  It's too late now.

Arvel

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 15:12:47 -0700

-Jim

Dave Crocker wrote:
>
>
> Jim Fenton wrote:
>> Dave Crocker wrote:
>>> Having non-normative text that describes it serves to promote the 
>>> idea but not couple it with the fate of ADSP.
>>
>> Making it non-normative means that two spec-compliant implementations 
>> of ADSP would return completely different results for non-existent 
>> domains.
>
>
> Since the record(s) in question are not created with ADSP in mind, 
> then the domain owner cannot be said to be participating in ADSP, with 
> respect to this check.
>
> SO how is inter-operation hurt or hindered by this specification's 
> making the check normative vs. non-normative?
>
> d/
>
>

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 15:55:21 -0700

Dave Crocker wrote:
>
> For example, let's say that a receiver chooses either not to do the 
> NXDomain test or chooses to process the result differently than the 
> document specificies.
>

It produces the outcome "unknown" for non-existent domains, that is 
subject to misinterpretation.

It's more important when coupled with the parent domain check.  If DKIM 
has the parent domain check (the misleadingly named "tree walk" in 
common parlance), referencing the parent domain's ADSP without checking 
for the existence of either the parent or subdomain makes it impossible 
to protect against the multilevel (a.b.c.d.e.example.com) attack.

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 15:43:31 -0700

Tony Finch wrote:

That's true, which is one of the reasons I wasn't crazy about allowing
AAAA records to define valid mail domains, in addition to the fact that
the use of A records is really for legacy reasons.  It adds one more
thing to check, both here and when sending mail.

NXDOMAIN does what might be considered a "sloppy" check since it some
domains that aren't valid mail domains might look OK.  I don't have a
sense for how many such domains there are; probably not many at the
registrar level but perhaps quite a few domains that are intended for
internal use and not for mail routing.

This is one of the reasons the ADSP specification needs to define how
this is done:  just saying "don't use it on non-existent domains" isn't
precise enough.

-Jim

---

From: Jim Fenton <[email-masked]>
Date: Thu, 01 May 2008 16:24:36 -0700

> publicly standardized, and changes.

Exactly what terrible outcome does this produce if this is done wrong?  
It's unlikely that com, ac.uk, or org.au are going to publish ADSP 
records.  So there is an unnecessary query to the parent, which is 
probably cached anyway (15 minutes for com, 1 day for ac.uk).

-Jim

---

From: Tony Finch <[email-masked]>
Date: Fri, 2 May 2008 00:01:12 +0100

> this is done:  just saying "don't use it on non-existent domains" isn't

I disagree that the ADSP spec should define valid mail domains. The SMTP
specification already defines them in section 5. There's no need for ADSP
to have a different specification for the same thing - in fact that would
be harmful. If you want to argue about what is and isn't a valid mail
domain, then get involved with the SMTP revision process: here is the
wrong place.

ADSP's current NXDOMAIN language is a particularly bad example of a
specification that disagrees with the SMTP spec, and it also seems
to be based on a misunderstanding of how the DNS works.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
TYNE DOGGER FISHER GERMAN BIGHT: SOUTHERLY 4 OR 5 BECOMING VARIABLE 3 OR 4.
MODERATE OR ROUGH DECREASING SLIGHT OR MODERATE. SHOWERS. MAINLY GOOD.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 1 May 2008 16:57:16 -0700

On May 1, 2008, at 3:16 PM, Arvel Hathcock wrote:
>> The other question is what the existence check should consist of:   
>> check for an NXDOMAIN response or check for MX/A/AAAA which more  
>> precisely defines mail domains?
>

When domain validity checks discover whether SMTP might be supported  
by the domain's DNS, MX and then A records could be queried without  
"retasking" their purpose.  Not looking for _positive_ confirmation of  
possible SMTP support means empty nodes could be considered "valid"  
domains, which is wrong, problematic from the standpoint of depending  
upon negative results that might overlook local resolutions, and of  
course wasted overhead.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Thu, 1 May 2008 16:57:11 -0700

On May 1, 2008, at 3:31 PM, Arvel Hathcock wrote:

>> Text which states that ADSP is to be applied only to Author  
>> Domains  which _actually exist in DNS_ would suffice since there's  
>> only one way to determine whether that condition is meet.
>
> lookup.  I realize there are many different types of lookups (MX/A,  
> etc) that _could_ be done.  All we care is that one is performed so  
> we have NXDOMAIN (or not).

Publishing ADSP should mandate MX records be published when the domain  
intends to receive email.  By making this mandate, not finding an MX  
record will prevent subsequent transactions attempting to retrieve  
policy records.   Such a convention reduces the amount of undesired  
traffic for domains not involved with SMTP.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Thu, 1 May 2008 17:09:21 -0700

On May 1, 2008, at 4:01 PM, Tony Finch wrote:

>> This is one of the reasons the ADSP specification needs to define  
>> how this is done:  just saying "don't use it on non-existent  
>> domains" isn't precise enough.
>
> I disagree that the ADSP spec should define valid mail domains. The  
> SMTP specification already defines them in section 5. There's no  
> need for ADSP to have a different specification for the same thing -  
> in fact that would be harmful. If you want to argue about what is  
> and isn't a valid mail domain, then get involved with the SMTP  
> revision process: here is the wrong place.
>

Tony,

Agreed (almost).  Publishing ADSP should also mandate the publishing  
of MX records when SMTP is supported.  This helps reduce undesired  
policy traffic directed toward spoofed domains.

ADSP should also take another look using large text labels and record  
placement under "_domainkey".  When ADSP records become highly  
replicated, the unnecessary use of labels and text quickly increase  
the zone size.

-Doug

---

From: [email-masked] (Wietse Venema)
Date: Thu, 1 May 2008 20:26:08 -0400 (EDT)

> The same way we do now: annoying, manually maintained case statements.

And who will provide updates to all the ADSP verifiers in the field?

	Wietse

---

From: [email-masked] (Wietse Venema)
Date: Thu, 1 May 2008 20:34:19 -0400 (EDT)

> probably cached anyway (15 minutes for com, 1 day for ac.uk).

Jim, 

You deleted the context of the original question: a mechanism that
allows organizations to advertise a policy in one place that applies
to their entire DNS tree.

In the absence of a solid algorithm that determines the top of an
arbitrary organization's DNS tree, verifiers will have to walk up
the entire DNS tree from the bottom.

Thus, ADSP becomes a tool thay can be mis-used for trivial
amplification attacks by sending rfc2822.from addresses with many
domain levels. That is not a good thing for a protocol that attempts
to improve security. The prime directive should be "do no harm".

	Wietse

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 20:55:18 -0500

> to be based on a misunderstanding of how the DNS works.

What is the misunderstanding?

Arvel

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 21:48:30 -0500

> making the check normative vs. non-normative?

Some food for thought on this question:

One of the results returned by ADSP is "domain does not exist" (see 
section 3.2).  If you predicate one of the outcomes of the protocol upon 
an optional "it might be a good idea to do a lookup, you know, if you 
feel like it" then what you'll have are different implementations of the 
protocol each able to truthfully claim compliance yet each returning a 
different result _for the same input data_.  Bad.

This is what I have been referring to in earlier posts saying that ADSP 
results should be "reliable" and is why, if we're stuck on the question 
of normative vs non-normative, either normative must win or we must 
remove "the domain does not exist" from the list of ADSP results (which 
creates all the problems my side in this debate has been preaching about 
in other posts).

Hence, the wisdom of Wietse's solution which is to have the spec assert 
that ADSP is to be applied only to Author Domains which exist in DNS. 
This nicely eliminates the need for language like "you MUST (or SHOULD) 
do a DNS check" thus satisfying (one would hope) one side of the debate 
while at the same time requiring the acquisition of the data which 
"domain does not exist" needs - thus satisfying the other side of the 
debate.

It's beautiful.  I note with great concern that my opponents in this 
debate have not commented on Wietse's proposal.

Arvel

---

From: Arvel Hathcock <[email-masked]>
Date: Thu, 01 May 2008 21:55:48 -0500

> So your issue is with wanting to ensure that different implementations
 > use the result of the test in the same way.

Clearly, that's _not_ what he said.  He said this:

 > sufficient for different implementations _to return the same result_
 > under the same test conditions.

Obviously, "return the result" does not mean "use the result"

Arvel

---

From: "Al Iverson" <[email-masked]>
Date: Thu, 1 May 2008 22:14:07 -0500

> domains with tens of thousands of hosts without new DNS tools.

Can you help me to understand what tool may be required beyond
/usr/bin/bash? I need to understand what the problem is beyond
potentially requiring an ability to scriptly generate a one time
update of many new DNS records for bunches of FQDNs or domains. That
alone is hardly a new thing, and is a problem for those who manage DNS
for many hosts or domains; it is not specific to ADSP, nor DKIM, nor
SPF, nor A record changes due to connectivity changes, etc. So I'm
struggling to grasp what makes this different and potentially worse.

Regards,
Al

-- 
Al Iverson on Spam and Deliverability, see http://www.spamresource.com
News, stats, info, and commentary on blacklists: http://www.dnsbl.com
My personal website: http://www.aliverson.com -- Chicago, IL, USA
Remove "lists" from my email address to reach me faster and directly.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 02 May 2008 11:09:31 +0100

> unrelated standards is an issue for a standard.

Fine! Then let us write a BCP.

It could be a separate BCP draft specifying "Best Practice" for Verifiers  
who wished to take advantage of ADSP.

Or it could be a distinct section within the ADSP draft, clearly marked as  
being Non-Normative but nevertheless describing Verifiers "Best Practice".

In either case, it could still use RFC 2119 language "MUST" and "SHOULD",  
but with a clear proviso that those REQUIREMENTS/RECOMMENDATIONS were only  
obligatory for a Verifier who wished to claim "Compliance with this BCP".  
A Verifier would remain free to do whatever he saw fit (as at present),  
provided he did not claim to adhere to the BCP.

And the BCP would not be applicable at all to receivers who did not take  
notice of ADSP at all (so it is not a "behind the scenes" attempt to  
change basic SMTP).

Proper topics to be discussed within this BCP document/section would  
include
1. The extent of treewalking to be done (if any), and the consequences for  
cousin domains.
2. What to do about non-existent domains (if anything), including just how  
"non-existent" was to be defined.
3. What addresses were to be checked for ADSP compliance (currently, we  
have every address in the From header).
4. All other things Verifiers should or should not be doing.

And everything regarding Verifiers would be removed from the ADSP draft  
(or from the Normative sections thereof). The (normative parts of) the  
ADSP draft would be limited to

1. Describing what was to be contained in the ADSP records.
2. Which domains and subdomains should have them
3. Possible requirement that MX records MUST be provided alongisde ADSP  
records.

And, of course, the contents of the normative ADSP draft and the  
non-normative BCP draft/section would be carefully coordinated to  
interoperate with each other (so as to a provide well-understood treatment  
for cousins).

---

From: Tony Finch <[email-masked]>
Date: Fri, 2 May 2008 12:08:27 +0100

> What is the misunderstanding?

There are two kinds of "not found" responses in the DNS: NXDOMAIN and
NODATA. The current draft only treats one of them as an error, which is
very unusual for applications. It's also a common mistake for someone to
say NXDOMAIN when they mean NODATA. If the spec is deliberately being
clever about DNS semantics then it should have a couple of sentences to
explain this.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
PLYMOUTH: SOUTHWEST BACKING SOUTHEAST 4 OR 5, INCREASING 6 OR 7 IN WEST LATER.
SLIGHT OR MODERATE, OCCASIONALLY ROUGH IN WEST LATER. SHOWERS, OCCASIONAL RAIN
LATER. GOOD, OCCASIONALLY MODERATE LATER.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Eliot Lear <[email-masked]>
Date: Fri, 02 May 2008 14:02:10 +0200

> Fine! Then let us write a BCP.

No.  The more documents people need to read to understand the mail 
system, the more difficult it is to implement and deploy.  Let us help 
those who actually are trying to implement this stuff to do the right 
thing.  Especially when we know omitting the check would be precisely 
the wrong thing.

Eliot

---

From: <[email-masked]>
Date: Fri, 2 May 2008 15:24:56 +0300

John Levine wrote:

> > Assume, say, one million people who regularly receive valid emails
> > from their bank ([email-masked]). If they received an
> > email from [email-masked], how many of them would
> > believe the email is really from the bank?
> 
> Well, now we have another question -- who's going to be using ADSP,
> mail system operators or end users?  It's always been my impression
> that the main audience is MTA operators, who will use it in
> filtering decisions.

I think I agree with you here. I was just trying to give an example
why BigBank would like to publish ADSP records that cause MTA
operators to treat mails from [email-masked] as
"suspicious" (or something).

Best regards, 
Pasi

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Fri, 2 May 2008 09:07:52 -0400

> [email-masked]] On Behalf Of Dave Crocker
> Sent: Thursday, May 01, 2008 6:03 PM
> To: J D Falk
> Cc: [email-masked]

But is it such a bad thing Dave? This is why I'm suggesting specifying
how the domain owner can articulate the policy but not specifying (at
this point) how a receiver might use it. It's that old King Canute thing
that John likes to bring up.

Different receivers will take different approaches for taking advantage
of "A=Y" initially. Why would this be an issue? I have a strong feeling
that the domain owners most likely to take advantage of something like
this do not have tons of subdomains in their trees. 

I expect ADSP records to generally have (relatively) long TTLs. Do we
expect most adopters to be changing their policies willy nilly? If I
really wanted to make a change I would shorten up the TTL and then wait
until well after the original TTL had passed to make the change. It's
only an issue if someone has already published an ADSP policy - wouldn't
it be nice if we could get ADSP out the door so people could actually
start implementing?

The overall hit in terms of lookups, tree walking, etc is not likely to
be significant. I would expect (early) implementers to cache the results
locally for the duration of the TTL rather than going externally for an
ADSP lookup for each and every piece of email.

There is a reason the name was changed from SSP to ADSP. With respect to
that we should be asking ourselves how to empower author domains to
express their signing policies in ways that then empower receivers to
make rational decisions about how to handle (validly) signed vs unsigned
email.

J.D. and several others have indicated that they would determine base
domains manually with regard to various TLD practices. I go back to my
original question to receivers. Would an "A=Y" (or however syntactically
constructed) assertion be sufficiently useful to receivers and
reputation service providers that they would take advantage of it?
Would it make sense to require an ADSP publisher wishing to utilize this
to publish it for all (that would be a MUST) subdomains in a tree making
such an assertion? 

If receivers and reputation service providers don't feel such an
assertion is particularly useful then we can drop the discussion and
move on to other things.

Mike

---

From: John Levine <[email-masked]>
Date: Fri, 2 May 2008 09:21:22 -0400 (EDT)

> "suspicious" (or something).

Oh, I have no doubt that bigbank.com would like to blanket the DNS with 
discardable ADSP records.  The question is whether any receivers would 
care.  I wouldn't, since I don't consider a.b.foo.com any more similar to 
foo.com than a-b-foo.com.

Among the many conceptual problems that plague ADSP is the fantasy among 
some senders that this will somehow give them more control over what 
receivers do with their mail.

R's,
John

---

From: "Al Iverson" <[email-masked]>
Date: Fri, 2 May 2008 09:45:11 -0500

>  receivers do with their mail.

I think it would help the productivity here to drop the "senders
inherently have an agenda and are all wrong for that reason alone"
line of discussion.

I don't know that you count as a receiver any more than I (or my
employer) does. So I'm unsure what you're trying to distance yourself
from. Both you and I are receivers or represent receivers, probably to
a similar extent. Host random stuff for random people. In addition, ET
hosts inbound mail handling for many clients, hosts many domains, and
receives a stunning amount of blowback. I don't know that this makes
either of us "receivers" along the lines of top consumer ISPs. As far
as sending, historically I have understood you to consult with
numerous senders on occasion. So I think it's a bit unfair to take
issue purely based on which MAAWG badge somebody might be
wearing....when if you set those aside, we're seemingly a lot harder
to tell apart.

Senders have an agenda. As do receivers, and everybody else. We're
here to figure it all out, but I'm certainly starting to feel that
"bleck, senders" is a useless, unhelpful response.

The larger picture of that "fantasy among senders" seems to actual
have been started, and in a large part driven, by large receiving
sites, looking for newer, better tools and assistance to help improve
identification of phish and spoof mail. Look historically at the email
authentication effort, specifically Yahoo and DK, Hotmail and Sender
ID. These both give senders more control over what receivers do with
their mail, yet they are not something that some secret group of
senders jammed down anybody's throat. They were receiver designed and
pushed.

It's not a fantasy, nor is it "senders" anything alone.

Regards,
Al Iverson

---

From: Arvel Hathcock <[email-masked]>
Date: Fri, 02 May 2008 10:37:24 -0500

> NODATA. 

NXDOMAIN (RCODE=3) means the name does not exist in DNS.  NODATA 
(RCODE=0 + ANCOUNT=0) means the name is valid but no records of the 
requested type were found.  We're only interested in the first one.

So, our side in the debate is not operating from a misunderstanding of 
DNS as far as I can tell.

-- 
Arvel

---

From: Dave Crocker <[email-masked]>
Date: Fri, 02 May 2008 08:49:12 -0700

> DNS as far as I can tell.

I understand the reason for the test to be verifying that the From: field 
address is  (probably) valid.  That's an email semantic questions.

Looking for domain existence, rather than domain use of an email-related DNS 
record, seems like a significantly less appropriate test, given the goal of 
the test.

Why is it sufficient for the domain to have no RR relevant to email, just as 
long as it has some RR?

d/

-- 

   Dave Crocker

---

From: Arvel Hathcock <[email-masked]>
Date: Fri, 02 May 2008 11:32:07 -0500

> Why is it sufficient for the domain to have no RR relevant to email, 
> just as long as it has some RR?

Because it just seems the path of least resistance.  If I were to 
suggest that every domain taken from From: needs to have an MX RR then 
somebody would point out that this violates SMTP RFC 2821 Section 5 
which, in it's wisdom, treated the need for MX RR's as an optional "we 
think it's a good idea" type of thing.

Arvel

---

From: Tony Finch <[email-masked]>
Date: Fri, 2 May 2008 17:34:01 +0100

> DNS as far as I can tell.

Yes, but that isn't clear from reading the spec.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SHANNON ROCKALL: SOUTH OR SOUTHEAST 4 OR 5 INCREASING 5 TO 7, OCCASIONALLY
GALE 8. MODERATE OR ROUGH INCREASING ROUGH OR VERY ROUGH. RAIN AT TIMES. GOOD
BECOMING MODERATE, OCCASIONALLY POOR.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Arvel Hathcock <[email-masked]>
Date: Fri, 02 May 2008 11:37:45 -0500

> think it's a good idea" type of thing.

Sorry, perhaps that statement wasn't fair.

2821 does require clients to query and use MX records.  However, 
immediately thereafter it says, in effect, "but if there aren't any MX 
records, that's ok, use an A record instead."

Arvel

---

From: Jim Fenton <[email-masked]>
Date: Fri, 02 May 2008 10:01:22 -0700

> to their entire DNS tree.

I didn't delete anything but Dave's signature line and the mailing list 
footer.  Please check the thread.  I'm very sensitive to being taken out 
of context, and try not to do the same.

What is in the ssp-03 draft is not a mechanism to advertise a policy in 
one place that applies to their entire DNS tree.  It is a mechanism to 
advertise a policy that applies to leaf nodes (only) one level down, 
such as A records within the domain.

> the entire DNS tree from the bottom.

Or those publishing ADSP records will need to do so for each of their 
subdomains (as distinct from hostnames).  Of course, hostnames are much 
more numerous than subdomains, so this reduces the publication 
requirement significantly.

> to improve security. The prime directive should be "do no harm".

Several revisions back, SSP had provision for a record search that was 
either unbounded or 5 layers deep, which was abandoned for this (among 
other) reasons.  The current draft, and the proposal on the table, is to 
perform a maximum of one additional lookup if an ADSP record is not 
found and the domain exists.  I don't see that as introducing a 
significant amplification or make-work attack.

This is one of the reasons I react so strongly to the term "tree walk" 
for this.  It's a single additional query, maximum.

-Jim

---

From: Douglas Otis <[email-masked]>
Date: Fri, 2 May 2008 11:25:18 -0700

On May 2, 2008, at 9:32 AM, Arvel Hathcock wrote:

>> Why is it sufficient for the domain to have no RR relevant to  
>> email, just as long as it has some RR?

An empty node with no resource records will not cause an NXDOMAIN  
response.

> suggest that every domain taken from From: needs to have an MX RR  
> then somebody would point out that this violates SMTP RFC 2821  
> Section 5 which, in it's wisdom, treated the need for MX RR's as an  
> optional "we think it's a good idea" type of thing.

While it may be reasonable to check for A records after checking for  
MX records, before considering a domain as not supporting SMTP, a  
requirement to query A records should not be included within the ADSP  
specification.  Although SMTP does not require domains to publish MX  
records, publish ADSP policy should still be predicated upon the  
domain also publishing MX records in conjunction with their ADSP  
records.  Such a mandate thereby protects all domains not involved  
with SMTP from being flooded by policy related DNS transactions  
whenever some sub-domain within their domain is spoofed within email  
addresses.

With an MX record mandate made by ADSP (that should also be made by  
SMTP at some point), the first check should be to discover whether an  
MX record exists.  When used legitimately, MX records have a fairly  
high probability of existing and being cacheable.  When an MX record  
is not returned, the ADSP mandate allows recipients to assume ADSP  
policy is not published at the domain, and thereby forgo attempts to  
retrieve SMTP related policies.  If ADSP becomes accepted practice,  
other policy records will be deployed as well.  Limiting the impact  
caused by policy records transactions generated by each email received  
becomes an extremely important security consideration.

The ADSP draft could recommend the support for SMTP at the From domain  
be confirmed before accepting messages, but such a recommendation has  
little to do with the application of ADSP policy or DKIM.  This would  
have more to do with a general issue of not accepting messages  
containing spoofed addresses.

-Doug

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 05 May 2008 10:46:27 +0100

> containing spoofed addresses.

ALmost. The ADSP draft could recommend the support for SMTP at the From  
domain
be confirmed before accepting messages by sites who claimed to be  
Verifiers of ADSP/DKIM (because those sites are the only ones that could  
be considered "our customers").

BUT it could also add, parenthetically, this this particular practice was  
often regarded as beneficial even by sites not claiming to do such  
verification.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 05 May 2008 10:29:23 +0100

On Fri, 02 May 2008 13:02:10 +0100, Eliot Lear <[email-masked]> wrote:

> system, the more difficult it is to implement and deploy.

Fine! Then adopt my second suggestion, which was to make the BCP a clearly  
designated section within the ADSP document.

> ...  Let us help

Exactly, so write it all as a BCP section, as I suggested. It cannot be  
100% normative because we have no means of REQUIRING Verifiers to behave  
in any particular way, or even to Verify at all. But we can make it very  
clear just what they are _expected_ to do.

---

From: Hector Santos <[email-masked]>
Date: Mon, 05 May 2008 13:06:58 -0400

> designated section within the ADSP document.

BCP - "Best Current Practice"

What's best, current about the practice nor the practice itself, has yet 
to be established.  So why would you call it a BCP?

Or does BCP mean something else"

    "Best Consideration (for) Practical" application of theory?

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 06 May 2008 10:14:45 +0100

> to be established.  So why would you call it a BCP?

We need some means of saying "ADSP has defined what Senders may put in an  
ADSP record. Here is what Verifiers are intended/expected to do with it",  
but without using Normative Language for the Verifiers (since we cannot  
even REQUIRE that they Verify at all, let alone do it any particular way).

If you don't like the term BCP because the practices we are proposing are  
not yet "current", then let us use some other term implying  
non-normativeness. Particularly if this is done in a specially designated  
section of the ADSP draft (rather than in a separate document), we do not  
need to use that term. Perhaps something like:

"This section describes a Code of Practice for verifiers who wish to  
fulfil the expectations of sites that have supplied ADSP records. This  
Code is not normative (since it is open to any site whether it chooses to  
perform any verification at all), and the words "MUST" and "SHOULD", when  
used within this section, are intended to apply only to those sites which  
claim to be following this particular Code."

---

From: "J D Falk" <[email-masked]>
Date: Tue, 6 May 2008 17:37:08 -0400

On 2008-06-05 3:14, Charles Lindsey wrote:

> non-normativeness.

It seems as if this is something the IETF tends to be spectacularly bad at.
I agree that there's a need, but there are other organizations which might
be more suited to writing this particular document.

At least one of 'em is already considering it, but we're waiting for the
IETF to finish ADSP first.  Will that ever happen?

-- 
J.D. Falk
Return Path

Work with me!

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 May 2008 12:20:57 +0200

> designated section within the ADSP document.

That would second best, but these are games.  Put the information where 
it logically belongs, AS VIEWED FROM THE READER'S PERSPECTIVE.  Right 
now this group is playing bureaucratic games while we all know the right 
answer.  It's just astonishing.

Eliot

---

From: Douglas Otis <[email-masked]>
Date: Mon, 12 May 2008 11:54:27 -0700

On May 12, 2008, at 3:20 AM, Eliot Lear wrote:

>> Fine! Then adopt my second suggestion, which was to make the BCP a  
>> clearly designated section within the ADSP document.
>
> That would second best, but these are games.  Put the information  
> where it logically belongs, AS VIEWED FROM THE READER'S  
> PERSPECTIVE.  Right now this group is playing bureaucratic games  
> while we all know the right answer.  It's just astonishing.

Eliot,

Are you suggesting games prevent adoption of a process expecting MTAs  
to query parent domains for ADSP records whenever these records are  
not found within the From email-address domain?

In general, the ADSP draft should describe specifically the public  
protocol protected by its policy assertions, and how policy should be  
discovered and applied.

Even in a minority of cases where an email-address is not spoofed,  
parent ADSP checks will cause transactions to be generated against  
domains _not_ involved with SMTP for _each_ incoming email.  Also,  
these transactions will typically offer little benefit due to ADSP  
scarcity making these empty transactions nearly worthless.

However, a BCP may be needed to aid domains not implementing SMTP,  
DKIM and ADSP in how to protect themselves from transactions caused by  
an egregious level of transactions associated with SMTP, such as ADSP  
policy discovery processes. : (

How can uninvolved domains seek protection?  Domains may exist without  
containing resource records.  Must all domains publish ADSP records to  
terminate what might become a torrent of transactions?  A high  
percentage of SMTP messages are sent by opportunistic bad-actors that  
might also have a malevolent intent.

Don't expect the world will publish ADSP records at every domain any  
time soon.  A more reasonable, immediate, and obtainable goal would be  
to insist the few domains supporting SMTP that have yet to publish MX,  
to publish or their public messages might not be accepted.  This type  
of mandate impacts only a small number of domains.  These domains  
could be white-listed to assist with an interim MX mandate  
transition.  ADSP should support the MX record first philosophy by  
specifying a discovery process that first checks for MX records in the  
email-address domain before proceeding with other related transactions.

With such an MX mandate, domains not publishing MX records are  
protected from additional transactions and connections caused by  
spoofed email-addresses.  This mandate does not alter the SMTP  
discovery process.  It only establishes an expectation that any public  
email-address exchanged by SMTP be confirmable by the existence of an  
MX record.  This avoids the myriad of SMTP opt-out strategies such as  
"MX .", "TXT v=spf1 -all", or "TXT DKIM=DISCARDABLE".

-Doug

---

From: Steve Atkins <[email-masked]>
Date: Thu, 15 May 2008 10:18:21 -0700

On May 15, 2008, at 9:18 AM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> This avoids the myriad of SMTP opt-out strategies such as
> Digression, finally I recall a technical problem (above the
> clear disadvantages of opt-out at least for IPv6) with the
> null-MX idea, one of those weird "alternate roots"

Then that's not the domain name system, so is likely out of scope for  
anything DNS related.

> managed
> to have an IPv4 (A record) for its wannabe-root.

If the person doing the DNS query is falsifying data, there's not a  
lot you can do to defend against that, but it's their problem, not  
yours.

Cheers,
   Steve

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 15 May 2008 20:26:37 +0200

Steve Atkins wrote:

>> one of those weird "alternate roots"
 
> Then that's not the domain name system, so is 
> likely out of scope for anything DNS related.

Just because you and me do not like "alternate
roots" is not the same as a hardcore technical
argument based on say RFC 1034:  Can "a" root
in any class of "a" DNS have an A-record ?  

The SMTP RFCs are not limited to "the" DNS, it
is perfectly fine to roll a LAN with "a" DNS -
ideally after folks have read and understood
what 2821bis says about gateways.  

Sorry for the digression, but Doug is on record
with worrying about the fine print for DKIM.

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 15 May 2008 18:18:13 +0200

Digression, finally I recall a technical problem (above the
clear disadvantages of opt-out at least for IPv6) with the
null-MX idea, one of those weird "alternate roots" managed
to have an IPv4 (A record) for its wannabe-root.

 Frank

---

From: John Levine <[email-masked]>
Date: 22 May 2008 16:03:28 -0000

We don't seem to have resolved the question of whether the ADSP should
define a specific existence check, or just say that you should check
but leave the definition for other places.

Personally, I think it's severe mission creep to try to define an
existence check.  It's straightforward to check for a NXDOMAIN or
NODATA result, but I see no reason to think that such a check has the
semantics an ADSP user would want.

To touch on some of the issues (and try not to rehash them all), the
majority of A and AAAA records don't name domains used in mail and you
can't check short of sending a test message and waiting a week to see
if it bounces, there's many ways a name can exist but again not for
mail (what if there's just a TXT record), and any check we defined
would just be wrong if, e.g., next year we make MX . the no-mail
standard.

So I like Arvel and Wietse's approach, say to do it but don't try to
define it since any definition would be wrong.  Other thoughts?

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Thu, 22 May 2008 10:36:16 -0700

On May 22, 2008, at 9:03 AM, John Levine wrote:

> It's straightforward to check for a NXDOMAIN or NODATA result, but I  
> see no reason to think that such a check has the semantics an ADSP  
> user would want.

Reliance upon NXDOMAIN or NODATAT precludes use of wildcards for _any_  
record by _any_ other protocol.

> majority of A and AAAA records don't name domains used in mail and  
> you can't check short of sending a test message and waiting a week  
> to see if it bounces, there's many ways a name can exist but again  
> not for mail (what if there's just a TXT record), and any check we  
> defined would just be wrong if, e.g., next year we make MX . the no- 
> mail standard.

Why re-experience the lessons regarding opting-out for email?  Opting  
out is _not_ practical.  Reliance upon "MX ." requires _all_ other  
domains wishing to opt-out (perhaps to avoid transactions generated  
looking for ADSP or DKIM keys) to publish bogus MX records targeting  
the root.

> define it since any definition would be wrong.  Other thoughts?

Not defining the public protocol protected by ADSP would be wrong.   
Once SMTP is defined as the protocol protected by ADSP, then checking  
for A and MX records limits where ADSP is required and therefore  
sought.  Leaving this issue undefined and suggest all domain could  
publish ADSP or "MX ." already implies the use of SMTP!  The goal is  
not to _absolutely_ ensure SMTP is supported.  The goal is to limit  
the number of domains required to publish ADSP.

An approach requiring domains to opt-out does not scale!   Mandating  
inclusion of discovery records in conjunction with policy does.  It is  
simply not practical to extend an "opt-out" strategy.  Imagine what  
DNS would become once other protocols follow suit and also adopt an  
"opt-out" strategy.  A strategy that ADSP/SMTP should take would be to  
require MX records to ensure the acceptance of public message  
exchanges.  Until such time the MX mandate is adopted, at least limit  
domains where ADSP is required to those that include A records.  The  
ADSP draft does just this.

-Doug

---

From: Eric Allman <[email-masked]>
Date: Thu, 22 May 2008 10:27:56 -0700

I remain convinced that it is absolutely essential to retain the 
existence check.  Making it advisory would be a serious mistake.

eric

--On May 22, 2008 4:03:28 PM +0000 John Levine <[email-masked]> wrote:

> We don't seem to have resolved the question of whether the ADSP
> should define a specific existence check, or just say that you
> should check but leave the definition for other places.
>
> NODATA result, but I see no reason to think that such a check has
> the semantics an ADSP user would want.
>
> defined would just be wrong if, e.g., next year we make MX . the
> no-mail standard.
>

---

From: Steve Atkins <[email-masked]>
Date: Thu, 22 May 2008 10:21:56 -0700

> semantics an ADSP user would want.

The reason for the existence check is simply to make it possible
for an ADSP user to specify consistent policy across all their
space in the DNS tree. A simple check for NXDOMAIN is
sufficient to fill that need. Any semantics beyond that are moving
beyond the reason that the check is needed.

> define it since any definition would be wrong.  Other thoughts?

If you don't define it then the sender has to assume the most
pessimistic implementation, in order for their wishes to be communicated
reliably to all checkers who implement their own form of check
for existence.

That most pessimistic implementation is to assume that the sender
checks only for NXDOMAIN, so the sender will need to explicitly
publish ADSP records for anything that's syntactically valid for
use in email and does not return an NXDOMAIN.

So, that's exactly the same operationally as specifying checking
just for NXDOMAIN, but less clearly communicated, as it replaces
a simple hard rule with unclear language that implies
exactly the same behavior for senders but is more likely to be
misunderstood and misimplemented.

Cheers,
   Steve

---

From: Dave Crocker <[email-masked]>
Date: Thu, 22 May 2008 11:41:31 -0700

> beyond the reason that the check is needed.

Each form of pre-test provides a narrow degree of enhanced protection across a 
domain tree.  Note that no pre-test at all is necessary for 'protecting' a 
single domain name, since the presence or absence of the ADSP record does that 
entirely sufficiently.

(Again, to the extent that one is seeking protection with sites that do not 
use ADSP, one is completely beyond the scope of this working group.)

Narrow does not mean useless, but it does mean narrow, as in incomplete.  This 
vigorous insistence on demanding a use of a particular mechanisms that 
nonetheless provides incomplete enforcement is quite odd, since its utility 
amongst the broader spectrum of abuse exploits is so small.

d/

-- 

   Dave Crocker

---

From: "J D Falk" <[email-masked]>
Date: Thu, 22 May 2008 17:50:33 -0400

> define it since any definition would be wrong.  Other thoughts?

I'm confused.  Arvel and Wietse's approach seems to make perfect sense to
me, but some other very smart people who I also have a lot of respect for
are disagreeing with them...so I have to assume that I'm missing something.

Most of the thread about it involved people disagreeing with Doug about
entirely unrelated issues, so it's very difficult to determine what the
actual arguments are.  Could someone please summarize, perhaps with
suggested solutions to allow us to move forward?

-- 
J.D. Falk
Return Path

Work with me!

---

From: "Al Iverson" <[email-masked]>
Date: Thu, 22 May 2008 17:47:16 -0500

> suggested solutions to allow us to move forward?

+1

Regards,
Al

---

From: Tony Finch <[email-masked]>
Date: Thu, 22 May 2008 23:23:40 +0100

> semantics an ADSP user would want.

NODATA is even more wrong than NXDOMAIN :-(

Tony.
FORTIES: NORTHEASTERLY 3 OR 4, OCCASIONALLY 5 IN WEST. SLIGHT OR MODERATE.
FAIR. GOOD.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 22 May 2008 16:31:34 -0700

On May 22, 2008, at 2:50 PM, J D Falk wrote:

>> So I like Arvel and Wietse's approach, say to do it but don't try  
>> to define it since any definition would be wrong.  Other thoughts?
>
> I'm confused.  Arvel and Wietse's approach seems to make perfect  
> sense to me, but some other very smart people who I also have a lot  
> of respect for are disagreeing with them...so I have to assume that  
> I'm missing something.

Arvel has suggested disqualifying the innocent absence of an ADSP  
record can rely upon NXDOMAIN when requesting _any_ record from the  
domain ADSP is below.  More than one DNS transaction is therefore  
required.  Such a strategy creates an unfortunate situation in that  
domain-wide ADSP protection then requires publishing ADSP records  
directly below _each_ domain within the parent domain seeking  
protection.  Such an "existence" test also provides little to merit  
the check.  In addition, there would not be any means to curb these  
checks when they prove troublesome for domains being spoofed.

>  Could someone please summarize, perhaps with suggested solutions to  
> allow us to move forward?

A complete ADSP draft has been submitted to permit the review of an  
alternative path forward.

As a separate topic, this ADSP draft eliminates restrictions on DKIM  
identity parameters.  When the message has a valid signature by the  
Key Domain, identity affirmation MUST be based upon the trust of the  
Key Domain to exclude spoofing.  Determining whether the Key Domain  
has affirmed an identity within the message remains a separate, and  
fully independent function, completely unrelated to whether messages  
containing the Author Domain are always signed.

It is no wonder people have become confused, even though the charter  
is rather clear.  Placing restrictions on DKIM identity parameter use  
is clearly at odds with the charter, as this forces DKIM to confirm  
Author identities, even when such confirmation should not occur.   
Unfortunately, compliance with SSP provides unnecessary coercion and,  
at times, requires multiple signatures. : (

-Doug

---

From: Jim Fenton <[email-masked]>
Date: Thu, 22 May 2008 16:13:53 -0700

> NODATA is even more wrong than NXDOMAIN :-(

Definitely, which is largely the reason why I feel this needs to be 
specified in ADSP.

I agree that checking for the existence of MX, A, and/or AAAA records is 
a better approximation to those domains that use email than is mere 
existence (lack of NXDOMAIN).  There will of course be domains having A 
or AAAA records that don't use email, so even if we do the better check 
we don't know for sure that the From address is valid, although they 
could still publish an ADSP record.

But the NXDOMAIN check is attractive in its simplicity, and it does 
cover the case where it isn't possible to publish an ADSP record (since 
the domain would then exist).

My question:  How many domains exist that don't have MX, A, and/or AAAA 
records?  Is the additional coverage of these domains important, or is 
the NXDOMAIN check good enough?

-Jim

---

From: Douglas Otis <[email-masked]>
Date: Thu, 22 May 2008 17:51:39 -0700

On May 22, 2008, at 4:13 PM, Jim Fenton wrote:
>
> I agree that checking for the existence of MX, A, and/or AAAA  
> records is a better approximation to those domains that use email  
> than is mere existence (lack of NXDOMAIN).  There will of course be  
> domains having A or AAAA records that don't use email, so even if we  
> do the better check we don't know for sure that the From address is  
> valid, although they could still publish an ADSP record.
>
> cover the case where it isn't possible to publish an ADSP record  
> (since the domain would then exist).

It would always be possible to publish an ADSP record within a domain  
one controls?

> My question:  How many domains exist that don't have MX, A, and/or  
> AAAA records?

When a domain publishes a single wildcard for reasons separate from  
SMTP, DKIM, or ADSP, the number of domains that could result is about  
1 x 10^74.  The number of valid email domains that do not have an MX  
record represents a small percentage.  It might be worth the effort to  
list these domains to create a white-list used in conjunction with an  
MX record mandate for all new MTAs.  As indicated in the ADSP draft,  
when DNS is attacked and fails to function, crucial SMTP clients  
require their IP addresses white-listed to bypass ADSP requirements  
during such an event.

>  Is the additional coverage of these domains important, or is the  
> NXDOMAIN check good enough?

Would requiring that every domain have an ADSP record published be  
good enough?

Checking for MX and A records offers a means to limit the number of  
domains where ADSP records is required.  In addition, such an ADSP  
check would offer valuable information even when an ADSP record is not  
found.  Ensuring merit in the ADSP discover process in the absence of  
ADSP records seems rather critical for adoption.

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 23 May 2008 02:00:06 +0200

+1

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 23 May 2008 12:44:29 +0100

> beyond the reason that the check is needed.

Or, to put it another way, the site that created the ADSP record has some  
"expectations" that verifiers will thereby catch assorted misuses of his  
domain(s). In order to fulfil the record owner's "expectations" it is  
NECESSARY for the verifier to check for the existence of the domain, and  
therefore our document MUST explain that necessity. It might well add,  
parenthetically, that even if recipients are not trying to verify DKIM  
signatures, it is still a sensible/useful check to perform.

Whether the document uses normative language (MUST/SHOULD) to described  
what verifiers are expected to do is orthogonal to what it says about how  
to do it properly.

---

From: Tony Finch <[email-masked]>
Date: Fri, 23 May 2008 19:01:20 +0100

> are disagreeing with them...so I have to assume that I'm missing something.

My opinion is that the correct way to check for the existence of a mail
domain using just the DNS is described in RFC 2821 section 5. I think this
is simplest from the wordsmithing point of view, likely to be easiest to
implement using existing code, and safer because it avoids the tricky
distinction between NODATA and NXDOMAIN (which even serious experts can
get wrong as John Levine recently illustrated).

AFAICT Wietse thinks the full check is unnecessarily complicated, and
since a purely DNS check cannot give an even remotely accurate answer
(because of the A/AAAA fallback) it might as well be simplified to just
checking for NXDOMAIN. (The additional invalid domains that NXDOMAIN fails
to identify include intermediate labels without records, such as
csx.cam.ac.uk, or domains with records none of which are mail-related,
such as cambridge.ac.uk and almost all of the in-addr.arpa tree.)

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
SOLE: EAST OR NORTHEAST 3 OR 4, INCREASING 5 TO 7, PERHAPS GALE 8 LATER.
MODERATE, OCCASIONALLY ROUGH. RAIN OR THUNDERY SHOWERS. MODERATE OR GOOD,
OCCASIONALLY POOR.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Arvel Hathcock <[email-masked]>
Date: Fri, 23 May 2008 20:05:19 -0500

> are disagreeing with them...so I have to assume that I'm missing something.

Some people (including me) would prefer that the NXDOMAIN check be left 
as it currently is - a required piece of the ADSP algorithm.  There are 
others who think it shouldn't be part of the algorithm at all but should 
be replaced by advisory text stating that the check is recommended.  All 
parties understand the importance that the check be performed.

A compromise proposal has been laid out which is to remove the NXDOMAIN 
step from the algorithm but add text defining ADSP as applicable only to 
domains which actually exist in DNS.  This removes the need for ADSP to 
specify how (or by what means) such a check is determined, does not 
introduce normative language, addresses all the objections yet put 
forth, and still provides a basis for believing that a check will be done.

What we need at this point is for the chairs to call for a straw-poll on 
whether the NXDOMAIN check should be removed from the ADSP algorithm in 
the first place.  I'd personally like to know where the numbers are for 
both sides in this debate.  If we're deadlocked over this NXDOMAIN issue 
then the way forward is consensus on the compromise proposal.  This is 
the only way forward that I can see and this issue is really the last 
thing holding up the spec.

Arvel

---

From: Mark Delany <[email-masked]>
Date: Fri, 23 May 2008 20:44:54 -0700

On May 23, 2008, at 6:05 PM, Arvel Hathcock wrote:

> forth, and still provides a basis for believing that a check will  

If we are to have or to imply a check, then surely the worst  
situation is for a domain advertising ADSP to have no clue as to what  
verifiers will do, apart from the fact that they will do something  
that is effectively random.

Surely a technical spec that guarantees a random result is not a good  
spec.

Rather than create a vague definition of "existence" why not make a  
precise definition but make it optional. A verify MAY do an existence  
test and if they do so, it will be done as follows...

Mark.

---

From: Arvel Hathcock <[email-masked]>
Date: Sat, 24 May 2008 10:16:43 -0500

> test and if they do so, it will be done as follows...

I think an _optional_ existence check actually creates the very problem 
you're talking about - "a domain advertising ADSP to have no clue as to 
what verifiers will do".

 > Surely a technical spec that guarantees a random result is not a good

If the existence check is rendered optional then a random result is 
exactly what we'll get.  You'll have two different implementations of 
the same spec, one doing the existence check, and one not, both able to 
legitimately claim spec compliance, yet each producing a different 
algorithmic result for the same input.

Arvel

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 26 May 2008 11:13:55 +0100

> forth, and still provides a basis for believing that a check will be  

But I want more than a "belief" that a check will be done. It must be made
abundantly clear that a check MUST be done. I do not mind if the exact
details of the check are left open, just so long as verifiers NEVER let
through a message with an Author Domain that does not exist.

---

From: <[email-masked]>
Date: Tue, 27 May 2008 09:01:18 -0400

I am imperfectly signing messages with DKIM that I am sending via my home machine on a dhcp address purported to be from bill.oxley.home.com a vanity non existent domain. According to DKIM that message is to be treated as unsigned, why do you wish to drop it?
Thanks,
Bill Oxley

---

From: Douglas Otis <[email-masked]>
Date: Tue, 27 May 2008 10:29:38 -0700

On May 27, 2008, at 6:01 AM, <[email-masked]> wrote:

> I am imperfectly signing messages with DKIM that I am sending via my  
> home machine on a dhcp address purported to be from  
> bill.oxley.home.com a vanity non existent domain. According to DKIM  
> that message is to be treated as unsigned, why do you wish to drop it?

ADSP changes what is accepted over SMTP when implemented by receiving  
hosts.  Receiving hosts are free to verify that SMTP support records  
exist when the receiving host wish to limit message-addresses to those  
that might be supported by SMTP.  This will not require most sending  
domains to make any change.  Perhaps there will be a few cases where a  
"converted" NNTP (RFC3977) message has been issued by a domain that  
does not support SMTP, or in your case from a made-up domain.  The  
otis-dkim-adsp draft does suggest that a message signed by an non- 
Author-Key-Domain can still be used as a basis for acceptance.   
Stringent tests are unlikely to be imposed by larger providers, at  
least until abuse complaints exceed those of not receiving non-SMTP  
messages over SMTP.  This change will require time.

Expecting the rest of the world to publish records declaring SMTP as  
not supported at every domain is unfair and also does not scale.  The  
ADSP discovery algorithm must be based upon positive assertions of  
support for SMTP.

-Doug

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 29 May 2008 11:27:45 +0100

On Tue, 27 May 2008 14:01:18 +0100, <[email-masked]> wrote:

> home machine on a dhcp address purported to be from bill.oxley.home.com  
> a vanity non existent domain. According to DKIM that message is to be  
> treated as unsigned, why do you wish to drop it?

Many sites will already be dropping such messages..

With ADSP, many more will be dropping it, because there is no way to tell  
that iwhether it should have had an ADSP record (except by building in  
special knowledge that ".com" is a special case). Consider the case where  
your vanity domain was bill.oxley.home.co.uk.