Subject: Re: [ietf-dkim] 2821bis and AAAA (was: Thoughts on latest SSP draft)
From: Douglas Otis <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: Frank Ellermann <[email-masked]>, [email-masked], [email-masked]
Date: Fri, 28 Sep 2007 10:54:30 -0700
List-Id: <ietf-smtp.imc.org>

On Sep 27, 2007, at 1:36 PM, John C Klensin wrote:

[In so many words, I said  deprecating A and AAAA records for SMTP  
discovery would allows email policy records be limited to MX record  
locations.]

> I think you have the cart and horse turned around backward.  If  
> (and I'm not going to express an opinion at this point), one really  
> needs MX records if DKIM (and its near and distant header-signing  
> relatives) are to be supported in a reasonable and efficient way,  
> then it would be perfectly sensible to impose that requirement on  
> DKIM users.  In other words, one makes provision, in the DKIM  
> specs, that,
>
> 	(i) if one is going to insert DKIM header records, one
> 	MUST have MX records for the appropriate hosts.

Email policy records are published to cover cases for when some added  
email requirement is _not_ met.  Email policy records pose a degree  
of risk and added overhead when they must be separately discovered.   
It would be much safer to limit a discovery process to that of MX  
records.  Several possible policy discoveries can thereby be  
eliminated by mandating use of MX records for the discovery of all  
public (port 25) SMTP servers.  Any desired policy would then be  
adjacent to discovery (MX) record.  This scheme could work with many  
other protocols.

> 	(ii) if one encounters DKIM header records, does an MX
> 	lookup, and does not get one or more MX records back,
> 	then one SHOULD just give up and treat the DKIM records
> 	as trash (whatever that happens to imply).

Woe! Turn the cart around.  Policy is needed when additional  
requirements are desired.  A DKIM header is better confirmed with the  
offered key.  Not having a DKIM header is when policy is needed.

> This makes the "mandatory MX" issue a DKIM (and friends) issue, not  
> a requirement that zillions of hosts that do "MX, then address"  
> lookups consistent with 2821 (and 1123, and...) change what they  
> are doing because of some proposed words in 2821bis that change a  
> 20-odd-year-old spec.  Won't happen, whether 2821bis is changed or  
> not.

Email is faltering under an ever increasing onslaught of organized  
abuse often using forged domains.  2821bis is proposing a change  
where now AAAA records are to be allowed for discovery.  This change  
adds to the overhead related to discerning valid email domains for  
little other benefit.

Perhaps soon messages from domains lacking MX records will be  
refused.  Even DSNs are being dropped.  Times have changed.  It is  
far less expensive and much safer to not wonder whether an address  
record might locate an SMTP server.  20 years ago an address record  
at an email domain was much more likely to point to an SMTP server.   
This is no longer a safe assumption.

-Doug

Date: Thu, 27 Sep 2007 16:36:04 -0400
From: John C Klensin <[email-masked]>
To: Douglas Otis <[email-masked]>, Frank Ellermann <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: [ietf-dkim] 2821bis and AAAA (was: Thoughts on latest SSP draft)
Sender: [email-masked]

(someone will probably need to forward this to the DKIM list;
I'm not subscribed to it)

--On Thursday, 27 September, 2007 12:12 -0700 Douglas Otis
<[email-masked]> wrote:

>...
> In addition to checking for a policy record of some sort, A,
> AAAA, and MX records must also be queried.  Every message
> referencing a randomly spoofed domain will thereby lead to a
> series of expensive DNS transactions.  DNS overhead could be
> reduced by 2/3 thirds at least by requiring an MX record for
> acceptance.  This would preclude A or AAAA record for
> acceptance.  The impact of this change should be limited to
> message acceptance.

I think you have the cart and horse turned around backward.  If
(and I'm not going to express an opinion at this point), one
really needs MX records if DKIM (and its near and distant
header-signing relatives) are to be supported in a reasonable
and efficient way, then it would be perfectly sensible to impose
that requirement on DKIM users.  In other words, one makes
provision, in the DKIM specs, that, 

	(i) if one is going to insert DKIM header records, one
	MUST have MX records for the appropriate hosts.  
	
	(ii) if one encounters DKIM header records, does an MX
	lookup, and does not get one or more MX records back,
	then one SHOULD just give up and treat the DKIM records
	as trash (whatever that happens to imply).

This makes the "mandatory MX" issue a DKIM (and friends) issue,
not a requirement that zillions of hosts that do "MX, then
address" lookups consistent with 2821 (and 1123, and...) change
what they are doing because of some proposed words in 2821bis
that change a 20-odd-year-old spec.  Won't happen, whether
2821bis is changed or not.

#include <some cliche about rocket science>

      john

Cc: [email-masked], [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: [ietf-dkim] 2821bis and AAAA (was: Thoughts on latest SSP draft)
Date: Thu, 27 Sep 2007 12:12:13 -0700
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Sep 27, 2007, at 1:38 AM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> Email policy solutions assume policy can be asserted for parent  
>> domains and all sub-domains.
>
> Not for SPF, and it took me some time to figure this out in 2004:
>
> A receiver getting mail with an alleged evelope sender [email-masked]  
> can reject it as soon as it's clear that there's no MX, no A, and  
> no AAAA for y.example.  Zone cuts, DNS tree walks, or SPF aren't  
> needed for this decision.  Email policy solutions like SPF don't  
> need to worry about "impossible" sub-domains, the receicers are  
> anyway obliged to reject mail from an "impossible" [email-masked] at  
> least temporarily, otherwise they couldn't report later errors.

Increasing levels of spam is requiring receivers to add resources.   
Additional transactions for random domains will amplify the need for  
additional resources.

In addition to checking for a policy record of some sort, A, AAAA,  
and MX records must also be queried.  Every message referencing a  
randomly spoofed domain will thereby lead to a series of expensive  
DNS transactions.  DNS overhead could be reduced by 2/3 thirds at  
least by requiring an MX record for acceptance.  This would preclude  
A or AAAA record for acceptance.  The impact of this change should be  
limited to message acceptance.

>> a policy record needs to be published at every existing node
>
> For some definitions of "existing" by "has MX, A, or AAAA", yes.
>
> OTOH receivers could check that the A / AAAA without MX cases have  
> an SMTP listening on port 25, and otherwise decide that an alleged  
> envelope sender address isn't acceptable for the purpose of sending  
> later error reports to the originator "as indicated by the reverse  
> path", the keystone of SMTP.  This check might be simpler than  
> trying to find and interprete sender policies, and the A / AAAA  
> without MX cases should be rare.

Attempting to also connect to an SMTP server would also require  
receivers to increase resources.  To limit the impact of this  
increase, specialized caching would be needed.  Unlike DNS, each SMTP  
connection requires TCP setup.  Many of the inbound systems are  
overloaded with spam, where without retries, a failure to connect  
will reduce email's delivery integrity.  In addition, inbound  
connections should be held during this process.

>> Publishing a policy record adjacent every existing node will be  
>> difficult to manage.
>
> It depends on what you want.  If you never use any @www.example  
> address in email, and there's no MX for or SMTP at www.example, you  
> could decide that's good enough.  An admin for bank.example likely  
> decides that www.bank.example has to be covered.

The goal would be to limit spoofing.  This requires every address  
record to have an adjacent policy record that indicates this address  
record is not used for email.  Policy records of every sort would  
need to be published adjacent to each address record.  Cluttering  
zones with email policy records is sub-optimal.  This problem is best  
resolved by requiring MX records.

>> Walking even a small portion of the label tree might negatively  
>> impact SLD and TLDs.
>
> Yes, tree walks are odd when they cross zone cuts, that's not  
> limited to TLDs and SLDs like ac.uk.

Currently checking for MX, A, and AAAA records would be required to  
avoid tree walking.

>>> AFAIK it's not planned to remove the "A fallback" from 2821bis,  
>>> in fact 2821bis will augment all discussions of A records with  
>>> AAAA for IPv6 compatibility.
>
>> AAAA record discovery could be excluded in 2821bis and require the  
>> use of MX records.
>
> Okay, for that section 2.3.5 at the moment offers:
>
> | Only resolvable, fully-qualified, domain names (FQDNs) are
> | permitted when domain names are used in SMTP.  In other words,
> | names that can be resolved to MX RRs or address (i.e.  A or
> | AAAA) RRs (as discussed in Section 5) are permitted, as are
> | CNAME RRs whose targets can be resolved, in turn, to MX or
> | address RRs.
>
> For your proposal it should say ..."MX RRs or IPv4 addresses (i.e.  
> A) are permitted"..."resolved, in turn, to MX or IPv4 address RRs."

At least.  It would be nice to obtain enough consensus that would  
allow use of A records to hereby be deprecated.  I suspect this must  
be done in a separate draft, or by de-facto behaviour.

> And the details in section 5 would have to fixed, limiting the "no- 
> MX" procedure to the legacy IPv4 case, plus something about IPv4  
> addresses as they're seen in an IPv6 environment.

That would still demand an inordinate level of policy records.

>> At some point, even A records for discovery should be deprecated.
>
> I'm pretty sure that the 2821bis author would insist on saying that  
> this point isn't *now*.  But for IPv6 (and any IPvFuture) there  
> might be enough wiggle room.

I agree.

-Doug

Cc: [email-masked], [email-masked]
From: Douglas Otis <[email-masked]>
Subject: Re: [ietf-dkim] Re: Thoughts on latest SSP draft
Date: Wed, 26 Sep 2007 15:31:42 -0700
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

On Sep 22, 2007, at 6:38 PM, Frank Ellermann wrote:

> Douglas Otis wrote (2007-07-25 on the DKIM list):
>
>> At this point in time, it should be rather rare for incoming SMTP  
>> servers to depended upon a AAAA record for locating their servers.  
>> The DKIM WG should push to have A or AAAA record discovery  
>> deprecated.  Deprecating address record discovery techniques will  
>> eventually simplify where policy needs to be published.  At some  
>> point in the future, not publishing an MX record for the  
>> originating domain might cause a message to be rejected.
>
> Hi, scanning old messages I saw that you said this more than once  
> on the DKIM mailing list.  I'm also aware that Meng Weng Wong and  
> others proposed something in this direction on the SPF and MARID  
> list back in 2004.  It's also related to the expired "null-MX" I-D,  
> and because of that it might affect various "NOMAIL" solutions  
> (4408 "v=spf1 -all" and Phil's I-D.hallambaker-nomail).

Email policy solutions assume policy can be asserted for parent  
domains and all sub-domains.  This is done with DNS wildcard records,  
by walking some portion of the DNS tree, or checking for discovery  
records.  Any existing node within DNS prevents synthesis of a DNS  
wildcard policy record.  As such, either the domain tree must be  
walked, a policy record needs to be published at every existing node,  
or at every possible discovery record.  Publishing a policy record  
adjacent every existing node will be difficult to manage.  Walking  
even a small portion of the label tree might negatively impact SLD  
and TLDs.  The level of impact would depend upon consistency of the  
implementation of the negative caching of the missing address record  
transactions.  Some domains disable negative caching for faster  
transient error recovery.

> I'm not strictly against it, quite the contrary.  *But* AFAIK it's  
> not planned to remove the "A fallback" from 2821bis, in fact  
> 2821bis will augment all discussions of A records with AAAA for  
> IPv6 compatibility.

AAAA record discovery could be excluded in 2821bis and require the  
use of MX records.  One solution for resolving whether email policy  
might apply can then be validated by discovering an MX record.  At  
some point, even A records for discovery should be deprecated.  The  
presences of address records should not necessitate the publishing  
email policy.

> If you and others feel that the no-MX fallback should be limited to  
> IPv4 in 2821bis, as it arguably is in 2821, then please say so on  
> the SMTP list.  Fixing the SMTP spec. for IPv6-only senders is  
> something between tricky and impossible, and your proposal could  
> shift this task from impossible towards tricky.

The impact of the deprecation would not cause discovery to fail, as A  
records could still be used.  The impact would likely be felt when  
acceptance of a message fails due to the lack of an MX record.    
Systems sending diagnostic messages within an organization might be  
white-listed to alleviate the publishing of an MX record.  Often,  
these systems are not intended to communicate with some random set of  
domains.

-Doug

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  accept everything, send DSNs as needed later (was: acceptings partically a message)
Date:  Sun, 23 Sep 2007 07:09:38 +0200
Organization:  http://purl.net/xyzzy
Lines: 30
Sender: [email-masked]

John C Klensin wrote (2007-07-17):
=20
> Arnt Gulbrandsen <[email-masked]> wrote:
>>=20
>> My code to implement this does
>>=20
>>      if all rejected
>>         answer "CR LF . CR LF" with 5xx
>>     else if all accepted
>>         2xx
>>     else
>>         2xx, and send a DSN with the exact details
>>=20
>> The third case is very, very uncommon in practice, so I'm not
>> all that worried by its ill effects.
=20
> yep.  It is also no worse than "accept everything, send DSNs as
> needed later" which, for better or worse, 2821 explicitly
> contemplates.

BTW, while I'm still catching up with old messages:  The BSI, a
federal agency for IT security in Germany, allegedly recommends
that all federal organizations in Germany cease to send NDRs,
ignoring RFC 2821 for better or worse:

<http://www.heise.de/ix/news/meldung/96350> (German article)
<http://purl.net/net/de2en/heise.de/ix/news/meldung/print/96350>
The second link is a crude translation provided by Google.

 Frank