Subject: [ietf-dkim] dkim-base-01: 6.2 - DNS error

From: "Hector Santos" <[email-masked]>
Date: Fri, 14 Apr 2006 10:36:47 -0400

In section 6.2 "Get The Public Key, we have step #2

|   2.  If the query for the public key fails to respond, the verifier
|       SHOULD defer acceptance of this email (normally this will be
|       achieved with a 451/4.7.5 SMTP reply code).

This assumes dynamic SMTP level operational (DATA call out) implementation.
In my view, this is the ideal and the prefer mode of DKIM operation.

But what about post SMTP/transport DKIM implementations?

This requires mail acceptance and DNS query errors/rejections can promote
bounce attacks.  This dilemma also promotes unreliable mail
delivery/notification practice (dropping mail rather than take chance
bouncing mail as required by SMTP).  This is a tough one, hence why there is
the direction to do SMTP level operations since the response is established
and the design resolves all SMTP technical (and legal) expectations.
Nonetheless, DKIM, as a payload solution, there will be operations in post
smtp modes too. So this needs to mentioned/addressed as well.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 18 Apr 2006 13:35:00 -0400

On 04/14/2006 10:36, Hector Santos wrote:
> bouncing mail as required by SMTP).  This is a tough one, hence why there
> is the direction to do SMTP level operations since the response is
> established and the design resolves all SMTP technical (and legal)
> expectations. Nonetheless, DKIM, as a payload solution, there will be
> operations in post smtp modes too. So this needs to mentioned/addressed as
> well.
>
There is another potential issue with this approach if we get to a dedicated 
RR type.  While not an issue when using TXT, there are resolvers that will 
fail to respond when queried with an unknown RR type (no I don't know which 
one, just that it happens).

In this case if the verifier defers based on lack of response, it may be 
indefinite.

Scott K

---

From: Jim Fenton <[email-masked]>
Date: Wed, 19 Apr 2006 20:51:23 -0700

>> In my view, this is the ideal and the prefer mode of DKIM operation.
The fact that this is a SHOULD also seems to presume that the verifier
is the first-hop MTA in the domain (i.e., one of the MX hosts). 
Otherwise it wouldn't matter whether or not you defer acceptance, or
accept and queue the message.  SHOULD seems a little strong in the
general case.
> In this case if the verifier defers based on lack of response, it may be 
This points out another problem:  if a verifier defers verification or
acceptance of a given message, it SHOULD maintain enough state so that
the message may be accepted after some number of retries, so that
messages with key retrieval problems are not rejected entirely.

-Jim

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 20 Apr 2006 07:12:41 -0400

----- Original Message ----- 
From: "Jim Fenton" <[email-masked]>

> messages with key retrieval problems are not rejected entirely.

Jim,

Wouldn't that create a loophole?

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Apr 2006 07:53:58 -0400

> > There is another potential issue with this approach if we get to a
> > dedicated RR type.  While not an issue when using TXT, there are
> > resolvers that will fail to respond when queried with an unknown RR type
> > (no I don't know which one, just that it happens).
> >
For SPF, which is where I ran across this, their problem was that they'd 
specified that queries for SPF records could be made both to TXT and their 
dedicated RR type (Type 99/SPF) either in parallel or in series and that if 
there was no response, then it should be considered a temporary error.  This 
approach resulted in domains that used resolvers that wouldn't respond to a 
Type 99/SPF query always getting a temporary error, even if they had a 
perfectly good TXT record but Type 99/SPF was also queried.

Their solution was to change the draft to say that it was only a temporary 
error if neither TXT nor Type 99/SPF responded.  The idea being that if you 
didn't get any response for TXT (not even NXDOMAIN), then their DNS was down 
and temporary error was an appropriate response.

First, I think we need to make sure that any error condition that we suggest 
deferring message acceptance for is truly a temporary condition.  No DNS 
response to a new RR type does not fall into this catagory.  It isn't clear 
to me whether we need to worry about this now or if it can wait until we get 
to defining the new record?

WRT your point, I agree.  Perhaps we need to add another bit along the lines 
of, "If an email is deferred based on lack of response to the query for the 
public key, the verifier SHOULD NOT indefinitely defer the message.  While 
messages SHOULD be deferred for temporary DNS issues, lack of response to a 
query for a public key alone SHOULD NOT result in messages being permanently 
rejected."

I realize that as written the proposed text is pretty proscriptive for the 
receiver, more so than we generally have been.  I couldn't come up with a way 
to phrase it that wasn't.  I put the word alone in the second sentence to 
leave it open to receivers to consider this along with other factors in local 
policy and perhaps indefinitely defer based on this and other factors.

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Thu, 20 Apr 2006 06:53:51 -0700

On Thu, 2006-04-20 at 07:53 -0400, Scott Kitterman wrote:
> WRT your point, I agree.  Perhaps we need to add another bit along the
> lines of, "If an email is deferred based on lack of response to the
> query for the public key, the verifier SHOULD NOT indefinitely defer
> the message.  While messages SHOULD be deferred for temporary DNS
> issues, lack of response to a query for a public key alone SHOULD NOT
> result in messages being permanently rejected."

Deferred  handling allows recovery from equipment faults which may take
hours or days.  Holding state for such situations could create a sizable
overhead.  Unless there is a reason to believe there may be some issue
at the receiver, continued deferral seems appropriate.

Deferring before attempting to utilize an alternative RR type is a
separate issue.

-Doug

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Apr 2006 10:21:10 -0400

> at the receiver, continued deferral seems appropriate.
For some definition of sizeable I'm sure that's true.  

Greylisting, which requires that the state be held, is popular enough that I 
don't think this overhead is a significant issue for most.  The arguments I 
see against greylisting are almost exclusively about message delivery 
reliability.  I don't see people not using greylisting due to the processing 
overhead.

>From my perspective, as long as the deferrals are for true temporary errors, 
then if the length of the temporary error exceeds the time period for which 
the sending MTA will retry, that's OK (so I don't personally feel obligated 
to mandate that message state be maintained).  My concern is that there are 
DNS errors that are not temporary and so we need to be careful with this 
section not to mandate eventual message rejection due to lack of a reply to a 
query for public key.

Scott K

---

From: Michael Thomas <[email-masked]>
Date: Thu, 20 Apr 2006 15:53:49 -0700

>>query for a public key alone SHOULD NOT result in messages being permanently 
Hold on a sec... with normal 400's the sender is the one who's supposed 
to eventually
give up, not the receiver. For a DNS entry that keeps timing out, why 
should we
special case this?

       Mike

---

From: Douglas Otis <[email-masked]>
Date: Thu, 20 Apr 2006 16:46:41 -0700

On Apr 20, 2006, at 3:53 PM, Michael Thomas wrote:
>>> WRT your point, I agree.  Perhaps we need to add another bit  
>>> along the lines of, "If an email is deferred based on lack of  
>>> response to the query for the public key, the verifier SHOULD NOT  
>>> indefinitely defer the message.  While messages SHOULD be  
>>> deferred for temporary DNS issues, lack of response to a query  
>>> for a public key alone SHOULD NOT result in messages being  
>>> permanently rejected."
>
> Hold on a sec... with normal 400's the sender is the one who's  
> supposed to eventually
> give up, not the receiver. For a DNS entry that keeps timing out,  
> why should we
> special case this?

An apparent server fault of the signer should not have a special case  
where the receive stops deferring.  I agree with Mike, the  
transmitter should report a delivery failure, where the faulty  
equipment is then repaired once reported.  It would seem rather bad  
to effectively ignore such failures, especially when the alternative  
may be to place the message into the junk folder.

-Doug

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 20 Apr 2006 20:26:23 -0400

----- Original Message -----
From: "Douglas Otis" <[email-masked]>
To: "Michael Thomas" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, April 20, 2006 7:46 PM
Subject: Re: [ietf-dkim] dkim-base-01: 6.2 - DNS error

> may be to place the message into the junk folder.

Keep in mind that a DSN (AKA Bounce) is only activated in an Accept/Reject
mode of operation.

With a SMTP level rejection, no DSN is required simply because its not even
technically possible.

It is the Sender's job to send a DSN it its original author or technically
speaking, the return path (2821.Mail From:) once the sender sees a permanent
negative response code (55x).

The main reason this is the preferred way to reject transactions is that it
eliminates the SORBIG generation eViral Bounce Attack which uses a dual
distribution offensive concept:

  Offensive #1 - Throw junk at that Wall and see what sticks

                 Address #1 (RCPT TO:)

  Offensive #2 - Hope it finds receivers that do not check
                 address #1 at the SMTP level, and has a
                 accept/reject mode of operation and
                 throws the junk at someone else.

                 Address #2 (MAIL FROM:)

Older, larger operations with an accept all first concept runs into these
issues. Many run in this mode for scalability reasons.  Many are still
running with a UUCP/SLIP mind set too. :-) I still see support reports on
"how do I stop all these bounces!?" and all it took to eliminate a good
portion of them was to turn on SMTP level Validation!  "Oh, I didn't even
now that option was available."

Well, most, if not all, software do today, offer SMTP level validation
features.  Whether it is the best way for one site to operate, that's a
different question.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Apr 2006 21:24:31 -0400

On 04/20/2006 18:53, Michael Thomas wrote:
> >> query for the public key, the verifier SHOULD NOT indefinitely defer the
> >> message.  While messages SHOULD be deferred for temporary DNS issues,
> >> lack of response to a query for a public key alone SHOULD NOT result in
> >> messages being permanently rejected."
>
Yes, but if the receiver indefinitely defers, it doesn't matter who gives up, 
the eventual result is the same as a 500 something.

The one special case is that no response to a query for the new RR should not 
be treated as no response since some DNS resolvers will not reply at all to a 
query for an unknown RR.  

I don't know if we need to deal with this now or when we are talking about the 
new RR, but we need to make sure that the totality of the specs do not add up 
to defer if you get no response for a query to the new RR.  It's probably as 
simple as don't treat it as no response unless there is no reponse for both 
the new RR and TXT.

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Thu, 20 Apr 2006 19:45:13 -0700

> With a SMTP level rejection, no DSN is required simply because its  

This was about issuing 4xx errors.

-Doug

---

From: Jim Fenton <[email-masked]>
Date: Fri, 21 Apr 2006 17:36:58 -0700

> the eventual result is the same as a 500 something.
The longer I think about this, the more I am of the opinion that we
shouldn't talk about 400-ing in the -base specification.  One of the
advantages of DKIM is that it the signer and verifier can be anywhere
within their administrative units, and in particular the verifier
doesn't need to be one of the MX hosts.  The use of a 400 in this way
only makes sense if the verifier is in the process of receiving the
message from outside; otherwise, the verifier's domain already "owns"
the message so it might just as well accept and queue it.

I'd suggest that we just remove the mention of special responses to key
retrieval failures in -base, and put them in the overview document as a
deployment option.

-Jim

---

From: Dave Crocker <[email-masked]>
Date: Fri, 21 Apr 2006 17:51:23 -0700

> shouldn't talk about 400-ing in the -base specification.  
...
> retrieval failures in -base, and put them in the overview document as a

Exactly right.  This is a specification concerning the message object, not 
concerning message transport, even though it pertains to the object "while in 
transit."

Anything about DKIM that involves SMTP is probably going to need an ESTMP 
option.  I suspect that that is out of scope.

d/

-- 

Dave Crocker
Brandenburg InternetWorking
<http://bbiw.net>
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Jim Fenton <[email-masked]>
Date: Fri, 21 Apr 2006 17:56:40 -0700

> Wouldn't that create a loophole?  
If you mean, how would the verifier know how many deferrals are
acceptable, you're right that's a problem.  If the key can't
(permanently) be retrieved, it's a signature verification failure, and
not in general a reason to reject the message outright, so I don't
consider it to be a loophole in that sense.

-Jim

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 21 Apr 2006 23:26:17 -0400

----- Original Message -----
From: "Jim Fenton" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, April 21, 2006 8:56 PM
Subject: Re: [ietf-dkim] dkim-base-01: 6.2 - DNS error

> consider it to be a loophole in that sense.

I was referring to the bad guy finding out:

        'Hey Fellas! All we need to is create error conditions,
         beat down the verifier enough so that eventually it
         give up and accept our payload.  The beauty?  No
         need to have a correct DKIM signature! Just fake it."

A DKIM Greylisting-like concept is a terrible idea Jim. :-)

IMV, DKIM is a new era of expectations. A domain with a DKIM signature not
only is claiming responsibility for the message, it is also claiming it is
compliant with DKIM.

PS: It is a reason to reject the "transaction."

---

From: "Hector Santos" <[email-masked]>
Date: Sat, 22 Apr 2006 00:03:29 -0400

----- Original Message -----
From: "Dave Crocker" <[email-masked]>
To: "Jim Fenton" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, April 21, 2006 8:51 PM
Subject: Re: [ietf-dkim] dkim-base-01: 6.2 - DNS error

> Exactly right.  This is a specification concerning the message
> object, not concerning message transport, even though it pertains
> to the object "while in transit."

Then there is going to be alot of editing in both the base and Threat
documents because it all and intent and purposed modeled around a SMTP
system.   Then you need to invent define BCP for all other transport
concepts as well.

How will you word step 2, section 6.2?  Simply remove the parenthetical SMTP
related statement?

   2.  If the query for the public key fails to respond, the verifier
       SHOULD defer acceptance of this email.

Ok, SMTP people can see what that means. But you should have insight for
threats too.

   2.  If the query for the public key fails to respond, the verifier
       SHOULD defer acceptance of this email.  Verifiers SHOULD track
       any abuse of non-stop errors.

Or would you like to add Jim's GreyList Concept?

   2.  If the query for the public key fails to respond, the verifier
       SHOULD defer acceptance of this email.  Verifiers SHOULD track
       continuous errors and SHOULD eventually accept the message
       object after a number of tries.

> option.  I suspect that that is out of scope.

SMTP or ESMTP is out of scope?

Lets see what else you need to remove now:

Section 5.1:  Remove SMTP AUTH reference.

Section 6.2:  Removal of SMTP 451 advice

Section 6.5:  Removal of asynchronous SMTP session discussion.
              Removal of response codes.  This section would
              need a massive editing.

Section 8.2:  Removal SMTP Authentication.

Appendix A.3:  Removal of critical statement of inbound SMTP
               server as a verifier.  Maybe SMTP examples
               or show other method examples or remove all
               examples.

Appendix B.1: Removal discussion about how Received: headers
              are added.

And so on and so on.  The entire document reeks SMTP and ESMTP.  This system
is modeled on a SMTP/ESMTP system so who are we kidding?  9.9 out of 10
systems, if not all, will begin using DKIM within a mail framework that
includes SMTP, including GoodMail.

If we want to generalized it, that's fine, but you need to atleast provide
insight and guidance and make sure there is no obvious "got chas" making it
too late to correct in the future not even a BCP will correct.  If it was
that easy, to use incremental BCP documents to correct the design issues
that you knew existed, we wouldn't have these problems today.  20 years ago
I can understand.  Not today, we have hundreds of thousands of man-years
here and we know enough to know what the issues are.  Ignoring them today is
engineering neglect.

---

From: <[email-masked]>
Date: Sat, 22 Apr 2006 08:30:36 -0400

* 2.  If the query for the public key fails to respond, the verifier
   *    SHOULD defer acceptance of this email.  Verifiers SHOULD track
   *     continuous errors and SHOULD eventually accept the message
   *    object after a number of tries.

If the query for the public key fails to respond, the verifier SHOULD defer acceptance of this email. Verifiers MAY track continuous errors and determine the message has a broken signature.

thanks,
Bill

---

From: "Hector Santos" <[email-masked]>
Date: Sat, 22 Apr 2006 08:42:35 -0400

----- Original Message -----
From: <[email-masked]>

> If the query for the public key fails to respond, the verifier
> SHOULD defer acceptance of this email. Verifiers MAY track
> continuous errors and determine the message has a broken signature.

Bill,

Isn't it always the case, the signature always broken (or non-verifiable) in
step 2?

PS: The above step I wrote was basically something it should not say. I view
that as a loophole bad actors will exploit.

---

From: John Levine <[email-masked]>
Date: 22 Apr 2006 13:30:32 -0000

>If the query for the public key fails to respond, the verifier SHOULD defer
>acceptance of this email. Verifiers MAY track continuous errors and determine
>the message has a broken signature.

I don't know what "acceptance" means here.  Since we're talking about
verification, I suspect "verification" would be the appropriate word.

R's,
John

---

From: Jim Fenton <[email-masked]>
Date: Mon, 24 Apr 2006 13:07:12 -0700

> A DKIM Greylisting-like concept is a terrible idea Jim. :-)
I agree with you, and that isn't what I was proposing.

Signatures that can't be verified should be treated the same as invalid
signatures, which in turn should be treated the same, no better or
worse, than signatures that aren't there.  Otherwise an attacker will
just create a signature that references a name server that just doesn't
seem to be online, and if that causes the message to be treated better,
they'll take advantage of it.

I'm also concerned about the burden of maintaining state, possibly even
in the form of queued messages, when the public key is "temporarily
unavailable".  There is the potential to create a DoS attack by making
the verifier queue too much.  The verifier needs to be aware of this and
just give up on verification when things get too busy.
> PS: It is a reason to reject the "transaction."
One problem is that the verifier need not be the edge MTA (MX host), so
the recipient AU may have already accepted the message.  Another is that
rejecting a message outright based on the lack of a signature isn't
going to be practical in most cases for quite some time to come.

-Jim