Subject: [ietf-dkim] Expiration Tag (x=) is required to minimize DNS lookups.

From: "Hector Santos" <[email-masked]>
Date: Tue, 18 Apr 2006 06:39:49 -0400

Summary:

   x= is needed to offer a major DKIM verifier optimization to
   address expiring or revoked keys.

Background:

Overall we have this basic key rollover, revocation and removal time model
such as:

    DAY T1  :  First day mail is signed with KEY1

    DAY T2     Last day mail is signed with KEY1, and
               first day mail is signed with KEY2

    DAY X1  :  KEY1 is revoked (p= is empty)

    DAY Y1  :  KEY1 domain selector is removed from DNS.

Based on this, we can defined what "x=" means by what it COULD be:

   - x=T2

     The time that the signature may verify but no longer consider
     valid?   Currently, per specs,  T2 is X1-7.

   - x=X1

     The time that the signature will no longer verify because it
     was revoked (p= set to empty).

   - x=Y1

     The time that the selector itself is removed from DNS.

The main difference?

In the first two cases, a DNS query is successful and in the last case,
we have NXDOMAIN result.

This NXDOMAIN result is a major protocol DNS consideration for the verifiers
as well as for the wide-scale adoption network considerations.

Regardless of the true intent of the signer, avoiding DNS lookups, when
possible, is an important consideration that should not be ignored.

The preference would be for the responsible key domain to provide
information to fail a transaction without looking up this selector.

I understand that we still have the issue of the time-shifted applications.
I don't think much can be done about this to solve the issue 100% but
signers can do wonders to reduce the problem.

>From a key management standpoint, I think x= includes the representation and
sum of two times:

    x = f(R1 + R2)

where

    R1 = X1-T2

         Key retention time before key is revoked

    R2 = Y1-X1

         Revoked key selector domain DNS retention time.

R1 and R2 is based on the domain's key management/rotation policy, if
any, that has only meaning to the signer.

The point of all this is that from a signer standpoint, it helps to
have a model for rollover and rotations that will help domains decide how to
best manage keys to provide enough retention time for verifiers, including
MUAs.

But from an verifier standpoint, regardless of the domain key policy, if a
x=tag is not provided in the signature, the verifier is left with an high
overhead situation where the only way to determine if a key is valid or not
is by performing the lookup.  I don't think this would be desirable in a
wide-scale DKIM network.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Mark Delany <[email-masked]>
Date: Tue, 18 Apr 2006 14:34:43 +0000

>    address expiring or revoked keys.

This is surely an edge case that Knuth warns us about.

Mark.

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 18 Apr 2006 08:38:35 -0700

>This is surely an edge case that Knuth warns us about.

+1

The current spec has enough language to handle key rollover 
gracefully. Further, if a sender wants to minimize DNS lookups, the 
DNS TTL is the perfect tool, and is already implemented everywhere.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 18 Apr 2006 12:23:40 -0400

----- Original Message -----
From: "Mark Delany" <[email-masked]>

> This is surely an edge case that Knuth warns us about.

-10

You got to be kidding me? :-)

Besides, you mean Hoare and this has to do with premature optimization in
software engineering where you don't have a clue about the design,
especially when you don't have a handle on your performance bottlenecks or
tradeoffs.   Knuth simply repeated what Hoare wrote about approaching
optimization with caution which is basically related to a management
principle based on Pareto.

Paul likes to repeat how the semantics is currently written:

        "Signatures MUST NOT be considered valid if
          the current time at the verifier is past the expiration date."

The x=tag has a specific meaning from the signer and if it clearly says that
the signature is not valid;  the implicit semantic is that there is no need
to do an DNS lookup.

DNS lookups is a source of performance bottlenecks.   That is a sound
concept and not premature.   Experienced software developers  and engineers
will understand this and immediately apply it.   No need to wait "until
later" to "find out" if any of this is true.   It is not difficult to
realized where performance issues may take take. This is a clear
optimization concept.

Now if you want to explain how this optimization is premature I would gladly
love to discuss it.

I can see how it could be premature... iff we change the semantics:  If we
defined the x= tag as x=T2:

     The time that the signature may verify but no longer consider
     valid.  i.e. The key is stale but still valid for signature
verification.

In this case, the optimization is lost because now you have to give
verifiers the option to verify the signature, even if stale.  Do we want to
change the semantics to something that allows a stale key to persist?

Also, in section 6.2 Get the Public Key, it has for step 2:

   2.  If the query for the public key fails to respond, the verifier
       SHOULD defer acceptance of this email (normally this will be
       achieved with a 451/4.7.5 SMTP reply code).

If there is no x= tag, then the verifier has no choice but to send this
temporary response code (451).   However, if the x=tag is present and
expired, then this response code can change to a permanent negative response
code (551).

Therefore, step #2 would have to change to:

   2.  If the query for the public key fails to respond, and no x= tag
       is defined, the verifier SHOULD defer acceptance of this email
       (normally this will be achieved with a 451/4.7.5 SMTP reply code).

       If the query for the public key fails to respond, and the x=tag
       shows an expired time, the verifier  SHOULD permanently reject
       this email (normally this will be achieved with  a 551/5.7.5 SMTP
       reply code).   This condition is met when the  selected has been
       removed from DNS.

So in this case, optimization is lost too,  but the expiration tag is still
a necessary concept   It serves to improve the technical specification for
the SMTP response codes for NXDOMAIN results.

But if we keep the semantics as it is written today, it is an optimization
concept because it doesn't require the overhead in DNS lookup or the payload
hashing overhead.

Look,  I am trying to look for a solution here.  I'm trying to present all
the issues. If you guys want to remove the x=tag,   you will have to address
all the issues with this deprecation, including the high potential for
failed DNS lookups.  I failed to see why we would want to limit signers to
expose this important possible key 'event'.  I see negatives. No positives
in it.

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 18 Apr 2006 12:29:49 -0400

----- Original Message -----
From: "Paul Hoffman" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, April 18, 2006 11:38 AM
Subject: Re: [ietf-dkim] Expiration Tag (x=) is required to minimize DNS
lookups.

> DNS TTL is the perfect tool, and is already implemented everywhere.

But it still an optimization concept:

     - No need to DNS lookup, regardless of TTL state.
     - No need to do any SHA256 Hashing on a potential HUGE payload.

This is clearly an optimization any good engineer will see.

Plus Paul, I respect your input, but this +1 doesn't make sense with the +1
you voted on for:

| 6.2  Get the Public Key
|
|  ...
|
+   0.  If signature has an expiration (x=) tag, check if the signature
+       has expired. Signatures MUST NOT be considered valid if the
+       current time at the verifier is past the expiration date.
+
|   1.  Retrieve the public key as described in (Section 3.6) using the
|       domain from the "d=" tag and the selector from the "s=" tag.

But then again, maybe I don't understand the "nit system" here which is a
high possibility.

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 18 Apr 2006 09:46:39 -0700

>      - No need to DNS lookup, regardless of TTL state.

A cached DNS lookup only goes to a local DNS responder. There is no 
need to optimize past that.

>      - No need to do any SHA256 Hashing on a potential HUGE payload.

This has nothing to do with minimizing DNS lookups, the topic of this thread.

>This is clearly an optimization any good engineer will see.

Then I guess I'm a bad engineer in your eyes. Oh, well.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Steve Atkins <[email-masked]>
Date: Tue, 18 Apr 2006 09:55:04 -0700

>      - No need to DNS lookup, regardless of TTL state.

Yes. DNS is cheap, and NXDOMAIN is cached, though. How often, in
practice, do you think that even an MUA, let alone an MTA, will be
seeing a significant fraction of expired signatures?

>      - No need to do any SHA256 Hashing on a potential HUGE payload.

If the public key has expired from DNS, then you wouldn't be
doing that anyway, would you? (If your MTA were ill-designed
enough to do that it would also be doing so for the significantly
larger fraction of email that has falsified DKIM headers, which would
seem a much bigger problem.)

> This is clearly an optimization any good engineer will see.

I don't really care one way or the other, but describing this
feature as "clearly an optimization" seems to overstate the case.

Cheers,
   Steve

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 18 Apr 2006 13:32:25 -0400

----- Original Message -----
From: "Steve Atkins" <[email-masked]>

> seeing a significant fraction of expired signatures?

But are we ready to presume it is foregone that this will be low?  Low or
high, I still don't think it substracts from the optimization consideration
based on the current semantics of the protocol.

What are saying anyway?  That a DNS lookup should always be done anyway,
therefore a x=tag or any other short-circuiting concept that would provide
the same optimozed result is not required?

A side nit: We have no 100% reliable conclusion that all verifying sites
will have the same 100% reliable DNS operations.

> seem a much bigger problem.)

I think this all depends on the semantics and what time x= points too in the
cycle of key life and in combination with TTL, who know what chaotic timing
and caching issue we are dealing with.

But yes, you are correct, The current semantic for making a key invalid if
when the key p= tag is empty or a NXDOMAIN and in this case, there is no
hashing overhead.  Also keep in mind that this (NXDOMAIN) means with Section
6.2 Step 2.

> feature as "clearly an optimization" seems to overstate the case.

Probably. But it also depends on what a verifier is experiencing and I am
for using our insights to make this an efficient system without changing the
end results.   The concept is clearly an optimization because the results
are not changed.  It is clear step to removing any additional processing or
consideration required.

I am not disagreeing that it net effect is major. By no means, but if you
don't have to do an DNS lookup, and the specs has an implicit semantics to
this effort, and the end results are the same whether you do or not, then
why enforce it?

Thanks

---

From: Steve Atkins <[email-masked]>
Date: Tue, 18 Apr 2006 10:54:02 -0700

> based on the current semantics of the protocol.

It adds complexity. That's a cost (and given the confused semantics,
quite a significant one).

The only benefit it adds is the ability to avoid a DNS lookup in some
cases. Based on how I believe that the system will be used by senders
and recipients I think that will be an extremely rare case. <1%.

Even if it saved a DNS lookup 20% of the time I don't think it would be
a good tradeoff.

> will have the same 100% reliable DNS operations.

The only possible design failure that could have any effect on this  
particular
case is a recursive resolver that caches results for significantly  
longer than
the TTL. (Failure to handle NXDOMAIN correctly would increase external
traffic somewhat, but, eh.)

I'm not aware of any widely deployed implementations DNS caching  
resolvers
that fail in that way.

> this effort, and the end results are the same whether you do or  

It makes the signature header bigger. That's mildly bad, and in the  
common
case (bulk mail) the increase in payload size will outweigh the DNS  
traffic,
so any argument based on reduction of network traffic is not convincing.

The network traffic issue means hat the really large mailers will  
likely not
use the feature (why do you think they use domain names like m0.net?).
That means recipients will not see it on the vast majority of their  
bulk mail
load.

It adds complexity to the validator, and simpler is always better  
than more
complex, both in terms of having people write code, having them agree
on standards and deploying systems.

Unless I'm misunderstanding something it adds a clock-skew vulnerability
which wasn't previously there.

It's also a rat hole under a bike shed.

If the x= tag only value is the ability to occasionally avoid DNS  
lookups
(as opposed to it having some other semantic value) then the additional
complexity doesn't seem worth it.

I've not chimed in before because I really, really don't care. It's  
clearly
a misfeature in the protocol, but arguing about it for a month and
delaying the eventual draft by at least that long is far, far more  
damaging
than adding one extra piece of unneeded chrome.

Simple, functional and deployed beats complex, over-designed and
theoretical.

Cheers,
   Steve

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 18 Apr 2006 15:54:59 -0400

----- Original Message -----

> Simple, functional and deployed beats complex, over-designed and

Agree, but I also recognize the school of thought that simplicity promotes
complexity which promotes over-designed functionality when resolving
foreseeable but neglected deployment problems.  Sorry, but its my old
corporate trained "Getting it right the first time" mindset, that I am
quickly coming to terms doesn't work very well in the IETF WG arena. This
truncated response is an example of that realization. :-)

I don't believe we have a complex situation with the x=tag.  You have the
same time-shifted issues with it or without it, but in my view, even more
issues are introduced without it.

If you don't think so, see section 6.2 step 2.  Do you any rat-hole there?
:-)

Thanks