Subject: Re: Fwd: Re: [ietf-dkim] Introducing myself

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 06 Dec 2006 13:13:08 -0000

On Mon, 06 Nov 2006 10:47:45 -0000, Charles Lindsey <[email-masked]>  
wrote:

> On Sat, 04 Nov 2006 02:03:38 -0000, John Levine <[email-masked]> wrote:

That was quite some time ago, so to refresh your memories, I had been  
claiming that DKIM-base would fail to verify if some message had its  
Content-Transfer-Encoding changed en route, and that it proposed to get  
around this by saying that all messages SHOULD be sent as 7bit, or encoded  
into 7bit. In these days when 8BITMIME is now almost universally supported  
and widely used (with BINARYMIME coming along as well), that seemed to be  
a very backward step. So I proposed a canonicalization that would reverse  
all those encodings before hashing.

>> You can sign whatever you want, but if the message is 7bit, your
>> signature is more likely to survive transit to the verifier.

But of course I don't want them to be "likely to survive". I want a system  
that is robust enough that they "always survive".
>
>> DKIM doesn't understand MIME.  If DKIM signers and verifiers had to
>> unpack MIME parts they would be orders of magnitude more complicated.
>> In practice, I think that nearly everyone uses the simple body canon
>> anyway.
>
> Not at all. Going through the MIME structure of a message body and  
> undoing
> all Q-P or Bas64 encodings is fairly straightforward, and if you hash and
> sign the result of doing that, then it is guaranteed to pass straight
> through all those systems which (quite legitimately under RFC 1652)
> re-encode stuff en route, without breaking the signature. I shall try to
> write a demonstration implementation in the next day or so, and it
> certainly won't be "orders of magnitude more complicated".

So it was an issue of whether such a canonicalization really would be  
"orders of magnitude more complicated". Anyway, I have been working off  
and on on this since then, and I have written a demonstration  
implementation, as promised, of what it would take, which you can find at  
<http://www.cs.man.ac.uk/~chl/uncode/uncode.html>.

It is less that 140 lines of Perl (excluding comments and empty lines).  
Hardly any "orders of magnitude" in evidence there.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: <[email-masked]>
Date: Wed, 6 Dec 2006 09:58:55 -0500

Nice code, now during your testing how many messages (average message size today 3k) per second were you able to process and on what machine. I need something that can do about 1200 messages per second per second.
Thanks,

Bill Oxley
Messaging Engineer
Cox Communications
404-847-6397

---

From: [email-masked] (Wietse Venema)
Date: Wed, 6 Dec 2006 11:36:32 -0500 (EST)

> all those encodings before hashing.
...
> Hardly any "orders of magnitude" in evidence there.

Actually, it's 128 lines. But that's a minor detail.

With real production MTAs such as Sendmail and Postfix, the MIME
processor takes about 900 lines of C code (sans comments, formatted
in the K&R coding style). That's 900 lines of opportunity for error.

My concern is about interoperabilitity.  With the present design,
senders and recipients who exchange QP or Base64 content only need
bug-compatible MIME processors in their respective MUAs.

When DKIM signers and verifiers are requird to up-convert QP or
Base64 content before computing signatures, we also require that
all DKIM signers and verifiers have bug-compatible MIME processors.
That is, bug-compatible with every MUA.

Introducing this extra requirement is unlikely to help with the
successful adoption of DKIM.

	Wietse

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 06 Dec 2006 16:58:41 +0000

So it looks to me like you're suggesting a new c14n algorithm for
the WG to consider.

If so, I think the proper course would be to write that up as a
draft-lindsey-dkim-foo and then we can see if we like it or not.
(While having the perl code is great, its not necessarily the
easiest thing for everyone to analyse.)

And we do have pluggability in base for c14n, so that if you're
right, and this new c14n produces significantly less brittle
signatures, then your new c14n algorithm would probably get
adopted fairly quickly in any case.

That suggests not trying to replace what we have in base, but
rather proposing your new scheme as an alternate. In the meantime,
we should proceed with getting more deployment experience based
on the current approach.

To me, that seems like a better approach than trying to make such
a significant last minute change, which I would have a problem with
in any case, mainly on the grounds that we'd need a reasonable time
to do a security analysis of any new c14n proposal.

Does that sound like a way forward?

Stephen.

Charles Lindsey wrote:
> On Mon, 06 Nov 2006 10:47:45 -0000, Charles Lindsey 
> <[email-masked]> wrote:
> 
> around this by saying that all messages SHOULD be sent as 7bit, or 
> encoded into 7bit. In these days when 8BITMIME is now almost universally 
> supported and widely used (with BINARYMIME coming along as well), that 
> seemed to be a very backward step. So I proposed a canonicalization that 
> would reverse all those encodings before hashing.
> 
> But of course I don't want them to be "likely to survive". I want a 
> system that is robust enough that they "always survive".
>>
> implementation, as promised, of what it would take, which you can find 
> at <http://www.cs.man.ac.uk/~chl/uncode/uncode.html>.
> 
> --Charles H. Lindsey ---------At Home, doing my own thing------------------------ 
> 
> Tel: +44 161 436 6131                      
> NOTE WELL: This list operates according 
> tohttp://mipassoc.org/dkim/ietf-list-rules.html
>

---

From: Dave Crocker <[email-masked]>
Date: Wed, 06 Dec 2006 09:51:30 -0800

...
> successful adoption of DKIM.

Canonicalization has been recognized as a *very* challenging topic for at least 
15 years of Internet mail work.  It was a major focus for MIME, it was a major 
focus for DomainKeys and it was a major focus for DKIM.  (I'm sure it's been a 
major focus elsewhere, but this list will suffice.)

My own summary is that we know we can trade between high qualitysimplicity and 
robustness/fragility. There seems to be no perfect choice.

This invites infinite discussion.  We should decline the invitation.

DKIM permits more than one canonicalization scheme to be defined.  The current 
set is the result of lengthy discussion and even experience. As Stephen notes, 
the list can be extended, without requiring that we replace any existing entry.

If the current set proves problematic *in the field* then we can add more... later.

We most certainly do *not* need to add consideration of additional schemes to 
the current public discussion, given that the focus now should be on adoption 
and use of the current specification that has benefited from a couple of years 
substantial effort.

That said, as Stephen notes, anyone is of course free to write an Internet-Draft 
proposing additional schemes.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: John Levine <[email-masked]>
Date: 6 Dec 2006 20:25:39 -0000

>that is robust enough that they "always survive".

As I recall, we agreed that is specifically not a goal of DKIM.  If
you want a signing scheme designed to survive all sorts of hostile
gateways, there's already S/MIME.  The limited c18n in DKIM is
intended to survive only the most common sorts of transit relays.

Honestly, I'd be more inclined to go in the other direction and
deprecate the relaxed body c18n, since it is my impression that the
simple one works in practice for nearly any message that relaxed does,
and relaxed is more complicated and may be vulnerable to ASCII art
hacks.

R's,
John

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 07 Dec 2006 09:56:41 -0000

On Wed, 06 Dec 2006 14:58:55 -0000, <[email-masked]> wrote:

> Nice code, now during your testing how many messages (average message  
> size today 3k) per second were you able to process and on what machine.  
> I need something that can do about 1200 messages per second per second.
> Thanks,

I haven't done any speed testing yet, but will try some now.

But bear in mind my program was intended to show clearly what needs to be  
done. You would not write a production version in Perl, though you might  
use that Perl version as a guide to ensure that the production version had  
covered all the corners.

But having said that, the question to ask is whether adding this feature  
is going to make things significantly slower that all the other extra  
stuff you have to add to implement DKIM. My expectation is that the  
machine cycles for finding and reversing the MIME encodings will be much  
less than the machine cycles consumed by performing the SHA-256 hash,  
which has to be done over the whole body in any case.

But that bit is easily checked with my program, since the decoding and  
hashing within the Perl library are written in C anyway.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 07 Dec 2006 10:07:21 -0000

> easiest thing for everyone to analyse.)

Yes, I might do that in due course, but we need to toss the idea around  
here a little bit more first (as we seem to be doing).
> adopted fairly quickly in any case.

My concern is that people will tend not to implement stuff that is not in  
the base standard. And any c14n has to be implemented at both ends in  
order to be of any use.

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 07 Dec 2006 10:27:27 +0000

Charles,

Charles Lindsey wrote:
 > Yes, I might do that in due course, but we need to toss the idea
 > around here a little bit more first (as we seem to be doing).

Good. I think that that's the right approach (and has the nice
side-effect of being a good check on whether we've done the
pluggability stuff well in base).

If draft-lindsey-dkim-better-c14n were ready for discussion in
Prague or before that'd be about right, IMO.

> My concern is that people will tend not to implement stuff that is not 
> in the base standard. And any c14n has to be implemented at both ends in 
> order to be of any use.

I understand. However, in the case of xmlsig I think the 2nd c14n
spec (which was demonstrably more useful) basically won out even
though it was done about a year later.

So there is an existence proof that, where there's a real benefit,
then the market moves to use c14n that works.

And in this case, as was pointed out, and I think, agreed by you
above, yours is a late, though interesting, proposal, that needs some
more work, (e.g. in terms of security & performance analysis, field
testing etc.). That's a fairly strong argument for going with the
current proposal in base since that's already been through those
hoops.

Cheers,
S.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 07 Dec 2006 10:25:35 -0000

On Wed, 06 Dec 2006 20:25:39 -0000, John Levine <[email-masked]> wrote:

>> system

Unfortunately, S/MIME already suffers from exactly the same  
bug^H^H^Hfeature, which is why I was surprised to see that DKIM has  
followed that same broken path.

DKIM will have no effect on the present spam/phishing/malware scene unless  
it is widely adopted. It will not be widely adopted unless it is seen to  
be robust. In particular, it will not be adopted in countries (esp those  
in Asia) where the character sets used are totally unlike ASCII if it can  
only be made to work by forcing everything to be sent as 7bit. They just  
cannot survive in an environment where textual messages 'on the wire'  
cannot easily be read in that form. They will just resort to "send 8bits  
anyway" which is already happening, even with headers, to a large extent,  
because 99.9% of the time it actually works like that without problem.

That is why the parallel EAI effort has been mentined so often in these  
discussions, because it is pulling in exactly the opposite direction to  
this WG, and it is the Chinese and the Japanese who are pulling the  
hardest.
> and relaxed is more complicated and may be vulnerable to ASCII art

It has been standard practice in PGP, since its inception, to ignore  
trailing whitespace (unless you explicitly ask it not to). I have never  
heard of a Bad Guy who managed to create a correctly signed message  
message with usefully different content by taking advantage of that.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 07 Dec 2006 10:41:32 -0000

On Wed, 06 Dec 2006 16:36:32 -0000, Wietse Venema <[email-masked]>  
wrote:

>> Content-Transfer-Encoding changed en route,....

> Actually, it's 128 lines. But that's a minor detail.

Hmmm! I actually counted 138 :-( .

> bug-compatible MIME processors in their respective MUAs.

I have little sympathy with implementations that don't adhere to standards.
> That is, bug-compatible with every MUA.

However, it is not as bad there as you suggest. Provided the c14n is  
correctly implemented at both ends (and there is never any room for  
incorrectly implemented c14n), it does not matter if some buggy MUA  
produces bad Q-P or Base64, because the c14n will treat it the same way at  
both ends. But the specification of the c14n has to be very tightly drawn.

It *does* matter if some MTA that downgrades 8BITMIME en route gets it  
wrong. And I need to look into that (I have the source code of sendmail to  
hand). Fortunately, RFC 2045 defines pretty exactly how Q-P and Base 64 is  
to be done, especially as regards which CRLFs belong to the text being  
(en/de)coded, and which to the structure of the multipart.

---

From: Hector Santos <[email-masked]>
Date: Thu, 07 Dec 2006 05:45:33 -0500

> in the base standard. And any c14n has to be implemented at both ends in 

Speaking for myself, I can almost guarantee you, from on my long held 
mail product design philosophical and ethical standpoint, we are not 
going to implement anything that will require us to a) touch the 
original mail integrity of the message and b) more importantly anything 
that will require us to 'bring apart' a MIME message for the purpose of 
DKIM signing.

This pretty much a non-starter for us.  DKIM is not going to be the 
"thing" that will change our product design in screwing around with 
original mail integrity.  The exception is any CR/LF conversion on 
original submissions, but in principle passthrus will never will never 
be touched, altered or whatever any in shape or form and I hope others 
don't begin get into this dangerous game as well.  I have confident this 
is not going to happen.

Please excuse me for asking this, but are you attempting to develop a 
"total product solution" with far reaching change requirements across 
the board or a backend protocol that fits with the current framework as 
best it can, yet offers the highest adoption potential?

---
HLS

---

From: Hector Santos <[email-masked]>
Date: Thu, 07 Dec 2006 06:14:16 -0500

> followed that same broken path.

I don't believe it did not follow the same path. Its one reason we are 
interested in its completion and implementation. The MUA version is one 
that was easily seen as broken simply because MUA has no control on 
backend server hosting systems.  That was a no-brainer.

> DKIM will have no effect on the present spam/phishing/malware scene 
> unless it is widely adopted. 

I disagree with that premise. DKIM can have a very highly effective 
domain protection potential depending on the domain signature signing 
policies.  Its not a cure all, but can be highly effective in 
eliminating the obvious failures.  The DSAP I-D illustrates this:

http://tools.ietf.org/wg/dkim/draft-santos-dkim-dsap-00.txt

 > It will not be widely adopted unless it is seen to be robust.

which says that exclusivity does plays a big role in DKIM success. 
Relaxing policies has always failed (been exploited) protocols with 
relaxed provisions in practice.  This idea is like adding $25K in home 
security, yet, you leave a key under a potted plant on the porch.

> In particular, it will not be adopted in countries 
> (esp those in Asia) where the character sets used are totally unlike 
> ASCII if it can only be made to work by forcing everything to be sent as 
> 7bit. 

You're assuming MUA are involved. Right? This all can be 100% 
transparent if we keep DKIM out of the Presentation Layer.

 > They just cannot survive in an environment where textual messages
> 'on the wire' cannot easily be read in that form. They will just resort 
> to "send 8bits anyway" which is already happening, even with headers, to 
> a large extent, because 99.9% of the time it actually works like that 
> without problem.

Ok, if a transformation has to be done, then the DKIM owner will know 
how do deal with this when it finds out that a failed outcome can not be 
avoided.   Or maybe they have worked out a route so that there will be 
known middle ware who will alter and but also resign the mail.  As long 
as the final system validates and authorizes the domain signature , its 
fine.  You can't use certain technologies in all systems and that 
applies in many areas.

> this WG, and it is the Chinese and the Japanese who are pulling the 

I don't see how it applies and IMV, its no different when there are 
other mid-stream transformation requirements and a DKIM owner who is or 
not aware of it.  Regardless of the transformation requirements, whether 
its for Americans, British, Chinese or Martians, DKIM still needs 
knowledge of any mail integrity change and signing/resigning entities.

===
HLS

---

From: Barry Leiba <[email-masked]>
Date: Thu, 07 Dec 2006 10:26:40 -0500

Please stop this thread with this subject line, and see my next message.

--
Barry Leiba, DKIM Working Group chair  ([email-masked])

---

From: Barry Leiba <[email-masked]>
Date: Thu, 07 Dec 2006 10:31:01 -0500

If this discussion is to continue, please use this subject line (or 
another one like it that you prefer).  Looking for it and keeping track 
of it with that "introducing myself" subject is silly.

Apart from that, I'd like to see the discussion stop (though we chairs 
are not going to put any feet down about it yet) until there's an 
Internet Draft documenting a new proposed canonicalization.  Then we 
have something clear to discuss.

Also note that the base spec has completed last call and is scheduled 
for the next IESG telechat on 12 Dec.  Charles has brought this up in 
last-call comments, and the IESG will be considering it, along with the 
work the WG has already done on canonicalization.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 08 Dec 2006 13:38:23 -0000

Note change of Subject at request of Barry Leiba. The original thread
(Introducing Myself) was fine when I first joined this list with a long  
list
of issues I was concerned about, but it is well past its sell-by-date now.

In due course, this needs an I-D draft with a definite proposal, but the  
issues
could use a little more informal discussion first.

>> In particular, it will not be adopted in countries (esp those in Asia)  
>> where the character sets used are totally unlike ASCII if it can only  
>> be made to work by forcing everything to be sent as 7bit.
>

No. AIUI DKIM is supposed to operate mainly between the 1st MTA after the  
sending MUA and the last MTA before the receiving MUA (though smarter MUAs  
that sign their own, or verify their own are welcome to try). But  
nevertheless the form of the message as processed by MTAs 'on the wire'  
does get looked at and needs to be looked at for all sorts of odd reasons,  
and if the people looking at it find it squashed into 7bits that they  
cannot readily interpret, they are going to revolt (and are doing so). It  
is ridiculous that in the 21st century the mail protocols are still  
cramming 8bit material into 7bits using abo9ut 6 different coding  
mechanisms. That is the logjam that EAI is trying to break.

> fine.  You can't use certain technologies in all systems and that  

That might work for an 'owner' who regularly emails to the same group of  
people and knows the routes by which they can be reached with 8BITMIME  
supported all the way. But knowledge of detailed routes largely  
disappeared 20 years ago when 'bang' paths went out of fashion, and any  
need for such knowledge was finally outlawed with RFC 2821. And it is no  
use at all if the 'owner' wants to communicate reliably, with the full  
protection of DKIM, with arbitrarily selected people the world over.
> knowledge of any mail integrity change and signing/resigning entities.

How DKIM will work in an EAI context is not yet clear. For messages which  
remain in an EAI (aka UTF8SMTP) environment throughout their journey, DKIM  
should work OK, provided implemetors heed the advice in dkim-base to  
maintain 8bit cleanliness in strings. But if a UTF8SMTP message has to be  
downgraded by some MTA en route, then secondary signing by that MTA is  
just not an option. The best solution so far seems to be for the verifier  
to upgrade the message to its original form before checking it. That  
should in principle be possible as things are being defined, but whether  
with sufficient robustness to always work is as yet far from clear.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 08 Dec 2006 13:38:51 -0000

Note change of Subject at request of Barry Leiba. The original thread  
(Introducing Myself) was fine when I first joined this list with a long  
list of issues I was concerned about, but it is well past its sell-by-date  
now.

In due course, this needs an I-D draft with a definite proposal, but the  
issues could use a little more informal discussion first.

> I haven't done any speed testing yet, but will try some now.

I did some experimentation last night, but the outcome was that, whilst  
Perl is a fine tool for setting out clearly the essential features of an  
algorithm, it is of no help in estimating how fast it might run.

Being an interpretive language, which calls library subroutines to do the  
interesting stuff, it can run terribly slowly, but then do something  
blindingly fast when it hits a subroutine written in C.

So yes, I could just about tell that decoding Base64 was faster than  
generating a SHA-256 hash, but not reliably by how much.

Essentially, however, the inner loop of what I am suggesting would look  
like this:

Go through the input stream looking for CRLF.
    When you find one
       Look for whitespace before it (to delete is at per the 'relaxed'  
c14n)
       Look for '--' after it to check whether you have possibly reached
           the end of the current part (of some multipart)
       Copy what you have got to the output stream with or without decoding
           of Q-P or Base 64 according to the CTE in force
Put the output stream through SHA-256.

Of all that, everything but looking for the '--' and the possible  
decodoing of Q-P/Base64 have to be done for the present Relaxed c14n. My  
belief is that the SHA-256 will consume most of the machine cycles in  
that, with the search for the CRLF uning quite a lot. Hence the addition  
of the decoding should not make a huge addition percentagewise. But it  
would be necessary to rewrite the whole thing in C to get exact figures,  
and it would be premature to do that just yet.

---

From: <[email-masked]>
Date: Fri, 8 Dec 2006 10:05:14 -0500

Slight disagreement
" No. AIUI DKIM is supposed to operate mainly between the 1st MTA after
the  
sending MUA and the last MTA before the receiving MUA (though smarter
MUAs  
that sign their own, or verify their own are welcome to try)."

I would suggest that DKIM operates between the signing MTA and the edge
boundary MTA of the receiving domain that is the certifier of DKIM
signatures which may be a smart MUA but is more likely a filtering MTA
at the ISP.

Thanks, 

Bill Oxley
Messaging Engineer
Cox Communications

---

From: Douglas Otis <[email-masked]>
Date: Fri, 8 Dec 2006 09:32:52 -0800

On Dec 8, 2006, at 7:05 AM, <[email-masked]> <[email-masked]>  
wrote:

> Slight disagreement
>> " No. AIUI DKIM is supposed to operate mainly between the 1st MTA  
>> after the sending MUA and the last MTA before the receiving MUA  
>> (though smarter MUAs that sign their own, or verify their own are  
>> welcome to try)."
>
> I would suggest that DKIM operates between the signing MTA and the  
> edge boundary MTA of the receiving domain that is the certifier of  
> DKIM signatures which may be a smart MUA but is more likely a  
> filtering MTA at the ISP.

Signing is not limited to the MTA, it can be done at the MUA.  In  
addition, protections afforded by DKIM requires the MUA to verify  
signatures or obtain trustworthy signaling from the MDA.  Blocking at  
the MTA can not offer adequate protection.  It would be wrong to  
expect blocking at the MTA via restrictive policy produces a  
significant effect on the level of abuse.  Blocking via policy  
definitely does _not_ offer much in the way of protection, but will  
require a significant level of support explaining why various  
messages are being rejected.

-Doug

---

From: Hector Santos <[email-masked]>
Date: Fri, 08 Dec 2006 12:55:29 -0500

Douglas Otis wrote:
> signatures or obtain trustworthy signaling from the MDA.  

I'm sorry. What section in the DKIM specification does it say it 
"requires the MUA to verify signatures"?

 > Blocking at the MTA can not offer adequate protection.

Why not?

 > It would be wrong to expect blocking at the MTA via restrictive
 > policy produces a significant effect on the level of abuse.

Bad Guy uses my domain.com at site XYZ. Site XYZ looks up my policy and 
finds he wasn't suppose to use my DOMAIN.

Whas wrong with expecting this is not a highly probably event?

> Blocking via policy definitely does _not_ offer 
> much in the way of protection, but will require a significant level of 
> support explaining why various messages are being rejected.

It will?

- A domain does not expect mail.  Pretty good protection
- A domain requires mail to be sign. Pretty good protection

Those two along will cut down a very significant amount of the most 
common exploitations without requiring any feedback whatsoever.

--
HLS

---

From: Douglas Otis <[email-masked]>
Date: Fri, 8 Dec 2006 13:42:13 -0800

> "requires the MUA to verify signatures"?

The DKIM specification does not indicate how protective benefits are  
derived.  It surely does not say the MUA can not verify signatures.   
DKIM use at the MUA has an advantage over SPF that must often depend  
upon Received headers including the optional IP address of the SMTP  
client.

>> Blocking at the MTA can not offer adequate protection.

You can not safely tell customers they are protected at the MTA from  
spoofs with enforcement of DKIM's policies.  Currently MUAs do not  
adequately display email-addresses to safely allow visual  
verification.  When your customer assume they are protected with  
assurances of MTA policy blocking, they more easily fall victim to  
visual obfuscation techniques.  These techniques are not handled by  
DKIM when limited to just email-address domains.  Protection must  
also work when email-addresses are not using ASCII as well.

> Bad Guy uses my domain.com at site XYZ. Site XYZ looks up my policy  
> and finds he wasn't suppose to use my DOMAIN.
>
> Whats wrong with expecting this is not a highly probably event?

Because bad actors adapt where then you might then detect a few  
percent of lazy ones as with SPF.  All you have done is add more  
overhead, where now removing it places your customers at greater risk  
after hearing your initial promise of applying DKIM policy to block.   
When bad actors adapt, can you stop making searches up label trees  
for each message you receive?  You have created two bad problems  
based upon this assumption:

1) Increased overhead for the MTA for little benefit.
2) Increased susceptibility to spoofing due to the false claim of  
protection.

>> Blocking via policy definitely does _not_ offer much in the way of  
>> protection, but will require a significant level of support  
>> explaining why various messages are being rejected.
>
> It will?
>

Only when message originators are recognized and verified by the MUA,  
or by the MUA in conjunction with the MDA where annotation protection  
can be achieved.  Visual examination of a From header does not offer  
"Pretty good protection".  DKIM signatures themselves are invisible  
by design and there should be no assumption about the method of  
header display.

> common exploitations without requiring any feedback whatsoever.

There is plenty of evidence that differs with this assessment.   
Sorry, but DKIM does not enable safe visual examination of headers,  
nor does it offer a viable means for reducing abuse.  In fact,  
without some method to associate the SMTP client with the signing- 
domain, DKIM does not even offer safe white-listing in many cases.

Associative techniques can establish safer annotation for a more  
secure protective scheme that does not depend upon visual examination  
or a high MTA overhead.  In fact, the MTA can stop checking DKIM when  
the domain is not white-listed.  Associative techniques eliminate any  
need to share private keys, while ensuring DKIM feedback is directed  
to the transmitting entity.  Associative techniques protect use of  
DKIM for white-listing.  Associative techniques can be for specific  
headers for differentiating various services.  Associative techniques  
can also protect the return-path. An association can be established  
within a single and small DNS transaction.

-Doug

---

From: Dave Crocker <[email-masked]>
Date: Fri, 08 Dec 2006 14:24:54 -0800

Charles Lindsey wrote:
> How DKIM will work in an EAI context is not yet clear. For messages 
> which remain in an EAI (aka UTF8SMTP) environment throughout their 
> journey, DKIM should work OK, provided implemetors heed the advice in 
> dkim-base to maintain 8bit cleanliness in strings. But if a UTF8SMTP 
> message has to be downgraded by some MTA en route, then secondary 
> signing by that MTA is just not an option. ...

It occurs to me that this is probably not a DKIM topic at all.  I don't mean 
that it isn't relevant to DKIM, but rather that it is not *specific* to DKIM.

EAI is a long-standing problem and canonicalization of email text is a 
long-standing issue.  I suspect that your focus is appropriate to a venue with 
that mix of interest, rather than DKIM, per se.

Let me suggest this more strongly:  These arenas of internationalization and 
canonicalization have proved exceptionally difficult and the sort of thing you 
are attempting to pursue *should* be of benefit -- and therefore interest -- to 
the larger email text-handling community.

That said, I'm not sure what venue to suggest, and I don't want to guess, lest 
it confuse things further.

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Fri, 08 Dec 2006 14:32:20 -0800

> signatures which may be a smart MUA but is more likely a filtering MTA

This is the sort of question that prompted my to add the construct of 
Administrative Management Domain (ADMD) to the Internet Mail Architecture draft 
<http://bbiw.net/specifications/draft-crocker-email-arch-05.html>

DKIM is envisioned as having signing done within an originating ADMD -- that is, 
within a trust boundary associated with the author or at least with the author's 
email posting service, and having validation done by a similarly-scoped 
environment at the recipient end.  (Validation by intermediaries is fine, but 
hasn't been a focus.)

Exactly which host within an ADMD will do signing or validating is not 
constrained by DKIM's design.

There are operationally realities that will constrain the choices for many 
ADMDs, but this is not a matter of DKIM design, but rather of handling (or 
perhaps MIShandling) behaviors within the ADMD.

Any other statements about host choices are a matter of preference, rather than 
need.  That the statements might prove true doesn't make them less an 
administrative choice.

So, yeah, a scenario that is viewed as highly likely is signing by the outbound 
boundary MTA and validating by the inbound boundary MTA.  Lots of good reasons 
for do that that.  None of them makes this scenario mandatory, however.

d/

-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Fri, 08 Dec 2006 18:05:57 -0500

Douglas Otis wrote:
> upon Received headers including the optional IP address of the SMTP client.

Whatever, it does not say DKIM "requires the MUA to verify signatures."

> Because bad actors adapt where then you might then detect a few percent 
> of lazy ones as with SPF.  

Who's talking about SPF?

> Only when message originators are recognized and verified by the MUA, 

Nope, once again, MUA are not required. I can do the above easily at the 
MDA.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 8 Dec 2006 15:33:37 -0800

On Dec 8, 2006, at 3:05 PM, Hector Santos wrote:

>>>> Blocking via policy definitely does _not_ offer much in the way  
>>>> of protection, but will require a significant level of support  
> Nope, once again, MUA are not required. I can do the above easily  
> at the MDA.

Is viewing the display name protected by this effort?

Is receiving non-ASCII email-addresses protected by this effort?

Are look-alike and cousin-domains prevented?

What happens when a domain wishes to allow users use of a mailing- 
list?  Should they setup different domain names, or use a sub- 
domain?  How will increased domain names of the same entity better  
allow a recipient to detect a spoof?

You can not offer "pretty good protection" at the MTA based upon  
policy blocking.  Simple schemes remain where your customers continue  
to be spoofed.  Annotation at the MUA can prevent these schemes,  
works with non-ASCII email-addresses, prevents look-alike and cousin  
domains exploits, and permits the use of mailing-lists without  
additional domain names.

Policy based blocking is not a desirable feature when it will likely  
make the situation worse at substantial costs to resources.

-Doug

---

From: Hector Santos <[email-masked]>
Date: Fri, 08 Dec 2006 19:22:06 -0500

Douglas Otis wrote:
>> Nope, once again, MUA are not required. I can do the above easily at 
>> the MDA.
> 

N/A - MUA are not part of the process for this protocol ratification!

> Is receiving non-ASCII email-addresses protected by this effort?

Unrelated to the basic QUESTION of DOMAIN protection via SSP.

> Are look-alike and cousin-domains prevented?

Doesn't APPLY!

> What happens when a domain wishes to allow users use of a mailing-list?  

Then the domain asking for trouble because MAILING-LIST are know to 
break the integrity of the mail.

> Should they setup different domain names, or use a sub-domain?  

Maybe, if that is what they want.

 > How will increased domain names of the same entity better

Doesn't apply. We are talking about 1 domain.  One transaction at a time.

> You can not offer "pretty good protection" at the MTA based upon policy 
> blocking.  

I sure can and if I didn't think so, I wouldn't be touching or even 
looking at DKIM/SSP.

> Simple schemes remain where your customers continue to be 
> spoofed.  

Not with a DKIM/SSP framework.  Phished? sure. not SPOOFED.  Plus
if a bad guy wanted to BREAK DKIM/SSP, all he has to do is AVOID using 
it and stay away from DKIM/SSP protected domains!

 > Annotation at the MUA can prevent these schemes,

This is like saying,

    "Mom, can I let the rabid dog in the house? He looks so cute!"

Mom is not going to let johnny get in trouble if she can help it. But 
just in case, she might give Johnny a rabbies shot.

> works with 
> non-ASCII email-addresses, prevents look-alike and cousin domains 
> exploits, and permits the use of mailing-lists without additional domain 
> names.

Out of scope!  You're stuck with this because you are looking for a MUA 
solution - unrealistic.

> make the situation worse at substantial costs to resources.

But that premise is highly false. So what bother to continue?  Its 
friday! Thats why!

---
HLS

---

From: Douglas Otis <[email-masked]>
Date: Fri, 8 Dec 2006 18:17:46 -0800

> N/A - MUA are not part of the process for this protocol ratification!

What element is ratified?  What benefit is established when  
recipients may not understand what is signed, and who actually sent  
the message?  Do you care this blocking scheme fails to ensure  
recipient protections?  This should not be just a check mark on a  
feature list.

> Unrelated to the basic QUESTION of DOMAIN protection via SSP.

Rather than just establishing restrictions, policy can also establish  
associations and permit a "recognized" email-address 'X' to be used  
in conjunction with signing-domain 'Y' to generate an annotated  
message.  Annotation can be a gold-star or placement into a folder or  
specialized mailbox handled by _either_ the MDA or the MUA.    
Recognition criteria can be established by the account's address-book  
or a DAC list.

>> Are look-alike and cousin-domains prevented?

What is the goal then?

>> list?

When a message is signed by a mailing-list, this establishes their  
signatures integrity.  Annotation can make it clear the header used,  
and that the originator is known (and trusted).  When annotation  
added by the MUA or MDA _is_ the protective mechanism, use of a  
mailing-list is never a problem and nothing is broken.  A mailing- 
list only becomes a problem when subjected to a highly flawed  
blocking scheme unable to provide adequate protection anyway.

> Maybe, if that is what they want.

This then increases recipient's confusion about what is being checked  
with a blocking scheme.

>> How will increased domain names of the same entity better allow a  
>> recipient to detect a spoof?
>

This is about protecting recipients from being spoofed.  This is not  
about making sure a message jumps through a set of hoops.

>> policy blocking.

It is hard to understand your goals.  What constitutes "pretty good"  
when recipients can still be spoofed?

>> Simple schemes remain where your customers continue to be spoofed.
>
> Not with a DKIM/SSP framework.  Phished? sure. not SPOOFED.

Huh?

> Plus if a bad guy wanted to BREAK DKIM/SSP, all he has to do is  
> AVOID using it and stay away from DKIM/SSP protected domains!

Bad actors would only need to provide the appearance of being from  
the trusted domain.  This is easily done with the proper HTML format  
and images.  It is also common to see display-names that make it  
appear the email-address is being displayed, when it is not.  The  
visual tricks are vast.

> Mom is not going to let johnny get in trouble if she can help it.  
> But just in case, she might give Johnny a rabbies shot.

Not really.  Annotation offers a clear indication about which dogs  
are known safe.  Your scheme lets in any dog and offers no clue which  
are safe.  Don't expect Johnny to look at the dog's tattoo under  
their lip.

>> works with non-ASCII email-addresses, prevents look-alike and  
>> cousin domains exploits, and permits the use of mailing-lists  
>> without additional domain names.
>
> Out of scope!  You're stuck with this because you are looking for a  
> MUA solution - unrealistic.

This approach can be applied at _both_ the MDA and MUA.  Message  
annotation has a realistic chance of offering real protection.  A  
blocking scheme does not!

>> Policy based blocking is not a desirable feature when it will  
>> likely make the situation worse at substantial costs to resources.
>
> But that premise is highly false.

DNS transactions required to discover whether policy is within any  
domain level for any message, signed or not, is not trivial  
overhead.  In addition, there will be support calls dealing with  
messages lost by various services.  There will be issues created when  
the use of EAI extensions is prevented.  There is information leaked  
to bad actors regarding the level of acceptance achieved at the MTA.   
There are still problems related to white-listing and the properly  
handling of DSNs not improved by a blocking strategy.  Comparisons  
between blocking and annotation are rather stark in terms of costs  
and benefits.

-Doug

---

From: Dave Crocker <[email-masked]>
Date: Sat, 09 Dec 2006 12:20:57 -0800

offlist.

Charles Lindsey wrote:
> unless it is widely adopted. It will not be widely adopted unless it is 
> seen to be robust. In particular, it will not be adopted in countries 
> 7bit. They just cannot survive in an environment where textual messages 

 From what I have seen, doing a basic web search on your history, you've been 
around and done a lot.

That said, I'm curious about the basis for your firm certitude about what the 
adoption barriers/requirements are for Internet mechanisms.

Just to be clear:  No one disputes the need for supporting a full range of 
characters.  But no one with experience also denies that it is a difficult problem.

d/

-- 

   Dave Crocker

---

From: Scott Kitterman <[email-masked]>
Date: Sat, 9 Dec 2006 15:27:17 -0500

> offlist.

or not.

Scott K

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 11 Dec 2006 12:01:14 -0000

On Fri, 08 Dec 2006 22:24:54 -0000, Dave Crocker <[email-masked]> wrote:

> It occurs to me that this is probably not a DKIM topic at all.  I don't  
> mean that it isn't relevant to DKIM, but rather that it is not  
> *specific* to DKIM.

This is an area where the work of hte DKIM WG and that of the EAI WG may  
conflict. I think all that can be done is for both groups to be on the  
lookout for such conflicts, and to avoid them where possible.

So in this group, it would be reasonable to point out "that feature might  
cause problems for EAI - here is how you might avoid them", and on the EAI  
group "this feature might cause difficulties if the message were to be  
DKIM-signed - here is a way to get around it".