Subject: Re: [ietf-dkim] Introducing myself

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 01 Nov 2006 15:04:22 -0000

On Tue, 31 Oct 2006 06:27:14 -0000, Eliot Lear <[email-masked]> wrote:

> Hi Charles, and welcome.  I can't answer many of your questions, but I  
> can certainly take a whack at a few.
>

>>> 3.5  The DKIM-Signature header field
>>

>> Can the various tags appear in this header in any order? OTOH, why is
>> there not an insistence that the b= tag should come last (since it has  
>> to
>> be easily joined to and separated from the rest)?
>
> I can't speak to anyone's implementation, but general SMTP folding rules  
> apply, no?

Yes, sensible folding would help. But it would make life easier for  
implementors if the b= tag was always last. That is, in any case, where  
most implementors are likely to put it simply because that is easiest, and  
it is also the easiest place to ignore it when a verifier needs to grab  
everything in the header _except_ that bit when constructing its hash.

>>>    l=  Body length count
>>
>>>            INFORMATIVE IMPLEMENTATION WARNING:
>>
>> I am very suspicious of the propriety of suggesting, in any IETF  
>> standard,
>> that it is legitimate to remove text from a message being conveyed
>> (certainly without the consent of the recipient). Surely marking it with
>> blood-red ink, or warnings in 32pt characters is as far as one should  
>> go?
>
> What if it's executable code inserted by an attacker?

A fair point, but labelling it clearly should surely be enough. One  
application of this tag is apparently for boilerplate added by a mailing  
list expander, and if that boilerplate is there, it is presumably intended  
that people should read it, and the list admin is entitled to be miffed if  
the recipients do not even get to see it.

>>
>>>    z=  Copied header fields
>>
>>>        Verifiers MUST NOT use the header field names or copied values
>>>        for checking the signature in any way.  Copied header field
>>>        values are for diagnostic use only.
>>
>> Why ever not? I can think of examples where a verifier might find it
>> exceedingly useful to be aware of the original state of some header  
>> which
>> might have been changed somewhere en route. And what potential
>> interoperability arises if a verifier makes some use of this  
>> information?
>
> Because z= field does NOT get rendered to the user, but rather the real  
> header fields will.  It makes no sense that the z= field should be  
> verified and the others are not.  Allowing otherwise would provide for a  
> malicious opportunity.

Yes, but it is the policy module of the verifier, rather than the user,  
which might find it useful. Some headers may legitimately get changed en  
route. These should probably not be signed, but knowing what such a header  
originally looked like might enable the policy module to spot some unusual  
situation and work around it. The example I have in mind is the use of  
UTF-8 in headers, being worked on by the ietf-EAI WG, where a message with  
'internationalized' headers will have a special header
     Header-Type: UTF8
If that message has to be downgraded en route, because it meets a legacy  
MTA that does not support UTF8, then that header gets changed to
     Header-Type: Downgraded
If a verifier can detect the change in that header, it can try to  
'upgrade' the message to its original form before checking the signature.  
I am not saying that is the only way that UTF8 headers might be made to  
work with DKIM, but it is certainly one possible approach that should be  
looked at.

>>> 3.6.1  Textual Representation
>>
>>>    h=  Acceptable hash algorithms
>>
>>>        ...  Signers and Verifiers MUST
>>>        support the "sha256" hash algorithm.  Verifiers MUST also  
>>> support
>>>        the "sha1" hash algorithm.
>>
>> Why MUST signers support the "sha256" hash algorithm (clearly, verifiers
>> MUST)? Surely a signer who, as a matter of policy, always chooses to use
>> sha-1 is not obliged to implement something he is never going to use?
>
> Again, for interoperability.  We picked one.  That was the one.

I think you misunderstand my point. AIUI, a signer is allowed to choose  
either sha-1 or sha-256. So evidently all verifiers MUST be able to accept  
whichever of those turns up.

But if a signer decides "my policy is always to use sha-1" then what is  
the point of saying he MUST include code in his implementation which he is  
never going to use. RFC 2119 says you can only say "MUST" where there is  
the possibility of some interoperability problem or other harm. But in  
this case there would be no way for an outsider with no access to the  
signers machine to be aware that he had omitted that code. Hence the use  
of "MUST" for the signer is meaningless, since violating it produces no  
visible effects.

The same argument applies to the places where it says the signer MUST  
support dns/ext and rsa. In those cases too, it is only the verifier who  
MUST support everything the signer is allowed to send.
>
> I'm sorry, but I've run out of time and must be on a plane shortly.

Have a good trip!

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 01 Nov 2006 16:03:55 -0000

On Tue, 31 Oct 2006 09:26:57 -0000, Jon Callas <[email-masked]> wrote:

> On 30 Oct 2006, at 1:42 PM, Charles Lindsey wrote:
>
> I'm going to cherry-pick a few things, particularly the ones I think I'm  
> best suited to answer.
>
>
>>> 3.2  Tag=Value Lists
>>
>>>       INFORMATIVE IMPLEMENTATION NOTE:  Although the "plain text"
>>>       defined below (as "tag-value") only includes 7-bit characters, an
>>>       implementation that wished to anticipate future standards would  
>>> be
>>>       advised to not preclude the use of UTF8-encoded text in tag=value
>>>       lists.
>>
>> Those future standards are nearer than you think. The currently active
>> ietf-eai WG is charged with producing an experimental protocol for  
>> writing
>> headers in UTF-8. ...
>>
>
> No, because we want DKIM to work with 7-bit-clean mail. We want broad,  
> fast deployment and that means working with old systems.

Sure, but that is no reason for making it _not_ work with new systems. If  
it is REQUIRED (rather then  merely suggested, as in the present draft)  
that UTF8-encoded text in tag=value lists is allowed, all existing mail  
will still work, but so will the new. That should make everybody happy,  
including the implementors who now have one test less to write.

>>>    h=  Signed header fields
>>
>> Why MUST NOT this list be empty? Suppose you want to sign the body, but
>> not any headers? Unusual, but perhaps sensible for some application. No
>> interoperability arises.
>
> Because you have to sign at least one header. Think of DKIM as a  
> header-signing system. It signs the body, too, but that's a means to an  
> end.

Fair enough, but that sounds like grounds for SHOULD NOT rather than MUST  
NOT, since nothing actually breaks if you violate it.

>>>    i=  Identity of the user or agent
>>
>> Must this tag, if a <local-part> is present, be a valid working email
>> address?
>>
>
> No, it can be anything you want.

OK.

>> that it is legitimate to remove text from a message being conveyed ...
>>
>
> The point of body lengths is that many systems add text to the end of a  
> message. In fact, the list server that is delivering this to you is  
> doing precisely that.

So I see, and a very sternly worded remark ("NOTE WELL: ...") it is. So  
why do you want to even suggest that is would be legitimate for verifiers  
(or their policy modules) to chop it out?

>>>    q= A colon-separated list of query methods used to retrieve the
>>>       public key
>>
>> Clearly, the use of DNS or some similar global database is the only
>> sensible PKI that is workable for DKIM. But am I right in saying that  
>> this
>> tag does not preclude the use of other PKIs for other applications (e.g.
>> attached certificates, web-of-trust, private agreements between the
>> communicating parties, etc.)?
>>
>
> A couple of comments on this one.
>
> DKIM is not a PKI. It has no trust model.

I would not say it has no trust model. The DNS is not all that easy to  
spoof, but it still gives a reasonably strong assurance you have the  
correct key, though clearly other PKIs are much stronger.

> Second, PKIs are not distribution mechanisms.

A mattter ot definition, I think. I would regard a PKI as encompassing  
either or both of a distribution and a trust mechanism.

>> Why MUST signers support "dns/ext" (clearly, verifiers MUST)? ...

See my reply to Eliot for this, and the similar wordings for hash and  
encryption algorithms. I think you have misunderstood the point of my  
concern. If it had said that signers SHOULD use dns/ext, ... rsa, ...  
sha-256, etc, (at least when the application is DKIM) then that would have  
been OK.

>>> 5.3  Normalize the Message to Prevent Transport Conversions
>>
>> I found this section absolutely astounding.
>>
>> Message bodies written in Non-ASCII charsets have been commonplace now  
>> for
>> 12 or more years, and they are most readily represented as 8-bit.  
>> 8BITMIME
>> has been around for the same length of time and is now almost  
>> universally
>> deployed. 8bit using 8BITMIME has become, or is well on the way to
>> becoming, the preferred CTE for charsets which will not fit into 7bits.
>> And yet you are now seriously proposing, for a protocol that needs to be
>> used in the great majority of future email messages if it is to fulfill
>> its purpose, to return to encodings that can be squashed into 7bits.  
>> That
>> is one monumental step backwards for the IETF.
>>
>
> You're confusing making sure that 7-bit systems don't break with  
> encouraging them. We're not encouraging them.

You say "signers SHOULD convert the message to a suitable MIME content  
transfer encoding such as quoted-printable or base64". That sounds to me  
like a pretty strong discouragement to continue using CTE 8bit.

> ... But we're being realistic and want DKIM to work with existing  
> systems.

Then is that case it has to work with CTE 8bit even when it gets changed  
to Q-P or Base64 by some non-8BITMIME MTA en route (I doubt there are many  
systems left that still do that, but there will be some).

And it is not as if this was a difficult problem to solve. All you have to  
do is to decode any Q-P or Base64 encountered during the canonicalization  
process and use that in the hash. By definition, changing the CTE does not  
alter the "meaning" of the message, so no harm can arise from that. Why  
was this not done?

>> Sorry to have gone on at such length. But some of these issues seem of
>> importance, so I hope someone will take the trouble to reply.
>>
>
> It's all right. Sorry for being a bit abrupt, but you need to catch up  
> to what we're doing.
>
> Go look at the web site at www.dkim.org, .......

Yes, I have already looked at all the sources you mention.

---

From: John Levine <[email-masked]>
Date: 4 Nov 2006 02:03:38 -0000

>>> writing headers in UTF-8. ...
>>
>Sure, but that is no reason for making it _not_ work with new systems.

It's still an open question how Unicode is going to show up in mail
headers, with 8 bit UTF8 being only one of multiple possibilities.
More likely there will be some kludge to smoosh it into 7 bits so it
can transit through old MTAs.  I don't think anyone is opposed to DKIM
handling whatever happens, but I also don't think it's productive to
try to guess at this point which way it'll turn out.

>(or their policy modules) to chop it out?

This has been very contentious.  Personally, I will never put an l=
into a signature, but there are some vocal people who insist that it's
important for signtures to survive (some) mailing list software, so
it's there if they want it.

>sha-256, etc, (at least when the application is DKIM) then that would
>have been OK.

The MUST is quite deliberate so that DKIM implementations will
interoperate.  You're welcome to do whatever you want to exchange
messages with your friends, but for mail to everyone else, you have
to use SHA-256 and dns/ext because that's what you know they'll be
able to handle.

>You say "signers SHOULD convert the message to a suitable MIME
>content transfer encoding such as quoted-printable or base64". That
>sounds to me like a pretty strong discouragement to continue using
>CTE 8bit.

I think that what the wording should say is that messages must be valid
RFC2822 (or maybe RFC822) messages.  This came up in connection with
what to do with messages with bare CR or LF characters, with the answer
being "don't do that".

You can sign whatever you want, but if the message is 7bit, your
signature is more likely to survive transit to the verifier.

>process ...

DKIM doesn't understand MIME.  If DKIM signers and verifiers had to
unpack MIME parts they would be orders of magnitude more complicated.
In practice, I think that nearly everyone uses the simple body canon
anyway.

R's,
John

---

From: Tony Finch <[email-masked]>
Date: Sat, 4 Nov 2006 13:38:44 +0000

On Sat, 4 Nov 2006, John Levine wrote:
> can transit through old MTAs.

It's much less of an open question than you seem to think. Transmitting
unicode over the current 7bit braindamage is handled with RFC 2047 which
has been around for many years, so your last sentence above is wrong in
its use of the future tense. The only proposed change is to allow raw UTF8
when the MTA advertises the capability.

> I don't think anyone is opposed to DKIM handling whatever happens, but I
> also don't think it's productive to try to guess at this point which way
> it'll turn out.

It just needs to be 8 bit clean.

> what to do with messages with bare CR or LF characters, with the answer

This implies DKIM cannot be used with 8bitmime or binarymime.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTH TYNE DOGGER: WEST 5 TO 7, OCCASIONALLY GALE 8. MODERATE OR ROUGH. FAIR.
GOOD.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: John L <[email-masked]>
Date: Sat, 4 Nov 2006 10:03:31 -0500 (EST)

> This implies DKIM cannot be used with 8bitmime or binarymime.

The "don't do that" was in the context of trying to kludge around MTAs 
that will change bare CR to something else such as CR LF.  8bitmime has 
the same problem, since I gather that some MTAs that will re-code it to 
7bit on the fly if they find they're speaking to another MTA that doesn't 
support 8bitmime.  DKIM works fine if the transmission path doesn't change 
the message much, for an extremely ill-defined definition of "much".

I presume we all agree that we do not want to go down the rathole of 
trying to survive all the re-coding tricks that MTAs and gateways do. 
Perhaps we should add language to the effect that you can sign whatever 
you want, and if your transmission path is sufficiently clean, it'll work.

This also suggests that we should ditch the relaxed body encoding, since 
the kinds of munging it tolerates is such a tiny fraction of the things 
that actually happen to messages.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://johnlevine.com, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 4 Nov 2006 07:53:41 -0800

At 1:38 PM +0000 11/4/06, Tony Finch wrote:
>It's much less of an open question than you seem to think.

Have you been following the EAI WG? If so, you have a different 
interpretation of "open question" than others of us. If not, then you 
really should do so before stating how things will be.

>  Transmitting
>its use of the future tense.

How does RFC 2047 handle non-ASCII on the left side of the @? Again, 
maybe go read the documents and discussion in EAI. The WG discussion 
is particularly useful for folks who are sure that they know the One 
True Way to solve the problem.

--Paul Hoffman, Director
--Domain Assurance Council
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Dave Crocker <[email-masked]>
Date: Sun, 05 Nov 2006 00:01:38 -0800

> really should do so before stating how things will be.

So it is probably a good thing that he said "more likely".  The semantics of 
that language is rather different that "how things will be".

As you well remember, the model he described has a solid and legitimate 
basis,both in terms of design and in terms of history.

That it is not the direction that EAI is taking is just fine. That the EAI 
approach might be the actual choice by the Internet is also fine.

But it is always worth being careful about taking a position concerning a future 
that relies on a particular standards effort succeeding, particularly one with a 
difficult history.

Maybe it will reach a critical mass of deployment.  That would be excellent, of 
course.

But there is no guarantee that it will happen.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 06 Nov 2006 10:47:45 -0000

> try to guess at this point which way it'll turn out.

Paul Hoffman has answered this well enough. All that is needed is that
DKIM should not fail to work just because it finds some octet with bit 8
set, because it is clear that whatever happens regarding unicode, such
octets are surely going to appear.

The present draft contains some advice:
        INFORMATIVE IMPLEMENTATION NOTE:  Although the "plain text"
        defined below (as "tag-value") only includes 7-bit characters, an
        implementation that wished to anticipate future standards would be
        advised to not preclude the use of UTF8-encoded text in tag=value
        lists.
Presumably whoever wrote that was satisfied that allowing such
UTF8-encoded text would do no harm. In which case, you may as well make
allowing it mandatory (or at least allowing the full 8bits, since the
question of the actual code doesn't matter at the moment, so long as ASCII
is a subset of it).

> it's there if they want it.

But the people who want it won't take kindly to having it deleted by
overly "helpful" verifier policy modules. Currently the draft suggests
that is a reasonable practice. It isn't, and the draft should not be
saying such things. By all means warn the user, and even provide the user
with tools to delete it. But don't chop bits out of his mail without his
approval.

> to use SHA-256 and dns/ext because that's what you know they'll be

But that is not what the draft says. It currenty says, in effect, that
signers MAY use either SHA-1 or SHA-256 (and in consequence verifiers MUST
accept both - that bit is not in dispute). But you cannot say, at the same
time, that signers MAY use SHA-1 and MUST use SHA-256 (or MUST implement
SHA-256 even if they have no intention of generating it). RFC 2119 just
does not allow you to use MUST in those sorts of ways.

> RFC2822 (or maybe RFC822) messages.  ...

Messages haven't been valid RFC 2822 for a long time (ever since 8BITMIME
and now BINARYMIME).

> In practice, I think that nearly everyone uses the simple body canon

Not at all. Going through the MIME structure of a message body and undoing
all Q-P or Bas64 encodings is fairly straightforward, and if you hash and
sign the result of doing that, then it is guaranteed to pass straight
through all those systems which (quite legitimately under RFC 1652)
re-encode stuff en route, without breaking the signature. I shall try to
write a demonstration implementation in the next day or so, and it
certainly won't be "orders of magnitude more complicated".

You said in a later message that the "relaxed" body canonicalization
should be ditched because the things it protected against rarely happen.
Surely it would be better to augment it with something that would make it
proof against things that regularly _do_ happen, and happen with the full
blessing of IETF standards.

And, moreover, I do not see why the 'simple' canonicalization is the
default (or even why it even exists at all, for both headers and bodies).
Can anybody suggest a scam or threat that would be facilitated if
"relaxed" rather than "simple" was used?

---

From: John Levine <[email-masked]>
Date: 6 Nov 2006 15:45:01 -0000

We're mostly talking past each other.  DKIM is intended to be 8 bit
clean, and you can sign whatever you want.  If you sign 8 bit data and
the path your message takes is transparent to 8 bit data, great, you
win, if not, you lose.  In what I expect is the common case that you
don't know what MTAs are going to handle your message, downcoding
everything to 7bit is a lot more robust.  Perhaps you could suggest
wording to make that clearer.

>with tools to delete it. But don't chop bits out of his mail without his

I don't see any place in the draft where it says anything about who's
approving or not approving of any particular operations.  If I were
one of the band of optimists who thinks that "l=" will be useful, I
expect I would heartily approve of my MTA chopping off the unsigned
bits in my incoming mail.

> [ re sha1 vs sha256 ]
>But that is not what the draft says.

Our intent is that signers have to sign with sha256 and, for backward
compatibility, can also add a sha1 signature.  Verifiers have to
handle either.  If the wording doesn't say that, we need to fix it.

>all Q-P or Bas64 encodings is fairly straightforward,

The people who have written DKIM signers and verifiers can comment on
how much complication would be involved if they had to add MIME
packers and unpackers to them.

>"relaxed" rather than "simple" was used?

May I direct your attention to the lengthy discussions of these topics
in the list archives?

R's,
John

---

From: Arvel Hathcock <[email-masked]>
Date: Mon, 06 Nov 2006 15:28:20 -0600

>> Not at all. Going through the MIME structure of a
 >> message body and undoing all Q-P or encodings
 >> is fairly straightforward,
 >
 > The people who have written DKIM signers and verifiers
 > can comment on how much complication would be involved
 > if they had to add MIME packers and unpackers to them.

Dealing with MIME would make things much much more complex.  I'm so glad 
we don't have to deal with that.  It would be a huge and needless 
implementation burden IMO.

Arvel