Subject: [ietf-dkim] Where is the formal definition of DKIM-Signature?

From: Alessandro Vesely <[email-masked]>
Date: Thu, 8 Feb 2018 11:14:04 +0100

Hi,

someone asked me about case sensitiveness of the header field name.  I looked
for an ABNF in RFC6376, but only found examples and informative notes.

Is that an error worth being reported?

Ale
_______________________________________________
NOTE WELL: This list operates according to
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 08:22:49 -0500

> for an ABNF in RFC6376, but only found examples and informative notes.

I was going to say that can't possibly be true, but it's true, there's no 
ABNF for the header.  So, for example, I don't know whether the v= field 
has to come first.  Send an erratum, we'll probably accept it as hold for 
update.

By the way, all field names are case insensitive.  RFC 5322 doesn't say 
that explicitly, but the ABNF for the field names makes it pretty clear, 
On the other hand, RFC 5322 also says that field names can be any printing 
ASCII character so this is be a valid header.

$"%\)': plugh

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
_______________________________________________
NOTE WELL: This list operates according to
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 10:07:38 -0500

> "v=1" doesn't have to come first.  It just usually does.  I think there was
> a version of RFC4871 that did that, but then when challenged we couldn't
> come up with a good reason to keep it that way.

I wonder how many DKIM libraries will accept a signature where it doesn't.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 8 Feb 2018 07:06:47 -0800

-MSK

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 07:42:33 -0800

>> for an ABNF in RFC6376, but only found examples and informative notes.

Header field name rules are in RFC 5322.  That deals with case 
sensitivity for field name strings.  Section 1.2.2 provides the basis 
for knowing whether a defined string is to be parsed in a case sensitive 
or insensitive manner.

> I was going to say that can't possibly be true, but it's true, there's 
> no ABNF for the header.  So, for example, I don't know whether the v= 
> field has to come first.  Send an erratum, we'll probably accept it as 
> hold for update.

In RFC 6376, note Section 3.2 on tag lists.  The ABNF shows no 
sensitivity to ordering. (The linkage to DKIM-Signature is Section 3.5, 
first paragraph.)

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net
_______________________________________________
NOTE WELL: This list operates according to
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 11:05:13 -0500

> Header field name rules are in RFC 5322.  That deals with case sensitivity 
> for field name strings.  Section 1.2.2 provides the basis for knowing whether 
> a defined string is to be parsed in a case sensitive or insensitive manner.

That's right, and all of the fields defined in 5322 have case insensitive 
names, but as far as I can tell, I could define a header like this:

  pickle-header = %d80.111.99.107.108.101 ":" CFWS ( "dill" / "garlic" / "kimchee" )

So this is a pickle-header

  Pickle: dill

but this is not:

  pickle: garlic

I'm not saying any sensible person would do that, but as far as I can 
tell, that's what the spec says.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 08:12:13 -0800

> tell, that's what the spec says.

 From a quick review of RFC 5322, I think you are correct.  I also 
believe (know) that this is not what has been intended for header field 
name specification, dating back to RFC 733.

That is, the capability you note is contrary to what I believe was 
intended in the RFC 5322 specification.  And deviation from iontent is 
the requirement for qualifying as an errata on an RFC.

I suggest you submit it.  It will be interesting to see the followup.

d/

---

From: "Mark Delany" <[email-masked]>
Date: 8 Feb 2018 16:17:54 +0000

> come up with a good reason to keep it that way.

Heh. I'm still waiting to hear a good reason as to why "v=" exists at all - apart
from exposing brittle parsers which mistakenly expect it to show up as the first
tag.

It's about as likely to change as "MIME-Version: 1.0", which is to say, never.

Mark.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 08:22:20 -0800

> from exposing brittle parsers which mistakenly expect it to show up as the first

There appears to be a genetically-encoded belief in the value of version 
numbers, independent of the logic against it.  The belief is pervasive 
and seems to cross all technical cultures, experiential sets, and 
protocol layers.

d/

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 11:52:36 -0500

> from exposing brittle parsers which mistakenly expect it to show up as the first

I had a draft that invented v=2, for headers with a tag syntax that is not 
quite backward compatible with the current spec.  I realize that we could 
change the header to DKIM-Improved-Signature but the change was small and 
it smelled to me like the same header.

See https://datatracker.ietf.org/doc/draft-levine-dkim-conditional/

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 09:00:35 -0800

On 2/8/2018 8:52 AM, John R. Levine wrote:
> I had a draft that invented v=2, for headers with a tag syntax that is 
> not quite backward compatible with the current spec.  I realize that we 
> could change the header to DKIM-Improved-Signature but the change was 
> small and it smelled to me like the same header.

What you realized goes to the heart of the reason we don't need version 
numbers.

The issue falls into the category of how to specify a processing fork. 
At each protocol layer, there is a mechanism for specifying which 
processing engine is appropriate for the next layer up.  Ethertype, 
Protocol ID, Port, etc.

Version numbers serve that purpose /within/ a protocol layer.

They seek to distinguish important differences in processing for what is 
claimed to be the /same/ protocol.

Except really they don't.

If modifications to a protocol are upward compatible, the the new 
features, themselves, self-identify and version numbers aren't needed. 
If no new features are present, you have the old 'version' of the 
protocol.  If new features are present, you have the new 'version'.

If modifications are /not/ upward compatible, you really have a new 
protocol.  Make a new entry in whatever forking mechanism was used to 
get to the previous version.  As you note, a new header-field would be 
appropriate here.

d/

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 12:09:35 -0500

> Except really they don't.

Except when they do.  I'm thinking, f'rinstance, that there is a bunch of 
code in things like Spamassassin that looks at headers and switches out to 
routines to do stuff with them.  There is nothing in Spamassassin that 
needs to care whether a DKIM signature is v=1 or v=2, that's all inside 
the DKIM library.  If it passes a v=2 signature to a library that only 
knows about v=1, the library will say it's invalid, which isn't ideal but 
isn't wrong.

Regards,

---

From: "Mark Delany" <[email-masked]>
Date: 8 Feb 2018 17:19:41 +0000

> it smelled to me like the same header.

Yes, one can certainly contrive a situation in which they prefer a v=2 solution,
but as you say, the requirement can be just as easily satisfied in a number of
other ways too.

A new header is one as is the presence of a new tag and v= is a third. There are
other ways as well.

Given we're talking about standards, it's de rigueur that we end up with at
least three competing ways of achieving the same outcome without any guidance
whatsoever as to which mechanism to choose when and why.

Underlying all this of course is the assumption that programmers intuit the
possibility of change inherent in these non-essential mechanisms and code and
test accordingly. Now that's a goodun.

Mark.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 09:31:48 -0800

On 2/8/2018 9:09 AM, John R. Levine wrote:
>> They seek to distinguish important differences in processing for what 
>> is claimed to be the /same/ protocol.
>>
> Except when they do.  I'm thinking, f'rinstance, that there is a bunch 
> of code in things like Spamassassin that looks at headers and switches 
> out to routines to do stuff with them.  There is nothing in Spamassassin 
> that needs to care whether a DKIM signature is v=1 or v=2, that's all 
> inside the DKIM library.  If it passes a v=2 signature to a library that 
> only knows about v=1, the library will say it's invalid, which isn't 
> ideal but isn't wrong.

the code that tests for the v= parameter could, just as easily, check 
for the presence of the new features.

d/

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Thu, 8 Feb 2018 17:24:28 +0000

The ones I wrote certainly didn't require v=1 to come first. ;-)

But you're right: there's probably cause to be concerned.

	Tony

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 12:45:43 -0500

> the code that tests for the v= parameter could, just as easily, check for the 
> presence of the new features.

Huh?  v=1 code doesn't know what the new features would be.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 09:49:39 -0800

> Huh?  v=1 code doesn't know what the new features would be.

Re-read what I wrote.

The code that knows to dispatch to v=2 can, just as easily, parse for 
the strings associated with the new features.

d/

---

From: "John R. Levine" <[email-masked]>
Date: 8 Feb 2018 13:03:48 -0500

> The code that knows to dispatch to v=2 can, just as easily, parse for the 
> strings associated with the new features.

True, but not very interesting.  In my spamassassin example, the outside 
code knows nothing about DKIM versions, it just sees a dkim-signature 
header and sends it to the DKIM library.

The point of a v=2 flag is to ensure that old v=1 code doesn't 
accidentally misinterpret new features.  In my example, I made a semantic 
change: in v=1 DKIM, verifiers ignore tags they don't understand.  In v=2, 
there's a new tag type that fails if a verifier can't handle it.  The new 
tags have new syntax that, in an ideal world, would make v=1 verifiers 
fail with a syntax error, but we all know that parse errors are often not 
well debugged.  I did look at a bunch of DKIM libraries and they all check 
for v=1 and fail if they don't find it.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 10:08:07 -0800

> header and sends it to the DKIM library.

Oh.  So v= doesn't dispatch to different code.

> accidentally misinterpret new features. 

"Accidentally misinterpret new features" seems to be synonymous with not 
being upward compatible.  In other words, the new features make the new 
version incompatible with the old.  Hence, the new features define a new 
protocol.

  In my example, I made a
> semantic change: in v=1 DKIM, verifiers ignore tags they don't 
> understand.  In v=2, there's a new tag type that fails if a verifier 
> can't handle it.

Change to basic semantics of the protocol.  Hence, new protocol.

d/

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 10:54:45 -0800

>> True, but not very interesting.  In my spamassassin example, the 
>> outside code knows nothing about DKIM versions, it just sees a 
>> dkim-signature header and sends it to the DKIM library.
> 

BTW, this topic tends to eventually produce a claim that the fact that 
the different versions share so much code justifies the versioning 
mechanism.

Except that code sharing happens in many circumstances that don't 
require conflating incompatible protocols and then using an internal 
demultiplexing switch.

The larger topic is the choice between high-level switching versus 
low-level, and the long-term costs of transition mechanisms.

d/

---

From: "Mark Delany" <[email-masked]>
Date: 8 Feb 2018 20:32:07 +0000

> The point of a v=2 flag is to ensure that old v=1 code doesn't

As a practical matter haven't you effectively invented a new header?

After all, what are most senders going to do? They will not want their
signatures to be suddenly unrecognized by 99% of the planet so they'll continue
to generate a v=1 header and they will also want to reap the bennies of your
fantastic SpamAssassin feature so they'll additionally generate a v=2 header.

The end result is two DKIM-Signature headers with different versions for decades
to come. This will no doubt tweak some receiver is a negative way.

I think this is the biggest flaw with the whole v= rationale. There is never
going to be a v=2 change that doesn't leave everyone continuing to
generate/validate a v=1 header. Is a new header by stealth better engineering
than formalizing a new header?

Mark.

---

From: "John Levine" <[email-masked]>
Date: 8 Feb 2018 19:34:23 -0500

>fantastic SpamAssassin feature so they'll additionally generate a v=2 header.

In my application, the feature that requires v=2 is double signing,
i.e. this signature is valid only if there's also a signature from X.  It
was intended as a workaround for the DMARC mailing list problem

The idea is that in addition to your regular v=1 signature, you'd also
put a weak v=2 signature that requires re-signing by the mailing list.
If you don't use conditional signing (or something else subsequently
added that depends on v=2 semantics), then v=1 signatures are fine
forever.

The reason to make then both DKIM-Signature headers is that there is
code, some of which I've written, that looks for DKIM-Signature
headers in a message and calls a DKIM verifier library if it sees
them.  DKIM is slow, do you don't want to waste time verifying
messages with nothing to verify.  If you don't change the
DKIM-Signature header, all of this code keeps working when you upgrade
your DKIM library.  If you invent a new header, it doesn't.

>to come. This will no doubt tweak some receiver is a negative way.

Only if their validators are broken.  I realize that's likely to be
the case now and then but I can't feel too sorry for them.

R's,
John

---

From: Michael Thomas <[email-masked]>
Date: Thu, 8 Feb 2018 16:42:40 -0800

> than formalizing a new header?
I dunno, it's not like there isn't precedent for this. oh say, like ipv4 
vs. ipv6?

Besides if you wanted to go from DKIM to EKIM, you'd be opening 
pandora's box imo.

Mike

---

From: Dave Crocker <[email-masked]>
Date: Thu, 8 Feb 2018 16:45:22 -0800

> Besides if you wanted to go from DKIM to EKIM, you'd be opening 

The pandora's box is creating an incompatible new version.  All the rest 
is just engineering and operations tradeoffs.

d/

---

From: Michael Thomas <[email-masked]>
Date: Thu, 8 Feb 2018 16:47:30 -0800

On 2/8/18 4:45 PM, Dave Crocker wrote:
> The pandora's box is creating an incompatible new version.  All the 
> rest is just engineering and operations tradeoffs.
>
Thankfully nobody's seriously talking about that from what i can tell, 
so this strikes me as so many angels on a pinhead.

Mike

---

From: "Mark Delany" <[email-masked]>
Date: 9 Feb 2018 20:26:21 +0000

> I dunno, it's not like there isn't precedent for this. oh say, like ipv4 

Oh I dunno. The precedent of RFC822 - the very standard we rely on - has
survived numerous upgraded and enhancements over a 36 year period and managed to
do just fine without a version.

I might add that RFC822 seems to have adapted a lot better than the v4 vs v6
crowd. Not sure you picked the best example of success there, Michael :-)

> Besides if you wanted to go from DKIM to EKIM, you'd be opening 

Indeed. As mentioned previously MIME-Version: 1.0 has set the standard for us to
emulate.

Mark.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 9 Feb 2018 13:03:38 -0800

> do just fine without a version.

and SMTP...

> crowd. Not sure you picked the best example of success there, Michael :-)

What's rather impressive is how aggressively the general Internet 
technically has worked to avoid learning any lessons from this disparity...

d/