Subject: [ietf-dkim] Clarification on z=

From: Arvel Hathcock <[email-masked]>
Date: Sat, 10 Jun 2006 15:19:45 -0500

How are multi-line header field values to be handled with z=?  Do we 
encode the fold (by adding =0D=0A for example) or do we unfold the 
header field value first and then add that result to z=?  Is preserving 
the fold important at all?

-- 
Arvel

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Michael Thomas <[email-masked]>
Date: Sat, 10 Jun 2006 13:46:54 -0700

Arvel Hathcock wrote:

> header field value first and then add that result to z=?  Is 
> preserving the fold important at all?
>
I treat a copy as just that -- a copy. There is some motivation for 
preserving the folding
insofar as with simple canonicalization the folding is important; so if 
you're trying to
figure out why simple broke, that would be an important (and probably 
common)
clue.

It might be worthwhile clarifying that in the description.

       Mike

---

From: Michael Thomas <[email-masked]>
Date: Sat, 10 Jun 2006 13:53:06 -0700

> preserving the fold important at all?
Another interesting question looking at your headers is that you include
the leading space of the header while I don't (due to that artifact of 
milters).
I'd say that you're probably right given my previous message.

Eric, what do you think?

          Mike

---

From: Arvel Hathcock <[email-masked]>
Date: Sat, 10 Jun 2006 16:00:04 -0500

> Another interesting question looking at your headers is that you
 > include the leading space of the header while I don't (due to that
 > artifact of  milters).  I'd say that you're probably right given my
 > previous message.

At the moment I think the way we're doing it is wrong actually because 
of this text in the spec:

"Copied header field values MUST immediately follow the header field 
name with a colon separator (no white space permitted)"

-- 
Arvel

---

From: Eric Allman <[email-masked]>
Date: Mon, 12 Jun 2006 14:53:24 +0100

--On June 10, 2006 2:53:06 PM -0700 Michael Thomas <[email-masked]> 
wrote:
> Arvel Hathcock wrote:
>
>> How are multi-line header field values to be handled with z=?  Do
>> we  encode the fold (by adding =0D=0A for example) or do we unfold
>> the  header field value first and then add that result to z=?  Is

I had intended that the format be unambiguous, so that all white 
space (which should be SWSP) is encoded.  Given the current ABNF, 
this means that the only folding permitted would be before a "|" or 
after a ":", which may not be what we want.

I think it's a bad idea to allow white space (including CRLF) to mean 
literal white space, since one of the things we are probably trying 
to preserve in z= is the original white space and folding of the 
original header fields.  So it seems that requiring all SWSP to be 
encoded and then allowing perhaps arbitrary addition of real SWSP 
simply to keep down line lengths may be the right thing to do.

Much of this violates RFC 2045 section 6.7, notably points (4) and 
(5).  Point (5) (soft line breaks) is especially problematic since 
they are indicated by "=CRLF" in 2045, but header folding requires at 
least one white space after the CRLF.

This clearly needs to be fixed up --- it is definitely ambiguous now. 
My suggestion is to require that all SWSP in the original header be 
encoded, and all physical SWSP in the z= tag (specifically, 
qp-hdr-value in the ABNF) be ignored.

eric

---

From: Michael Thomas <[email-masked]>
Date: Mon, 12 Jun 2006 07:49:39 -0700

Eric Allman wrote:

 > --On June 10, 2006 2:53:06 PM -0700 Michael Thomas <[email-masked]> wrote:
 >
 >> Arvel Hathcock wrote:
 >>
space (which should be SWSP) is encoded.  Given the current ABNF, this 
means that the only folding permitted would be before a "|" or after a 
":", which may not be what we want.

Definitely not what we want. The tags and values *must* be able to be 
folded as necessary.

 > I think it's a bad idea to allow white space (including CRLF) to mean 
literal white space, since one of the things we are probably trying to 
preserve in z= is the original white space and folding of the original 
header fields.

Right. Especially for simple. But what about:

"foo: bar" should that be "foo:bar" or foo:=20bar"?  This is sort of 
related to the
milter simple bug the separator white space. Mine currently does not 
insert this
at like it similarly assumes that the separator in simple is =20. It 
would be nice to
squash these once and forever.

 >   So it seems that requiring all SWSP to be encoded and then allowing 
perhaps arbitrary addition of real SWSP simply to keep down line lengths 
may be the right thing to do.

That's what I do.

 > Much of this violates RFC 2045 section 6.7, notably points (4) and 
(5).  Point (5) (soft line breaks) is especially problematic since they 
are indicated by "=CRLF" in 2045, but header folding requires at least 
one white space after the CRLF.

but what about =0a=0d? That's all I do. But I thought that the curret 
draft requires that swsp
be encoded? Then any swsp is clearly 822 syntax

My suggestion is to require that all SWSP in the original header be 
encoded, and all physical SWSP in the z= tag (specifically, qp-hdr-value 
in the ABNF) be ignored.

Can I make an ever humble suggestion that we also define simple to 
define the
simple separator character ws to be =20 and that we just elide the 
separator character for z=?

       Mike

---

From: Eric Allman <[email-masked]>
Date: Wed, 14 Jun 2006 06:08:53 -0700

> > I think it's a bad idea to allow white space (including CRLF) to
> mean literal white space, since one of the things we are probably
> trying to preserve in z= is the original white space and folding of
> the original header fields.
>
> "foo: bar" should that be "foo:bar" or foo:=20bar"?  This is sort
> of related to the milter simple bug the separator white space. Mine
> currently does not insert this at like it similarly assumes that
> the separator in simple is =20. It would be nice to squash these
> once and forever.

Personally, I would say it should be "foo:=20bar".  That's the only 
truly consistent answer.

> >   So it seems that requiring all SWSP to be encoded and then
> allowing perhaps arbitrary addition of real SWSP simply to keep
> down line lengths may be the right thing to do.
>
>
> That's what I do.

Great, sounds like the two of us, at least, are on the same page.

>  > Much of this violates RFC 2045 section 6.7, notably points (4)
> and (5).  Point (5) (soft line breaks) is especially problematic
> since they are indicated by "=CRLF" in 2045, but header folding
> requires at least one white space after the CRLF.
>
> but what about =0a=0d? That's all I do. But I thought that the
> curret draft requires that swsp be encoded? Then any swsp is
> clearly 822 syntax

Sorry, I don't understand.  I'm suggesting that if a wrapped header 
field is passed to DKIM, it should encode that in z= as =0a=0d --- 
that's from the above.  My point here is that the current spec says 
something like "Q-P as defined in 2045", but in fact we are using 
something slightly different.  2045 does not allow arbitrary 
non-significant white-space, requires that an existing CRLF sequence 
be encoded that way (not as =0a=0d), and requires "=CRLF" to mean a 
soft break.  This essentially violates all three of these.

So in some sense my question is: shall we continue to reference 2045 
and add in all the exceptions, or come up with yet-another encoding 
that might strongly resemble 2045 but which will not be identical? 
The result is the same, it's just the mechanics of the spec I'm 
questioning.

>  > This clearly needs to be fixed up --- it is definitely ambiguous
> now. My suggestion is to require that all SWSP in the original
> header be encoded, and all physical SWSP in the z= tag
> (specifically, qp-hdr-value in the ABNF) be ignored.
>
> define the simple separator character ws to be =20 and that we just
> elide the separator character for z=?

As in, replace "|" with "=20"?  That doesn't seem like a good idea to 
me because of the line wrapping problem.  I think it would be good to 
be able to encode a header such as:

        x: a
             b
        y: c  d

with

        ...  z=x:=20a=0a=0d=09b|y: =20c=20
                =20d;

Note that the actual white space in the encoding is /not/ part of the 
original input, which is all encoded.  If I'm understanding your 
suggestion, you would have to encode that as

        ...  z=x: a
                b=20y: c  d

or maybe

        ...  z=x:a=0a=0d=09b=20y:c  d

which I'm reasonably certain is not what you mean --- so I think I'm 
misunderstanding.

eric

---

From: Michael Thomas <[email-masked]>
Date: Wed, 14 Jun 2006 15:06:18 -0700

> Personally, I would say it should be "foo:=20bar".  That's the only 

I can live with that.

> that might strongly resemble 2045 but which will not be identical? The 
> result is the same, it's just the mechanics of the spec I'm questioning.

Given how simple our qp-like encoding is, I think you might be right
to just say in an informational way that it's similar to 2045, but really
define the encoding ourselves.

> be able to encode a header such as:

No, no. I meant that we simply define the white space separator as space,
both for this and simple regardless of whether it was space, tab, etc. Then
we wouldn't have to do any unnatural acts with milters :)

       Mike

---

From: Arvel Hathcock <[email-masked]>
Date: Wed, 14 Jun 2006 17:15:38 -0500

> Personally, I would say it should be "foo:=20bar".  That's the only

We'll need to strike this sentence then:

 > Great, sounds like the two of us, at least, are on the same page.
...
 > original input, which is all encoded.

I'm on the same page too.  That approach makes sense to me.  I'm for it.

-- 
Arvel

---

From: Arvel Hathcock <[email-masked]>
Date: Wed, 14 Jun 2006 17:18:59 -0500

> name with a colon separator (no white space permitted)"

And BTW, that's fine with me  :)  "foo:=20bar" is just fine in my view.

-- 
Arvel

---

From: Arvel Hathcock <[email-masked]>
Date: Wed, 14 Jun 2006 17:21:40 -0500

> to just say in an informational way that it's similar to 2045, but
 > really define the encoding ourselves.

I'm happy with that too if it's what you guys want to do.

-- 
Arvel

---

From: Eric Allman <[email-masked]>
Date: Fri, 16 Jun 2006 11:34:21 -0700

OK, enclosed is my proposed re-wording for z=.

eric

Copied header fields (plain-text, but see description; OPTIONAL, 
default is null). A vertical-bar-separated list of selected header 
fields present when the message was signed, including both the field 
name and value. It is not required to include all header fields 
present at the time of signing. This field need not contain the same 
header fields listed in the "h=" tag.  The syntax used resembles 
Quoted-Printable [RFC2045]: any character MAY be encoded as an "=" 
followed by two hexadecimal digits from the alphabet 
"0123456789ABCDEF" (no lower case characters permitted) representing 
the internal value of that character.  All control characters (those 
with values < %x20), eight-bit characters (values > %x7F), and the 
characters DEL (%x7F), SPACE (%x20), vertical bar ("|", %x7C), colon 
(":", %x3A), and semicolon (";", %x3B) MUST be encoded in both the 
header field name and the header field value.  Note that all white 
space, including CR and LF characters, must be encoded.  After 
encoding, SWSP MAY be added at arbitrary locations in order to avoid 
excessively long lines; such white space is NOT part of the value of 
the header field, and MUST be removed before decoding.

---

From: Arvel Hathcock <[email-masked]>
Date: Mon, 19 Jun 2006 17:23:12 -0500

We think this text is good here.  One of my engineers did suggest a 
small change which I thought I'd pass on for your judgement:

--SNIP--
It looks good but I don't see a reference to the colon separating the 
field name from the value.  Perhaps something like this would be ok:

"A vertical-bar-separated list of selected header fields present when 
the message was signed, including the field name, a colon, and the field 
value. "  ?
--SNIP--

-- 
Arvel Hathcock
CEO, Alt-N Technologies
http://www.altn.com