Subject: [art] Question on RFC 8823

From: Steffen Nurpmeso <[email-masked]>
Date: Thu, 06 May 2021 18:43:08 +0200

Hello.

(I am not subscribed to acme@, i hope it is ok to post here.)

I was reading RFC 8823, i think it is great that a technical way
has been found that seems to be acceptable for all.  I (who
personally still believes that passports should be government-run,
fwiw) looking forward, and hope that encrypted mail messages will
at some time be as easy and free for everybody as secured web is
now, as of Let's Encrypt (thanks).

While reading it i was astonished to see RFC 2231 not RFC 2047
mentioned for header field content (as opposed to field parameter
value content that does not occur here since all the value is
a base64url encoded string, see for example 3.1.1 -- i mean, yes!,
RFC 2231 is a straight good one compared to the older RFC 2047,
but it does not replace it), and the use of MIME in general even
for plain US-ASCII mails (without overlong lines and any other
such pitfalls which would qualify for using MIME in an RFC 5322
internet message).  And the DKIM-covered range extends to fields
which should never happen unless i am mistaken (like Resent-*,
List-* etc., the latter of which even seem explicitly forbidden).

(Also i do not understand S/MIME signing, the RFC does not speak
about the signing authority of the used certificate at all, and
here there could be a chicken and egg problem, or a "snake that
bites its own tail", shall you know that term, for self-signed
ones.)

I never have opened a RFC errata yet (i am usually very very
late to the party, and what has sailed has sailed, i mean, the
context is in the world and so i use that and leave the RFC as-is,
i surely remember some nits here and there), and given the names
and the WG and the sheer number of eyes in this area .. does the
above qualify for one?

Thank you in advance, and greetings from Germany,

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 6 May 2021 20:20:00 +0100

Hi Steffen,

Thank you for your comments on RFC 8823. My replies below.

On 06/05/2021 17:43, Steffen Nurpmeso wrote:
> internet message).

If you are talking about encoded word construct used in Subject header 
field, then RFC 2231 is backward compatible with RFC 2047 encoding, so 
any RFC 2047 encoded words are also valid RFC 2231 encoded words. The 
document requires support for RFC 2231 in order to have compatibility 
with most existing email clients.

> And the DKIM-covered range extends to fields
This is a feature of DKIM to prevent insertion of these header fields by 
signing their absence in the original message. So this is quite deliberate.
> bites its own tail", shall you know that term, for self-signed
I believe this is really outside the scope for the RFC.
> and the WG and the sheer number of eyes in this area .. does the

Hopefully my answers above clarified why I believe existing text in the 
RFC is correct, but I am happy to continue the discussion in case I 
missed something.

Best Regards,

Alexey

> _______________________________________________
> art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/art

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 May 2021 12:49:02 -0700 (PDT)

> ...

> any RFC 2047 encoded words are also valid RFC 2231 encoded words.

But not vice versa, which is why, in the unlikely event someone actually uses
the RFC 2231 extension, it will work.

Specifically, RFC 2231 adds an optional language tag to the charset
part of an encoded word. If that feature is used and only RFC 2047
forms are supported the charset field isn't going to match.

That said, if RFC 8823 is ever updated, it might be a good idea to say what to
do with the language tag: Don't generate it and if you find one, ignore it.

> The document requires support for RFC 2231 in order to have compatibility

There's a client that actually generates a language tag in an encoded word?
Wow. I don't think I've ever seen one in the wild.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 10 May 2021 11:43:48 +0100

Hi Ned,

> forms are supported the charset field isn't going to match.
Actually, I suspect there is more than one parser that is able to parse 
RFC 2231 syntax, but ignores the language tag.
> do with the language tag: Don't generate it and if you find one, 
Agreed.
> Wow. I don't think I've ever seen one in the wild.

I thought I've seen some, but a quick check of my INBOX archive for the 
last 3 years doesn't have any examples.

Best Regards,

Alexey

---

From: Steffen Nurpmeso <[email-masked]>
Date: Mon, 10 May 2021 22:03:26 +0200

Alexey Melnikov wrote in
 <[email-masked]>:
 |On 06/05/2021 20:49, Ned Freed wrote:
 ...
 |> Specifically, RFC 2231 adds an optional language tag to the charset
 |> part of an encoded word. If that feature is used and only RFC 2047
 |> forms are supported the charset field isn't going to match.

 |Actually, I suspect there is more than one parser that is able to parse 
 |RFC 2231 syntax, but ignores the language tag.

 |> That said, if RFC 8823 is ever updated, it might be a good idea to say 
 |> what to
 |> do with the language tag: Don't generate it and if you find one, 
 |> ignore it.

 |> There's a client that actually generates a language tag in an encoded 
 |> word?
 |> Wow. I don't think I've ever seen one in the wild.
 |
 |I thought I've seen some, but a quick check of my INBOX archive for the 
 |last 3 years doesn't have any examples.

We neither, and ignore it.
But still, RFC 2047 repeats terms needlessly whereas RFC 2231 then
uses continuations (also meaning you can first join the pieces and
act upon the decoded whole).
And, if i recall this right, RFC 2047 may force introduction of
artifical spaces shall line-breaking be needed at non-whitespace
(the leading whitespace of follow lines belongs to the value).

--steffen
|

---

From: Ned Freed <[email-masked]>
Date: Mon, 10 May 2021 13:58:24 -0700 (PDT)

>  |RFC 2231 syntax, but ignores the language tag.

You can also use a RFC 2047 parser and then remove the language tag yourself.
Part of the reason for saying it must be ignored is to suggest such approaches.

> act upon the decoded whole).

We're talking about encoded-words here, not content-type or
content-disposition parameters. RFC 2231 does not define
a continuation mechanism for encoded-words; it is identical
to RFC 2047 in this regard.

RFC 8823 is also quite specific in saying that RFC 2231 support
is needed for encoded-words for decoding the subject. It doesn't
require it anywhere else.

The only place where continuations might possibly be relevant in
the boundary (on multipart) or charset (on content-type) parameters.

Given there's no requirement for RFC 2231 support there, not to
mention there being no need to use continuations, if someone is
foolish enough to do this, they'll probably get what they deserve.

> (the leading whitespace of follow lines belongs to the value).

The maximum length of an encoded-word is restricted to 75 characters
in order to insure that the use of encoded-words is compatible with
the 78 character recommended line length. Beyond that, it is of course
the case that introduction of charset (and in RFC 2231, language)
information plus encoding is always going to increase the length
of the value, sometimes very substantially, and the folding needed
is going to be quite different.

Applications that (incorrecty) depend on line folding in headers being
preserved don't get along well with RFC 2047.

RFC 2231 changes none of this.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Mon, 10 May 2021 18:25:55 -0400

> act upon the decoded whole).

RFC2047 and its predecessors were designed to be compatible with the 
behavior of mail user agents then in existence, which could in some 
cases change the order of names/addresses in message headers when 
composing replies.  (I suspect that such behavior still exists, if for 
no other reason than to prevent duplicate recipients in the same message 
headers).   So repeating "terms" is not needless, it's required to deal 
with potential reordering.

> (the leading whitespace of follow lines belongs to the value).

Correct.   And this was a result of trying to make sure that MIME 
messages using encoded-words could still be encoded in lines of 79 
graphic characters or less, so that they would work with BITNET. (Though 
it's also an oversight, encoded-words could have been designed with some 
convention that meant "ignore white space that precedes this encoded-word".)

Keith

---

From: Steffen Nurpmeso <[email-masked]>
Date: Tue, 11 May 2021 18:49:52 +0200

Keith Moore wrote in
 <[email-masked]>:
 |On 5/10/21 4:03 PM, Steffen Nurpmeso wrote:
 |
 |> But still, RFC 2047 repeats terms needlessly whereas RFC 2231 then
 |> uses continuations (also meaning you can first join the pieces and
 |> act upon the decoded whole).
 |
 |RFC2047 and its predecessors were designed to be compatible with the 
 |behavior of mail user agents then in existence, which could in some 
 |cases change the order of names/addresses in message headers when 
 |composing replies.  (I suspect that such behavior still exists, if for 
 |no other reason than to prevent duplicate recipients in the same message 
 |headers).   So repeating "terms" is not needless, it's required to deal 
 |with potential reordering.
 |
 |> And, if i recall this right, RFC 2047 may force introduction of
 |> artifical spaces shall line-breaking be needed at non-whitespace
 |> (the leading whitespace of follow lines belongs to the value).
 |
 |Correct.   And this was a result of trying to make sure that MIME 
 |messages using encoded-words could still be encoded in lines of 79 
 |graphic characters or less, so that they would work with BITNET. (Though 
 |it's also an oversight, encoded-words could have been designed with some 
 |convention that meant "ignore white space that precedes this encoded-wor\
 |d".)

Yeah.  Just enforce another encoded word on the follow line should
do it.  Sorry.

--steffen
|

---

From: Steffen Nurpmeso <[email-masked]>
Date: Tue, 11 May 2021 20:14:38 +0200

Steffen Nurpmeso wrote in
 <[email-masked]>:
 |Keith Moore wrote in
 | <[email-masked]>:
 ||On 5/10/21 4:03 PM, Steffen Nurpmeso wrote:
 ||
 ||> But still, RFC 2047 repeats terms needlessly whereas RFC 2231 then
 ||> uses continuations (also meaning you can first join the pieces and
 ||> act upon the decoded whole).
 ||
 ||RFC2047 and its predecessors were designed to be compatible with the 
 ||behavior of mail user agents then in existence, which could in some 
 ||cases change the order of names/addresses in message headers when 
 ||composing replies.  (I suspect that such behavior still exists, if for 
 ||no other reason than to prevent duplicate recipients in the same message 
 ||headers).   So repeating "terms" is not needless, it's required to deal 
 ||with potential reordering.
 ||
 ||> And, if i recall this right, RFC 2047 may force introduction of
 ||> artifical spaces shall line-breaking be needed at non-whitespace
 ||> (the leading whitespace of follow lines belongs to the value).
 ||
 ||Correct.   And this was a result of trying to make sure that MIME 
 ||messages using encoded-words could still be encoded in lines of 79 
 ||graphic characters or less, so that they would work with BITNET. (Though 
 ||it's also an oversight, encoded-words could have been designed with some 
 ||convention that meant "ignore white space that precedes this encoded-wor\
 ||d".)
 |
 |Yeah.  Just enforce another encoded word on the follow line should
 |do it.  Sorry.

(P.S.: my statement was solely based upon "our" codebase (i had
stated that but removed the text from my email again, it was way
too long anyway); it is not
a deficiency of RFC 2047, as you have thankfully pointed out, but
now i feel i have to defend _my_self, different to you i stand in
the looser corner, however.

"We" cannot actually do this _yet_ because we operate on
iconv(3)ed data in an intransparent destination character set, and
because each RFC 2047 encoded word has to be "atomic" aka
self-describing, different to RFC 2231 where the parts can be
joined to form a whole, we cannot just split somewhere.  There is
also no (portable) mbtowc_l(3), and iconv(3) converts chunks of
data, not "one at a time".  So we operate on something entirely
intransparent, only being lucky that whitespace characters are
mostly portable across character sets, and have to adhere to RFC
2047 saying that each encoded word must be atomic.  This was
actual criticism regarding RFC 2047 once i read it.  With POSIX
standard methods (not to talk about ISO, a no-go) one could only
try brute-force to get this right, like McIlroy asking for
TeX-like paragraph formatting for plain roff.  I.e., like for
stateful encodings, where each RFC 2047 encoded word would need to
include reset sequences etc.  How much easier it is for RFC 2231,
where you just join.

I wholeheartly agree that the MIME part of the software i maintain
must be rewritten as such, this will happen in the future (if
nothing really bad happens to me), and if i have a good day it may
then be something that i like.)

Ciao from Germany,

--steffen
|