Subject: Re: 2822upd-04 "local" time and zone offsets
From: "Charles Lindsey" <[email-masked]>
To: [email-masked]
Date: Tue, 29 Jan 2008 13:09:37 GMT
List-Id: <ietf-822.imc.org>

In <[email-masked]> [email-masked] writes:

>I'm 100% behind Pete on this one - using UTC is usually a terrible idea (unless
>of course your local time zone happens to be UTC) bcause it causes important
>information to be lost.

+1

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: obs-commas
Date:  Tue, 29 Jan 2008 14:37:39 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 9
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

> that is comma-separated <address>es,
..........^^^^^^^^^^^^^^^
[...]
>| Reply-To: The Big Mailing List: (foo); my list: (bar);
..........................................^
Missing comma, but the effect with comma is not better:

| Reply-To: The Big Mailing List: (foo);, my list: (bar);

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: obs-commas
Date:  Tue, 29 Jan 2008 14:15:31 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 55
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Frank Ellermann wrote:

> I refuse to tell the DKIM WG that their "SSP magic" might
> have to deal with a From: The Big Mailing List: (yeah ;-)

Nonsense.  The <group> case of an <address> makes no sense
in "From:", and therefore <from> and <sender> permit only
<mailbox-list> and <mailbox> respectively.

I can't add a CFWS to <mailbox-list> just to get rid of the
<group-list>, CFWS has to be added to <group> for this job:

: group           =3D   display-name ":" [mailbox-list / CFWS] ";" =
[CFWS]
| mailbox-list    =3D   (mailbox *("," mailbox)) / obs-mbox-list
: obs-mbox-list   =3D   [mailbox] *("," [mailbox / CFWS])
234567.10.234567.20.234567.30.234567.40.234567.50.234567.60.234567

That's too long, getting rid of <group-list> was a bad idea.

Interesting:  Unlike <from> and <sender> the <reply-to> can
be an <address-list>, that is comma-separated <address>es,
each <address> can be a <group>, and that is not guaranteed
to contain a single <mailbox>:

| Reply-To: The Big Mailing List: (foo); my list: (bar);

Is that as it should be ?

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: obs-commas
Date:  Tue, 29 Jan 2008 13:45:31 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 68
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

> Here's all of the syntax in question:
> group           =3D   display-name ":" [group-list] ";" [CFWS]
.......................................^..........^
ACK, I missed that the complete group-list is already optional.

> Now, here's my new obs-mailbox syntax:
> obs-mbox-list   =3D   *([CFWS] ",") mailbox *("," [mailbox / CFWS])

Looks good, unnecessary [CFWS] omitted using the leading and
trailing [CFWS] in <mailbox>, and at least one <mailbox> as
you wanted it...

> The only thing not described by the above is a mailbox list
> with no mailboxes at all (i.e., simply commas with optional
> surrounding CFWS).

...or maybe not.  If no <mailbox> at all is okay, how about:

  obs-mbox-list   =3D   [mailbox] *("," [mailbox / CFWS])

But in -05 with <obs-commas> it was "at least one <mailbox>",
either in <mailbox-list> or in <obs-mbox-kist>.  The former
with one comma per <mailbox> pair, and the latter allowing
unnecessary commas. =20

Parsing left to right it's no problem if each <mailbox-list>
also matches <obs-mbox-list>.

> That's:
> obs-group-list  =3D   1*([CFWS] ",") [CFWS]

> I'm happy to simplify if possible. But so far, that's the
> best I can do.

Maybe I now see what you are talking about:  If you drop the
"at least one" from <obs-mbox-list> you could eliminate the
complete <obs-group-list>.

Except from the one case covered by "1" in <obs-group-list>,
which still gives you "if nothing else at least one comma".

But you don't need "at least one comma", because "nothing"
and "only CFWS" are already covered (and not obsolete):

1: "nothing" is "no <group-list>" in [group-list] of <group>.
2: "only CFWS" is the explicit CFWS in <group-list>
3: "one <mailbox>, no comma" is covered by <mailbox-list>
4: "more than one <mailbox>, good commas" is <mailbox-list>
5: "more than one <mailbox>, ugly commas" is <obs-mbox-list>
6: "no <mailbox>, but commas" does not need "at least one".

Putting it all together I get:

| group           =3D   display-name ":" [group-list] ";" [CFWS]
: group-list      =3D   mailbox-list / CFWS
| mailbox-list    =3D   (mailbox *("," mailbox)) / obs-mbox-list
: obs-mbox-list   =3D   [mailbox] *("," [mailbox / CFWS])

No more <obs-group-list>, and if that's really okay you could
also get rid of <group-list>:

: group           =3D   display-name ":" [mailbox-list] ";" [CFWS]
: mailbox-list    =3D   (mailbox *("," mailbox)) / CFWS / obs-mbox-list
: obs-mbox-list   =3D   [mailbox] *("," [mailbox / CFWS])

 Frank

Date: Mon, 28 Jan 2008 21:33:26 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: obs-commas (was: 2822upd-05 just submitted)
Cc: [email-masked]
Sender: [email-masked]

On 1/29/08 at 3:40 AM +0100, Frank Ellermann wrote:

>IOW, is this really CFWS, and [CFWS] is not good enough ?

Uh, I think we're talking past each other (and apologies to the list 
if this discussion is getting a bit obscure). CFWS *is* optional in 
group. Here's all of the syntax in question:

group           =   display-name ":" [group-list] ";" [CFWS]
display-name    =   phrase
group-list      =   mailbox-list / CFWS / obs-group-list
mailbox-list    =   (mailbox *("," mailbox)) / obs-mbox-list

So, in the generate syntax, group-list is optional in group, and may 
contain either a mailbox-list or CFWS.

Now, here's my new obs-mailbox syntax:

obs-mbox-list   =   *([CFWS] ",") mailbox *("," [mailbox / CFWS])

The only thing not described by the above is a mailbox list with no 
mailboxes at all (i.e., simply commas with optional surrounding 
CFWS). That's:

obs-group-list  =   1*([CFWS] ",") [CFWS]

I'm happy to simplify if possible. But so far, that's the best I can do.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: obs-commas (was: 2822upd-05 just submitted)
Date:  Tue, 29 Jan 2008 03:40:42 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 22
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:
=20
>>| group-list      =3D mailbox-list / CFWS / obs-group-list
>>[...]
>>| obs-group-list  =3D obs-commas

>> The most obscure case, is "only CFWS" really *NOT* obsolete ?
=20
> Sure, this seems pretty reasonable to do:
=20
> To: The Big Mailing List: (Names removed to protect the innocent);

Admittedly <group-list> is one of the constructs I've never
seen "in the wild", let alone used.  Just to be sure, CFWS
is "comment or folding white space", and a <group-list>
can be one or more spaces among others, but *NOT* empty ?
IOW, is this really CFWS, and [CFWS] is not good enough ?

I refuse to tell the DKIM WG that their "SSP magic" might
have to deal with a From: The Big Mailing List: (yeah ;-)

 Frank

Date: Mon, 28 Jan 2008 20:08:59 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: obs-commas (was: 2822upd-05 just submitted)
Cc: [email-masked]
Sender: [email-masked]

On 1/28/08 at 8:05 PM -0600, I wrote:

>On 1/28/08 at 7:53 PM -0600, I wrote:
>
>Oooooo.....this works doesn't it:
>
>obs-mbox-list   =   *([CFWS] ",") mailbox *([CFWS] "," [mailbox]) [CFWS]

Or even better:

obs-mbox-list   =   *([CFWS] ",") mailbox *("," [CFWS] [mailbox])

OK, I'll simply these for -06 and get rid of obs-commas.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 28 Jan 2008 20:05:20 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: obs-commas (was: 2822upd-05 just submitted)
Cc: [email-masked]
Sender: [email-masked]

On 1/28/08 at 7:53 PM -0600, I wrote:

>...obs-domain-list, obs-mbox-list, and obs-addr-list was that each 
>require at least one domain/mailbox/address, but could have multiple 
>commas on either end or in the middle.
[...]
>I'm not sure how to do that with your form. I'd be happy if you had 
>a solution.

Oooooo.....this works doesn't it:

obs-mbox-list   =   *([CFWS] ",") mailbox *([CFWS] "," [mailbox]) [CFWS]

I'll play with this a bit more.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 28 Jan 2008 19:53:30 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: obs-commas (was: 2822upd-05 just submitted)
Cc: [email-masked]
Sender: [email-masked]

On 1/29/08 at 12:19 AM +0100, Frank Ellermann wrote:

>End of 3.3, maybe s/of zone/of the zone/
>
>For <ftext> kill the stray "/" after 126.

Fixed. Thanks.

>| obs-commas      = [CFWS] "," *([CFWS] ",") [CFWS]
>| obs-phrase-list = [phrase / CFWS] *(obs-commas phrase) [obs-commas]
>
>This is rather strange, it was simpler for 2831bis

Yeah, I made obs-phrase-list more complicated than it had to be by 
re-using obs-commas. I guess I could make obs-phrase-list not like 
the others.

>If my proposal is correct the same pattern could be used for all 
>similar obs-lists, eliminating the obscure <obs-commas>.

No, the problem I ran into obs-domain-list, obs-mbox-list, and 
obs-addr-list was that each require at least one 
domain/mailbox/address, but could have multiple commas on either end 
or in the middle. So this is legal:

To:    ,  ,  , [email-masked] ,  ,  ,

but this is not:

To:   ,  ,  ,  ,  ,

I'm not sure how to do that with your form. I'd be happy if you had a solution.

>| group-list      = mailbox-list / CFWS / obs-group-list
>[...]
>| obs-group-list  = obs-commas
>
>The most obscure case, is "only CFWS" really *NOT* obsolete ?

Sure, this seems pretty reasonable to do:

To: The Big Mailing List: (Names removed to protect the innocent);

>| obs-bcc = "Bcc" *WSP ":"
>|           [address-list / CFWS / obs-commas] CRLF
>
>: obs-bcc = "Bcc" *WSP ":"
>:           (address-list / ( *([CFWS] ",") [CFWS] )) CRLF

Again, that I can do if I wish to forego reuse of obs-commas. The 
other nice thing about re-using obs-commas here and in 
obs-phrase-list is that it makes it clear why it is there ("Another 
one of those silly collections of just commas").

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: obs-commas (was: 2822upd-05 just submitted)
Date:  Tue, 29 Jan 2008 00:50:59 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 13
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Frank Ellermann wrote:

> obs-phrase-list =3D [phrase] *([CFWS] "," [CFWS] [phrase]) [CFWS]

Too long, <phrase> has its own leading and trailing [CFWS]:

| obs-phrase-list =3D [phrase] *([CFWS] "," [phrase]) [CFWS]

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  obs-commas (was: 2822upd-05 just submitted)
Date:  Tue, 29 Jan 2008 00:19:17 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 48
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:
=20
> See:
[diff URL]

Thanks. =20

End of 3.3, maybe s/of zone/of the zone/=20

For <ftext> kill the stray "/" after 126.

| obs-commas      =3D [CFWS] "," *([CFWS] ",") [CFWS]
| obs-phrase-list =3D [phrase / CFWS] *(obs-commas phrase) [obs-commas]=20

This is rather strange, it was simpler for 2831bis, example:

 digest-challenge =3D LWSP d-c-e *(LWSP "," LWSP [d-c-e]) LWSP

Trying this for <obs-phrase-list> I get:

  obs-phrase-list =3D [phrase] *([CFWS] "," [CFWS] [phrase]) [CFWS]

Both versions produce adjacent [CFWS] needing the MUSTard in
the prose about never generating "apparently empty lines",
i.e. LWSP is only accepted in the form of <obs-FWS>, but that
is no real issue, as anything in obs-* MUST NOT be generated.

If my proposal is correct the same pattern could be used for
all similar obs-lists, eliminating the obscure <obs-commas>.

| group-list      =3D mailbox-list / CFWS / obs-group-list
[...]
| obs-group-list  =3D obs-commas

The most obscure case, is "only CFWS" really *NOT* obsolete ?
Otherwise if you join CFWS and <obs-commas> you get:=20

: obs-group-list  =3D *([CFWS] ",") [CFWS]=20

The same idea for <obs-bcc>, and likewise <obs-resent-bcc>:

| obs-bcc =3D "Bcc" *WSP ":"=20
|           [address-list / CFWS / obs-commas] CRLF

: obs-bcc =3D "Bcc" *WSP ":"
:           (address-list / ( *([CFWS] ",") [CFWS] )) CRLF

 Frank

Date: Mon, 28 Jan 2008 14:41:59 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Local copies (html/txt/xml) always on home page
Sender: [email-masked]

By the way:

I always keep local copies of the xml source, html, and txt files on 
my home page:

<http://resnick1.qualcomm.com>

If you want the current files, they are:

<http://resnick1.qualcomm.com/draft-resnick-2822upd-05.txt>
<http://resnick1.qualcomm.com/draft-resnick-2822upd-05.html>
<http://resnick1.qualcomm.com/draft-resnick-2822upd-05.xml>

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 28 Jan 2008 14:38:19 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: 2822upd-05 just submitted
Sender: [email-masked]

See:

<http://tools.ietf.org/rfcdiff?url1=http://resnick1.qualcomm.com/draft-resnick-2822upd-04.txt&url2=http://resnick1.qualcomm.com/draft-resnick-2822upd-05.txt>

for changes.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 28 Jan 2008 11:08:16 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: 2822upd-04 definition of terms needed
Sender: [email-masked]

At 12:24 27-01-2008, Bruce Lilly wrote:
>I don't pretend to know whether or not any given person wants to do
>the right thing.  I do know that a significant number of systems
>are doing something quite silly, and I suspect that that silliness

Agreed.

>is due in large measure to ambiguities in 2822 and/or conflicts
>with other relevant specifications, and I further suspect that the
>amount of such silliness will increase in the absence of
>clarification.

I suspect that this is due to people not reading the RFC to do the right thing.

>One example of the sort of suspected conflict, and the reason for the
>request for definitions, is illustrated by the following example of
>silliness:
>
>Party A sends a message (e.g. an error report of some sort) to party
>B.  Because A, a good net citizen, wishes to avoid mail loops, he
>sets the envelope address to a null path in accordance with the
>provisions established for that purpose by RFC 821 and affirmed by
>RFC 1123 (though curiously not permitted by 2821, but that's not
>an issue for the draft under discussion).  A does not want any
>delivery reports or disposition notifications or message tracking
>reports (or any response at all), so consequently does not include
>SMTP commands and/or message fields which would trigger those (RFCs
>3461-3464, 3798, 3888).  A specifically states (in the message body)
>that he does not want any response.  Upon receipt of the message from
>A (with null return path), B, a rather silly system, sends an automatic
>receipt message. Contrary to years of established practice and good
>sense, not to mention a documented strong recommendation against doing
>so in RFC 3834, B sends that automated reply to the mailbox(es) in the
> >From field of A's original message, completely ignoring the null return
>path.  Why does B do such a silly thing?  Because RFC 2822 tells him to
>do so (as does the current draft text, from which the following are
>quoted):

B does it because he/she has not read RFC 3834.

>section 3.6.2, omitting pagination text and whitespace for brevity):
>
>      In the absence of the "Reply-To:" field,
>    replies SHOULD by default be sent to the mailbox(es) specified in the
>    "From:" field unless otherwise specified by the person composing the
>    reply.
>
>and section 3.6.3:
>
>    When a message is a reply to another message, the mailboxes of the
>    authors of the original message (the mailboxes in the "From:" field)
>    or mailboxes specified in the "Reply-To:" field (if it exists) MAY
>    appear in the "To:" field of the reply since these would normally be
>    the primary recipients of the reply.
>
>
>Note that neither text distinguishes between "automatic" (as used in
>RFC 3834) replies vs. manually-initiated replies.  It appears (lacking
>a real definition) that the use of "automatic" and variants in 2822upd-04
>differs markedly from the meaning in RFC 3834; but (lacking a real
>definition of how they are intended to be interpreted in 2822upd-04)
>that's just a hunch.

When we reply to a message, it is usually addressed to the 
author(s).  Automated replies are usually addressed to the sender of 
the message.  There is a distinction between the two.

>What can A do to avoid these annoying automated replies?  Not much
>(at least if he "really wants to do the right thing") according to
>2822upd-04:

"A" cannot do much as he/she depends on B to do the right thing.

Regards,
-sm 

Date: Mon, 28 Jan 2008 17:55:19 +0000
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: 2822upd-04 "local" time and zone offsets
Sender: [email-masked]

On Sun, 27 Jan 2008, Bruce Lilly wrote:
>
> The issues arise from the difficulty in converting *correctly* to an
> unambiguous time reference (i.e. UTC) which in turn is partially due to
> an unfortunate choice in the sign of the offset, (resulting in
> erroneously adding the offset to local time to get UTC when it should be
> subtracted).

RFC 2822 and RFC 3339 and ISO 8601 use the same sign. AFAIK only the old
Unix TZ specifications use the wrong sign which I believe was for the
convenience of users in the same hemisphere as the developers.

Apart from this I'll just agree with Pete and Ned.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SHANNON: SOUTHWEST VEERING NORTHWEST 5 TO 7. ROUGH OR VERY ROUGH. RAIN THEN
SHOWERS. MODERATE OR GOOD.

To: [email-masked]
Xref: clerew local.mime:5256
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: 2822upd-04 section 1 vs. section 4.1 discrepancy
Date: Mon, 28 Jan 2008 14:36:17 GMT
Lines: 39
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Pete Resnick wrote:

>>> I suggest eliding the sentence in the note in section
>>> 4.1 which begins with "In the future,".

The words "in the future" clearly implied (and presumably intentionally
so) that 'Joe G. Public' was not only to be accepted, but was a deliberate
extension to RFC 822 which was to be allowed in the 'generate' syntax at
some future date, when all implementations could be presumed to have caught
up.

Moreover, discussions on this List a few years back that this "obs-"
construct should be redescribed as an "extended-" construct confirmed that
such was how people understood those words at that time.
> 
>> I just noticed that the other day myself. Happy to have
>> confirmation of my gut reaction to it. Will delete.

-1

>Good, IIRC this detail alone took USEFOR several weeks.

Well that was one of the changes in RFC 2822 that USEFOR took on board, by
requiring it as "accept but not generate(yet)" (as opposed to the rest of
the obs-syntax - with one other minor exception) which USEFOR does not
require to be accepted.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 28 Jan 2008 09:09:54 -0800
To: ietf-822 <[email-masked]>
From: SM <[email-masked]>
Subject: Re: 2822upd-04 definition of terms needed
Sender: [email-masked]

At 20:30 26-01-2008, Bruce Lilly wrote:
>local time     as used in section 3.3.  Specifically, what is "local time"
>if a (human) message composer is sitting in an airplane composing a
>message via a telnet link to a (ground-based) host computer which will
>then send the message via a separate, remote submission server?  [By
>the way, the airplane is crossing over the (North) polar ice cap as
>the composer travels an intercontinental great circle route to his
>destination.]

Local time is the time to user considers as local.  From the user's 
perspective, the local time does not change as the airplane crosses 
the ice cap.  We might adjust our watches before we reach the 
destination and not as we cross each time zone.

Regards,
-sm 

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 "local" time and zone offsets
Date:  Mon, 28 Jan 2008 17:57:12 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 37
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Ned wrote:

> not all systems run their clocks in UTC. (In fact the machine I'm=20
> using right now to enter this message uses a local time clock.

Same here.  I know the issue, in a httpd-script it forced me to
use HTTP/1.0 and -0000, because HTTP/1.1 does not permit stupid
servers.  Just knowing that it's "local time" isn't the same as
knowing the correct timezone.

  [2359 vs. 9959]
>> You could limit it to say 23 hours.  With that the syntax
>> would more obviously indicate that it's actually about hh:mm
>> and not 9999 minutes.
=20
> Again I'm with Pete - I see no reason for such a restriction.

It could help casual future readers, as you described them here:

| In most cases it isn't possible to communicate with the=20
| responsible parties so there's no way to know why they did it.
| But when it is and they open their mouths to respond what
| usually emerges is a loud sucking sound resulting from the
| total vacuum of salient information between their ears.

| It is nothing short of stunning how many people think they
| know how to implement a protocol (the specific protocol is
| irrelevant) just from a cursory examination of example protocol
| exchanges from some random book they flipped through one day at
| a local bookstore. And no amount of added prose, or more precise
| ABNF, or anything else we do is going to change this. In fact by
| making our specifications longer we if anything make the problem
| slightly worse, not better.

Maybe they would get the idea of 2359.  Or maybe not :-)

 Frank

Date: Mon, 28 Jan 2008 08:53:32 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: 2822upd-04 definition of terms needed
Sender: [email-masked]

[email-masked] wrote:
>     I run into incompliant behavior of this and many
> other sorts all the time. In most cases it isn't possible to communicate with
> the responsible parties so there's no way to know why they did it. 
...
> It is nothing short of stunning how many people think they know how to
> implement a protocol (the specific protocol is irrelevant) just from a cursory
> examination of example protocol exchanges from some random book they flipped
> through one day at a local bookstore. And no amount of added prose, or more
> precise ABNF, or anything else we do is going to change this. In fact by making
> our specifications longer we if anything make the problem slightly worse, not
> better.

+1

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 28 Jan 2008 08:51:52 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: 2822upd-04 "local" time and zone offsets
Sender: [email-masked]

[email-masked] wrote:
> I'm 100% behind Pete on this one - using UTC is usually a terrible idea (unless
> of course your local time zone happens to be UTC) bcause it causes important
> information to be lost.

To emphasize this point:

Knowing the author's local time of submission is often useful. (Knowing they 
posted at 3am their time versus 3pm is significant for some exchanges.)

With local time plus timezone, you get local time plus the ability to determine 
UTC.  With UTC as the encoded time, you can't tell local time.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: [email-masked]
Cc: ietf-822 <[email-masked]>
Date: Mon, 28 Jan 2008 07:59:22 -0800 (PST)
Subject: Re: 2822upd-04 body content restrictions
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

> 2822upd-04 section 2.3 reads in part:

>    o  CR and LF MUST only occur together as CRLF; they MUST NOT appear
>       independently in the body.

I, OTOH, think this is exactly what 2822bis should say. (As a side note, I'm
quite unhappy with 2821bis's handling of bare CR and LF, in particular the rule
that they MUST NOT be treated as line terminators under any circumstances flies
in the face of considerable operational experience and sometimes puts
intermediaries in an impossible position, but I put up with it because it does
reflect the consensus DRUMS reached.)

> This makes me uneasy for several reasons:

> 1. it bucks the trend at liberalizing body content (e.g. 8BITMIME and BINARY
>    SMTP extensions).

That's just a matter of layering/timing. MIME extends these specifications
to allow stuff that used to be prohibited.

I note in passing that only binary MIME gets you to a place where bare CR and
LF are allowed without encoding. The extension 8bitMIME makes to 2822 is
allowing 8bit.

> 2. it (together with the disclaimer that appears further down the page)
>    implies that MIME messages do not conform to the IMF specification,
>    which is counter to the intent of MIME.

8bitMIME and binary MIME messages don't conform and it is pointless to pretend
otherwise.

> 3. the use of RFC 2119 terms implies that MIME messages used with extended
>    SMTP transport cannot interoperate; that impugns a great deal of careful
>    work to ensure that ESMTP negotiation and MIME are backward-compatible.

I'm not sure I fully understand what you're saying but I'm pretty sure I
disagree.

The way these specifications are layered on top of one another is a complex
compromise that attempts to balance the need for backwards compatibility and
not declaring old implemenations to be invalid with the need to be able to move
forward with stuff thqt wasn't envisioned when the antecedants to these
specifications where first created. It is ugly in places? You bet! And there's
no doubt this makes some people uncomfortable.

But validation of the approach lies in the results it has produced. MIME is now
widely deployed - although I have no proof, my sense is that the majority of
email handling software in use today supports it. Contrast this with previous
attempts at multimedia mail, none of which deployed to any significant degree
even though the network was smaller and simpler at the time and hence more
amenable to change. And while there's no way to be sure to what extent MIME's
focus on backwards compatibility helped in getting it deployed, there is
anecdotal evidence that it was an important factor.

Moreover, the actual confusion this approach has caused (and please remember
that when people have questions about MIME my email address in RFC 2045 is the
only one that still works) seems to have been pretty minimal. I'm all for
solving problems, but I don't think think there's a real problem here.

				Ned

From: [email-masked]
Cc: [email-masked]
Date: Mon, 28 Jan 2008 07:34:59 -0800 (PST)
Subject: Re: 2822upd-04 definition of terms needed
To: Bruce Lilly <[email-masked]>
Sender: [email-masked]

> > As I've said before to others, people can purposely
> > misread any specification we right, so I'm not inclined to try to
> > complicate things to try plug some perceived hole unless it's
> > actually going to stop someone who really wants to do the right thing
> > from doing something silly.

> I don't pretend to know whether or not any given person wants to do
> the right thing.  I do know that a significant number of systems
> are doing something quite silly, and I suspect that that silliness
> is due in large measure to ambiguities in 2822 and/or conflicts
> with other relevant specifications, and I further suspect that the
> amount of such silliness will increase in the absence of
> clarification.

I'm sorry, but absent a bunch of specific examples I'm going to have to
disagree with you on this. I run into incompliant behavior of this and many
other sorts all the time. In most cases it isn't possible to communicate with
the responsible parties so there's no way to know why they did it. But when it
is and they open their mouths to respond what usually emerges is a loud sucking
sound resulting from the total vacuum of salient information between their
ears.

It is nothing short of stunning how many people think they know how to
implement a protocol (the specific protocol is irrelevant) just from a cursory
examination of example protocol exchanges from some random book they flipped
through one day at a local bookstore. And no amount of added prose, or more
precise ABNF, or anything else we do is going to change this. In fact by making
our specifications longer we if anything make the problem slightly worse, not
better.

There are problems that we simply cannot solve and I'm very much afraid this is
one of them.

				Ned

From: [email-masked]
Cc: [email-masked]
Date: Mon, 28 Jan 2008 06:56:07 -0800 (PST)
Subject: Re: 2822upd-04 "local" time and zone offsets
To: Frank Ellermann <[email-masked]>
Sender: [email-masked]

> Pete Resnick wrote:

> >>> they can pick UTC.  Maybe 2822upd could drop the SHOULD (?)

> >> I'd like to see it dropped, or better still, replaced by a
> >> recommendation to use UTC.  That would partially remove the
> >> need to define "local".

> > Using UTC is a terrible idea.

> My example was a lunar base, but I think it's generally a good
> idea.

I'm 100% behind Pete on this one - using UTC is usually a terrible idea (unless
of course your local time zone happens to be UTC) bcause it causes important
information to be lost.

> Looking at say IETF WG meeting times I often have no
> clue what they mean in UTC (from where I could do the math for
> my local time).

This is a completely different problem and irrelevant to the matter at hand -
if the proper time zone isn't clearly identified that's a bug in the
information published about the meeting, one which cannot occur in 2822bis
without violaating the specification.

I will also point out that the issues with future meeting times is one which in
general is NOT solvable by giving UTC times or any times with any specific
offset. This is the old zone offset versus zone rule issue: An IETF meeting is
going to be held at times curresponding to whatever zone offset is _in effect_
at the venue where the meeting is held. Suppose it is announced that next
year's meeting is being held in, oh, let's say Israel. But between then and now
the Knesset decides to change the applicable daylight savings time rules and
this causes the zone offset to change. Times published in _any_ fixed offset
will now be incorrect. (I chose Israel for this example because it's a place
that piddles with these things more often than most.)

> For Date header fields it's not as bad, at least there is an
> offset, I have only to get the sign magic right for UTC, and
> from UTC it's again easy.

Exactly - there's no real upside, and a considerable downside, to using UTC.

> > allow the author to convey the time at which the message
> > was created. It is often important information to the
> > author (and to the reader) what local time the message
> > was created.

> For readers in a completely different time zone it's a matter
> of taste.  Likely I'm biased because I live near to UTC.  But
> UTC has no DST, a huge advantage.

Only in the sense that it makes things a tiny bit easier to implement on some
systems. If the system clock is run in UTC then of course it is easier to stuff
in a UTC time value. But unless he system can't add or substract properly, the
worst that can happen in converting to local time is that the wrong offset to
UTC is chosen and the point in time is identified correctly but the offset is
wrong.

And not all systems run their clocks in UTC. (In fact the machine I'm using
right now to enter this message uses a local time clock.) Such systems are at a
disadvantage because if they don't know the correct offset they cannot get the
right point in time into the header. However, if such systems are forced to
convert to UTC and mess it up there's now no indication in the header of
problem because then there's no offset you can look at and say, "I know this
was sent from California and that's the wrong offset for California right now". 

The bottom line is more information is almost always better.

> > hence the SHOULD

> <shrug />  It's no important point, I wouldn't miss it if you
> drop the SHOULD.
 
I would strongly object to this SHOULD being removed.

> >> Is +2586 legal (it IS within the numeric range -9959 through
> >> +9959)?
 
> > Yeah, someone else mentioned this to me. I've changed it to:
> > "and the last two digits of zone MUST be within the range 00
> > through 59."

> +1

>  [2718]
> > It's semantically valid, whether or not it is currently used.
> > There is no good reason to disallow such a value.

> 99 hours is rather obscure, four days aren't enough for an
> attempt to live in "local Julian time".  You could limit it
> to say 23 hours.  With that the syntax would more obviously
> indicate that it's actually about hh:mm and not 9999 minutes.

Again I'm with Pete - I see no reason for such a restriction.

				Ned

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 "local" time and zone offsets
Date:  Mon, 28 Jan 2008 13:21:58 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 52
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

>>> they can pick UTC.  Maybe 2822upd could drop the SHOULD (?)

>> I'd like to see it dropped, or better still, replaced by a=20
>> recommendation to use UTC.  That would partially remove the
>> need to define "local".

> Using UTC is a terrible idea.

My example was a lunar base, but I think it's generally a good
idea.  Looking at say IETF WG meeting times I often have no=20
clue what they mean in UTC (from where I could do the math for
my local time).

For Date header fields it's not as bad, at least there is an
offset, I have only to get the sign magic right for UTC, and=20
from UTC it's again easy.

> allow the author to convey the time at which the message
> was created. It is often important information to the
> author (and to the reader) what local time the message
> was created.

For readers in a completely different time zone it's a matter
of taste.  Likely I'm biased because I live near to UTC.  But
UTC has no DST, a huge advantage.

> hence the SHOULD

<shrug />  It's no important point, I wouldn't miss it if you
drop the SHOULD.
=20
>> Is +2586 legal (it IS within the numeric range -9959 through
>> +9959)?
=20
> Yeah, someone else mentioned this to me. I've changed it to:
> "and the last two digits of zone MUST be within the range 00
> through 59."

+1

 [2718]=20
> It's semantically valid, whether or not it is currently used.
> There is no good reason to disallow such a value.

99 hours is rather obscure, four days aren't enough for an
attempt to live in "local Julian time".  You could limit it
to say 23 hours.  With that the syntax would more obviously
indicate that it's actually about hh:mm and not 9999 minutes.

 Frank

Date: Mon, 28 Jan 2008 00:17:35 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 body content restrictions
Sender: [email-masked]

On 1/27/08 at 4:54 PM -0500, Bruce Lilly wrote:

>RFC 822 has been amended by RFC 1123 and RFC 2047 among others, and 
>both of those have published errata.  I'm sure that you're well 
>aware of this, as 2822 incorporated many of those widely-scattered 
>amendments to 822.

2822 never incorporated anything from MIME, and that was by charter 
of the working group that wrote it.

>My concern is that (ignoring for the moment the conflict between 
>what 1.2.3 says that section 2 describes and what the actual section 
>2 text states)

What conflict is that?

>a parser using the full 2822 parse grammer will not be able to parse 
>1994-era and modern messages.

You bet. One had better read and understand the MIME specs before 
implementing a parser for messages with characters in the range 
128-255 (leaving aside that you also better understand what those 
octets mean and which ones are actually legal depending on the 
charset of the message; many of them are not legal in certain 
charsets, and many have different meanings depending on the charset). 
2822[upd] is a baseline, not the entire story.

>2822 and the draft under discussion are supposed to update 822 in 
>accordance with changes to Internet mail since 822's publication.

No, that is false. 2822 was written under a specific charter, and 
2822upd is an attempt to fix actual errors in 2822. If you want to 
have a document that updates [2]822 in accordance with changes to 
Internet mail to date, either go to the IESG and charter a new 
working group, or write the document yourself.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Mon, 28 Jan 2008 00:06:27 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 definition of terms needed
Sender: [email-masked]

On 1/27/08 at 3:24 PM -0500, Bruce Lilly wrote:

>On Sunday 27 January 2008 13:04:52 Pete Resnick wrote:
>
>>As I've said before to others, people can purposely  misread any 
>>specification we right, so I'm not inclined to try to  complicate 
>>things to try plug some perceived hole unless it's  actually going 
>>to stop someone who really wants to do the right thing from doing 
>>something silly.
>
>I don't pretend to know whether or not any given person wants to do 
>the right thing.  I do know that a significant number of systems are 
>doing something quite silly, and I suspect that that silliness is 
>due in large measure to ambiguities in 2822 and/or conflicts with 
>other relevant specifications, and I further suspect that the amount 
>of such silliness will increase in the absence of clarification.

Again, you're going to have to a *real* example of an implementation 
that intended to do the right thing but was unable to due to this 
so-called ambiguity to convince me that much of this needs 
clarification.

>...B, a rather silly system, sends an automatic ***receipt*** 
>message. Contrary to years of established practice and good sense, 
>not to mention a documented strong recommendation against doing so 
>in RFC 3834, B sends that automated reply to the mailbox(es) in the 
>From field of A's original message, completely ignoring the null 
>return path.  Why does B do such a silly thing?  Because RFC 2822 
>tells him to do so...
[My emphasis added]

First of all, B not only does the wrong thing, but B thinks (for some 
reason as yet unclear to me) that a receipt message is a reply. I'm 
not sure how B made that leap (you'd think the author of B might have 
wondered, "Gee, I wonder whether there's a document defining 
receipts, as 2822 only talks about replies, which seem like a 
different thing."), but even so, I have to wonder what implementation 
this is: I would be flabbergasted if such an implementor even found 
section 3.6.2 and would love to hear of such an example.

Either way, I am completely unconvinced that saying *anything* in 
3.6.2 about what a reply is would convince any implementor to do 
something different in the scenario above.

>Note that neither text distinguishes between "automatic" (as used in 
>RFC 3834) replies vs. manually-initiated replies.

You will note that neither 2822 nor 3834 ever use the term "automatic 
replies". 2822 talks (in a non-normative note) about applications 
with "automatic reply commands", whereas 3834 only ever uses the word 
"automatic" in the phrase "automatic responses".

>What can A do to avoid these annoying automated replies?

He can point the sender to 3834, which talks about sending automatic responses.

>section 3.6.2:
>
>    In all cases, the "From:" field SHOULD NOT contain any mailbox that
>    does not belong to the author(s) of the message.
>
>so A is discouraged from setting the from field to something like 
><[email-masked]>

And if A, being someone who reads RFCs, does not understand what the 
term SHOULD NOT means, there's not a whole lot we can do about it. 
Again, additional text won't help.

>and while the earlier requirement for at least one destination field 
>(e.g. an empty Bcc field) has been dropped without explanation, viz.:
>
>    8.   Requirement for destinations removed.

Please let's stick to one discussion at a time. But FYI: It was 
discussed and noted that there existed implementations that created 
messages without destinations for perfectly valid reasons and 
consensus was reached that the requirement was neither necessary nor 
reflected current practice.

>there is no provision for not providing an originator field (in 
>order to preclude the aforementioned silly replies, not to mention 
>support for anonymity (e.g. for whistleblowers)).

Again, there are valid reasons to put a mailbox that does not belong 
to the author into the "From:" field, and that's why it is a SHOULD 
NOT. As was pointed out elsewhere, <anonymous@[127.0.0.1]> is a 
syntactically valid mailbox. (BTW: Any whistle blower who thinks 
changing the "From:" field of a message for anonymity purposes has 
different problems.)

>Are you saying that you intended the meaning to be the same as in RFC 3834?

No, I'm saying the English word "automatic" is being used in two 
completely different contexts in the two documents.

>N.B. the draft text "SHOULD by default [...] be sent to [...] the 
>'From:' field" vs. RFC 3834 "the From field SHOULD NOT be used [...] 
>for automatic responses".

Since the draft talks about "replies" and 3834 talks about "automatic 
responses", (two very different terms), I am not surprised that they 
say to do two different things with those. To wit: Section 3.1.2 of 
3834 says that replies should go to the From address.

>I have specific reasons for asking about each of the terms for which 
>I requested definitions.

Which you have yet to give. What is the reason that these terms need 
better defining in this document? If you can demonstrate some real 
potential for confusion (and examples of where this confusion has 
occurred would help), I'm sure the bunch of us would be happy to come 
up with some text.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 23:30:43 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 "local" time and zone offsets
Sender: [email-masked]

On 1/27/08 at 8:23 PM -0500, Bruce Lilly wrote:

>On Sunday 27 January 2008 11:18:37 Frank Ellermann wrote:
>
>>they can pick UTC.  Maybe 2822upd could drop the SHOULD (?)
>
>I'd like to see it dropped, or better still, replaced by a 
>recommendation to use UTC.  That would partially remove the need to 
>define "local".

Using UTC is a terrible idea. The entire idea of the "Date:" field is 
to allow the author to convey the time at which the message was 
created. It is often important information to the author (and to the 
reader) what local time the message was created. ("Was this written 
in the morning or the evening for the author?") There *are* valid 
reasons not to use local time (when you want to hide the information 
for some reason, or when you cannot reliably calculate it), hence the 
SHOULD, but implement the spec in such a way that you always use UTC 
creates information loss to the author and reader, and "the full 
implications must be understood".

>Regardless, there are security implications related to use of 
>"local" rather than UTC.  The issues arise from the difficulty in 
>converting *correctly* to an unambiguous time reference (i.e. UTC) 
>which in turn is partially due to an unfortunate choice in the sign 
>of the offset, (resulting in erroneously adding the offset to local 
>time to get UTC when it should be subtracted).  They also partially 
>arise from the "local" time issue of "Daylight Savings Time" (or 
>equivalent by any other name) and the horrendously complicated rules 
>(which vary from year to year and place to place) for additional 
>offsets.

The above are not security implications; they are reasons people 
sometimes get local time wrong. Programming is sometimes hard.

>I recall one instance where a person was wrongly accused because his 
>ISP made the offset conversion incorrectly (unfortunately I cannot 
>now find a reference), and here is a link to a case (not 
>mail-related) of the DST issue, also resulting in wrongful 
>accusation:
>http://yro.slashdot.org/article.pl?sid=07/04/17/1240237

That's not a security implication; that's a legal implication.

>Aside from those issues, there are ambiguities in the 2822upd-04 text:
>
>     and the zone MUST be
>    within the range -9959 through +9959.
>
>Is +2586 legal (it IS within the numeric range -9959 through +9959)?

Yeah, someone else mentioned this to me. I've changed it to:

"and the last two digits of zone MUST be within the range 00 through 59."

>What about semantics; what does +2718 mean (there are no actual time 
>zones outside the range -1200 to +1400, and none whose offset is not 
>a multiple of precisely 15 minutes [*])?

It's semantically valid, whether or not it is currently used. There 
is no good reason to disallow such a value.

>It seems highly inconsistent that the 2822upd-04 text is so pedantic 
>about semantic validity of day-of-week, day-of-month, and 
>time-of-day, yet so lax about semantic validity of zone offsets.

Using a "Bob" for the day-of-week, or "31 Feb", or "75:75:75" for the 
time-of day has the potential to really screw up code. I'm not clear 
on how having strange zone offsets does.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: 2822upd-04 line length limits
Date: Sun, 27 Jan 2008 22:02:00 -0500
Sender: [email-masked]

On Sunday 27 January 2008 21:02:33 Tony Hansen wrote:

> But MIME sort of blurs the lines between the identity definitions 
> and the encodings.

Agreed to the extent that a single keyword either indicates an encoding
transformation which has an implicit domain, or the domain of unencoded
content.

> And you sort of blurred the lines by only talking  
> about MIME encoding and then using that to say that "extending the line 
> length limits" was incorrect.

No, I stated some objections to the implication that "implementations 
which display messages" must "handle an arbitrarily large number of characters
in a line" (whereas MIME specifically limits lines using non-identity
encodings to 76 characters, specifically and as explicitly stated in the
MIME RFCs to provide a bound on what implementations must handle) and stated
that "extend this specification" might be misinterpreted in that context.
I did not use the term "incorrect" in that regard.

> However, I don't think it's necessary to mention all potentially 
> relevant RFCs and errata. This particular paragraph was using the MIME 
> RFCs as examples.

Agreed that it's unnecessary to mention all potentially relevant RFCs.
However where normative references have errata or are amended by other
RFCs, those errata and/or amending RFCs should be mentioned, as they are
essential to correct interpretation of the (amended) normative references.

Speaking of the particular examples chosen, two of them are media type
registration procedure specifications, which are not relevant to the message
format specification per se.

Date: Sun, 27 Jan 2008 21:02:33 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: 2822upd-04 line length limits
Sender: [email-masked]

Bruce Lilly wrote:
> On Sunday 27 January 2008 15:28:08 Tony Hansen wrote:
>> Bruce Lilly wrote:
>>> ...
> 
> 2045 overloads the terms used in a Content-Transfer-Encoding field to
> indicate either an actual encoding transformation or the domain of
> unencoded ("identity transformation" in 2045-speak) content:
> 
>     (3)   A Content-Transfer-Encoding header field, which can be
>           used to specify both the encoding transformation that
>           was applied to the body and the domain of the result.
>           Encoding transformations other than the identity
>           transformation are usually applied to data in order to
> 
> I was referring specifically to the *actual* encoding transformations.
> Thanks for the clarification.

Maybe. But MIME sort of blurs the lines between the identity definitions 
and the encodings. And you sort of blurred the lines by only talking 
about MIME encoding and then using that to say that "extending the line 
length limits" was incorrect.

Until MIME was defined, the notion of binary email was totally 
non-standard and there was no way of expressing such email. MIME gave us 
those tools, and subsequently *did* extend the limits.

Yes, "binary" is in the identity domain. But it does indeed extend the 
limits and it was MIME that did it.

 > ...
>> I suggest changing the wording "extend this specification" to be 
>> "shorten or extend this line length limitation".
> 
> That works for part of the problem, although it only applies to the
> generation (not parsing) part of the 2822 grammars, contrary to the
> description of section 2 provided in section 1.2.3 (the 2822upd-04
> parse grammar contains neither of the restrictions specified in
> draft section 2.3).  And as noted above, there are relevant RFCs
> and errata not mentioned in the draft.

Okay.

However, I don't think it's necessary to mention all potentially 
relevant RFCs and errata. This particular paragraph was using the MIME 
RFCs as examples.

	Tony Hansen
	[email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: 2822upd-04 "local" time and zone offsets
Date: Sun, 27 Jan 2008 20:23:16 -0500
Sender: [email-masked]

On Sunday 27 January 2008 11:18:37 Frank Ellermann wrote:

[regarding 2822upd-04 text
   The date and time-of-day SHOULD express local time.
] 

> they can pick UTC.  Maybe 2822upd could drop the
> SHOULD (?)

I'd like to see it dropped, or better still, replaced by a recommendation
to use UTC.  That would partially remove the need to define "local".

Regardless, there are security implications related to use of "local"
rather than UTC.  The issues arise from the difficulty in converting
*correctly* to an unambiguous time reference (i.e. UTC) which in turn
is partially due to an unfortunate choice in the sign of the offset,
(resulting in erroneously adding the offset to local time to get UTC
when it should be subtracted).  They also partially arise from the
"local" time issue of "Daylight Savings Time" (or equivalent by any
other name) and the horrendously complicated rules (which vary from year
to year and place to place) for additional offsets.

I recall one instance where a person was wrongly accused because his ISP
made the offset conversion incorrectly (unfortunately I cannot now find
a reference), and here is a link to a case (not mail-related) of the DST
issue, also resulting in wrongful accusation:
http://yro.slashdot.org/article.pl?sid=07/04/17/1240237

These security considerations should be mentioned in the draft.

Aside from those issues, there are ambiguities in the 2822upd-04 text:

    and the zone MUST be
   within the range -9959 through +9959.

Is +2586 legal (it IS within the numeric range -9959 through +9959)?

What about semantics; what does +2718 mean (there are no actual time
zones outside the range -1200 to +1400, and none whose offset is not a
multiple of precisely 15 minutes [*])?

It seems highly inconsistent that the 2822upd-04 text is so pedantic
about semantic validity of day-of-week, day-of-month, and time-of-day,
yet so lax about semantic validity of zone offsets.

* in fact there are none whose last two digits in 2822 notation would be 15,
  only 00, 30, and 45. Since the date field was first defined in RFC 561
  (September 1973) there have been no time zones whose offset from UTC is
  not a multiple of 5 minutes.  The last zone whose offset was not a
  multiple of 5 minutes was Liberia which went out of effect on May 1, 1972;
  prior to that date the Liberian offset was -44 minutes and 30 seconds
  (which cannot be represented with 2822upd-04 zone offsets as there is no
  provision for seconds). The last zone which was not a multiple of 15
  minutes offset from UTC was Kiritimati (-1040) and which went out of
  effect in 1979 (predating RFC 822, Pete).  See the latest version of the
  tzdata* file on
  ftp://elsie.nci.nih.gov/pub for all of the gory time zone offset details.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 body content restrictions
Date:  Sun, 27 Jan 2008 23:56:06 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 23
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:
=20
> Look for the marked text above in
> http://www.imc.org/ietf-822/mail-archive/msg06221.html
> which is my original message to which you responded

Apparently you say that <obs-body> can only handle "half
 of what can appear in a MIME application/octet-stream=20
 body transported via ESMTP BINARY transport"

That doesn't convince me, because anything in "obs" not
limited to <obs-body> is "MUST NOT generate".  That it
could somehow match half of the potential octets in any
binary content is beside the point. =20

It also somehow matches the first 128 UTF-8 characters,
so what ?  Binary 0D 0A is no lineend, and <obs-body>
explains the concept "line" for obsolete message bodies.

<obs-body> =3D *OCTECT wouldn't help for anything, it is
wrong for UTF-8, why do you (in essence) propose this ?

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 body content restrictions
Date:  Sun, 27 Jan 2008 23:28:45 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 15
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:

> Looking towards the future, ideally we won't be forced by today's
> decisions into a situation where there are once again umpteen
> widely-scattered documents that amend the basic IMF format in
> various ways.  One way to avoid that undesirable scenario is to
> make at least the parse grammar as liberal as practical in
> accordance with the robustness principle.

Looking towards the future I'd pick "net-utf8" as a least common
denominator for line oriented data (not binary or XML).  Net-utf8
would move any HTAB to "obs" (unless John changes this detail),=20
and move all CTL apart from CRLF, LF, HTAB, and FF to /dev/null.

 Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: 2822upd-04 body content restrictions
Date: Sun, 27 Jan 2008 17:14:49 -0500
Sender: [email-masked]

On Sunday 27 January 2008 16:50:41 Frank Ellermann wrote:
> 
> Bruce Lilly wrote:

> >>> 0x00-0x7f (with no CRLF restriction), but not 0x80-0xff.

See this ^?

> >> Where have you found the US-ASCII restriction ?  Excluding half of
> >> the octets would be odd for the purpose of a *binary* transport. 
>  
> > Please read the original, where the specific ABNF and the reference
> > to the section where is is located can be found.
> 
> "Where" as in "what original" - there's no "7f" or "80" in RFC 3030.

Look for the marked text above in
http://www.imc.org/ietf-822/mail-archive/msg06221.html
which is my original message to which you responded,
then read the part immediately above that where I showed where
the draft under discussion limits the content as described.

It has nothing to do with RFC 3030.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: 2822upd-04 body content restrictions
Date: Sun, 27 Jan 2008 16:54:30 -0500
Sender: [email-masked]

On Sunday 27 January 2008 14:07:52 Pete Resnick wrote:
 
> This is demonstrably false with regard to 8-bit and binary messages. 
>  From RFC 822:

Hardly surprising, since RFC 822 obviously predated RFCs 1651-2
etc.  Internet mail transport has changed appreciably since 1982.
Also, one should be rather cautious in quoting directly from
published RFC text; in many cases that text contains errors which
are documented elsewhere (because as a matter of policy, published
RFCs are never changed) (errata, other RFCs).  RFC 822 has been
amended by RFC 1123 and RFC 2047 among others, and both of those
have published errata.  I'm sure that you're well aware of this,
as 2822 incorporated many of those widely-scattered amendments to
822.

> MIME is an extension of [2]822, which allows for more stuff than 
> those documents. Characters above 127 do not belong in 2822[upd].

My concern is that (ignoring for the moment the conflict between what
1.2.3 says that section 2 describes and what the actual section 2
text states) a parser using the full 2822 parse grammer will not
be able to parse 1994-era and modern messages.  I.e., I think that
the parse grammar *should* handle 8bit/binary content in message
bodies, because that is the reality of modern Internet mail.  2822 and
the draft under discussion are supposed to update 822 in accordance
with changes to Internet mail since 822's publication.  Looking
towards the future, ideally we won't be forced by today's decisions
into a situation where there are once again umpteen widely-scattered
documents that amend the basic IMF format in various ways.  One way
to avoid that undesirable scenario is to make at least the parse
grammar as liberal as practical in accordance with the robustness
principle.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 body content restrictions
Date:  Sun, 27 Jan 2008 22:50:41 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 39
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:

> The MIME RFCs work harmoniously with RFC 822 as every MIME message
> is a valid RFC 822 message and every component of MIME header fields
> is a valid component of an RFC 822 field

Peter already answered that, and at the time RFC 2045 was published
they knew perfectly well that binary doesn't fly with ordinary SMTP:

| Such data cannot be transmitted over some transfer protocols.  For=20
| example, RFC 821 (SMTP) restricts mail messages to 7bit US-ASCII
| data with lines no longer than 1000 characters including any=20
| trailing CRLF line separator.

RFC 821/822 and 2821/2822 came as tandems, if all goes well that
will be also the case for 2821bis/2822upd, despite of a premature
2821bis Last Call.  Theories about appealing premature Last Calls
(as discussed on the general list) not withstanding.

An ordinary message/rfc822 without MIME magic is by definition in
RFC 2045 US-ASCII (header and body) text/plain (body).  MIME gives
us the means to transform anything into this basic pattern, where
MIME-agnostic 7bit software won't know what hit them.  And 2822upd
specifies what the "basic pattern" precisely is.  MIME only has a
normative 822 reference for this job, and 2822upd will be the new
STD 11 in about two years - if all goes well.  (And after that it
makes sense to promote MIME to STD, using 4234bis ABNF, the works)

>>> 0x00-0x7f (with no CRLF restriction), but not 0x80-0xff.
=20
>> Where have you found the US-ASCII restriction ?  Excluding half of
>> the octets would be odd for the purpose of a *binary* transport.=20
=20
> Please read the original, where the specific ABNF and the reference
> to the section where is is located can be found.

"Where" as in "what original" - there's no "7f" or "80" in RFC 3030.

 Frank

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: 2822upd-04 line length limits
Date: Sun, 27 Jan 2008 16:19:39 -0500
Sender: [email-masked]

On Sunday 27 January 2008 15:28:08 Tony Hansen wrote:
> 
> Bruce Lilly wrote:
> > ...
> > No!  The MIME RFCs specifically state that lines (in header or body)
> > which use MIME encoding are limited to 76 (N.B. not 78 or 998) characters.
> > Please do not suggest, imply, or leave to the reader's imagination that
> > MIME-encoded text can be placed in long lines. Do not under any
> > circumstances imply that MIME-conforming implementations must handle
> > encoded text in lines "at least up to the 998 character limit".
> > 
> > Section 2.3 states in part (I'll address the other part in a separate message):
> > 
> >    o  Lines of characters in the body MUST be limited to 998 characters,
> >       and SHOULD be limited to 78 characters, excluding the CRLF.
> > 
> >       Note: As was stated earlier, there are other documents,
> >       specifically the MIME documents ([RFC2045], [RFC2046], [RFC2049],
> >       [RFC4288], [RFC4289]), that extend this specification to allow for
> >       different sorts of message bodies.  Again, these mechanisms are
> >       beyond the scope of this document.
> > 
> > Note (as stated above) that the MIME RFCs *limit* line lengths to 76
> > characters when MIME encoding is used.  The text "extend this specification"
> > might be misinterpreted (76 characters is a *limitation*, not an
> > *extension*) w.r.t. line length limits.  At a minimum, some clarification
> > of this point is needed in the draft text.
> 
> Bruce,
> 
> you are both correct and incorrect. The MIME specifications cover a 
> number of transfer encodings. Two of those (quoted-printable and base64) 
> limit the lines to 76 characters plus a CRLF, as you describe.

2045 overloads the terms used in a Content-Transfer-Encoding field to
indicate either an actual encoding transformation or the domain of
unencoded ("identity transformation" in 2045-speak) content:

    (3)   A Content-Transfer-Encoding header field, which can be
          used to specify both the encoding transformation that
          was applied to the body and the domain of the result.
          Encoding transformations other than the identity
          transformation are usually applied to data in order to

I was referring specifically to the *actual* encoding transformations.
Thanks for the clarification.

> Two of the MIME encodings (7bit and 8bit) retain the 998 line limit. 
> (RFC 2045 sections 2.7 & 2.8)
> 
> However, one of the MIME encodings (binary) REMOVES all line length 
> limitations. (RFC 2045 section 2.9) It does indeed *extend* the 
> specification.

These "identity transformations" describe the domain of unencoded
body content.  See also section 6.2 of RFC 2045.
 
> And there is room left within the specification for other encodings to 
> be defined that have other limitations, although none are currently 
> defined within the MIME specs, nor is it likely that there ever will be any.

[off-topic]
The 2045-defined (actual, not "identity") encoding transformations both
result in an implicit 7bit domain (of the encoded text). I recall that
Ned was at one time working on a binary to 8bit transformation, but
there seemed little interest in it from precisely the community that
would have benefitted from it.

[back on-topic]
In addition to 2045's discussion of body content, RFC 2047 (as amended by
2231 and errata (http://www.rfc-editor.org/cgi-bin/errata.pl)) also
restricts the length of header field lines containing an encoded-word
to 76 characters.  As can be seen from the quoted draft text above,
RFC 2047, RFC 2231, and the RFC errata are not mentioned in the draft.
Many of the RFCs referenced by the draft ARE amended by errata (q.v.)
and/or other RFCs (the rfc-index provides some hints, but is not
exhaustive).
 
> I suggest changing the wording "extend this specification" to be 
> "shorten or extend this line length limitation".

That works for part of the problem, although it only applies to the
generation (not parsing) part of the 2822 grammars, contrary to the
description of section 2 provided in section 1.2.3 (the 2822upd-04
parse grammar contains neither of the restrictions specified in
draft section 2.3).  And as noted above, there are relevant RFCs
and errata not mentioned in the draft.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 section 1 vs. section 4.1 discrepancy
Date:  Sun, 27 Jan 2008 22:11:59 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 11
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

>> I suggest eliding the sentence in the note in section
>> 4.1 which begins with "In the future,".
=20
> I just noticed that the other day myself. Happy to have
> confirmation of my gut reaction to it. Will delete.

Good, IIRC this detail alone took USEFOR several weeks.

 Frank

Date: Sun, 27 Jan 2008 15:28:08 -0500
From: Tony Hansen <[email-masked]>
To: ietf-822 <[email-masked]>
Subject: Re: 2822upd-04 line length limits
Sender: [email-masked]

Bruce Lilly wrote:
> ...
> No!  The MIME RFCs specifically state that lines (in header or body)
> which use MIME encoding are limited to 76 (N.B. not 78 or 998) characters.
> Please do not suggest, imply, or leave to the reader's imagination that
> MIME-encoded text can be placed in long lines. Do not under any
> circumstances imply that MIME-conforming implementations must handle
> encoded text in lines "at least up to the 998 character limit".
> 
> Section 2.3 states in part (I'll address the other part in a separate message):
> 
>    o  Lines of characters in the body MUST be limited to 998 characters,
>       and SHOULD be limited to 78 characters, excluding the CRLF.
> 
>       Note: As was stated earlier, there are other documents,
>       specifically the MIME documents ([RFC2045], [RFC2046], [RFC2049],
>       [RFC4288], [RFC4289]), that extend this specification to allow for
>       different sorts of message bodies.  Again, these mechanisms are
>       beyond the scope of this document.
> 
> Note (as stated above) that the MIME RFCs *limit* line lengths to 76
> characters when MIME encoding is used.  The text "extend this specification"
> might be misinterpreted (76 characters is a *limitation*, not an
> *extension*) w.r.t. line length limits.  At a minimum, some clarification
> of this point is needed in the draft text.

Bruce,

you are both correct and incorrect. The MIME specifications cover a 
number of transfer encodings. Two of those (quoted-printable and base64) 
limit the lines to 76 characters plus a CRLF, as you describe.

Two of the MIME encodings (7bit and 8bit) retain the 998 line limit. 
(RFC 2045 sections 2.7 & 2.8)

However, one of the MIME encodings (binary) REMOVES all line length 
limitations. (RFC 2045 section 2.9) It does indeed *extend* the 
specification.

And there is room left within the specification for other encodings to 
be defined that have other limitations, although none are currently 
defined within the MIME specs, nor is it likely that there ever will be any.

I suggest changing the wording "extend this specification" to be 
"shorten or extend this line length limitation".

	Tony Hansen
	[email-masked]

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: 2822upd-04 definition of terms needed
Date: Sun, 27 Jan 2008 15:24:31 -0500
Sender: [email-masked]

On Sunday 27 January 2008 13:04:52 Pete Resnick wrote:
> 
> On 1/26/08 at 11:30 PM -0500, Bruce Lilly wrote:
> 
> >2822upd-04 uses several terms in ways which may or may not conflict 
> >with related specifications and documents;
> 
> Overall comment: For most of these, you're going to have to show me a 
> specific example of a conflict to convince me that any change needs 
> to be made.

First I need to know how you're using the terms to determine whether
or not there IS a conflict.

> As I've said before to others, people can purposely  
> misread any specification we right, so I'm not inclined to try to 
> complicate things to try plug some perceived hole unless it's 
> actually going to stop someone who really wants to do the right thing 
> from doing something silly.

I don't pretend to know whether or not any given person wants to do
the right thing.  I do know that a significant number of systems
are doing something quite silly, and I suspect that that silliness
is due in large measure to ambiguities in 2822 and/or conflicts
with other relevant specifications, and I further suspect that the
amount of such silliness will increase in the absence of
clarification.

One example of the sort of suspected conflict, and the reason for the
request for definitions, is illustrated by the following example of
silliness:

Party A sends a message (e.g. an error report of some sort) to party
B.  Because A, a good net citizen, wishes to avoid mail loops, he
sets the envelope address to a null path in accordance with the
provisions established for that purpose by RFC 821 and affirmed by
RFC 1123 (though curiously not permitted by 2821, but that's not
an issue for the draft under discussion).  A does not want any
delivery reports or disposition notifications or message tracking
reports (or any response at all), so consequently does not include
SMTP commands and/or message fields which would trigger those (RFCs
3461-3464, 3798, 3888).  A specifically states (in the message body)
that he does not want any response.  Upon receipt of the message from
A (with null return path), B, a rather silly system, sends an automatic
receipt message. Contrary to years of established practice and good
sense, not to mention a documented strong recommendation against doing
so in RFC 3834, B sends that automated reply to the mailbox(es) in the
>From field of A's original message, completely ignoring the null return
path.  Why does B do such a silly thing?  Because RFC 2822 tells him to
do so (as does the current draft text, from which the following are
quoted):

section 3.6.2, omitting pagination text and whitespace for brevity):

     In the absence of the "Reply-To:" field,
   replies SHOULD by default be sent to the mailbox(es) specified in the
   "From:" field unless otherwise specified by the person composing the
   reply.

and section 3.6.3:

   When a message is a reply to another message, the mailboxes of the
   authors of the original message (the mailboxes in the "From:" field)
   or mailboxes specified in the "Reply-To:" field (if it exists) MAY
   appear in the "To:" field of the reply since these would normally be
   the primary recipients of the reply.

Note that neither text distinguishes between "automatic" (as used in
RFC 3834) replies vs. manually-initiated replies.  It appears (lacking
a real definition) that the use of "automatic" and variants in 2822upd-04
differs markedly from the meaning in RFC 3834; but (lacking a real
definition of how they are intended to be interpreted in 2822upd-04)
that's just a hunch.

What can A do to avoid these annoying automated replies?  Not much
(at least if he "really wants to do the right thing") according to
2822upd-04:

section 3.6.2:

   In all cases, the "From:" field SHOULD NOT contain any mailbox that
   does not belong to the author(s) of the message.

so A is discouraged from setting the from field to something like
<[email-masked]>, and while the earlier requirement
for at least one destination field (e.g. an empty Bcc field) has been
dropped without explanation, viz.:

   8.   Requirement for destinations removed.

there is no provision for not providing an originator field (in order to
preclude the aforementioned silly replies, not to mention support for
anonymity (e.g. for whistleblowers)).

[...]

> >automatic (and related terms)    as used in sections 3.6.3 and 3.6.6.
> 
> Common use. I see no issue here.

Are you saying that you intended the meaning to be the same as in RFC
3834?  If so, the text of the draft directly conflicts with provisions
of RFC 3834 (which has been extensively discussed on this list) which
are specifically designed to prevent such silly and undesired responses.
[specifically, RFC 3834 section 4 (titled "Where to send automatic
responses (and where not to send them)"), which states in part]:

   Use of the From field as the destination for automatic responses has
   some of the same problems as use of Reply-To.  In particular, the
   From field may list multiple addresses, while automatic responses
   should only be sent to a single address.  In general, the From and
   Reply-To addresses are used in a variety of ways according to
   differing circumstances, and for this reason Personal or Group
   Responders cannot reliably assume that an address in the From or
   Reply-To field is an appropriate destination for the response.  For
   these reasons the From field SHOULD NOT be used as a destination for
   automatic responses.

N.B. the draft text "SHOULD by default [...] be sent to [...] the 'From:'
field" vs. RFC 3834 "the From field SHOULD NOT be used [...] for
automatic responses".  I.e. one apparently says "do", the other clearly says
"don't".  The one that says "don't" gives good definitions and rationale for
that recommendation.  To date I see no definitions or rationale for the
apparently contrary recommendation in the draft under discussion.  I
hope that the discrepancy is caused by different usage of "reply"/"response"
and/or "automatic", but I seem as yet unable to elicit a serious response
to my request for definitions corresponding to that usage.

See also RFC 3834 section 1 for an explanation of this issue.
See also RFC 3834 section 4 in its entirety, the last paragraph of which
provides a specific well-reasoned recommendation for where to send such
responses, one that is not mentioned at all in the draft currently under
discussion.

Given the specific text ("SHOULD [...] be sent to [...] From") in the
draft under discussion combined with the lack of mention of the use of the
Return-Path field for automated responses, the lack of concrete
definitions for "automated" and "reply" in the draft, as well as the lack
of even mere mention of RFC 3834 and/or Dave Crocker's mail architecture
draft, I cannot see how a reader of the document under discussion who is
not already intimately familiar with the latter unmentioned documents would
NOT be misled by the draft under discussion as it now stands into generating
silly unwanted automated responses as described above and in RFC 3834
section 1.

[...]
> >reply (and related terms)     as used in sections 3.6.2, 3.6.3, and 3.6.6.
[...]

> Flip answer: If you don't understand how these terms are being used, 
> you shouldn't be reading this document.

Flip (tongue-in-cheek) counterpoint:  If you don't understand what you
mean by the terms, to the point where you cannot define how they are used
and/or how they relate to well-defined usage of the same or similar
terms in related specifications...[you can figure out the rest] :-)

> More serious answer: Definition of such things belongs in a serious 
> applicability statement on e-mail. I think it will only add possible 
> confusion points, not remove them.

I disagree.  I have specific reasons for asking about each of the terms
for which I requested definitions.  It may well be the case that some
of the issues cannot be resolved definitively (in which case it should
at least be possible to include some non-normative text explaining the
issues), but I suspect that at the end of discussion, it will be possible
to resolve most conflicts between the draft text and other specifications.

Date: Sun, 27 Jan 2008 13:07:52 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 body content restrictions
Sender: [email-masked]

On 1/27/08 at 11:41 AM -0500, Bruce Lilly wrote:

>The MIME RFCs work harmoniously with RFC 822 as every MIME message 
>is a valid RFC 822

This is demonstrably false with regard to 8-bit and binary messages. 
 From RFC 822:

      message     =  fields *( CRLF *text )       ; Everything after
                                                  ;  first null line
                                                  ;  is message body

      text        =  <any CHAR, including bare    ; => atoms, specials,
                      CR & bare LF, but NOT       ;  comments and
                      including CRLF>             ;  quoted-strings are
                                                  ;  NOT recognized.

      CHAR        =  <any ASCII character>        ; (  0-177,  0.-127.)

MIME is an extension of [2]822, which allows for more stuff than 
those documents. Characters above 127 do not belong in 2822[upd].

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 12:17:30 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: typos in 2822upd-04
Sender: [email-masked]

On 1/27/08 at 1:00 PM -0500, Bruce Lilly wrote:

>On Sunday 27 January 2008 12:36:01 Pete Resnick wrote:
>
>>[off-topic] A spell-checker found exactly *one* similar error in the text...
>
>Here (from memory) are two+ more:
>
>1. "versionof" should be "version of"

That's the one I found.

>2+. [Aa]cknowledgement* should be [Aa]cknowledgment (multiple forms 
>and places)

Not my department. See 
<http://xml.resource.org/authoring/README.html#anchor7>, specifically 
discussion of "rfcedstyle".

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 12:13:14 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 field-name vs. line length and obs-optional
Sender: [email-masked]

On 1/27/08 at 12:29 AM -0500, Bruce Lilly wrote:

>2822upd-04 section 3.6.8 reads in part:
>
>    field-name      =   1*ftext
>
>That specifies unlimited length, which conflicts with line length 
>limits specified in section 2.1.1.
>
>    field-name      =   1*77ftext

OK, this is amusing, and I think I'm OK with it. I do wonder whether 
the obs- syntax should retain the length limit.

>2822upd-04 section 4.5.8 reads:
>
>4.5.8.  Obsolete optional fields
>    obs-optional    =   field-name *WSP ":" unstructured CRLF
>
>[off-topic] unlike most sections, this one lacks vertical whitespace between
>section heading and text.

It's an issue with xml2rfc I'm trying to deal with. Obviously we'll 
make sure it gets cleared up before release.

>This one probably needs an obs-field-name as RFC 733 permitted 
>space/tab within (but not at the start of, for obvious reasons) a 
>field-name (and it is occasionally necessary to parse old messages).

No. 2822 starts with 822 as a base. We don't deal with many 733 constructs.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 12:04:52 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 definition of terms needed
Sender: [email-masked]

On 1/26/08 at 11:30 PM -0500, Bruce Lilly wrote:

>2822upd-04 uses several terms in ways which may or may not conflict 
>with related specifications and documents;

Overall comment: For most of these, you're going to have to show me a 
specific example of a conflict to convince me that any change needs 
to be made. As I've said before to others, people can purposely 
misread any specification we right, so I'm not inclined to try to 
complicate things to try plug some perceived hole unless it's 
actually going to stop someone who really wants to do the right thing 
from doing something silly.

>structured field bodies     as used in section 2.2.2 and later; 
>specifically, is this definition meant to apply to (extension) 
>fields defined in other specifications?

I could put a note in 2.2.2, but I do wonder whether it would do much 
to clarify things. Has anyone ever written an extension document with 
a structured field where it wasn't clear if an implementation *that 
implements the extension* should treat it as structured?

>local time     as used in section 3.3.  Specifically, what is "local 
>time" if a (human) message composer is sitting in an airplane 
>composing a message via a telnet link to a (ground-based) host 
>computer which will then send the message via a separate, remote 
>submission server?  [By the way, the airplane is crossing over the 
>(North) polar ice cap as the composer travels an intercontinental 
>great circle route to his destination.]

The DRUMS archive is replete with information on why this ambiguity 
appears in this section, and your example actually points to the 
reason: "local time" is whatever the implementation best thinks is 
local time. It may make sense for certain specialized applications to 
figure out what time zone it is in at the precise moment the message 
is generated. For others, it might make sense to always use the 
ground-based time of the host computer. The short answer is, this is 
purposely ambiguous.

>unlimited     w.r.t. the number of optional-fields as specified in 
>section 3.6.  Specifically, does it mean that the specification for 
>extension fields is not allowed to specify a maximum (or minimum, 
>for that matter) field count for extension fields?

Again, I'm curious where this might have ever caused confusion. 
Anything strictly implementing (2)822(upd) better put absolutely no 
limits on any optional-field. Anything implementing an extension 
which gives a limit better honor that limit if it wants to conform to 
that specification. I don't see how any sane person could point to 
"unlimited" as a reason to ignore a MUST in another document.

>creator      as used in sections 1.2.3 and 3.6.1.
>[...]
>automatic (and related terms)    as used in sections 3.6.3 and 3.6.6.

Common use. I see no issue here.

>final form       as used in section 3.6.1.

See discussion of local time above.

>user      as used in sections 3.6.1 and 3.6.6.
>
>reply (and related terms)     as used in sections 3.6.2, 3.6.3, and 3.6.6.
>
>sender        as used in sections 3.4 and 3.6.4.
>
>reintroduced       as used in section 3.6.6.

Flip answer: If you don't understand how these terms are being used, 
you shouldn't be reading this document.

More serious answer: Definition of such things belongs in a serious 
applicability statement on e-mail. I think it will only add possible 
confusion points, not remove them.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: typos in 2822upd-04
Date: Sun, 27 Jan 2008 13:00:33 -0500
Sender: [email-masked]

On Sunday 27 January 2008 12:36:01 Pete Resnick wrote:
> 
> On 1/26/08 at 10:47 PM -0500, Bruce Lilly wrote:
[...]
> >[off-topic] "incumbant" should be "incumbent".  A spell-checker can 
> >find this and many other similar errors in the text...
> 
> [off-topic] A spell-checker found exactly *one* similar error in the text...

Here (from memory) are two+ more:

1. "versionof" should be "version of"

2+. [Aa]cknowledgement* should be [Aa]cknowledgment (multiple forms and places)

Date: Sun, 27 Jan 2008 11:36:01 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 line length limits
Sender: [email-masked]

On 1/26/08 at 10:47 PM -0500, Bruce Lilly wrote:

>Section 2.1.1 states in part:
>
>      Again, even though this limitation is
>    put on messages, it is incumbant upon implementations which display
>    messages to handle an arbitrarily large number of characters in a
>    line (certainly at least up to the 998 character limit) for the sake
>    of robustness.
>
>[off-topic] "incumbant" should be "incumbent".  A spell-checker can 
>find this and many other similar errors in the text...

[off-topic] A spell-checker found exactly *one* similar error in the text...

>No!  The MIME RFCs specifically state that lines (in header or body) 
>which use MIME encoding are limited to 76 (N.B. not 78 or 998) 
>characters.

I agree with Frank's comments in later messages with regard to this 
(and 3.5 below): MIME can extend (or limit) this document in any way 
it seems fit. However, it most certainly is incumbent upon 
implementations to deal with longer lines than MIME allows. I don't 
think you're going to find much support at all for your position.

>Section 2.3 states in part (I'll address the other part in a 
>separate message):
>
>       Note: As was stated earlier, there are other documents,
>       specifically the MIME documents ([RFC2045], [RFC2046], [RFC2049],
>       [RFC4288], [RFC4289]), that extend this specification to allow for
>       different sorts of message bodies.  Again, these mechanisms are
>       beyond the scope of this document.
>
>Note (as stated above) that the MIME RFCs *limit* line lengths to 76 
>characters when MIME encoding is used.  The text "extend this 
>specification" might be misinterpreted (76 characters is a 
>*limitation*, not an *extension*) w.r.t. line length limits.  At a 
>minimum, some clarification of this point is needed in the draft 
>text.

Happy to clarify.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 11:26:59 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 "field" vs. "line"
Cc: "ietf-822" <[email-masked]>, Bruce Lilly <[email-masked]>
Sender: [email-masked]

On 1/26/08 at 10:22 PM -0500, Bruce Lilly wrote:

>2822upd-04 section 2.1 states in part:
>
>       ...but instead always uses "header field" to refer to the
>       individual line and "header section" to refer to the entire
>       collection.
>
>"individual line" is technically incorrect.  Either "individual 
>field" (preferred) or "logical line" would avoid the difficulty.

Fine with me. Fixed.

>Section 2.2 begins with:
>
>    Header fields are lines beginning with a field name, followed by a
>    colon (":"), followed by a field body, and terminated by CRLF.
>
>I suggest beginning the sentence with "Header fields begin with a 
>field name at the start of a line," and continuing after the 
>existing text's first comma [same issue as above].

This change I don't like. The paragraph is defining header fields 
("Header fields are..."), not describing how they are formed ("Header 
fields begin with..."). Not only that, it is explained 2 subsections 
later that header fields are a logical line and how they can be 
folded. I don't think this general explanation will cause any 
confusion.

>Section 2.2.1 contains:
>
>    Semantically, unstructured field bodies are simply to be treated as a
>    single line of characters with no further processing (except for
>    "folding" and "unfolding" as described in section 2.2.3).
>
>I suggest changing "single" to "logical" [same issue as above].

Again, the reference to 2.2.3 I think deals with any possible confusions.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 27 Jan 2008 11:20:43 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: 2822upd-04 section 1 vs. section 4.1 discrepancy
Cc: "ietf-822" <[email-masked]>, Bruce Lilly <[email-masked]>
Sender: [email-masked]

On 1/26/08 at 10:08 PM -0500, Bruce Lilly wrote:

>Section 4.1 states in part:
>
>       Note: The "period" (or "full stop") character (".") in obs-phrase
>       is not a form that was allowed in earlier versions of this or any
>       other specification. [...]  In the future,
>       period may appear in the regular syntax of phrase.
>
>I suggest eliding the sentence in the note in section 4.1 which 
>begins with "In the future,".

I just noticed that the other day myself. Happy to have confirmation 
of my gut reaction to it. Will delete.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

Date: Sun, 27 Jan 2008 08:41:28 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Pete Resnick <[email-masked]>
CC: ietf-822 <[email-masked]>
Subject: Re: Comma-seperated lists
Sender: [email-masked]

Pete Resnick wrote:
> On 1/25/08 at 6:55 AM -0800, Dave Crocker wrote:
>> I read the above as at least implying that an empty list that 
>> nonetheless contains commas is allowed.
> 
> OK. I'm having fun making this work. It also means that a list can begin 
> or end with commas. Fun. Now, on a more interesting front:

FWIW, I think that these cases would be odd to have happen, but I don't see 
any reason to prohibit them.  Given the basic idea of a list syntax, it 
actually seems more complicated to prohibit them.

>> I read this combination as rather explicitly allowing white space 
>> around a comma.
> 
> So, do you think the following is a legal group address in 822's syntax?
> 
> GroupName: [email-masked], (missing address), [email-masked], 

Yes, thanks. I think you have nicely demonstrated its utility.  The comment 
might contain an explanation about the occurrence and reason for having 
excised an  address.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: 2822upd-04 body content restrictions
Date: Sun, 27 Jan 2008 11:41:01 -0500
Sender: [email-masked]

On Sunday 27 January 2008 10:47:17 Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
>  
> > 2822upd-04 section 2.3 reads in part:
> >    o  CR and LF MUST only occur together as CRLF; they MUST NOT
> >       appear independently in the body.
> 
> > This makes me uneasy for several reasons:
> > 1. it bucks the trend at liberalizing body content (e.g. 8BITMIME
> > and BINARY SMTP extensions).
> 
> IMO 2822upd is about message/rfc822 with transmission as SMTP DATA
> in mind.  It's not about an UTF8SMTP message/global, it's also not
> about BINARYMIME BDAT, NNTP message/news, or (arguably) SMTP SEND.
[...]

You seem to have missed the point.  Let me try to explain again.
The MIME RFCs work harmoniously with RFC 822 as every MIME message
is a valid RFC 822 message and every component of MIME header fields
is a valid component of an RFC 822 field (and can be specified
without having to disambiguate ambiguities in 822 -- about which
more in a separate message).  Per contra, the referenced 2822upd-04
sections state that a significant subset of valid RFC 822/MIME
messages are no longer valid Internet Message Format messages -- not
only for generation (which would itself be a giant step backward),
but for parsing as well (section 2 is claimed to be "general", i.e.
not restricted to generation).

> > 0x00-0x7f (with no CRLF restriction), but not 0x80-0xff.
> 
> Where have you found the US-ASCII restriction ?  Excluding half of
> the octets would be odd for the purpose of a *binary* transport. 

Please read the original, where the specific ABNF and the reference
to the section where is is located can be found.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 definition of terms needed
Date:  Sun, 27 Jan 2008 17:18:37 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 30
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:
=20
> Specifically, what is "local time" if a (human) message composer is
> sitting in an airplane composing a message via a telnet link to a
> (ground-based) host computer which will then send the message via a=20
> separate, remote submission server? =20

Let the human message composer figure it out, the prose allows them
to pick whatever they consider as "local".

> By the way, the airplane is crossing over the (North) polar ice cap
> as the composer travels an intercontinental great circle route to
> his destination.

Amusing, but no problem for 2822upd.  Let them compose messages from
a future lunar base, they can pick UTC.  Maybe 2822upd could drop the
SHOULD (?)

> unlimited     w.r.t. the number of optional-fields as specified in
> section 3.6.  Specifically, does it mean that the specification for
> extension fields is not allowed to specify a maximum (or minimum,
> for that matter) field count for extension fields?

IMO clear, an agent not knowing the optional field is supposed to
accept "one or more".  The specification can say what it wishes.

For today I pass on the notorious "sender", somehow I managed to
buy coffee without caffeine :-(

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 field-name vs. line length and obs-optional
Date:  Sun, 27 Jan 2008 16:56:20 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 25
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:
=20
>    field-name      =3D   1*77ftext
=20
> would conform to the lower recommended limit. [for reference, the
> longest field name that I know of defined in a published RFC is
> Delivery-Report-Content-Reported-Recipient-Info (47 characters)
> defined in RFC 987.]

LOL, yes, 1*77ftext could define an upper bound.

>    obs-optional    =3D   field-name *WSP ":" unstructured CRLF
[...]
> This one probably needs an obs-field-name as RFC 733 permitted
> space/tab within (but not at the start of, for obvious reasons)
> a field-name

No, for the same reasons why there's no  obs-at =3D *WSP "at" *WSP

> it is occasionally necessary to parse old messages

If you occasionally parse RFC 733 messages you will manage that
without syntax in 2822upd.

 Fran

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: 2822upd-04 body content restrictions
Date:  Sun, 27 Jan 2008 16:47:17 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 40
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:
=20
> 2822upd-04 section 2.3 reads in part:
>    o  CR and LF MUST only occur together as CRLF; they MUST NOT
>       appear independently in the body.

> This makes me uneasy for several reasons:
> 1. it bucks the trend at liberalizing body content (e.g. 8BITMIME
> and BINARY SMTP extensions).

IMO 2822upd is about message/rfc822 with transmission as SMTP DATA
in mind.  It's not about an UTF8SMTP message/global, it's also not
about BINARYMIME BDAT, NNTP message/news, or (arguably) SMTP SEND.

It's possible to transfom BINARYMIME BDAT into message/rfc822 DATA,
and then the 2822upd rules for bare CR and bare LF apply.=20

> 2. it (together with the disclaimer that appears further down
> the page) implies that MIME messages do not conform to the IMF=20
> specification, which is counter to the intent of MIME.

2822upd is mostly MIME-agnostic, extensions (the "E" in MIME) need
to say where and under which conditions they derive from the basic
format.  Or as 2822upd 1.1 puts it:

| Those mechanisms are outside of the scope of this specification.

> 2822upd-04 section 4.1 reads in part:
>    obs-body =3D *((*LF *CR *((%d0 / text) *LF *CR)) / CRLF)

A perfectly fine definition of "bare LF" and "bare CR" cases which
could make it without being confused with CRLF.  And RFC 3030 has
it clear that BINARYMIME is unsuited for line oriented SMTP DATA.

> 0x00-0x7f (with no CRLF restriction), but not 0x80-0xff.

Where have you found the US-ASCII restriction ?  Excluding half of
the octets would be odd for the purpose of a *binary* transport.=20

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Comma-seperated [sic] lists
Date:  Sun, 27 Jan 2008 15:47:04 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 14
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:
=20
> There's nothing there about commas.

I've quoted the #-rule about ASCII-art commas and LWS.
=20
> the commas remain intact.

ASCII-art commas, NUL, and LWSP showing up as "obs"
are fundamentally broken.  Maybe mentinining this in
an informative appendix about pre-Y2K messages could
make sense, for this millennium it's simply broken.

 Frank

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Comma-seperated [sic] lists
Date: Sun, 27 Jan 2008 08:55:10 -0500
Sender: [email-masked]

On 2008-01-27, Frank Ellermann wrote:
> 
> Bruce Lilly wrote:
> 
> >> With optional FWS you get ASCII art.  With LWS you would
> >> get "apparently empty lines"
>  
> > No, you get a list with empty elements
> 
> LWS in 2068 ff. is [CRLF] 1*( SP | HT ), and "implied *LWS"
> means zero or more LWS.  That's the same as LWSP in 4234bis:
> 
> | LWSP           =  *(WSP / CRLF WSP)

There's nothing there about commas.

> A sequence of zero or more SP, HT, CRLF in any order, where
> each CRLF is followed by either SP or HT.
> 
> That means "apparently empty lines"

No, the commas remain intact.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Comma-seperated lists
Date:  Sun, 27 Jan 2008 14:24:05 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 45
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Bruce Lilly wrote:

>> With optional FWS you get ASCII art.  With LWS you would
>> get "apparently empty lines"
=20
> No, you get a list with empty elements

LWS in 2068 ff. is [CRLF] 1*( SP | HT ), and "implied *LWS"
means zero or more LWS.  That's the same as LWSP in 4234bis:

| LWSP           =3D  *(WSP / CRLF WSP)

A sequence of zero or more SP, HT, CRLF in any order, where
each CRLF is followed by either SP or HT.

That means "apparently empty lines" CRLF SP CRLF or similar
can occur within LWSP or *LWS.  Such constructs are far too
fragile for an environment such as mail header fields sent
via multiple relays and passing through dubious software
doing "something" with messages.

It's no problem where "really empty lines" CRLF CRLF have no
different effect from "apparently empty lines".  But that's
not the case for mail, where a "really empty line" indicates
the end of the header.

Like NUL in various obs-rules the LWSP in <obs-FWS> *cannot*
work in practice, it was never a good idea, 30 years ago or
in 2008. =20

You don't neeed to quote 2821-rules about not modifying the
header in transit, everybody knows this.  And apparently some
folks feel that "mailing lists" are entitled to ignore RFCs
as it pleases them. =20

IMO you can't get rid of "sh*t happens" by a MUST NOT, that
would be irresponsible if you know that it will nevertheless
happen, as a plausible bug.

> Your hypothetical "something" is nonconforming and=20
> non-interoperable.

IBTD, we wouldn't need 2822(upd) if RFC 822 was good enough. =20

 Frank

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 body content restrictions
Date: Sun, 27 Jan 2008 01:06:48 -0500
Sender: [email-masked]

2822upd-04 section 2.3 reads in part:

   o  CR and LF MUST only occur together as CRLF; they MUST NOT appear
      independently in the body.

This makes me uneasy for several reasons:
1. it bucks the trend at liberalizing body content (e.g. 8BITMIME and BINARY
   SMTP extensions).

2. it (together with the disclaimer that appears further down the page)
   implies that MIME messages do not conform to the IMF specification,
   which is counter to the intent of MIME.

3. the use of RFC 2119 terms implies that MIME messages used with extended
   SMTP transport cannot interoperate; that impugns a great deal of careful
   work to ensure that ESMTP negotiation and MIME are backward-compatible.

2822upd-04 section 4.1 reads in part:

   obs-body        =   *((*LF *CR *((%d0 / text) *LF *CR)) / CRLF)

text is defined in section 3.5 as:

   text            =   %d1-9 /            ; Characters excluding CR
                       %d11 /             ;  and LF
                       %d12 /
                       %d14-127

Together they handle only half of what can appear in a MIME
application/octet-stream body transported via ESMTP BINARY transport, viz.
0x00-0x7f (with no CRLF restriction), but not 0x80-0xff.

obs-body (i.e. parsing requirement) really ought to include 8-bit content
(N.B. that's body, not header field, content).  And the section 2.3 text
quoted above (and its companion text dealing with "line length") ought to
be elaborated upon to explicitly state that it applies only to generation
(not parsing requirements, for which it is directly contradicted by the
obs- grammar) of non-MIME, i.e. implicitly text/plain us-ascii body content.

Note that 2822upd-04 section 1.2.3 reads in part:

   Section 2 lays out the general description of a message and its
   constituent parts.  This is an overview to help the reader understand
   some of the general principles used in the later portions of this
   document.  Any examples in this section MUST NOT be taken as
   specification of the formal syntax of any part of a message.

N.B. that says nothing about generate-only; it states (by lack of explicit
limitation) that the section 2.3 text applies to parsing as well as
generation.

Related issues appear in sections 2.1 (limited to ASCII 1-127, divided into
lines) and 2.1.1 ("line" length limits).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 field-name vs. line length and obs-optional
Date: Sun, 27 Jan 2008 00:29:31 -0500
Sender: [email-masked]

2822upd-04 section 3.6.8 reads in part:

   field-name      =   1*ftext

That specifies unlimited length, which conflicts with line length limits
specified in section 2.1.1.

   field-name      =   1*77ftext

would conform to the lower recommended limit. [for reference, the
longest field name that I know of defined in a published RFC is
Delivery-Report-Content-Reported-Recipient-Info (47 characters)
defined in RFC 987.]

2822upd-04 section 4.5.8 reads:

4.5.8.  Obsolete optional fields
   obs-optional    =   field-name *WSP ":" unstructured CRLF

[off-topic] unlike most sections, this one lacks vertical whitespace between
section heading and text.

This one probably needs an obs-field-name as RFC 733 permitted space/tab
within (but not at the start of, for obvious reasons) a field-name (and it
is occasionally necessary to parse old messages).

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 definition of terms needed
Date: Sat, 26 Jan 2008 23:30:49 -0500
Sender: [email-masked]

2822upd-04 uses several terms in ways which may or may not conflict
with related specifications and documents; unfortunately, as 2822upd-04
does not define these terms, it is unclear whether or not there are in
fact conflicts.  Moreover, lacking precise definitions of these terms, it
is not clear exactly what the specifications using these terms in the
draft mean.

The terms and sections where they are used are:

structured field bodies     as used in section 2.2.2 and later;
specifically, is this definition meant to apply to (extension) fields
defined in other specifications?

local time     as used in section 3.3.  Specifically, what is "local time"
if a (human) message composer is sitting in an airplane composing a
message via a telnet link to a (ground-based) host computer which will
then send the message via a separate, remote submission server?  [By
the way, the airplane is crossing over the (North) polar ice cap as
the composer travels an intercontinental great circle route to his
destination.]

unlimited     w.r.t. the number of optional-fields as specified in
section 3.6.  Specifically, does it mean that the specification for
extension fields is not allowed to specify a maximum (or minimum, for
that matter) field count for extension fields?

creator      as used in sections 1.2.3 and 3.6.1.

user      as used in sections 3.6.1 and 3.6.6.

automatic (and related terms)    as used in sections 3.6.3 and 3.6.6.

final form       as used in section 3.6.1.

reply (and related terms)     as used in sections 3.6.2, 3.6.3, and 3.6.6.

sender        as used in sections 3.4 and 3.6.4.

reintroduced       as used in section 3.6.6.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 line length limits
Date: Sat, 26 Jan 2008 22:47:07 -0500
Sender: [email-masked]

Section 2.1.1 states in part:

     Again, even though this limitation is
   put on messages, it is incumbant upon implementations which display
   messages to handle an arbitrarily large number of characters in a
   line (certainly at least up to the 998 character limit) for the sake
   of robustness.

[off-topic] "incumbant" should be "incumbent".  A spell-checker can find
this and many other similar errors in the text...

No!  The MIME RFCs specifically state that lines (in header or body)
which use MIME encoding are limited to 76 (N.B. not 78 or 998) characters.
Please do not suggest, imply, or leave to the reader's imagination that
MIME-encoded text can be placed in long lines. Do not under any
circumstances imply that MIME-conforming implementations must handle
encoded text in lines "at least up to the 998 character limit".

Section 2.3 states in part (I'll address the other part in a separate message):

   o  Lines of characters in the body MUST be limited to 998 characters,
      and SHOULD be limited to 78 characters, excluding the CRLF.

      Note: As was stated earlier, there are other documents,
      specifically the MIME documents ([RFC2045], [RFC2046], [RFC2049],
      [RFC4288], [RFC4289]), that extend this specification to allow for
      different sorts of message bodies.  Again, these mechanisms are
      beyond the scope of this document.

Note (as stated above) that the MIME RFCs *limit* line lengths to 76
characters when MIME encoding is used.  The text "extend this specification"
might be misinterpreted (76 characters is a *limitation*, not an
*extension*) w.r.t. line length limits.  At a minimum, some clarification
of this point is needed in the draft text.

Section 3.5 begins with:

   A message consists of header fields, optionally followed by a message
   body.  Lines in a message MUST be a maximum of 998 characters
   excluding the CRLF, but it is RECOMMENDED that lines be limited to 78
   characters excluding the CRLF.  (See the note in section 2.1.1 for
   explanation.)

Here again, some note of the lower length limit imposed on MIME-encoded
text is needed.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 "field" vs. "line"
Date: Sat, 26 Jan 2008 22:22:11 -0500
Sender: [email-masked]

2822upd-04 section 2.1 states in part:

      Note: Common parlance and earlier versions of this specification
      use the term "header" to either refer to the entire header section
      or to refer to an individual header field.  To avoid ambiguity,
      this document does not use the terms "header" or "headers" in
      isolation, but instead always uses "header field" to refer to the
      individual line and "header section" to refer to the entire
      collection.

"individual line" is technically incorrect.  Either "individual field"
(preferred) or "logical line" would avoid the difficulty.

Section 2.2 begins with:

   Header fields are lines beginning with a field name, followed by a
   colon (":"), followed by a field body, and terminated by CRLF. 

I suggest beginning the sentence with "Header fields begin with a field
name at the start of a line," and continuing after the existing text's
first comma [same issue as above].

Section 2.2.1 contains:

   Semantically, unstructured field bodies are simply to be treated as a
   single line of characters with no further processing (except for
   "folding" and "unfolding" as described in section 2.2.3).

I suggest changing "single" to "logical" [same issue as above].

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>
Subject: 2822upd-04 section 1 vs. section 4.1 discrepancy
Date: Sat, 26 Jan 2008 22:08:12 -0500
Sender: [email-masked]

Section 1 states in part: 

   Section 4 of this document specifies an "obsolete" syntax.  There are
   references in section 3 to these obsolete syntactic elements.  The
   rules of the obsolete syntax are elements that have appeared in
   earlier versions of this specification or have previously been widely
   used in Internet messages.  As such, these elements MUST be
   interpreted by parsers of messages in order to be conformant to this
   specification.  However, since items in this syntax have been
   determined to be non-interoperable or to cause significant problems
   for recipients of messages, they MUST NOT be generated by creators of
   conformant messages.

N.B. no mention of syntax to be changed in future revisions.

Section 4.1 states in part:

      Note: The "period" (or "full stop") character (".") in obs-phrase
      is not a form that was allowed in earlier versions of this or any
      other specification.  Period (nor any other character from
      specials) was not allowed in phrase because it introduced a
      parsing difficulty distinguishing between phrases and portions of
      an addr-spec (see section 4.4).  It appears here because the
      period character is currently used in many messages in the
      display-name portion of addresses, especially for initials in
      names, and therefore must be interpreted properly.  In the future,
      period may appear in the regular syntax of phrase.

I suggest eliding the sentence in the note in section 4.1 which begins
with "In the future,".  Any message generator which does not quote or
encode special characters in a phrase is and has always been non-conforming.
Such implementations should be fixed. This part of the specification should
not be changed now or in the future.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Comma-seperated lists
Date: Sat, 26 Jan 2008 21:58:15 -0500
Sender: [email-masked]

On 2008-01-26, Frank Ellermann wrote:
> 
> Pete Resnick wrote:
> 
>  [# and LWSP] 
> > What? What does one thing have to do with the other?
> 
> ,,,,,   ,,,,,    ,,,,    ,,,,,
>   ,     ,       ,          ,
>   ,     ,,,,     ,,,       ,
>   ,     ,           ,      ,
>   ,     ,,,,,   ,,,,       ,   
> 
> | The full form is "<n>#<m>element " indicating at least
> | <n> and at most <m> elements, each separated by one or
> | more commas (",") and optional linear whitespace (LWS).
>  [RFC 2068 2.1, similar in 2069, 2616, 2617, 2831]  ^^^
> 
> With optional FWS you get ASCII art.  With LWS you would
> get "apparently empty lines"

No, you get a list with empty elements (N.B. not allowed for message
generation for any lists in RFC 2822 or draft-resnick-2822upd-04.txt).

> - causing havoc if something  
> in transit manages to transform them into a "really empty
> line" (indicating the end of the header, CRLF CRLF).

Quoting draft-klensin-rfc2821bis-06.txt:
"
   As discussed in Section 6.4, a relay SMTP has no need to inspect or
   act upon the header section or body of the message data and MUST NOT
   do so except to add its own "Received:" header field (Section 4.4)
   and, optionally, to attempt to detect looping in the mail system (see
   Section 6.3).  Of course this prohibition also applies to any
   modifications of these header fields or text (see also Section 7.9).
"

Your hypothetical "something" is nonconforming and non-interoperable.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Space after colon
Date: Sat, 26 Jan 2008 21:43:19 -0500
Sender: [email-masked]

On 2008-01-23, Tony Hansen wrote:
> 
> Pete Resnick wrote:
> > 
> > I'm separating out these two issues, as I want to hear yeas and nays 
> > (and explanations) from people other than Charles and Frank.
[...]
> > [On space after the ":"...]
> > 
> > [Charles originally wanted:]
> >>>> Although in all header fields the CFWS following the colon is 
> >>>> optional, it is customary to place at least one SP there; moreover 
> >>>> such a SP is mandatory in some related protocols (notably 
> >>>> [NETNEWS]). In order to facilitate interoperability with such 
> >>>> related protocols, that SP SHOULD normally be present.
> > 
> > [Frank suggested:]
> >>> Although all message header fields allow either [FWS] or [CFWS] after 
> >>> the colon, it is customary to place at least one white space 
> >>> character after the colon.  In NetNews articles [RFC xxxx] it's 
> >>> mandatory to use a space character after the colon, and in order to 
> >>> faciliate interoperability implementors might wish to separate the 
> >>> header field name plus colon from the header field body by a space in 
> >>> all Internet messages.
[...]

> Requiring a SP character after the : is a non-starter for me. I've used 
> a number of systems that use TAB instead of SP; they're still out there 
> and should not be considered invalid.

I tend to agree.  Moreover, as noted in draft-resnick-2822upd-04.txt
section 3.6.3, one use of the Bcc field is to leave the field body empty.
I have examined a large number of messages, and of those using the Bcc field
in this manner, only three included a space after the colon; the overwhelming
majority indeed had a truly empty field body (i.e. nothing between colon
and terminating CRLF.  Claiming space-after-colon as "customary" is flatly
wrong, at least for this widespread usage of the Bcc field.  It would be
silly and wasteful (using 7 octets to do what can be accomplished in 6) to
suggest that there should be a space in an otherwise empty field body.

If some non-normative text is included, it should be clearly labeled as
such AND it should be clearly reiterated that any message parser claiming 
conformance with the IMF specification MUST accept header fields with
an empty field body if permitted by the specific field syntax, with a
non-whitespace character immediately after the colon (including the
opening parenthesis of a parenthesized comment in structured fields which
use parentheses to delimit comments), with any reasonable (i.e. within
line length limits specified in the specification) number of space and/or
tab characters after the colon, and with line folding immediately following
the colon and preceding a non-empty field body.

It is also not correct to raise "interoperability" as an excuse for
attempting to push use of space-after-colon.  Gateways to/from other
protocols have to deal with a number of issues, and any specific
special-case requirements w.r.t. space-after-colon clearly fall into
the gateway implementors lap.

Since RFC 724 (May 1977, i.e. more than three DECADES!), whitespace
after the colon has been optional, and any implementation that does not
properly handle zero whitespace, or tab-after-colon is non-conforming
(i.e. broken) and therefore has no bearing on interoperability among
conforming implementations.  That said, it would be correct to point
out that both RFC 561 (the original IMF RFC) and RFC 680 (see RFC 724
for a discussion of RFC 680) did both require space-after-colon;
however, I would not want to burden readers of the next iteration of
the specification with such historical trivia as the specification
draft is already five dozen pages long, and there are far more important
issues that should be covered.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Comma-seperated lists
Date:  Sat, 26 Jan 2008 10:54:52 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 33
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

 [# and LWSP]=20
> What? What does one thing have to do with the other?

,,,,,   ,,,,,    ,,,,    ,,,,,
  ,     ,       ,          ,
  ,     ,,,,     ,,,       ,
  ,     ,           ,      ,
  ,     ,,,,,   ,,,,       ,  =20

| The full form is "<n>#<m>element " indicating at least
| <n> and at most <m> elements, each separated by one or
| more commas (",") and optional linear whitespace (LWS).
 [RFC 2068 2.1, similar in 2069, 2616, 2617, 2831]  ^^^

With optional FWS you get ASCII art.  With LWS you would
get "apparently empty lines" - causing havoc if something=20
in transit manages to transform them into a "really empty
line" (indicating the end of the header, CRLF CRLF).

>> Invalid: "sp ce"@example   Valid: "sp\ ce"@example  =20
>> Is that as it should be ?
=20
> Of course, both of those are valid in 2822.

Oops, indeed, 2822(upd) has *([FWS] qcontent), unescaped
SP matches FWS, no problem.  The 2821(bis) issue is using
<qcontent> instead of *(SP qcontent).  I fear I confused
2821 <Quoted-string> and 2822 <quoted-string> assuming
that they are "identical modulo CFWS", but that's wrong.

 Frank

To: [email-masked]
Xref: clerew local.mime:5211
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Space after colon - text requested
Date: Fri, 25 Jan 2008 17:20:27 GMT
Lines: 58
Sender: [email-masked]

In <[email-masked]> Mark Martinec <[email-masked]> writes:

>Here is my attempt:

A bit wordy, and not quite correct, so some emendations below:

>> 3.6.  Field definitions
>>
>>  The header fields of a message are defined here.  All header fields
>>  have the same general syntactic structure: A field name, followed by
>>  a colon, followed by the field body.  The specific syntax for each
>>  header field is defined in the subsequent sections.
>>
>>     Note: In the ABNF syntax for each field in subsequent sections,
>>     each field name is followed by the required colon.  However, for
>>     brevity sometimes the colon is not referred to in the textual
>>     description of the syntax.  It is, nonetheless, required.

>      Note: all structured and unstructured header field bodies defined
>      in this document allow an optional WSP as their first character.
>      It became a customary practice to place one SP there. Some other
>      protocols such as Netnews require such a SP, ...
                                ^
               [USEFOR (or RFC1036 at a pinch)], including NNTP [RFC3977],

>      ........ so a gateway needs
>      to modify header fields which do not conform to that stricter
>      specification. ..........
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
       ... and gateways cannot be relied upon to insert it .....

>      ....... Moreover, some popular electronic mail message
>      transportation programs like to modify header fields to conform to
>      their idea of a pretty header field. Such necessary or frivolous
>      transformations can be detected by recipients and may rise suspicion
>      about message integrity or authenticity. ......
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
       [I would be happy to omit that bit, but it does no harm]

>      ...... Continuance of the practice
>      by implementors to place one SP at the beginning of newly generated
>      header field bodies is encouraged in order to facilitate
>      interoperability.
+1

>>  It is important to note that the header fields are not guaranteed to
>>  be in a particular order.  ...

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:5212
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Date: Fri, 25 Jan 2008 17:31:21 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> Graham Klyne <[email-masked]> writes:

>[*] Question for the reader:  Is this still an implementation choice?  I parse 
>headers by first running an unescaping pass over the entire header, then parsing 
>the resulting tokenized stream, I know I'll accept some headers that are 
>strictly incorrect, but do I end up mis-interpreting or rejecting any valid 
>headers?  I'd like the answer to be No.

That strategy would actually cause Netnews to fail utterly if applied to
the Message-ID header, which is exactly why USEFOR has had to outlaw those
quoted-pairs and severely restrict quoted-strings. Which in turn is the
reason (or one reason) why it is now proposed to remove both those things
from the relevant parts of RFC 2822bis.

>I agree that it's not necessary, but non-normative text is OK.

>Reading this thread, it seems to me that this is a news/mail gateway issue and 
>not really applicable 'within the framework of "electronic mail" messages' (cf. 
>abstract), and as such the information might better belong in a separate 
>informative document about implementing mail/news gateways, which could do more 
>justice to a range of issues in this area.

Unfortunately, mail/news gateways, which tend to be ad-hoc scripts written
by each gatewayer, are unlikely to fix theis problem (since, in practice,
99.9999% of email messages already insert that SP). Hence the better
solution is to "encourage" implementors to continue doing what 99.9999%
of them already do.

In the case of the few implemenmtors that use HTAB, there may be problems
(IIRC correctly, there is some advice in USEFOR to accept HTAB as well as
SP, but don't try holding your breath until all News sites conform to that).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Fri, 25 Jan 2008 15:20:48 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Comma-seperated lists
Cc: [email-masked]
Sender: [email-masked]

On 1/25/08 at 11:16 AM +0100, Frank Ellermann wrote:

>>were the following ever legal instantiations of <#element>?
>>
>>, element ,
>>element, element,
>>, , ,
>
>Yes.

Got that. See my reply to Dave.

>That's the reason why 4234bis deprecates LWSP in an ABNF-comment [...]

What? What does one thing have to do with the other?

>If you want to "allow" series of comma (as "obs") it's one thing, 
>but if you want to reintroduce LWSP in any shape or form I think you 
>need an IAB majority.

I have no idea what you are talking about here. First of all, 
something akin to LWSP *is* allowed (and has been allowed since 2822) 
in "obs"; see section 4.2 of 2822 and of the present document. 
However, I don't see how any of that involves the IAB. Did you mean 
the IESG?

>What about the more interesting question of a "bare SP" within 
><quoted-string> asked on the SMT list ?  RFC 2822 demands that SP 
>cannot occure outside of <quoted-pair> in a <quoted-string>:
>
>Invalid: "sp ce"@example   Valid: "sp\ ce"@example   Is that as it should be ?

Of course, both of those are valid in 2822. That's a bug in 2821 that 
should be fixed.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 25 Jan 2008 15:09:55 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: Comma-seperated lists
Cc: ietf-822 <[email-masked]>, Dave Crocker <[email-masked]>
Sender: [email-masked]

On 1/25/08 at 6:55 AM -0800, Dave Crocker wrote:

>I read the above as at least implying that an empty list that 
>nonetheless contains commas is allowed.

OK. I'm having fun making this work. It also means that a list can 
begin or end with commas. Fun. Now, on a more interesting front:

>I read this combination as rather explicitly allowing white space 
>around a comma.

So, do you think the following is a legal group address in 822's syntax?

GroupName: [email-masked], (missing address), [email-masked], (another) ;

That is, how do you like comments between the commas?

This is all do-able. I just want to get it right.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 25 Jan 2008 06:55:41 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Pete Resnick <[email-masked]>
CC: ietf-822 <[email-masked]>
Subject: Re: Comma-seperated lists
Sender: [email-masked]

Pete Resnick wrote:
> RFC 2822 not withstanding, were the following ever legal instantiations 
> of <#element>?
> 
> , element ,
> element, element,
> , , ,
> 
> 822 is not clear on this point to me, but it seems like those might be 
> allowed. My obs- lists currently don't allow them. Thoughts?

1.  Empty List

> Wherever this construct is used, null elements
>      are allowed, but do not  contribute  to  the  count  of  elements
>      present.   That  is,  "(element),,(element)"  is  permitted,  but
>      counts as only two elements.  Therefore, where at least one  ele-
>      ment  is required, at least one non-null element must be present.
>      Default values are 0 and infinity so that "#(element)" allows any
>      number,  including  zero;  "1#element" requires at least one; and
>      "1#2element" allows one or two.

I read the above as at least implying that an empty list that nonetheless 
contains commas is allowed.

2.  White Space Around Commas

> 3.4.2.  WHITE SPACE
> 
>         Note:  In structured field bodies, multiple linear space ASCII
>                characters  (namely  HTABs  and  SPACEs) are treated as
>                single spaces and may freely surround any  symbol. 

and

>      specials    =  "(" / ")" / "<" / ">" / "@"  ; Must be in quoted-
>                  /  "," / ";" / ":" / "\" / <">  ;  string, to use
>                  /  "." / "[" / "]"              ;  within a word.

I read this combination as rather explicitly allowing white space around a comma.

I'll note that all of this matches what I would have expected -- and even 
wished that 822 had been more clear about.

The goal with the lexical model and rule for white space was to allow its use 
to be extremely free.  If there was anywhere it might get used, without the 
ability to do parsing, the goal was to permit it.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Comma-seperated lists
Date:  Fri, 25 Jan 2008 11:16:04 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 27
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:
=20
> were the following ever legal instantiations of <#element>?
=20
> , element ,
> element, element,
> , , ,

Yes.  That's the reason why 4234bis deprecates LWSP in an ABNF-
comment, it's also one of the reasons why the HTTP RFCs have
to be fixed, and it's one of the reasons why DIGEST-MD5 (2831)
will be declared to be obsolete.

> My obs- lists currently don't allow them. Thoughts?

If you want to "allow" series of comma (as "obs") it's one thing,
but if you want to reintroduce LWSP in any shape or form I think
you need an IAB majority. =20

What about the more interesting question of a "bare SP" within=20
<quoted-string> asked on the SMT list ?  RFC 2822 demands that
SP cannot occure outside of <quoted-pair> in a <quoted-string>:

Invalid: "sp ce"@example   Valid: "sp\ ce"@example   Is that as
it should be ? =20

 Frank

Date: Fri, 25 Jan 2008 03:00:11 -0600
To: ietf-822 <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Comma-seperated lists
Sender: [email-masked]

in RFC 822 and previous, there are several list constructs using the form:

1#element

or

#element

RFC 2822 not withstanding, were the following ever legal 
instantiations of <#element>?

, element ,
element, element,
, , ,

822 is not clear on this point to me, but it seems like those might 
be allowed. My obs- lists currently don't allow them. Thoughts?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: quoted-pair in dcontent / Space after colon
Date:  Fri, 25 Jan 2008 09:24:17 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 12
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:
=20
> Messages with a TAB rather than a SP will interoperate with Netnews
> better than messages with neither, but some Netnews software will
> still baulk at the TAB.

When I tested it (in the "magic SP" epoch of the USEFOR saga) HTAB
failed, and that killed the proposal to migrate the "magic SP" to a
"mandatory FWS" as something that could also fly in a successor of
RFC 2822.

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Space after colon - text requested
Date:  Fri, 25 Jan 2008 09:14:39 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 6
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Mark Martinec wrote:
[...]

+1

To: [email-masked]
Xref: clerew local.mime:5199
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Date: Thu, 24 Jan 2008 17:13:18 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Tony Hansen <[email-masked]> writes:

>-1

>Requiring a SP character after the : is a non-starter for me. I've used 
>a number of systems that use TAB instead of SP; they're still out there 
>and should not be considered invalid.

Well we are not proposing to REQUIRE it.

Messages with a TAB rather than a SP will interoperate with Netnews better
than messages with neither, but some Netnews software will still baulk at
the TAB.

But all we are doing is pointing out to implementors that SP is the
"usual" practice, and explaining the consequences (non-interoperability
with many/most Netnews systems) if they do not follow it. It is up to
implementors to decide whether that disadvantage matters enough for them
to change their system. More realistically, my wording should encourage
them to include SP rather than TAB in any new software that gets written.

I don't expect the world to be 100% perfect, but OTOH I don't want it ever
to get less perfect than it already is due to ignorance of some small
feature that incurs negligible cost to implement "properly".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: Mark Martinec <[email-masked]>
Organization: Jozef Stefan Institute
To: [email-masked]
Subject: Re: Space after colon - text requested
Date: Fri, 25 Jan 2008 02:28:50 +0100
Sender: [email-masked]

On Thursday 24 January 2008 21:45:07 Pete Resnick wrote:
| There seems to be a not very strong consensus for some non-normative text.
|
| I would like some text ****and a specific place in the current
| document to put it****. If you want text, you had better provide me
| with an example and a location.

Here is my attempt:

> 3.6.  Field definitions
>
>  The header fields of a message are defined here.  All header fields
>  have the same general syntactic structure: A field name, followed by
>  a colon, followed by the field body.  The specific syntax for each
>  header field is defined in the subsequent sections.
>
>     Note: In the ABNF syntax for each field in subsequent sections,
>     each field name is followed by the required colon.  However, for
>     brevity sometimes the colon is not referred to in the textual
>     description of the syntax.  It is, nonetheless, required.

      Note: all structured and unstructured header field bodies defined
      in this document allow an optional WSP as their first character.
      It became a customary practice to place one SP there. Some other
      protocols such as Netnews require such a SP, so a gateway needs
      to modify header fields which do not conform to that stricter
      specification. Moreover, some popular electronic mail message
      transportation programs like to modify header fields to conform to
      their idea of a pretty header field. Such necessary or frivolous
      transformations can be detected by recipients and may rise suspicion
      about message integrity or authenticity. Continuance of the practice
      by implementors to place one SP at the beginning of newly generated
      header field bodies is encouraged in order to facilitate
      interoperability.

>  It is important to note that the header fields are not guaranteed to
>  be in a particular order.  ...

  Mark

Date: Thu, 24 Jan 2008 14:45:07 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Space after colon - text requested
Sender: [email-masked]

There seems to be a not very strong consensus for some non-normative text.

I would like some text ****and a specific place in the current 
document to put it****. If you want text, you had better provide me 
with an example and a location.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Thu, 24 Jan 2008 14:42:21 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Remove quoted-pair domain literals
Sender: [email-masked]

Consensus seems to have been pretty clear on removing quoted-pair 
completely from domain literals. Unless I hear objection, consider 
this done.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Thu, 24 Jan 2008 15:18:57 +0000
From: Tony Finch <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Pete Resnick <[email-masked]>, Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New	2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

On Thu, 24 Jan 2008, Arnt Gulbrandsen wrote:
> Pete Resnick <[email-masked]>
> > 1. Remove quoted-pair entirely from dcontent in the generate syntax
> > (meaning "[", "]", and "\" may never appear in dcontent).
>
> I like this.
>
> There are too many weird corners in address literals as it is. Too
> much syntax in the gray area between meaningful and illegal. So I say
> yes to dropping this.

Makes sense to me too.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FAIR ISLE: SOUTHWEST 7 TO SEVERE GALE 9, OCCASIONALLY STORM 10, PERHAPS
VIOLENT STORM 11 LATER. VERY ROUGH BECOMING HIGH OR VERY HIGH. RAIN OR SQUALLY
SHOWERS. MODERATE OR POOR.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: quoted-pair in dcontent / Space after colon
Date:  Thu, 24 Jan 2008 13:56:14 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 28
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Arnt Gulbrandsen wrote:

>> I've used a number of systems that use TAB instead of SP;
>> they're still out there and should not be considered invalid.
=20
> I am persuaded by this argument.

Charles proposed SHOULD, without SHOULD it's just an advice,=20
nothing remotely related to "invalid".
=20
 [you have]
> sent umpty million colon-tab messages. It attracted more than
> its fair share of problems, and I wouldn't recommend doing it.

In other words:  If implementors go to the trouble to insert a
WSP after the colon (excluding cases of folding immediately=20
after the colon, and ignoring some simple minded scripts) they
better pick SP instead of HTAB for this WSP.

Your argument supports Charles' proposal to mention this, the
current net-utf8 draft also "dislikes" HTAB-incarnations of WSP.

I was unhappy with this detail precisely because of WSP in mail,
but "accept HTAB, better don't create it" makes a lot of sense
for the discussed "after colon" position.  For "after CRLF" in
a folded header field I'd be more stubborn in defending HTAB :-)

 Frank

Date: Thu, 24 Jan 2008 13:30:31 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: Pete Resnick <[email-masked]>
Cc: Charles Lindsey <[email-masked]>, [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

Pete Resnick <[email-masked]>
> 1. Remove quoted-pair entirely from dcontent in the generate syntax 
> (meaning "[", "]", and "\" may never appear in dcontent).

I like this.

There are too many weird corners in address literals as it is. Too
much syntax in the gray area between meaningful and illegal. So I say
yes to dropping this.

Arnt

Date: Thu, 24 Jan 2008 13:22:07 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Cc: Tony Hansen <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

Tony Hansen <[email-masked]>
> Requiring a SP character after the : is a non-starter for me. I've used a 
> number of systems that use TAB instead of SP; they're still out there and 
> should not be considered invalid.

I am persuaded by this argument.

FWIW, I was postmaster/listmaster/more at a Zmailer site for many
years and have sent umpty million colon-tab messages. It attracted
more than its fair share of problems, and I wouldn't recommend doing
it. But I agree: There is longstanding minority usage of colon-tab and
so it should not be disallowed now. Gateways just need to cope as they
always have.

Arnt

Date: Thu, 24 Jan 2008 11:17:44 +0000
From: Graham Klyne <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

With reference to:
   http://www.imc.org/ietf-822/mail-archive/msg06171.html

I was asked to comment.  I've reviewed the ensuing thread, but am not fully 
aware of all prior discussion.

Pete Resnick wrote:
> [On quoted-pair in dcontent...]
> 
> So, there are two forms to this request:
> 
> 1. Remove quoted-pair entirely from dcontent in the generate syntax 
> (meaning "[", "]", and "\" may never appear in dcontent).
> 2. Only allow the quoted-pairs "\[", "\]", and "\\" in dcontent in the 
> generate syntax.
> 
> My personal opinion: #2 makes me queasy and I will argue tooth and nail 
> against it. If people are heavily in favor of #1, I won't raise a stink, 

The coder in me would prefer (1) that escaping be applied consistently 
*everywhere* in a message header, or (2) applied in as few different places as 
possible.

Since (1) is a long gone option [*], I'd favour #1 in the absence of any 
compelling reason to retain the escape chars in domains or message ids.  I've 
scanned the thread forward from this message and in a fair amount of discussion 
have not seem any reason to keep these escapes.  If not applied consistently per 
option (1), escaping in message ids seems like a particularly poor option, 
complicating as it does the very notion of identity comparison.

cf.
   http://resnick1.qualcomm.com/draft-resnick-2822upd-04.html#addrspec
   http://resnick1.qualcomm.com/draft-resnick-2822upd-04.html#ident

I agree with other comments that limiting to escapes to \[, \\, \], or simply 
allowing \ as a non-escape character are not desirable options.  The former 
created yet another special case, and the I think latter has the potential for 
great nasty surprises - the use of \ as an escape in this context is, I think, 
just too deeply ingrained.

[*] Question for the reader:  Is this still an implementation choice?  I parse 
headers by first running an unescaping pass over the entire header, then parsing 
the resulting tokenized stream, I know I'll accept some headers that are 
strictly incorrect, but do I end up mis-interpreting or rejecting any valid 
headers?  I'd like the answer to be No.

> [On space after the ":"...]
> My personal feeling on this is again it is not necessary, and that a 
> gateway should be dealing with this anyway. But if people want text, I 
> would be OK with a non-normative note in either 2.2 or 3.6. (Anything 
> with a SHOULD I will make a stink over.) As editor, I'd want specific 
> text agreed to and instructions on where to put it.

I agree that it's not necessary, but non-normative text is OK.

Reading this thread, it seems to me that this is a news/mail gateway issue and 
not really applicable 'within the framework of "electronic mail" messages' (cf. 
abstract), and as such the information might better belong in a separate 
informative document about implementing mail/news gateways, which could do more 
justice to a range of issues in this area.

cf.
   http://resnick1.qualcomm.com/draft-resnick-2822upd-04.html#fields
   http://resnick1.qualcomm.com/draft-resnick-2822upd-04.html#fielddefs

#g

-- 
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Wed, 23 Jan 2008 16:24:08 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

Pete Resnick wrote:
> 
> I'm separating out these two issues, as I want to hear yeas and nays 
> (and explanations) from people other than Charles and Frank.
> ...
> 
> So, there are two forms to this request:
> 
> 1. Remove quoted-pair entirely from dcontent in the generate syntax 
> (meaning "[", "]", and "\" may never appear in dcontent).
 > 2. Only allow the quoted-pairs "\[", "\]", and "\\" in dcontent in the
 > generate syntax.
 >
 > Of course, quoted-pair would be in the obs- (interpret) syntax.

+1 for #1. I agree with removing these from the generate syntax. I see 
no need for us to ever generate them in a domain address literal.

> [On space after the ":"...]
> 
> [Charles originally wanted:]
>>>> Although in all header fields the CFWS following the colon is 
>>>> optional, it is customary to place at least one SP there; moreover 
>>>> such a SP is mandatory in some related protocols (notably 
>>>> [NETNEWS]). In order to facilitate interoperability with such 
>>>> related protocols, that SP SHOULD normally be present.
> 
> [Frank suggested:]
>>> Although all message header fields allow either [FWS] or [CFWS] after 
>>> the colon, it is customary to place at least one white space 
>>> character after the colon.  In NetNews articles [RFC xxxx] it's 
>>> mandatory to use a space character after the colon, and in order to 
>>> faciliate interoperability implementors might wish to separate the 
>>> header field name plus colon from the header field body by a space in 
>>> all Internet messages.
>>
>> But if we can agree that a wording of some form along the lines of 
>> what I and Frank are suggesting, then the exact wording can be worked 
>> on by further discussion. As I said, either wording is netter than 
>> nothing.
> 
> My personal feeling on this is again it is not necessary, and that a 
> gateway should be dealing with this anyway. But if people want text, I 
> would be OK with a non-normative note in either 2.2 or 3.6. (Anything 
> with a SHOULD I will make a stink over.) As editor, I'd want specific 
> text agreed to and instructions on where to put it.

-1

Requiring a SP character after the : is a non-starter for me. I've used 
a number of systems that use TAB instead of SP; they're still out there 
and should not be considered invalid.

	Tony Hansen
	[email-masked]

Date: Wed, 23 Jan 2008 14:41:18 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Sender: [email-masked]

Thanks Nick. Those goals are still worth following, even after this many 
years, and it's worth being reminded of them.

	Tony Hansen
	[email-masked]

Nick Shelness wrote:
> Frank,
> 
> You wrote on 01/18/2008 12:59:47:
> 
>> ...
>> For a DS it has to be implemented and needs to be interoperable in
>> some way, and "breaks in NetNews" would be suspicious, UAs for mail
>> and news likely support only one approach where possible.
>> ...
> 
> I realize that the drums effort began so long ago that it is easy to 
> forget its objective.
> 
> To paraphrase, it was not to CHANGE existing Full Standards (822, 1123, 
> etc) and other RFCs that it was expected would reach Full Standard status 
> (even though they haven't yet), but to produce a small set of documents 
> that superseded, and where necessary (and where possible) clarified, them 
> all. The working group reached early consensus that it could not outlaw 
> constructs collectively allowed by these earlier standards even though the 
> group would deprecate their use. Pete's (if memory serves me correctly, it 
> was his invention) obs- non-obs approach was then accepted as the best 
> vehicle to achieve this. So the only remaining issue viz the obs- syntax 
> is simply whether it accurately captures the syntax explicitly allowed by 
> these earlier standards and not whether these obs- constructs are good, 
> bad or indifferent.
> 
> Nick
> 

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: quoted-pair in dcontent / Space after colon
Date:  Wed, 23 Jan 2008 19:28:58 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 48
Reply-To: "Frank Ellermann" <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

Charles Lindsey wrote:
=20
> The whole of the <msg-id> syntax in USEFOR is a "mess",
> and we knew it was so when we wrote it.

Well, it used to be completely wrong for some years, simply
copying whatever RFC 2822 said without checking very basic=20
requirements like "does it work anywhere, e.g. with NNTP ?"

After it was clear that the USEFOR WG didn't bother for six
years to get at least this fundamental concept for NetNews
right, the Message-ID syntax was fixed to reflect a minimal
subset of RFC 2822 still expected to survive NNTP:  Killing
NO-WS-CTL and unnecessary <quoted-pair> everywhere, keeping
fictitious <no-fold-literal>s with \[, \\, or \], because
RFC 2822 demanded that they are "valid".

As far as that was still messy it was the RFC 2822 mess, and
IIRC you wanted it in the spec.  Similarly some wanted [FWS]
in places where it made no sense, until I posted an erratum
about one pointless [FWS] in RFC 2822, and it was pulled.

If you are now ready to pull also \[, \\, and \] please do,
but don't introduce a new bare backslash in domain literals
while you're at it.

> I hope we will find some way (AUTH48 or otherwise) to fix
> USEFOR so as to match <msg-id> in 2822bis exactly.

I'll believe it when I see it, confirmed by the responsible
area director.  It took ten years so far, what the approved
RFC says is already supposed to work, plausible Message-IDs
up to very ugly ideas match the approved syntax, while worse
constructs breaking NNTP won't match.

> My wording will still make an improvement in=20
> interoperability in 99% of cases, and I don't think we are
> going to persuade this List to go beyond that.

I've no idea how you determined that 99% of all cases wihout
SP after the colon are a missing "magic SP".  The only cases
where I ever did this in e-mail were related to colon-CRLF-SP,
or in other words folding, e.g. when that's reqired for long
2047-encoded words due to 2047 line length limits.  An issue
also relevant for 2231-encoded words with long 4646 language
tags, trapped in its "magic SP" heritage USEFOR ignored this.

 Frank

Date: Wed, 23 Jan 2008 09:13:16 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Sender: [email-masked]

Pete Resnick wrote:
> 1. Remove quoted-pair entirely from dcontent in the generate syntax 
> (meaning "[", "]", and "\" may never appear in dcontent).

+1

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
Xref: clerew local.mime:5193
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon
Date: Wed, 23 Jan 2008 14:12:47 GMT
Lines: 43
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Charles Lindsey wrote on the 822 list:

>> Yes, I agree that #2 is a mess.

>How on earth did it end up in the NetNews RFC if you agree that
>it is messy ?  Cc: USEFOR, maybe that is a simple AUTH48 fix (?)

The whole of the <msg-id> syntax in USEFOR is a "mess", and we knew it was
so when we wrote it. The only reason it was put that way is that we were
trying to make the minimum alteration from RFC 2822.

But now we are writing RFC 2822bis, and there is no reason for putting
"messiness" in there. If RFC 2822bis comes out with a really clean syntax
for <msg-id> (and it has gone a long way in that direction already), then
I hope we will find some way (AUTH48 or otherwise) to fix USEFOR so as to
match <msg-id> in 2822bis exactly.

> [magic SP]

>And a field name-colon-space-folding-WSP-body contruct is
>not a "magic SP", for NetNews a non-empty part of the body
>has to exist in the first line (before the first folding) -
>some header fields are better not folded at all in NetNews.

Yes my proposed wording, even if followed by implementors, will still
leave a slight difference as regards folding leaving empty header bodies,
but I think we have to live with that. My wording will still make an
improvement in interoperability in 99% of cases, and I don't think we are
going to persuade this List to go beyond that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 22 Jan 2008 17:18:30 -0500 (New York)
From: Gary Welles <[email-masked]>
To: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Sender: [email-masked]

The first sentence of the second paragraph of Section 3.6.2 
appears to suggest that "Reply-to:" indicates "source":

> The originator fields indicate the mailbox(es) of the source of the
> message.

In that "source of the message" could be a source of 
misinterpretation, it might be best to eliminate the sentence.

-- Gary

Date: Tue, 22 Jan 2008 12:55:09 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Sender: [email-masked]

Pete Resnick wrote:
> things by showing only that line. Here's the new wording of the entire 
> note:
> 
> Note: The transmitter information is always present. The absence of the 
> "Sender:" field is sometimes mistakenly taken to mean that the agent 
> responsible for transmission of the message has not been specified. The 
> absence of the "Sender:" field merely means that the transmitter is 
> identical to the author and is therefore not redundantly placed into a 
> separate field.

+1

d/

ps. my motivation for pursuing this issue is that, in effect, I want to 
highlight that the agent responsible for posting really has been stated 
"explicitly".  But it merely has not be coded "differentially"...

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
Xref: clerew local.mime:5189
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon
Date: Tue, 22 Jan 2008 11:48:40 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Mark Martinec <[email-masked]> writes:

>Charles Lindsey wrote:
>> I think it is clear (3 people against it) that the wording would have to
>> avoid "SHOULD". So here is a suggested wording avoiding 2119 language:
>>
>> Although in all header fields the CFWS following the colon is optional, it
>> is customary practice to place at least one SP there; moreover such a SP
>> is mandatory in Netnews [UESFOR] (and possibly in other protocols).
>> Continuance of that practice by implementors is encouraged in order to
>> facilitate interoperability.
>>
>> That could probably be tuned further.

>The text looks fine, but I would like to point out one catch
>which could result from placing too much emphasis on that space.
>Namely, a header field consists of a header field name, a colon,
>and a header field body, i.e. everything after a colon belongs
>to a field body, including that [CFWS]. It would be wrong if
>it could somehow be undestood that a field body starts only
>*after* a space.

I think the "man in the street" would be surprised to be told that initial
(or even trailing) whitespace was a part of the field-body (though he
would assuredly regard any <comment> as being part of it).

So first question, is whether that is really the best definition of "field
body"?

And the second question is what differences would arise if it were defined
differently?

I think we need to understaand the implications of question 2 before
deciding whether some further tuning of the text is needed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: quoted-pair in dcontent / Space after colon
Date:  Tue, 22 Jan 2008 13:08:27 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 82
Reply-To: "Frank Ellermann" <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

Charles Lindsey wrote on the 822 list:

>> 2. Only allow the quoted-pairs "\[", "\]", and "\\" in dcontent
>> in the generate syntax.
[...]
>> My personal opinion: #2 makes me queasy and I will argue tooth
>> and nail against it.
=20
> Yes, I agree that #2 is a mess.

How on earth did it end up in the NetNews RFC if you agree that
it is messy ?  Cc: USEFOR, maybe that is a simple AUTH48 fix (?)

> 3. As an alternative to #2, allow "\" as a character in <dtext>
> in its own right (but with no special semantic interpretation).
> I am not pushing for that

Good, because introducing a new "\"-semantics is even worse.  We
are talking about *hypothetical* new domain literals, and there
is no backslash in an STD 66 IPvFuture:

| IPvFuture  =3D "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )
[...]
| unreserved =3D ALPHA / DIGIT / "-" / "." / "_" / "~"
[...]
| sub-delims =3D "!" / "$" / "&" / "'" / "(" / ")"
|                  / "*" / "+" / "," / ";" / "=3D"

There's also no "[" or "]" within an IPvFuture, it's beyond me
why we need to talk for months in USEFOR (and now again here)
about this cruft.

> it would allow everything that is allowed at present, if
> people care about that.

How could they care about non-existing IPvFuture "features"
not permitted in STD 66 ?  Where was <quoted-pair> in <dtext>
ever used outside of theoretical discussions ?  Implementors
wishing to ignore a "\" as (an unnecessary) <quoted-pair> in
<dtext> can still do this, even if it's "not allowed".

The opposite approach, "allow" it and then say why it breaks
e.g. octect-comparison of Message-ID, is far more convoluted.

> 4. As an addition to any of #1, #2 or #3, make the change
> only in <no-fold-literal>, leaving <domain-literal> alone.

Good for Message-ID (etc.), but not good enough for 2821bis.

> But of those, my preference would still be for #1, or for
> #1+#4 as second best.

Yes, #1 is KISS, and if the inventors of a new IP version or
another "domain literal" concept really want backslashes or
square brackets *within* their constructs let them "fix" the
relevant RFCs (STD 66, 2822upd, 2821bis, NetNews, ...)

> Every gateway is, in practice, a hand-crafted script=20
> tailored for its specific purpose.

+1  And gateway operators won't like to "fix" any Message-ID,
there are simpler ways to commit net suicide.  I also don't
see how gateway operators are supposed to "fix" or replace
non-existing domain literals in addresses for a peer network,
see 2821bis 3.7.4 for some MUSTard and SHOULDs about this.

 [magic SP]
> Same answer as regards gateways. People writing ad hoc
> scripts are just not going to bother to check for such
> unlikely circumstances.

The script won't be able to inject any message without the
"magic SP" for critical header fields into NetNews, a "GiGo"
approach won't fly.  It's no "unlikely" scenario, a header
field folded immediately after the colon makes sense. =20

And a field name-colon-space-folding-WSP-body contruct is
not a "magic SP", for NetNews a non-empty part of the body
has to exist in the first line (before the first folding) -
some header fields are better not folded at all in NetNews.

 Frank

Date: Mon, 21 Jan 2008 12:42:32 -0800
To: Pete Resnick <[email-masked]>
From: SM <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked]
Sender: [email-masked]

Hi Pete,
At 12:28 19-01-2008, Pete Resnick wrote:
>No. That's not how RFC 2119 MUSTs and SHOULDs are used. MUST NOT 
>means that doing so is known to cause harm or interfere with 
>interoperation. That simply isn't the case here. We can imagine 
>cases where having redundant Sender information might cause 
>confusion, but we say SHOULD NOT because there may exist valid 
>reasons where the behavior is acceptable.

Agreed.  Thank you for the clarification.

Regards,
-sm 

From: Mark Martinec <[email-masked]>
Organization: J. Stefan Institute
To: [email-masked]
Subject: Re: quoted-pair in dcontent / Space after colon
Date: Mon, 21 Jan 2008 18:55:34 +0100
Sender: [email-masked]

Charles Lindsey wrote:
> I think it is clear (3 people against it) that the wording would have to
> avoid "SHOULD". So here is a suggested wording avoiding 2119 language:
>
> Although in all header fields the CFWS following the colon is optional, it
> is customary practice to place at least one SP there; moreover such a SP
> is mandatory in Netnews [UESFOR] (and possibly in other protocols).
> Continuance of that practice by implementors is encouraged in order to
> facilitate interoperability.
>
> That could probably be tuned further.

The text looks fine, but I would like to point out one catch
which could result from placing too much emphasis on that space.
Namely, a header field consists of a header field name, a colon,
and a header field body, i.e. everything after a colon belongs
to a field body, including that [CFWS]. It would be wrong if
it could somehow be undestood that a field body starts only
*after* a space.

  Mark

To: [email-masked]
Xref: clerew local.mime:5186
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Date: Mon, 21 Jan 2008 13:07:10 GMT
Lines: 92
Sender: [email-masked]

In <p0625011ec3b5dfaa7044@[74.134.5.163]> Pete Resnick <[email-masked]> writes:

>On 1/17/08 at 1:48 PM +0000, Charles Lindsey wrote:

>[On quoted-pair in dcontent...]

>So, there are two forms to this request:

>1. Remove quoted-pair entirely from dcontent in the generate syntax 
>(meaning "[", "]", and "\" may never appear in dcontent).
>2. Only allow the quoted-pairs "\[", "\]", and "\\" in dcontent in 
>the generate syntax.

>Of course, quoted-pair would be in the obs- (interpret) syntax.

>My personal opinion: #2 makes me queasy and I will argue tooth and 
>nail against it.

Yes, I agree that #2 is a mess. But there are some other possibilities:

3. As an alternative to #2, allow "\" as a character in <dtext> in its own
right (but with no special semantic interpretation). I am not pushing for
that, but it would allow everything that is allowed at present, if people
care about that.

4. As an addition to any of #1, #2 or #3, make the change only in
<no-fold-literal>, leaving <domain-literal> alone.

But of those, my preference would still be for #1, or for #1+#4 as second
best.

> ... If people are heavily in favor of #1, I won't raise 
>a stink, but I don't see the point: If you're writing a gateway from 
>e-mail to Netnews, you're going to have to go through the messages 
>looking for things you don't like anyway, so why not clean up these 
>along with everything else?

Because, unlike MTAs and MUAs, you do not buy gateways "off the shelf".
Every gateway is, in practice, a hand-crafted script tailored for its
specific purpose. The WG's USEPRO document contains a long list of
'gotchas' to avoid when gatewaying to/from Netnews, and fixing syntactic
differences is way below the horizon in comparison with them. In fact,
writing the 'perfect' bidirectional gateway between news and mail that
will work in every conceivable combination of circumstances is just about
impossible.

In practice, such gateways are nearly always limited in scope. For
example, I gateway this list into a local newsgroup on my machine, and the
script to do it is just 19 lines of Bourne shell.

>In any event, doing #1 is an easy edit. I just want to know what 
>people want to do.

>[On space after the ":"...]

>>But if we can agree that a wording of some form along the lines of 
>>what I and Frank are suggesting, then the exact wording can be 
>>worked on by further discussion. As I said, either wording is netter 
>>than nothing.

>My personal feeling on this is again it is not necessary, and that a 
>gateway should be dealing with this anyway. But if people want text, 
>I would be OK with a non-normative note in either 2.2 or 3.6. 
>(Anything with a SHOULD I will make a stink over.) As editor, I'd 
>want specific text agreed to and instructions on where to put it.

Same answer as regards gateways. People writing ad hoc scripts are just
not going to bother to check for such unlikely circumstances.

I think it is clear (3 people against it) that the wording would have to
avoid "SHOULD". So here is a suggested wording avoiding 2119 language:

Although in all header fields the CFWS following the colon is optional, it
is customary practice to place at least one SP there; moreover such a SP
is mandatory in Netnews [UESFOR] (and possibly in other protocols).
Continuance of that practice by implementors is encouraged in order to
facilitate interoperability.

That could probably be tuned further.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Please comment on draft-duerst-mailto-bis-04.txt
Date:  Sun, 20 Jan 2008 12:57:09 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 262
Reply-To: "Frank Ellermann" <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

Martin Duerst wrote:
=20
> I expect to submit it to the IESG soon.

The draft doesn't depend on the improvements in 2822upd
or 4234bis, referencing RFC 2822 and 4234 is no issue.

 [RFC 2368]
> contains some advice against using a bcc field in a
> mailto: URI, but this doesn't seem to be followed,
> and we were unable to find any reason, so we removed it.

MUAs might not support Bcc, not display it in their=20
default configuration, or reject attempts to preset=20
Bcc.  Better keep the advice, in an ordinary public
mailto: URL Cc: has more or less the same effect.

> I'm not subscribed to [email-masked], so please
> keep me (and my coauthors) in the cc.

Can't do, will send separate copy from the Reply-To
address, URI list added to the Cc:

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=

=3D Review of mailto-bis-04 (top down single pass) =3D

You use %2C to separate <addr-spec>s derived from
the RFC 2368 syntax #mailbox based on RFC 822 2.7.
Among other # oddities that forbids runs of comma.

How does %2C match the overall STD 66 syntax ?  If
xxx in a mailto:xxx?yyy pattern is a <hier-part>,
and that's a <path-rootless>, finally arriving at
<segment-nz>, then an unencoded comma is a <pchar>=20
matching <sub-delims>, why do you use %2C here ?

My old browser had issues with comma in URLs, but
it was implemented years before STD 66 was written.

The difference between your <some-delims> and the=20
<sub-delims> plus ":" and "@" in STD 66 appears to
be "&" and "=3D" (you need them for &hname=3Dhvalue)
PLUS "/" and "?".  What's the reason to exclude=20
"/" and "?" from <some-delims> ?

Whatever you end up with, please explain it in the
draft, comparing obscure ASCII subsets is a PITA.

A note about <addr-spec> says that some characters
have to be percent-encoded because they are not=20
allowed in an STD 66 URL.  It took me about a year
to understand that that's beside the point for the
purpose of news: URLs.  Some characters have to be
percent-encoded because they otherwise don't match
<pchar> in STD 66.

That's a subtle difference.  You write that of 'the
characters in sub-delims, at least the following
also have to be percent-encoded: "&", ";", and "=3D"'

I don't see why, you need "&" and "=3D" only after the
"?", not before in the address list, and you don't
do anything special with ";".  Percent-encoding ","
*within* <addr-spec> would make sense, if you use it
as delimiter for an address list, but at the moment
the draft uses %2C.

Testing mail to "co,ma"@example + "am,oc"@example=20
1: mailto:%22co,ma%22@example%2C%22am,oc%22@example

That would be IMO strange, cleaner versions could be
2: mailto:%22co,ma%22@example,%22am,oc%22@example
3: mailto:%22co%2Cma%22@example,%22am%2Coc%22@example

(2) keeps comma as is no matter what its purpose is,
(3) uses an unencoded comma to separate addresses.

You forbid NO-WS-CTL and <obs-local-part>, please add
<obs-domain>.  That's crap like user@example(oops).com
with comments, folding, and white space on both sides
of the dots separating domain labels. =20

IMO you need a MUST NOT for all obs-cenities mirroring
the same MUST NOT in RFC 2822 and 2822upd.  If 2822upd
adds NO-WS-CTL to "obs" and you upgrade the normative
reference remove the then redundant MUST NOT NO-WS-CTL.

Note (3) about comments and whitespace in a local part
is ambiguous, you want to forbid CFWS, but likely not
<quoted-pair> horrors like "sp\ ce"@example - I think
that works, mailto:%22sp%5C%20ce%22@example is okay (?)

  BTW, for reasons unknown to me "bare space" is not=20
  allowed in <quoted-string>, recently discussed on the
  SMTP list.  This could be a bug in RFC 2822 not yet
  fixed in 2822upd, maybe a missing SP in <qtext> (?!?)

Note (4) is odd, there are no "non-ASCII" characters
in domains used for (non-EAI) e-mail, so why discuss=20
it, or limit it to domains ?  You could just say that=20
all percent-encoded characters that are not ASCII are
supposed to be UTF-8 for compatibility with RFC 3987
and ongoing I18N work (EAI, IDN, IRI).=20

While you're at it please note that this won't work
as expected with many UAs and so SHOULD be avoided at
the moment.  All mailto: URI producers SHOULD use the
A-label form of domains, URI consumers might have no
idea what U-labels are, percent-encoding doesn't help
with this issue.  It only helps in non-UTF8 documents
for native mailto: IRIs not in the document charset.

Or maybe for *all* mailto: IRIs in non-UTF8 documents,
the Firefox 2 bug for your non-UTF-8 IRI tests likely
also affects mailto:, not only http:

| When the internationalized domain name is used to
| compose a message, the name must be transformed to
| the IDNA encoding where appropriate [RFC3490].

The appropriate place is IMO the mailto URI producer.

| The considerations for reg-name in [STD66] apply.

I'm not aware of a registry permitting %-characters
in their <reg-name>s.  Please let's focus on the LDH-
labels as required for SMTP in mailto: URLs.  We can
do UTF8SMTP etc. later, mailto is complex enough. :-(

Okay, you have "should A-label" at the end of (4), I
proposed a kind of "temporary" SHOULD above.  Notes
4 and 5 are far too long and confusing, chapter 2 is
for folks trying to understand the mailto: syntax,
incl. users not familiar with EAI / IDN / IRI / I18N.

Notes 4 and 5 could be subsections of a section with
"I18N considerations".  IMO any I18N in 2368bis is
irrelevant before it doesn't at least work for ASCII
and STD 66.

| Percent-encoding is needed for the same characters
| as listed above for "addr-spec".

Stupid question, why ?  Behind the "?" you don't need
to worry about "/" and "?" anymore (just an example).

| The "body" hname should contain the content for the
| first text/plain body part of the message.

s/hname/hvalue/  The body=3D concept is weird.  It's not
clear what the charset is, if you assume UTF-8 it might
not fly for old UAs, assuming document charset is worse,
assuming local charset of the UA also makes no sense.

Better deprecate it.  Who implemented body=3D anyway, and
how bad was it ?

| Non-ASCII characters can be encoded in hvalue as follows:

Indeed, it works like a charme, but not for body=3D hvalue.

| Non-ASCII characters can be encoded according to UTF-8
| [STD63], and then each octet of the corresponding UTF-8
| sequence is percent-encoded to be represented as URI
| characters.

But that doesn't fly.  The MUA started by the browser (if=20
that's what happens) can assume that it's running in the
local charset of the operating system, it doesn't need to
support UTF-8 at all.  The only interoperable solution is
what you have as (1), RFC 2047 + 2231.  (2) doesn't work,
not before UTF-8 is the only charset worldwide outside of
museums.  You cannot decree this above what RFC 2277 did.

It's the job of the URI producer to get it right, mailto:
URLs prepare a message/rfc822 with an US-ASCII header.

The URI consumer is the weaker part - you have to protect
them for interoperability, not force them to upgrade when
they're not ready for it.  Shift all "UTF-8 and beyond"
issues into "I18N considerations" - they muddy the water
for the job at hand, define a STD 66 compatible mailto:
URL preparing a 2822upd + MIME compatible message/rfc822.

| mailto:?to=[email-masked][email-masked]

If my <segment-nz> theory has merits this is not "nz",
i.e. syntactically invalid.  I'm too lazy to check this
against the regular expression in STD 66.

A hname "to" like "bcc", let alone "body", is a bad idea=20
and best avoided.  The "to" function belongs to "mailto",
not into the query part - a very simple mailto: approach
could be to ignore query parts.

| A mailto URI designates an "internet resource", which
| is the mailbox specified in the address.

I'm not sure about this, the "resource" appears to be a
"proto"-message/rfc822, which can be sent to one or more
mailboxes with SMTP (or whatever, but IIRC 2822upd only
mentions SMTP, not UUCP / LMTP / ...) when it's ready.=20

Other URI schemes and a MIME access-method deal with a
"mailbox" as a "resource".  Maybe I'm confused, or maybe
it would help if you say "one or more".

| The operation of how any URI scheme is resolved is not
| mandated by the URI specifications.

Depends, the nntp: URL scheme is designed for NNTP, it
would be slightly more complex if it were designed for
article numbers on "non-NNTP" news servers (example).
The NNTP details are not mandated, but the design fits.
JFTR, I guess we agree.

You have a good In-Reply-To example in 7.1, please add
this hname to the "save and useful" list in section 4.

Please remove body=3D from "save and useful", it's neither
save nor useful, it's a bad idea to start with.  While
keywords=3D are save they are rarely used.  IMO cc=3D is a=20
more realistic candidate for "save and useful".  Other
bad ideas (in addition to body=3D, bcc=3D, and to=3D) are=20
date=3D or message-id=3D for obvious reasons. =20

Maybe enumerate all potentially "save and useful" header
fields:  subject=3D, cc=3D, in-reply-to=3D, keywords=3D, is that
really all ?

| When producing mailto: URIs, all spaces SHOULD be=20
| encoded as %20.

The given reasons are compelling, justifying MUST.  The
"+" or "_" hacks are for Google or Wikipedia.  Not too
bad in subject=3D or keywords=3D, but not in addresses etc.

| The mailto URI scheme is limited in that it does not
| provide for substitution of variables.

That's not a specific mailto: limitation, you could say
"URI schemes are"...

Section 6 is where you could put all UTF-8 notes, maybe
rename it to "Internationalization considerations" as
proposed in RFC 2277.  As is section 6 is a lame excuse
for breaking existing software and annoying poor users.

7.1 + 7.2 are excellent. =20

| Applicable protocol:
|  None. This registration is made to assure that
|  this header field name is not used at all, in order
|  to not create any problems for mailto: URIs.

It suffices to reserve it for "mail", I don't see how it
could affect "http" or "news".  SIP has apparently its
own way to register header fields =3D> not your problem.

 Frank

Date: Sat, 19 Jan 2008 14:28:20 -0600
To: SM <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked]
Sender: [email-masked]

On 1/19/08 at 11:22 AM -0800, SM wrote:

>At 10:23 19-01-2008, Pete Resnick wrote:
>>Note: The transmitter information is always present. The absence of 
>>the "Sender:" field is sometimes mistakenly taken to mean that the 
>>agent responsible for transmission of the message has not been 
>>specified. The absence of the "Sender:" field merely means that the 
>>transmitter is identical to the author and is therefore not 
>>redundantly placed into a separate field.
>
>I prefer the above text to what is in Section 3.6.2 of the draft.
>
>The current draft (draft-resnick-2822upd-04) has:
>
>   "If the originator of the message can be indicated
>    by a single mailbox and the author and transmitter are identical, the
>    "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
>    appear."

That text will still appear in the next draft. The above "Note" is in 
addition to that text.

>I viewed RFC 2822 as saying that we "must" put in a Sender if it's 
>different from the author.

Well, sort of. It's saying, "having a Sender field means that the 
author is different from the transmitter".

>If the author and the sender are the same, we "must not" put in the 
>Sender header as it would be redundant.
>
>On reading of the current draft, one may interpret that "SHOULD NOT" 
>as saying that it is recommended not to put in a Sender field unless 
>the circumstances dictate it.  To avoid "the circumstances" from 
>being stretched, we can either use a "MUST NOT" or else use your 
>note.

No. That's not how RFC 2119 MUSTs and SHOULDs are used. MUST NOT 
means that doing so is known to cause harm or interfere with 
interoperation. That simply isn't the case here. We can imagine cases 
where having redundant Sender information might cause confusion, but 
we say SHOULD NOT because there may exist valid reasons where the 
behavior is acceptable.

>I favor your note as it's a clarification of the issue that was 
>sought.  Here's a slightly modified version:
>
>Note: The transmitter information is always present. The absence of 
>the "Sender:" field is sometimes mistakenly taken to mean that the 
>agent responsible for transmission of the message has not been 
>specified.  This absence merely means that the transmitter is 
>identical to the author and is therefore not redundantly placed into 
>the "Sender:" field.

I'm fine with that as a bit clearer. But that's in addition to the 
normative information still in 3.6.2.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sat, 19 Jan 2008 11:22:49 -0800
To: Pete Resnick <[email-masked]>
From: SM <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked]
Sender: [email-masked]

At 10:23 19-01-2008, Pete Resnick wrote:
>Note: The transmitter information is always present. The absence of 
>the "Sender:" field is sometimes mistakenly taken to mean that the 
>agent responsible for transmission of the message has not been 
>specified. The absence of the "Sender:" field merely means that the 
>transmitter is identical to the author and is therefore not 
>redundantly placed into a separate field.

I prefer the above text to what is in Section 3.6.2 of the draft.

The current draft (draft-resnick-2822upd-04) has:

   "If the originator of the message can be indicated
    by a single mailbox and the author and transmitter are identical, the
    "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
    appear."

I viewed RFC 2822 as saying that we "must" put in a Sender if it's 
different from the author.  If the author and the sender are the 
same, we "must not" put in the Sender header as it would be redundant.

On reading of the current draft, one may interpret that "SHOULD NOT" 
as saying that it is recommended not to put in a Sender field unless 
the circumstances dictate it.  To avoid "the circumstances" from 
being stretched, we can either use a "MUST NOT" or else use your 
note.  I favor your note as it's a clarification of the issue that 
was sought.  Here's a slightly modified version:

Note: The transmitter information is always present. The absence of 
the "Sender:" field is sometimes mistakenly taken to mean that the 
agent responsible for transmission of the message has not been 
specified.  This absence merely means that the transmitter is 
identical to the author and is therefore not redundantly placed into 
the "Sender:" field.

Regards,
-sm 

To: Pete Resnick <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
From: "Nick Shelness" <[email-masked]>
Date: Sat, 19 Jan 2008 18:37:15 +0000
Sender: [email-masked]

Pete Resnick <[email-masked]> wrote on 01/19/2008 18:23:29:

> ...
> Note: The transmitter information is always present. The absence of 
> the "Sender:" field is sometimes mistakenly taken to mean that the 
> agent responsible for transmission of the message has not been 
> specified. The absence of the "Sender:" field merely means that the 
> transmitter is identical to the author and is therefore not 
> redundantly placed into a separate field.
> ... 

Works for me.

Nick

Date: Sat, 19 Jan 2008 12:23:29 -0600
To: "Nick Shelness" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked]
Sender: [email-masked]

On 1/19/08 at 12:54 PM +0000, Nick Shelness wrote:

>I'm not sure that this works for me. Could I suggest:
>
>... to mean that the agent responsible for transmission of the
>message is specified implicitly rather than explicitly.

No, that's wrong. That reverses the meaning of the paragraph. I 
should have pasted the whole paragraph in, as I think I might have 
confused things by showing only that line. Here's the new wording of 
the entire note:

Note: The transmitter information is always present. The absence of 
the "Sender:" field is sometimes mistakenly taken to mean that the 
agent responsible for transmission of the message has not been 
specified. The absence of the "Sender:" field merely means that the 
transmitter is identical to the author and is therefore not 
redundantly placed into a separate field.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: "Frank Ellermann" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
From: "Nick Shelness" <[email-masked]>
Date: Sat, 19 Jan 2008 12:54:25 +0000
Sender: [email-masked]

Frank,

You wrote on 01/18/2008 12:59:47:

> ...
> For a DS it has to be implemented and needs to be interoperable in
> some way, and "breaks in NetNews" would be suspicious, UAs for mail
> and news likely support only one approach where possible.
> ...

I realize that the drums effort began so long ago that it is easy to 
forget its objective.

To paraphrase, it was not to CHANGE existing Full Standards (822, 1123, 
etc) and other RFCs that it was expected would reach Full Standard status 
(even though they haven't yet), but to produce a small set of documents 
that superseded, and where necessary (and where possible) clarified, them 
all. The working group reached early consensus that it could not outlaw 
constructs collectively allowed by these earlier standards even though the 
group would deprecate their use. Pete's (if memory serves me correctly, it 
was his invention) obs- non-obs approach was then accepted as the best 
vehicle to achieve this. So the only remaining issue viz the obs- syntax 
is simply whether it accurately captures the syntax explicitly allowed by 
these earlier standards and not whether these obs- constructs are good, 
bad or indifferent.

Nick

To: Pete Resnick <[email-masked]>
Cc: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
From: "Nick Shelness" <[email-masked]>
Date: Sat, 19 Jan 2008 12:54:24 +0000
Sender: [email-masked]

Pete,

you wrote on 01/19/2008 01:27:46:

> 
> On 1/19/08 at 1:25 AM +0100, Mark Martinec wrote:
> 
> >  > ... to mean that the agent responsible for transmission of the
> >  > message has not been specified explicitly.
> >               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> >After all, it *is* true that this information has not been specified 
> >*explicitly*, it has been specified implicitly.
> 
> Ah, right. Strike the word "explicitly". Will do. Thanks.

I'm not sure that this works for me. Could I suggest:

... to mean that the agent responsible for transmission of the
message is specified implicitly rather than explicitly.

Nick

Date: Fri, 18 Jan 2008 19:27:46 -0600
To: Mark Martinec <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked]
Sender: [email-masked]

On 1/19/08 at 1:25 AM +0100, Mark Martinec wrote:

>  > ... to mean that the agent responsible for transmission of the
>  > message has not been specified explicitly.
>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>After all, it *is* true that this information has not been specified 
>*explicitly*, it has been specified implicitly.

Ah, right. Strike the word "explicitly". Will do. Thanks.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

From: Mark Martinec <[email-masked]>
Organization: Jozef Stefan Institute
To: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Date: Sat, 19 Jan 2008 01:25:36 +0100
Sender: [email-masked]

On Tuesday 15 January 2008 22:33:44 Pete Resnick wrote:
> How about this (to better conform to the current language):
>
> Note: The transmitter information is always present. The absence of
> the "Sender:" field is sometimes mistakenly taken to mean that the
> agent responsible for transmission of the message has not been
> specified explicitly. The absence of the "Sender:" field merely means
> that the transmitter is identical to the author, so it is not
> redundantly placed into a separate field.
>
> Does that convey the correct information?

Well, yes, except for the wording in:

> ... to mean that the agent responsible for transmission of the
> message has not been specified explicitly.
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

After all, it *is* true that this information has not been
specified *explicitly*, it has been specified implicitly.

  Mark

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: New 2822upd-04 - obs-NO-WS-CTL
Date:  Fri, 18 Jan 2008 13:59:47 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 150
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

> See my response to Charles regarding obs- items in general. This
> was (syntactically) allowed in previous versions.

For a DS it has to be implemented and needs to be interoperable in
some way, and "breaks in NetNews" would be suspicious, UAs for mail
and news likely support only one approach where possible.

> I don't see how continuing to give fair warning in the "you might
> have to interpret this" section is problematic.

UAs have to interpret any stream of random octets without crashing,
you could mention this in the prose if you think it helps.  That's
not the same as "MUST accept NUL" (etc.) in odd places, where the
UA might need to talk with other software (e.g. address books).

It would be also confusing if we end up with an 2822upd Message-ID,
where the LHS is a proper subset of NetNews, and the RHS a superset.

Ideally you arrive at something that can be copied "as is" to the
NetNews RFC replacing what it says now in AUTH48.  For that it has
to get rid of NO-WS-CTL (done, "obs" is by definition not allowed
in NetNews) and <quoted-pair> (not done, <dcontent> allows it).

Alternatively you could limit <quoted-pair> in the RHS to three
"necessary" cases (necessary for the hypothetical domain literals
"needing" \[, \\, or \] for some reason).  If you do nothing it's
not good enough for the canonical Mesage-IDs needed in NetNews:

The Message-ID <local@[who\knows]> with \k is bad for a simple
comparison with <local@[whoknows]>.  Don't ask me what a domain
literal [whoknows] is, you want it, IMO adopting the <IP-literal>
syntax from STD 66 (adding IPv4 of course) would be better.

If folks insist on using random garbage as RHS they don't need
to put it in square brackets to make it more interesting.  For
some peculiar effects see the news: URL draft (that's also one
reason why I favour to adopt STD 66 syntax for domain literals).

> What's the gain in ditching it?

Simpler parsers.  MIME boundaries and RFC 2231 are another dark
corner, where syntax going to the limits can result in "let's
forget it and find a good heuristics".  One of my scripts does
this, bad... :-(

>>* 3.6.4
[...]
> How about, "Though listed as optional in the table in section
> 3.6..."?

+1  Looking for the table in the wrong direction I stumbled over
the IANA consideratins, I think you could trim the header field
registrations:  IANA only has to update the existing pointers to
2822upd.  Mentioning the status and the change controller once
in 2822upd should do.  It's no exercise in bureaucracy, it's for
header field registry users looking for a relevant specification.

An exception is the obsolete Resent-Reply-To, IANA has to flag it
as obsolete.  IANA set the initial pointers *to* RFC 4021 instead
of the specifications and status listed *in* RFC 4021 ;-)

Back to your table, it doesn't say "optional", it says "0*" with
a note "SHOULD be present - see 3.6.4".  The lower case optional
makes me nervous, it could be confused with a 2119 OPTIONAL.

 [In-Reply-To]
> The conservative move seems to me to keep the language as-is.

Okay, let's say that a news2mail gateway is a good excuse to
ignore this SHOULD, trying to derive In-Reply-To from References
would be a bad idea.

> How about, "later in this section, the use of a domain
> identifier is RECOMMENDED...."?

+1

> Avoid the issue. I don't want to overly obfuscate the point
> here.

Somewhere you have to say that domain literals are generally a
bad idea, I don't find this statement at the moment in 2822upd.
It's so fundamental that a verbatim copy from 822 would be fun:

     THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.

>> But you got rid of obs-NO-WS-CTL in <dcontent>, good.

> Nope. <dcontent> has <dtext> which has <obs-dtext> which is=20
> obs-NO-WS-CTL.

Kill it, take no "obs" prisoners.  "Strongly discouraged" plus
"obs" plus "never used" is a clear kill.  If the interop report
says that NO-WS-CTL in domain literals is precisely what e-mail
needs in 2008 you can simply reinsert it, but I guess it won't. =20

The <dcontent> also affects 2821bis.  The last state I know is
to introduce a special <SMTP-dcontent> without (obs-) NO-WS-CTL
for SMTP's variant of "IPvFuture" - of course different from
the "IPvFuture" in STD 66, confusing implementors... <sigh />

> I like everything in section 4 to start with obs-. A stylistic
> thing.

Squeezing obs-NO-WS-CTL into 69 columns for the ABNF is tricky,
my own style requires all "=3D" in the same column.  "Beautiful"
ABNF tends to convince me that it can't be completely wrong ;-)

Just explaining why I proposed to drop the "obs" in this case,
if you don't run into length issues or just use other stylistic
priorities it's of course okay.

> (*Shrug*) Doesn't seem all that informative to me since
> quoted-pair, which is the only place that obs-qp appears,
> already has HTAB.

Yes, I thought that the <obs-qp> syntax isn't obvious for the
purpose to enumerate CTL minus HTAB, no important point, just
what I saw top down.

>> <VCHAR> in <obs-utext> is wrong, it's covered by=20
>> <unstructured>.
=20
> No, VCHAR is required there. obs-utext is not a alternative
> for utext. There is no utext.

Right, I first wrote utext, found that it's gone, and replaced
it by unstructured.  Missing that the "bare-magic" must also
work for VCHAR.

> If that makes it clearer, I could do that.

No, future readers won't know what <utext> used to be, the
<obs-utext> syntax is clear as is, my bad.

>> I think you have lost NO-WS-CTL in <obs-body>, I fear
>> you need it, as soon as you have NUL anything goes :-(
=20
> text already contains everything from obs-NO-WS-CTL.

%d6 :-)

> How about, "Semantically, none of the optional CFWS in
> the local-part and the domain..."?

+1

 Frank

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: New 2822upd-04 - obs-NO-WS-CTL
Date:  Fri, 18 Jan 2008 08:10:58 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 58
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

> OK, both you and Ned have stated that bare CR, LF, and NUL have indeed
> been seen in Real World messages in the past, and that is a sufficient
> answer to my point.

As it happens I think it answers nothing.  Of course "invalid" messages
exist, otherwise a one-liner message =3D *CHAR would suffice to permit
anything. =20

NUL crashed my good old Netscape 3 MUA, that would justify a bug report
(over a decade ago), it can exit with an error message, but it should
not crash.  Do we really need to add a NUL in "obs" syntax everywhere
only to convince implementors that messages can be worse than they
expect ? =20

The "obs" chapter boils down to "MUST accept".  I don't see why anybody
would wish, let alone must, accept NUL in Internet messages.  The same
goes for NO-WS-CTL uses in domain literals.  I understand "MUST accept"
if it was used somewhere, but I think it's now cruft left over from
RFC 822, when new exotic networks (Maus, Z, etc.) were still created:

Nobody ever used NO-WS-CTL in domain literals, and that won't change.

> we should be anticipating that some future standard (probably well
> after 2010) may well decide that the time has come to ditch it.

IMO the job *now* is to get rid of as much cruft as possible, shifting
it to "obs" is not the same as "get rid of".  They could reintroduce
it in the future if they miss it.   AFAIK the 2822 "obs"-chapter has
to pass in an "implementation and interoperability report".  If some
2822-obs-features are not only obsolete, but not implemented and maybe
causing havoc they have to be pulled *now*.

Or maybe I'm wrong, the "2821bis to DS" cabale was obviously annoyed
when I mentioned RFC 2026 and the "interop" rules.  It's still beyond
me how a "good PS" could be not always better than a "less good DS".

> And <msg-id>s are hardly of use in the email environment, except
> for threading using References, whereas they are fundamental to
> the operation of Netnews.

Now you're getting carried away, a <msg-id> is also useful in email,
for threading and mail2news as you said, also for some applications
of Archived-At (RFC 5064) or similar archiving purposes.  And my old
MUA had other local uses related to forwarding by mailbox URL, IIRC.

Another "feature" of <msg-id> is that nobody needs to reinvent it.

Wrt to the "magic SP", it's a known USEFOR rathole, and as far as=20
I'm concerned not important enough to delay 2822upd, which could
end up in incompatibilities with 2821bis if the IESG approves it
before 2822upd is ready.  After all they Last Called 2821bis, IMO
dangerously near to SNAFU for oddities like an "IPvFuture literal"
still allowing (in ABNF) 2822 NO-WS-CTL unless 2822upd fixes this,
and 2821bis fixes the normative reference (note a lot of IFs here).

 Frank

Date: Thu, 17 Jan 2008 23:32:54 -0600
To: "Charles Lindsey" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: quoted-pair in dcontent / Space after colon (Was: New 2822upd-04 - obs-NO-WS-CTL)
Cc: [email-masked]
Sender: [email-masked]

I'm separating out these two issues, as I want to hear yeas and nays 
(and explanations) from people other than Charles and Frank.

On 1/17/08 at 1:48 PM +0000, Charles Lindsey wrote:

[On quoted-pair in dcontent...]
>
>...nothing useful is served by retaining those <quoted-pairs>, 
>certainly not in <msg-id>s. And <msg-id>s are hardly of use in the 
>email environment, except for threading using References, whereas 
>they are fundamental to the operation of Netnews.
>
>And Frank, as well as myself, has consistently supported their 
>removal (and has just repeated his support in this thread).

So, there are two forms to this request:

1. Remove quoted-pair entirely from dcontent in the generate syntax 
(meaning "[", "]", and "\" may never appear in dcontent).
2. Only allow the quoted-pairs "\[", "\]", and "\\" in dcontent in 
the generate syntax.

Of course, quoted-pair would be in the obs- (interpret) syntax.

My personal opinion: #2 makes me queasy and I will argue tooth and 
nail against it. If people are heavily in favor of #1, I won't raise 
a stink, but I don't see the point: If you're writing a gateway from 
e-mail to Netnews, you're going to have to go through the messages 
looking for things you don't like anyway, so why not clean up these 
along with everything else?

In any event, doing #1 is an easy edit. I just want to know what 
people want to do.

[On space after the ":"...]

[Charles originally wanted:]
>>>Although in all header fields the CFWS following the colon is 
>>>optional, it is customary to place at least one SP there; moreover 
>>>such a SP is mandatory in some related protocols (notably 
>>>[NETNEWS]). In order to facilitate interoperability with such 
>>>related protocols, that SP SHOULD normally be present.

[Frank suggested:]
>>Although all message header fields allow either [FWS] or [CFWS] 
>>after the colon, it is customary to place at least one white space 
>>character after the colon.  In NetNews articles [RFC xxxx] it's 
>>mandatory to use a space character after the colon, and in order to 
>>faciliate interoperability implementors might wish to separate the 
>>header field name plus colon from the header field body by a space 
>>in all Internet messages.
>
>But if we can agree that a wording of some form along the lines of 
>what I and Frank are suggesting, then the exact wording can be 
>worked on by further discussion. As I said, either wording is netter 
>than nothing.

My personal feeling on this is again it is not necessary, and that a 
gateway should be dealing with this anyway. But if people want text, 
I would be OK with a non-normative note in either 2.2 or 3.6. 
(Anything with a SHOULD I will make a stink over.) As editor, I'd 
want specific text agreed to and instructions on where to put it.

Let's put these issues to bed.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Thu, 17 Jan 2008 11:13:39 -0800
To: Charles Lindsey <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Sender: [email-masked]

At 04:37 16-01-2008, Charles Lindsey wrote:
>So, a Question: Were NUL and naked CR and LF ever used in significant
>quantity in real emails (as opposed to being theoretically allowed by RFC
>822 and its predecessors)?

It's not really the quantity that matters.  I've seen them in emails 
coming from a specific  region.  There were a lot of legacy MUAs in 
use in that region.  It's not a problem as long as the application 
handles it gracefully.

Regards,
-sm 

From: [email-masked]
Cc: [email-masked]
Date: Thu, 17 Jan 2008 10:33:09 -0800 (PST)
Subject: Re: rfc2822bis -- adding clarifying Sender: text
To: Charles Lindsey <[email-masked]>
Sender: [email-masked]

> In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

> >Charles Lindsey wrote:
> >
> >> there are situations where an existing Sender currently
> >> IS modified, that being regarded as good practice.

> >The precise number of RFCs discussing to *modify* an
> >existing Sender header field I'm aware of is zero, what
> >do you have in mind ?

> That is because there is currently no RFC covering best practices for
> mailing list expansions (that is a gap that ought to be filled).

Ought to be but is unlikely to happen. This was tried some years back, with
the result that the working group imploded. I've seen nothing to indicate
a general shift of attitudes that would prevent this from playing out the
same way if it were tried again.

> But nevertheless, the practice of forcing the address of the mailing list
> expander into the Sender header (whether or not such a header was present
> before hand) is so widespread that any RFC that might appear in future
> would have a hard time it is tried to reverse that convention.

Actually, my observation has been that fashions in this area seem to shift back
and forth over time. RIght now I'm seeing an upswing in the number of lists I'm
on that do this, but this behavior is still a minority taste among the lists I
subscribe to. Before this current upswing there was a period where I observed a
slow drop in the number of lists that do it.

I suspect this is more a result in changed to what list handlers are popular
and their default behavior than it is a shift in attitudes of list managers.
(My guess is most people running lists neither know nor care about this issue
to any significant degree and just accept whatever defaults they are given.)
And while I'm on a lot of lists I cannot claim to have anything resembling
definitive data. Indeed, it hard to even know how to tally this behavior - do
we count lists, list operators (Yahoo Groups hosts a lot of lists), list
managers, or something else?

> De Facto, that is the accepted usage, and we have to live with that.

No, what we have to live with is the fact that some lists replace the Sender:
field and others do not. An expectation that it will always happen is every bit
as wrongminded as an expecation that it will never happen.

				Ned

To: [email-masked]
Xref: clerew local.mime:5170
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Date: Thu, 17 Jan 2008 12:22:06 GMT
Lines: 31
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Charles Lindsey wrote:
> 
>> there are situations where an existing Sender currently
>> IS modified, that being regarded as good practice.

>The precise number of RFCs discussing to *modify* an
>existing Sender header field I'm aware of is zero, what
>do you have in mind ?

That is because there is currently no RFC covering best practices for
mailing list expansions (that is a gap that ought to be filled).

But nevertheless, the practice of forcing the address of the mailing list
expander into the Sender header (whether or not such a header was present
before hand) is so widespread that any RFC that might appear in future
would have a hard time it is tried to reverse that convention.

De Facto, that is the accepted usage, and we have to live with that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:5171
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Date: Thu, 17 Jan 2008 13:48:36 GMT
Lines: 133
Sender: [email-masked]

In <p0625010ec3b3f0926679@[74.134.5.163]> Pete Resnick <[email-masked]> writes:

>On 1/16/08 at 12:37 PM +0000, Charles Lindsey wrote:

>>This is a good time to ask how long it is intended that the 
>>obs-syntax should remain in the standard. One hopes that it will no 
>>longer be a REQUIREMENT to recognize it in 1000 years time, but 
>>maybe some earlier deadline might be in order.

>A few things on this point (and I believe we've had this discussion 
>multiple times before):

>1. What would an implementation do, in the face of a bare CR, LF, or 
>NUL, if the specification did not have this stuff in the obs- syntax? 
>As it currently says:

>       Note: This section identifies syntactic forms that any
>       implementation MUST reasonably interpret.  However, there are
>       certainly Internet messages which do not conform to even the
>       additional syntax given in this section.  The fact that a
>       particular form does not appear in any section of this document is
>       not justification for computer programs to crash or for malformed
>       data to be irretrievably lost by any implementation.  It is up to
>       the implementation to deal with messages robustly.

>The reason that bare CR, LF, and NUL are in there is because these 
>things have cropped up in messages over the years and implementations 
>may need to deal with them. It's fair warning to have these 
>constructs in the spec.

OK, both you and Ned have stated that bare CR, LF, and NUL have indeed
been seen in Real World messages in the past, and that is a sufficient
answer to my point. And Ned claims that recognizing some of the more
obscure forms of addr-spec is more troublesome that dealing with CR, LF
and NUL (though I suspect that just accepting and displaying anything
printable with an '@' in the middle would suffice for software whose only
purpose when faced with obs-stuff was to display it legibly - i.e. nobody
expects replies to such obs-addresses to work).

>2. I find the whole concept of deadlines in protocol specifications 
>silly. If we say, "you MUST accept bare LF until January 1, 2010 at 
>12:00:00 UTC, at which point you MAY", what have we accomplished? If, 
>by any particular date, we find that it is unreasonable to ask 
>implementations to do any particular thing, we change the spec at 
>that time.

But I suggested no such thing. There is a lot of space between forbidding
it after January 1, 2010 and expecting it still to be working after 1000
years. I.e. we should be anticipating that some future standard (probably
well after 2010) may well decide that the time has come to ditch it.

And, actually, what I would like to see at the moment (and I think this
would be safe as of now) is that one could assume that obs-syntax would
not be seen coming off the wire, and hence that recognizing it should only
be required of MUAs for the purposes of displaying legacy messages, and
hence there should be no requirement for current MTAs to accept it.

>>A. I see that <quoted-pair> is still present in <dcontent>...
>>
>>B. As regards the use of SP after the ':' in header fields...

>I did not see support on the list for either of your positions on 
>these.

That is because there are only three (maybe four) people on this list who
are looking at this draft from the interoperability with Netnews POV.

As regards A, nothing useful is served by retaining those <quoted-pairs>,
certainly not in <msg-id>s. And <msg-id>s are hardly of use in the email
environment, except for threading using References, whereas they are
fundamental to the operation of Netnews.

And Frank, as well as myself, has consistently supported their removal
(and has just repeated his support in this thread).

> ... In fact, the only response to your Sept. 20 message on B was 
>an objection by Frank to doing what you said. Cite some support for 
>either of these positions if you would.

No, Frank agreed with what I wanted to do, but suggested an alternative
wording. Here is the full exchange, including my reply to Frank:

>>| Although in all header fields the CFWS following the colon is
>>| optional, it is customary to place at least one SP there; moreover
>>| such a SP is mandatory in some related protocols (notably [NETNEWS]).
>>| In order to facilitate interoperability with such related protocols,
>>| that SP SHOULD normally be present.

>I'd put it slightly different, AFAIK Netnews is the only case where
>that's necessary, and structured header fields defined outside of
>2822upd tend to use [FWS] instead of [CFWS] after the colon.  And
>unstructured header fields like Subject: also use [FWS].  What I get is:

>| Although all message header fields allow either [FWS] or [CFWS]
>| after the colon, it is customary to place at least one white space
>| character after the colon.  In NetNews articles [RFC xxxx] it's
>| mandatory to use a space character after the colon, and in order
>| to faciliate interoperability implementors might wish to separate
>| the header field name plus colon from the header field body by a
>| space in all Internet messages.

I don't think there is any serious difference between our two wordings
(except that I would still prefer "SHOULD" to "might"). As to other
protocols than Netnews that might be affected, I don't know of any, but
thought it might be useful to allow for the possibility.

There are two diffenences in Frank's text:

i) "might wish to" vs "SHOULD". Breaking a "SHOULD" is not a crime, but
should only be done with full knowledge of the consequences (e.g. in a
special-purpose script where it is clear that Netnews will never become
involved). But if people still want something weaker than "SHOULD", then I
would accept that as better than nothing.

ii) Although Netnews is the only case we know of which would be affected,
I wanted to allow for the possibility there might be others. Hence my
wording "some related protocols (notably [NETNEWS])".

But if we can agree that a wording of some form along the lines of what I
and Frank are suggesting, then the exact wording can be worked on by
further discussion. As I said, either wording is netter than nothing.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Wed, 16 Jan 2008 19:09:26 -0600
To: "Frank Ellermann" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Cc: [email-masked], "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

On 1/17/08 at 12:09 AM +0100, Frank Ellermann wrote:

>* 3.2.2
>   <ctext>

Yup, Charles mentioned. Fixed.

>* 3.4.1
>   <quoted-pair> in <dcontent>

I'll let this discussion play out.

>   <obs-dtext> is harmful, there are no domain literals with
>   NO-WS-CTL, please kill this obscenity or show me where it
>   was needed.

See my response to Charles regarding obs- items in general. This was 
(syntactically) allowed in previous versions. I don't see how 
continuing to give fair warning in the "you might have to interpret 
this" section is problematic. And you ought not be writing code that 
simply crashes if you encounter one of these. What's the gain in 
ditching it?

>* 3.5, 3.6.2, 3.6.3, 3.6.5
>   Missing blank lines between prose and syntax.

Yes. I mentioned that in my initial message. Will fix before release.

>* 3.6.4
>   "Though optional, every message SHOULD" [...].  Please
>   remove "though optional", an unqualified SHOULD already
>   takes care of obsolete implementations.

I wasn't refering to anything obsolete here. How about, "Though 
listed as optional in the table in section 3.6..."?

>   Why are In-Reply-To and References simultaneously
>   recommended (SHOULD), how about limiting the In-Reply-To
>   SHOULD to messages without References ?  Nothing's wrong
>   if a message has only References, or is it ?

(Scraping the bottom of my memory...) I believe the point was that 
(a) some implementations might depend on In-Reply-To and (b) 
In-Reply-To was necessary if a reply was to multiple messages. 
Personally, I only know one independent implementation of (b), and 
I'm not sure about (a). The conservative move seems to me to keep the 
language as-is.

>| later in this section, the use of a domain name or
>| literal Internet address is RECOMMENDED for the [RHS]
>
>   "Recommending" a domain literal

How about, "later in this section, the use of a domain identifier is 
RECOMMENDED...."? Avoid the issue. I don't want to overly obfuscate 
the point here.

>   For <no-fold-literal> see above, no <quoted-pair>, please.

Again, I'll let that discussion play out.

>   <obs-id-right> points back to <domain>, that forward to
>   <obs-domain>, back to dot-separated <atom>s with <CFWS>,
>   shudder.

Life's tough all over.

>   But you got rid of obs-NO-WS-CTL in <dcontent>,
>   good.

Nope. <dcontent> has <dtext> which has <obs-dtext> which is obs-NO-WS-CTL.

>Maybe move <obs-id-right> from 4.5.4 to 4.4, ditto
>   <obs-id-left>, historically that's where they belong to.

I like it where it is.

>* 3.6.8 (matter of taste)
>   You could now write 'VCHAR excl. ":"' in the <ftext>
>   comment, or 'printable US-ASCII excluding ":"' similar
>   to what you have for <qtext>.  It's not more necessary
>   to mention that <ftext> can't contain SP and control.

Will do.

>* 4.1
>   <obs-NO-WS-CTL> also doesn't include NUL, fortunately.
>   How about dropping the obs- prefix from obs-NO-WS-CTL ?
>   It's anyway not used outside of the the obs-chapter 4.

I like everything in section 4 to start with obs-. A stylistic thing.

>   Oops, 4234bis "forgot" to define NUL forcing you to say
>   %d0.  But 4234bis mentions NUL in a comment.  Maybe NUL
>   could be still added in AUTH48 to 4234bis (?)

That's up to Dave and Paul. It makes little difference to me, but 
I'll use it if it's there.

>   <obs-qp> boils down to "CTL minus HTAB", maybe note it
>   as ABNF comment.  The full syntax is necessarily clumsy.

(*Shrug*) Doesn't seem all that informative to me since quoted-pair, 
which is the only place that obs-qp appears, already has HTAB.

>   You got rid of <obs-text>, now that surprises me.  It
>   was used in <text>, indirectly <quoted-pair> (killed),
>   indirectly <body> (???), and <obs-utext>.

Since body is now the only place text is used, and we've got to have 
an obs-body, there was no longer a need for obs-text.

>   The bare CR or bare LF magic went from <obs-utext> to
>   <obs-unstruct>, that should be okay.  But <VCHAR> in
>   <obs-utext> is wrong, it's covered by <unstructured>.

No, VCHAR is required there. obs-utext is not a alternative for 
utext. There is no utext. obs-unstruct is an alternative for 
unstructured, and in that alternative, a string of bare LFs and or 
bare CRs can be followed by a VCHAR. I could make obs-unstruct:

obs-unstruct    =   *((*LF *CR *((%d0 / obs-NO-WS-CTL / VHAR) *LF *CR)) / FWS)

and ditch obs-utext entirely, but then I'd have to break obs-unstruct 
over two lines. If that makes it clearer, I could do that.

>   I think you have lost NO-WS-CTL in <obs-body>, I fear
>   you need it, as soon as you have NUL anything goes :-(

text already contains everything from obs-NO-WS-CTL.

>4.5.4
>| Semantically, none of the optional CFWS surrounding the
>| local-part and the domain are part of the obs-id-left
>| and obs-id-right respectively.
>
>   Ditto CFWS within <obs-domain>, see comment for 3.6.4.

How about, "Semantically, none of the optional CFWS in the local-part 
and the domain..."?

Thanks for the comments.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: New 2822upd-04 - obs-NO-WS-CTL
Date:  Thu, 17 Jan 2008 00:09:44 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 133
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

> Try this instead:

Thanks.  As soon as an I-D is available on the Tools server
the "diff" links in the HTML output can compare N+1 and N.
=20
> If you want the comparison to 2822 itself, try this:

For now I consider -03 as "known good", and try to figure=20
out if -04 is "better".  Points as I see them (top down):

* 3.2.2
  <ctext> is supposed to be VCHAR minus "(", ")", and "\",
  the syntax says decimal 33..39 and 93..126.  Please add
  41..91 again.

* 3.4.1
  <quoted-pair> in <dcontent>, compare Charles' and Ned's
  comments.  As far as this allows for \[, \\, and \] in a
  <domain-literal> I'm not aware of any case (incl. MIXER)
  where this is needed.  If this allows any <quoted-pair>,
  because that's as it was in RFC 2822 I'm curious what an
  "interop" report will say (good, bad, or ugly).  Outside
  of a Message-ID I'm not very hot about it.

  <obs-dtext> is harmful, there are no domain literals with
  NO-WS-CTL, please kill this obscenity or show me where it
  was needed.=20

* 3.5, 3.6.2, 3.6.3, 3.6.5
  Missing blank lines between prose and syntax.  I never
  got that effect with xml2rfc, your figure-artwork-ABNF
  magic must be different from what I use.

* 3.6.4
  "Though optional, every message SHOULD" [...].  Please
  remove "though optional", an unqualified SHOULD already
  takes care of obsolete implementations. =20

  Why are In-Reply-To and References simultaneously=20
  recommended (SHOULD), how about limiting the In-Reply-To
  SHOULD to messages without References ?  Nothing's wrong
  if a message has only References, or is it ?

| later in this section, the use of a domain name or=20
| literal Internet address is RECOMMENDED for the [RHS]

  "Recommending" a domain literal is not what RFC 822 did:
     THE USE OF DOMAIN-LITERALS IS STRONGLY DISCOURAGED.
  At least for a Message-ID a domain literal can be a bad
  idea, IMO it's a last resort before trying random garbage.

  For <no-fold-literal> see above, no <quoted-pair>, please.

  <obs-id-right> points back to <domain>, that forward to
  <obs-domain>, back to dot-separated <atom>s with <CFWS>,
  shudder.  But you got rid of obs-NO-WS-CTL in <dcontent>,
  good.  Maybe move <obs-id-right> from 4.5.4 to 4.4, ditto
  <obs-id-left>, historically that's where they belong to.

[I ignore further missing empty lines between prose + ABNF]

* 3.6.8 (matter of taste)
  You could now write 'VCHAR excl. ":"' in the <ftext>
  comment, or 'printable US-ASCII excluding ":"' similar
  to what you have for <qtext>.  It's not more necessary
  to mention that <ftext> can't contain SP and control.

* 4.1
  <obs-NO-WS-CTL> also doesn't include NUL, fortunately.
  How about dropping the obs- prefix from obs-NO-WS-CTL ?
  It's anyway not used outside of the the obs-chapter 4.

  Oops, 4234bis "forgot" to define NUL forcing you to say
  %d0.  But 4234bis mentions NUL in a comment.  Maybe NUL
  could be still added in AUTH48 to 4234bis (?)
=20
  <obs-qp> boils down to "CTL minus HTAB", maybe note it
  as ABNF comment.  The full syntax is necessarily clumsy.

  You got rid of <obs-text>, now that surprises me.  It
  was used in <text>, indirectly <quoted-pair> (killed),
  indirectly <body> (???), and <obs-utext>.

  The bare CR or bare LF magic went from <obs-utext> to
  <obs-unstruct>, that should be okay.  But <VCHAR> in
  <obs-utext> is wrong, it's covered by <unstructured>.

  I think you have lost NO-WS-CTL in <obs-body>, I fear
  you need it, as soon as you have NUL anything goes :-(

4.5.4
| Semantically, none of the optional CFWS surrounding the
| local-part and the domain are part of the obs-id-left
| and obs-id-right respectively.

  Ditto CFWS within <obs-domain>, see comment for 3.6.4.

Thanks, great progress wrt NO-WS-TL.  And yes, I think
that the NetNews "magic SP" should be limited to NetNews.

 Frank

 =20
=20

  =20

 =20

 =20

 =20

 =20
=20
 =20

 =20

From: [email-masked]
Cc: Charles Lindsey <[email-masked]>, [email-masked]
Date: Wed, 16 Jan 2008 10:37:18 -0800 (PST)
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
To: Pete Resnick <[email-masked]>
Sender: [email-masked]

> On 1/16/08 at 12:37 PM +0000, Charles Lindsey wrote:

> >In <p06250100c3b2cdde7d7b@[74.134.5.163]> Pete Resnick
> ><[email-masked]> writes:
> >
> >A couple of typos:
> >
> >In 2.2 s/ unfolding"/ "unfolding"/
> >
> >In 3.2.2, %d42-91 appears to have got omitted from <ctext>.

> Good catch! Don't know how I missed those.

> > This is a good time to ask how long it is intended that the
> > obs-syntax should remain in the standard. One hopes that it will no
> > longer be a REQUIREMENT to recognize it in 1000 years time, but
> > maybe some earlier deadline might be in order.

> > How burdensome is it to have to recognize this ancient stuff? I
> > suspect that NUL and naked CR and LF are the bits that cause the
> > most problem - the rest is probably easier.

I cannot claim to have a definitive answer here, but our experience has been
the exact opposite. Way back when issues with this stuff first cropped up
cleaning up our handling of CR and LF was trivial. NULs were only slightly
harder.

Compare this with support for, say, stuff like:

    "a"."b"@c.d
    a.(b)[email-masked]

Dealing with this stuff is nasty - it makes parsers much more complex, and even
more important the "right thing" to do when stuff like this is received can be
tricky. A bunch of iterations were required before we were content we had most
of the cases right. (And don't get me started on the heuristics needed to deal
with the stuff that's not even legal according to obs-syntax but commonly used
nevertheless.) This is in sharp contrast to the situation with illegal
characters, where to be blunt the major source of pain has come from shifting
winds in the standards community. (I note in passing that I am not a fan of the
present state of affairs in regards to bare CR and LF in 2821bis, but that's
not especially relevant here.)

> > So one might like to
> > remove the NUL and naked CR and LF sooner rather than later. For
> > sure, there should no longer be a need to recognize ANY obs-stuff
> > coming off the wire, even today. We are concerned only with ancient
> > emails that are still in people's archives.

I'm sorry, but this goes much too far. There's a fair amount of stuff in the
obsolete syntax that I have never seen used other than in examples (e.g.,
multiple quoted strings in local parts). But there's also a fair amount of
obsolete stuff that does occur commonly in practice (e.g., legitimate messages
containing bare CRs and LFs). And there's also a significant middle ground of
stuff that doesn't come up very often (e.g., comments stuck in odd places) but
when it does it  always seems it came from the client the company CEO insists
on using so not supporting it is not an option.

> A few things on this point (and I believe we've had this discussion
> multiple times before):

> 1. What would an implementation do, in the face of a bare CR, LF, or
> NUL, if the specification did not have this stuff in the obs- syntax?
> As it currently says:

>        Note: This section identifies syntactic forms that any
>        implementation MUST reasonably interpret.  However, there are
>        certainly Internet messages which do not conform to even the
>        additional syntax given in this section.  The fact that a
>        particular form does not appear in any section of this document is
>        not justification for computer programs to crash or for malformed
>        data to be irretrievably lost by any implementation.  It is up to
>        the implementation to deal with messages robustly.

> The reason that bare CR, LF, and NUL are in there is because these
> things have cropped up in messages over the years and implementations
> may need to deal with them. It's fair warning to have these
> constructs in the spec.

Exactly.

> 2. I find the whole concept of deadlines in protocol specifications
> silly. If we say, "you MUST accept bare LF until January 1, 2010 at
> 12:00:00 UTC, at which point you MAY", what have we accomplished? If,
> by any particular date, we find that it is unreasonable to ask
> implementations to do any particular thing, we change the spec at
> that time.

> I strongly object to protocol specifications trying to impose
> deadlines on implementations. Write an Informational RFC with
> prognostications if so desired.

You know, I don't think I'd ever considered the notion of embedding a flag day
requirement in a specification before. Now that it has come up I have to agree
with Pete that it seems extraordinarily silly.

> >A. I see that <quoted-pair> is still present in <dcontent>...
> >
> >B. As regards the use of SP after the ':' in header fields...

> I did not see support on the list for either of your positions on
> these. In fact, the only response to your Sept. 20 message on B was
> an objection by Frank to doing what you said. Cite some support for
> either of these positions if you would.

In my 30-Jul-2007 message I was mildly in favor of A but strongly opposed to B.
Reiterating what I said at the time, I favor A on the grounds that it cleans
things up and seems "mostly harmless" and I oppose B because there's are a huge
number of marginal SMTP clients out there in stuff other than full-fledged
MUAs, including a lot of embedded gizmos that cannot be upgraded easily if at
all, and this is precisely the sort of shortcut you find all the time in that
sort of code.

However, in regards to A and irespective of my own position, I concur that
there was insufficient support shown on the list for making the change.

				Ned

Date: Wed, 16 Jan 2008 11:52:13 -0600
To: "Charles Lindsey" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Cc: [email-masked]
Sender: [email-masked]

On 1/16/08 at 12:37 PM +0000, Charles Lindsey wrote:

>In <p06250100c3b2cdde7d7b@[74.134.5.163]> Pete Resnick 
><[email-masked]> writes:
>
>A couple of typos:
>
>In 2.2 s/ unfolding"/ "unfolding"/
>
>In 3.2.2, %d42-91 appears to have got omitted from <ctext>.

Good catch! Don't know how I missed those.

>This is a good time to ask how long it is intended that the 
>obs-syntax should remain in the standard. One hopes that it will no 
>longer be a REQUIREMENT to recognize it in 1000 years time, but 
>maybe some earlier deadline might be in order.
>
>How burdensome is it to have to recognize this ancient stuff? I 
>suspect that NUL and naked CR and LF are the bits that cause the 
>most problem - the rest is probably easier. So one might like to 
>remove the NUL and naked CR and LF sooner rather than later. For 
>sure, there should no longer be a need to recognize ANY obs-stuff 
>coming off the wire, even today. We are concerned only with ancient 
>emails that are still in people's archives.

A few things on this point (and I believe we've had this discussion 
multiple times before):

1. What would an implementation do, in the face of a bare CR, LF, or 
NUL, if the specification did not have this stuff in the obs- syntax? 
As it currently says:

       Note: This section identifies syntactic forms that any
       implementation MUST reasonably interpret.  However, there are
       certainly Internet messages which do not conform to even the
       additional syntax given in this section.  The fact that a
       particular form does not appear in any section of this document is
       not justification for computer programs to crash or for malformed
       data to be irretrievably lost by any implementation.  It is up to
       the implementation to deal with messages robustly.

The reason that bare CR, LF, and NUL are in there is because these 
things have cropped up in messages over the years and implementations 
may need to deal with them. It's fair warning to have these 
constructs in the spec.

2. I find the whole concept of deadlines in protocol specifications 
silly. If we say, "you MUST accept bare LF until January 1, 2010 at 
12:00:00 UTC, at which point you MAY", what have we accomplished? If, 
by any particular date, we find that it is unreasonable to ask 
implementations to do any particular thing, we change the spec at 
that time.

I strongly object to protocol specifications trying to impose 
deadlines on implementations. Write an Informational RFC with 
prognostications if so desired.

>A. I see that <quoted-pair> is still present in <dcontent>...
>
>B. As regards the use of SP after the ':' in header fields...

I did not see support on the list for either of your positions on 
these. In fact, the only response to your Sept. 20 message on B was 
an objection by Frank to doing what you said. Cite some support for 
either of these positions if you would.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Wed, 16 Jan 2008 11:32:56 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Sender: [email-masked]

On 1/15/08 at 3:12 PM -0600, Pete Resnick wrote:

>For differences between -03 and -04, try this:
>
><http://tools.ietf.org/rfcdiff?url1=http://www.ietf.org/internet-drafts/draft-resnick-2822upd-03.txt&url2=http://www.ietf.org/internet-drafts/draft-resnick-2822upd-04.txt>

Well, looks like they deleted the old version from the IETF web site. 
Try this instead:

<http://tools.ietf.org/rfcdiff?url1=http://resnick1.qualcomm.com/draft-resnick-2822upd-03.txt&url2=http://resnick1.qualcomm.com/draft-resnick-2822upd-04.txt>

If you want the comparison to 2822 itself, try this:

<http://tools.ietf.org/rfcdiff?url1=http://resnick1.qualcomm.com/rfc2822-fixed.txt&url2=http://resnick1.qualcomm.com/draft-resnick-2822upd-04.txt>

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

To: [email-masked]
Xref: clerew local.mime:5163
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: New 2822upd-04 - obs-NO-WS-CTL
Date: Wed, 16 Jan 2008 12:37:23 GMT
Lines: 69
Sender: [email-masked]

In <p06250100c3b2cdde7d7b@[74.134.5.163]> Pete Resnick <[email-masked]> writes:

>Things to note:

>1. obs-qp, obs-body, and obs-unstruct each have NUL. obs-ctext, 
>obs-qtext, and obs-dtext don't. Is that what's desired? (I'm happy to 
>move NUL into all of them. I'm more worried about removing it from 
>all of them.)

This is a good time to ask how long it is intended that the obs-syntax
should remain in the standard. One hopes that it will no longer be a
REQUIREMENT to recognize it in 1000 years time, but maybe some earlier
deadline might be in order.

How burdensome is it to have to recognize this ancient stuff? I suspect
that NUL and naked CR and LF are the bits that cause the most problem -
the rest is probably easier. So one might like to remove the NUL and naked
CR and LF sooner rather than later. For sure, there should no longer be a
need to recognize ANY obs-stuff coming off the wire, even today. We are
concerned only with ancient emails that are still in people's archives.

So, a Question: Were NUL and naked CR and LF ever used in significant
quantity in real emails (as opposed to being theoretically allowed by RFC
822 and its predecessors)?

>2. text moved into the body discussion, and specials moved into the 
>atom discussion. Glad to move them elsewhere if you think necessary.

You appear to have done a good job of getting rid of NO-WS-CTL.

A couple of typos:

In 2.2 s/ unfolding"/ "unfolding"/

In 3.2.2, %d42-91 appears to have got omitted from <ctext>.

Which leaves just two items, wearing my NETNEWS/USEFOR hat.

A. I see that <quoted-pair> is still present in <dcontent>. I had hoped
that, with the more recent discussions about domain-literals that resulted
in the disappearance of NO-WS-CTL, similar arguments would have persuaded
you to remove that anomaly. Actually, removing <quoted-pair> from
<no-fold-literal> is all that is actually needed for NETNEWS compatibility
- removing it from <domain-literal> as well would just be tidier.

B. As regards the use of SP after the ':' in header fields, I proposed the
following wording (or something like it) on Sept 20th:

   Although in all header fields the CFWS following the colon is optional,
   it is customary to place at least one SP there; moreover such a SP is
   mandatory in some related protocols (notably [NETNEWS]). In order to
   facilitate interoperability with such related protocols, that SP
   SHOULD normally be present.

{[NETNEWS] would then be an informative reference to the current USEFOR
draft, which has already passed IESG review.}

I would still like to see text like that included somewhere.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: rfc2822bis -- adding clarifying Sender: text
Date:  Wed, 16 Jan 2008 15:26:45 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 51
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:
=20
> there are situations where an existing Sender currently
> IS modified, that being regarded as good practice.

The precise number of RFCs discussing to *modify* an
existing Sender header field I'm aware of is zero, what
do you have in mind ?

> The best example is where the message is being forwarded
> by a mailing list expander

2821 and 2821bis say MUST NOT, and if anything else is
regarded as "good practice" folks supporting this view
didn't show up before the end of the 2821bis Last Call.

This is rather critical for DKIM SSP if they drop their
odd "first author" magic in favour of a 2822upd sender.

Just because some mailing lists do whatever they like
doesn't mean that Internet Standards have to sanction
these practices.  Manipulating "in transit" mails could
be considered as illegal where I live. =20

> (or, equivalently, it is being gatewayed into Netnews)

IMO gateways are not equivalent, they are discussed in
a different chapter of 2821bis, IIRC also in mail-arch.

> Those cases are customarily justified on the grounds
> that it is a brand new transmission (even though the
> original Message-ID is retained).

If it's "brand new" then retaining the Message-ID *is*=20
broken.  Even Resent-* has its own Message-ID, and the
old "de facto" NetNews standard s-o-1036 (its successor
waiting for an RFC number is likely not yet well known)
offers clear advice why that's a bad idea:

| Above all, prevent loops

"Same Message-ID" means "same article" for NetNews. =20

I don't say that it cannot be done as you want it, but
to do it this way requires substantial modifications of
2821bis, 2822upd, mail-arch, maybe 4409, PRA, and more.

Until that happens wannabe "mailing lists" mutilating
valid mails are non-standard junk, not "good practice".

 Frank

To: [email-masked]
Xref: clerew local.mime:5162
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Date: Wed, 16 Jan 2008 11:29:31 GMT
Lines: 26
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>IMO "let's replace Sender" schemes are broken
>by design.  If you know a good place to note
>that an existing Sender MUST NOT be modified
>please do.

But there are situations where an existing Sender currently IS modified,
that being regarded as good practice.

The best example is where the message is being forwarded by a mailing list
expander (or, equivalently, it is being gatewayed into Netnews). Those
cases are customarily justified on the grounds that it is a brand new
transmission (even though the original Message-ID is retained).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 15 Jan 2008 19:23:35 -0800
From: Dave Crocker <[email-masked]>
Organization: Brandenburg InternetWorking
To: Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Sender: [email-masked]

(30 seconds after hitting send, of course...)

damn.  I misread the rfc2822 paragraph.

ignore my expressed concern about 'transmitter' since indeed it does appear in 
the existing rfc2822 paragraph.

So, yeah, your text is fine.

grrr.

d/

Dave Crocker wrote:
> 
> 
> Pete Resnick wrote:
>> On 1/15/08 at 10:18 AM -0800, Dave Crocker wrote:
>>
>>>    Note: The Sender (responsible agent) information is always 
>>> present. The absence of the "Sender:" field merely means that the 
>>> information is redundant with the "From:" field, so it is not 
>>> redundantly encoded into a separate field. The absence of the 
>>> "Sender:" field sometimes confuses readers into believing that the 
>>> Sender responsible agent information has not been specified explicitly.
>>
>> How about this (to better conform to the current language):
>>
>> Note: The transmitter information is always present. The absence of 
>> the "Sender:" field is sometimes mistakenly taken to mean that the 
>> agent responsible for transmission of the message has not been 
>> specified explicitly. The absence of the "Sender:" field merely means 
>> that the transmitter is identical to the author, so it is not 
>> redundantly placed into a separate field.
>>
>> Does that convey the correct information?
> 
> I think so, except use of the term "transmitter" does not have a direct 
> copy in the preceding paragraph.  While the semantics are fine (is 
> fine?) I think that particularly pedantic writing is needed since this 
> is all about clarification.
> 
> That's why my suggested paragraph tried to echo the "responsible agent" 
> phrasing from the preceding paragraph.  Anything that makes it 
> essentially impossible to misconstrue the reference is fine with me.
> 
> d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 15 Jan 2008 19:21:33 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Pete Resnick <[email-masked]>
CC: [email-masked]
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Sender: [email-masked]

Pete Resnick wrote:
> On 1/15/08 at 10:18 AM -0800, Dave Crocker wrote:
> 
>>    Note: The Sender (responsible agent) information is always present. 
>> The absence of the "Sender:" field merely means that the information 
>> is redundant with the "From:" field, so it is not redundantly encoded 
>> into a separate field. The absence of the "Sender:" field sometimes 
>> confuses readers into believing that the Sender responsible agent 
>> information has not been specified explicitly.
> 
> How about this (to better conform to the current language):
> 
> Note: The transmitter information is always present. The absence of the 
> "Sender:" field is sometimes mistakenly taken to mean that the agent 
> responsible for transmission of the message has not been specified 
> explicitly. The absence of the "Sender:" field merely means that the 
> transmitter is identical to the author, so it is not redundantly placed 
> into a separate field.
> 
> Does that convey the correct information?

I think so, except use of the term "transmitter" does not have a direct copy 
in the preceding paragraph.  While the semantics are fine (is fine?) I think 
that particularly pedantic writing is needed since this is all about 
clarification.

That's why my suggested paragraph tried to echo the "responsible agent" 
phrasing from the preceding paragraph.  Anything that makes it essentially 
impossible to misconstrue the reference is fine with me.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: rfc2822bis -- adding clarifying Sender: text
Date:  Wed, 16 Jan 2008 03:44:27 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 16
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Pete Resnick wrote:

> Does that convey the correct information?

Yes, if it is understood that you are talking
about a message modulo Resent-* header fields.
 =20
Any "let's add a Sender" scheme starting with=20
RFC 4409 8.1 has to be aware of Resent-*.
 =20
IMO "let's replace Sender" schemes are broken
by design.  If you know a good place to note
that an existing Sender MUST NOT be modified
please do.

 Frank

Date: Tue, 15 Jan 2008 18:53:55 -0600
To: Mark Martinec <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2822bis: "originator of the message" and "transmitter" not defined
Cc: [email-masked]
Sender: [email-masked]

On 1/16/08 at 12:28 AM +0100, Mark Martinec wrote:

>  > 3.6.2.
>>  [...]
>>     The originator fields indicate the mailbox(es) of the source of the
>>     message.  The "From:" field specifies the author(s) of the message,
>>     that is, the mailbox(es) of the person(s) or system(s) responsible
>>     for the writing of the message.  The "Sender:" field specifies the
>>     mailbox of the agent responsible for the actual transmission of the
>>     message.  For example, if a secretary were to send a message for
>>     another person, the mailbox of the secretary would appear in the
>>     "Sender:" field and the mailbox of the actual author would appear in
>>     the "From:" field.  If the originator of the message can be indicated
>                                 ^^^^^^^^^^^^^^^^^^^^^^^^^
>>     by a single mailbox and the author and transmitter are identical, the
>                                             ^^^^^^^^^^^
>>     "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
>>     appear.
>
>In this paragraph a term "originator of the message" is used for the 
>first and the only time, and is nowhere defined. Other text only 
>speaks of "originator fields" (= From, Sender, Reply-To).
>
>Also, the term "transmitter" is used here for the first time too, 
>and is never defined.

You are getting into territory where I feel like you're trying to 
overstate the obvious, and I can't figure out what problem you're 
trying to solve.

"The originator" of the message is the entity or entities specified 
in the originator fields.

"The transmitter" of the message is the entity responsible for 
transmitting the message.

It seems redundant to say those things.

The originator might be one entity (who is both the author and the 
transmitter and specified in the "From:" field), or the originator 
might be one or more authors (specified in the "From:" field) plus a 
separate transmitter (specified in the "Sender:" field).

Is there something in the above that you can not glean from 3.6.2? 
And what problem are we trying to solve by attempting to overtly 
define these terms in this very circular way?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

From: Mark Martinec <[email-masked]>
Organization: Jozef Stefan Institute
To: [email-masked]
Subject: rfc2822bis: "originator of the message" and "transmitter" not defined
Date: Wed, 16 Jan 2008 00:28:51 +0100
Sender: [email-masked]

> 3.6.2.
> [...]
>    The originator fields indicate the mailbox(es) of the source of the
>    message.  The "From:" field specifies the author(s) of the message,
>    that is, the mailbox(es) of the person(s) or system(s) responsible
>    for the writing of the message.  The "Sender:" field specifies the
>    mailbox of the agent responsible for the actual transmission of the
>    message.  For example, if a secretary were to send a message for
>    another person, the mailbox of the secretary would appear in the
>    "Sender:" field and the mailbox of the actual author would appear in
>    the "From:" field.  If the originator of the message can be indicated
                                ^^^^^^^^^^^^^^^^^^^^^^^^^
>    by a single mailbox and the author and transmitter are identical, the
                                            ^^^^^^^^^^^
>    "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
>    appear.

In this paragraph a term "originator of the message" is used for the first
and the only time, and is nowhere defined. Other text only speaks of
"originator fields" (= From, Sender, Reply-To).

Also, the term "transmitter" is used here for the first time too,
and is never defined.

The proper term could be borrowed from the:

> The "Sender:" field specifies the mailbox of the agent responsible
> for the actual transmission of the message.

so
  'originator of the message'
should actually be:
  'mailbox of the agent responsible for the actual
   transmission of the message'
  (although this is unsightly long and repeating the just said)
or just:
  'transmitter' (if it were defined elsewhere)

Looks like a term needs to be invented for the agent specified in
a Sender field, or the text rephrased to avoid the need for a
new term.

  Mark

Date: Tue, 15 Jan 2008 15:33:44 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: Re: rfc2822bis -- adding clarifying Sender: text
Cc: [email-masked], Dave Crocker <[email-masked]>
Sender: [email-masked]

On 1/15/08 at 10:18 AM -0800, Dave Crocker wrote:

>    Note: The Sender (responsible agent) information is always 
>present. The absence of the "Sender:" field merely means that the 
>information is redundant with the "From:" field, so it is not 
>redundantly encoded into a separate field. The absence of the 
>"Sender:" field sometimes confuses readers into believing that the 
>Sender responsible agent information has not been specified 
>explicitly.

How about this (to better conform to the current language):

Note: The transmitter information is always present. The absence of 
the "Sender:" field is sometimes mistakenly taken to mean that the 
agent responsible for transmission of the message has not been 
specified explicitly. The absence of the "Sender:" field merely means 
that the transmitter is identical to the author, so it is not 
redundantly placed into a separate field.

Does that convey the correct information?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Tue, 15 Jan 2008 15:12:35 -0600
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: New 2822upd-04 - obs-NO-WS-CTL
Sender: [email-masked]

I'd be inclined for folks to go over this one with a fine toothed 
comb. (I know about some of the spacing issues; that's not what I 
mean. Review for content.)

For differences between -03 and -04, try this:

<http://tools.ietf.org/rfcdiff?url1=http://www.ietf.org/internet-drafts/draft-resnick-2822upd-03.txt&url2=http://www.ietf.org/internet-drafts/draft-resnick-2822upd-04.txt>

For an HTML copy, try this:

http://resnick1.qualcomm.com/draft-resnick-2822upd-04.html

Things to note:

1. obs-qp, obs-body, and obs-unstruct each have NUL. obs-ctext, 
obs-qtext, and obs-dtext don't. Is that what's desired? (I'm happy to 
move NUL into all of them. I'm more worried about removing it from 
all of them.)

2. text moved into the body discussion, and specials moved into the 
atom discussion. Glad to move them elsewhere if you think necessary.

Any other comments welcome.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

From: [email-masked]
Cc: [email-masked]
Date: Tue, 15 Jan 2008 13:00:42 -0800 (PST)
Subject: Re: rfc2822bis -- adding clarifying Sender: text
To: Dave Crocker <[email-masked]>
Sender: [email-masked]

> This is not about a problem with the current text, but rather a bit of
> confusion started by RFC 733 that might benefit from some clarifying language.

> I see the confusion rear its head regularly and think it time to have some
> text that tries to help people understand the difference between information
> and its encoding...

> Current:

> > 3.6.2.  Originator fields
> >    The originator fields indicate the mailbox(es) of the source of the
> >    message.  The "From:" field specifies the author(s) of the message,
> >    that is, the mailbox(es) of the person(s) or system(s) responsible
> >    for the writing of the message.  The "Sender:" field specifies the
> >    mailbox of the agent responsible for the actual transmission of the
> >    message.  For example, if a secretary were to send a message for
> >    another person, the mailbox of the secretary would appear in the
> >    "Sender:" field and the mailbox of the actual author would appear in
> >    the "From:" field.  If the originator of the message can be indicated
> >    by a single mailbox and the author and transmitter are identical, the
> >    "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
> >    appear.

> Add after this:

>     Note: The Sender (responsible agent) information is always present. The
> absence of the "Sender:" field merely means that the information is redundant
> with the "From:" field, so it is not redundantly encoded into a separate
> field. The absence of the "Sender:" field sometimes confuses readers into
> believing that the Sender responsible agent information has not been specified
> explicitly.

Seems like a reasonable clarification to me.

				Ned

To: [email-masked]
Cc: 
From: [email-masked]
Date: Tue, 15 Jan 2008 00:30:01 -0500
Subject: I-D Action:draft-resnick-2822upd-04.txt 
Reply-To: [email-masked]
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : Internet Message Format
	Author(s)       : P. Resnick
	Filename        : draft-resnick-2822upd-04.txt
	Pages           : 59
	Date            : 2008-01-15

This document specifies the Internet Message Format (IMF), a syntax
for text messages that are sent between computer users, within the
framework of "electronic mail" messages.  This specification is a
revision of Request For Comments (RFC) 2822, which itself superseded
Request For Comments (RFC) 822, "Standard for the Format of ARPA
Internet Text Messages", updating it to reflect current practice and
incorporating incremental changes that were specified in other RFCs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-resnick-2822upd-04.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of 
the message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then
	"get draft-resnick-2822upd-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-resnick-2822upd-04.txt".

NOTE:   The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body; access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID: <[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-resnick-2822upd-04.txt

--OtherAccess
Content-Type: Message/External-body; name="draft-resnick-2822upd-04.txt";
	site="ftp.ietf.org"; access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <2008-01-15002440.I-D\@ietf.org>

--OtherAccess--

--NextPart
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce

--NextPart--

Date: Tue, 15 Jan 2008 10:18:29 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: rfc2822bis -- adding clarifying Sender: text
Sender: [email-masked]

This is not about a problem with the current text, but rather a bit of 
confusion started by RFC 733 that might benefit from some clarifying language.

I see the confusion rear its head regularly and think it time to have some 
text that tries to help people understand the difference between information 
and its encoding...

Current:

> 3.6.2.  Originator fields
>    The originator fields indicate the mailbox(es) of the source of the
>    message.  The "From:" field specifies the author(s) of the message,
>    that is, the mailbox(es) of the person(s) or system(s) responsible
>    for the writing of the message.  The "Sender:" field specifies the
>    mailbox of the agent responsible for the actual transmission of the
>    message.  For example, if a secretary were to send a message for
>    another person, the mailbox of the secretary would appear in the
>    "Sender:" field and the mailbox of the actual author would appear in
>    the "From:" field.  If the originator of the message can be indicated
>    by a single mailbox and the author and transmitter are identical, the
>    "Sender:" field SHOULD NOT be used.  Otherwise, both fields SHOULD
>    appear.

Add after this:

    Note: The Sender (responsible agent) information is always present. The 
absence of the "Sender:" field merely means that the information is redundant 
with the "From:" field, so it is not redundantly encoded into a separate 
field. The absence of the "Sender:" field sometimes confuses readers into 
believing that the Sender responsible agent information has not been specified 
explicitly.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Sat, 05 Jan 2008 15:47:26 +0900
To: [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Please comment on draft-duerst-mailto-bis-04.txt 
Cc: "Larry Masinter" <[email-masked]>, [email-masked]
Sender: [email-masked]

Hello Email Experts,

As you can see in the announcement below, I have just submitted
an updated version of the Internet Draft updating the mailto
URI scheme, after a long delay.

I would greatly appreciate your comments on this draft; I expect
to submit it to the IESG soon.

In particular, the current spec for mailto:, RFC 2368, contains
some advice against using a bcc field in a mailto: URI, but this
doesn't seem to be followed, and we were unable to find any reason,
so we removed it. Comments on this (both positive and negative,
if possible with reasons) would be appreciated.

With kind regards,   Martin.

P.S.: I'm not subscribed to [email-masked], so please keep me
      (and my coauthors) in the cc.

>From: [email-masked]
>Date: Fri, 04 Jan 2008 01:20:01 -0500
>Subject: I-D Action:draft-duerst-mailto-bis-04.txt 
>List-Id: i-d-announce.ietf.org

>A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>       Title           : The 'mailto' URI Scheme
>       Author(s)       : M. Duerst, et al.
>       Filename        : draft-duerst-mailto-bis-04.txt
>       Pages           : 18
>       Date            : 2008-01-04
>
>This document defines the format of Uniform Resource Identifiers
>(URI) to identify resources that are reached using Internet mail.  It
>updates the syntax of 'mailto' URIs from [RFC2368] for better
>internationalization and compatibility with IRIs ([RFC3987]).
>
>A URL for this Internet-Draft is:
>http://www.ietf.org/internet-drafts/draft-duerst-mailto-bis-04.txt

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]