Subject: [ietf-822] utf8 messages

From: Brandon Long <[email-masked]>
Date: Mon, 11 Aug 2014 13:45:48 -0700

In our recent launch of support for EAI, we noticed an issue with 6532
"utf8" messages.

As near as I can tell, there is nothing about a 6532 message which tells
you it is such a message... except the existence of 8bit characters in the
headers.  Ie, 7bit -> 5322, 8bit -> 6532.

Our problem is that this isn't actually true in practice.  Prior to
launching support for 6532 messages, we've already had to support
widespread use of 8bit messages that were not always in utf8.  Since these
typically didn't specify which charset they were in, we used a variety of
techniques including direct charset detection on such messages.

The problem we're having with 6532 messages, is that we moved from
explicitly identified charsets via 2047/etc mechanisms, to "its just
utf8"... and sometimes we mis-detect the utf8 as cp1250 or other encodings.

Now, we can work on improving our detection and maybe start biasing it to
utf8 or even just assuming utf8 for any 8bit message which is in
interchange valid utf8.  Anything we do there will result in some potential
for mistakes, of course.

This would all be solved if 6532 messages were actually denoted as such,
and I recall seeing at least one such X header used by another service
we've been interoperability testing with:
X-CM-HeaderCharset: UTF-8
CM no doubt standing for CoreMail, which is the software used:
X-Mailer: Coremail Webmail Server Version XT3.0.4 build
 20140526(27182.6409.6185) Copyright (c) 2002-2014 www.mailtech.cn coremail

Thoughts?  It looks like there was a i-Email/Header-Type originally, but
was removed early in the utf8smtp timeframe:
http://www.ietf.org/mail-archive/web/ima/current/msg01358.html
The general consensus for removal seemed to be "you'll know because it was
specified at SMTP time", "just look for 8bit" and "its bad to duplicate
data between the envelope and the headers".

Looks like it goes nearly to the beginning of the utf8smtp time frame:
http://www.ietf.org/mail-archive/web/ima/current/msg00079.html

It seems that the pre-existence of 8bit messages was not considered by
those who felt it wasn't necessary, as least as far as I've read in the
discussions (wow do I wish the mhonarc had been updated with an easier to
explore/read model)

Now, as hinted at in the consensus to remove such a marker from the draft,
we can certainly add such a header when composing 6532 messages or when we
receive any message via SMTPUTF8 for our own utility, but I would think
there would be some utility in such a marker being mutually understood and
shared.

Brandon

---

From: Ned Freed <[email-masked]>
Date: Mon, 11 Aug 2014 16:08:15 -0700 (PDT)

> headers.  Ie, 7bit -> 5322, 8bit -> 6532.

Sure there is: The smtputf8 flag in the envelope and/or utf8 flag in the IMAP
metadata is what tells you that the message is in RFC 6532 format.

It's true that there's no explicit indicator in the message content. There
doesn't need to be, and since you have to have the indicator in the envelope
and in the message metadata, putting it in both places just creates the
possibility of a silly state.

The basic theory of operation of EAI is if decide to submit an RFC 6532
message, you do so with the SMTPUTF8 extension engaged. If you decide to APPEND
it to an IMAP folder, you do so with the utf8 data extension engaged. Same
thing if you decide to use CATENATE to create the message. This tells whicever
server that the message is in fact in RFC 6532 format so it can keep track of
that fact.

EAI only allows for the transport and movement of RFC 6532 messages if the
necessary extensions are present. If they aren't, the message cannot be
transferred or moved.

The only time a RFC 6532 message can escape the EAI world is inside of a DSN,
and in that case a message/global wrapper provides the indicator.

It is, or is supposed to be, a sealed system implemented as a set of
interlocking extensions to existing email facilities.

> techniques including direct charset detection on such messages.

It depends on what you mean by "8bit message". If you mean messages with 8bit
in the body data, then sure, that's fully standard and widespread.

But 8bit of any sort in a header at any level was a standards violation prior
to RFC 6532. And since there are many 8bit charsets, and telling them apart is
in general impossible (although intelligent guesses can be made) without
labeling (which implies conversion to 7bit), this was never a terribly
interoperable thing to be doing from day 1.

> utf8"... and sometimes we mis-detect the utf8 as cp1250 or other encodings.

No message created prior to the release of support for RFC 6532 can be assumed
to be a RFC 6532 message. Now, if you want to vet those messages using some
sort of process to insure such a message meets the syntax and at least looks
semantically sensible, then I suppose you could set the flag in the metadata.

But if you can't distinguish such messages from legitimate RFC 6532 created and
submitted by compliant clients, it sounds to me like you're not retaining a
really critical piece of envelope/metadata in your implementation.

> interchange valid utf8.  Anything we do there will result in some potential

Sure. There are always tradeoffs when handling invalid data with no clear
semantics. And those tradeoffs just got nastier in the case of EAI, because now
you can't simply violate the standards by puking out such messages to clients
willy-nilly. Why? Because a compliant EAI client is going to do an ENABLE
UTF8=whatever, and once it does that it's going to expect the messages it gets
that do contain 8bit in headers and addresses to actually comply with RFC 6532.
So you're going to have to do something to either downgrade those messages to
7bit headers or upgrade them to RFC 6532, and yes, mistakes will be made.

But this is not a lack of labeling issue. Rather, it's an issue of having
have to deal with the garbage you let in.

One thing you could argue is that there should be a way to fetch the UTF8
state of a message - AFAIK that's not possible at present, which means
that in some cases clients have to scan the message to figure that out.
I think this is an oversight in the standard, but not a major one.

>  20140526(27182.6409.6185) Copyright (c) 2002-2014 www.mailtech.cn coremail

Say what? This solves nothing, since nobody else is going to bother doing this,
and creates unnecessary silly states.

Legitimate RFC 6532 messages are, or should be, labeled, and the label
should be carried along with the message as it moves around.

I suppose if you wanted to you could make the label manifest as a header
field in the stored copy. You could even write a specification for such
a header, but you won't be able to count on its presence.

> data between the envelope and the headers".

Which was correct.

> discussions (wow do I wish the mhonarc had been updated with an easier to

The pre-existence of 8bit messages is a problem, but the problem is the
lack of a proper label, not where the label happens to be stored.

> there would be some utility in such a marker being mutually understood and

Please don't. This is the sort of thing that wrecks standards.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Tue, 12 Aug 2014 01:20:27 -0700

> interlocking extensions to existing email facilities.

So, if I have an "email" message, I can no longer just parse it.  Instead,
there are actually two
types of email messages, and the only way to know how to parse it is to
know a priori which type it is.
Because all systems are "sealed" and there's never any leakage.

As for just check for 8 bit messages... on to the next part.

> interoperable thing to be doing from day 1.

Yes, it wasn't a great idea.  Apparently, strict adherence to spec was not
a strong concern from the non-English speaking world.  And, this worked
fine before "global" services, such that local sites would just assume
local charsets for these broken messages.  We've had to do auto-detection
on messages for quite a while, and yes, its sometimes broken.

Last time we looked, it was 2% of the email in the subset we checked.

> really critical piece of envelope/metadata in your implementation.

Yes.  And the most obvious place for that information, to me, is in the
headers of the message.  Assuming that every mechanism for exchanging email
messages needs an explicit external piece of data...

Is there a separate three letter filename extension for 6532 messages, for
example? (for platforms which use such things).

> a header, but you won't be able to count on its presence.

[snip]

> Please don't. This is the sort of thing that wrecks standards.

So, its ok to write a specification to have such a header but doing so
wrecks standards?

Or are you saying it would be fine for us to have such a header, but not to
leak it?

Brandon

---

From: Mark Martinec <[email-masked]>
Date: Tue, 12 Aug 2014 12:27:34 +0200

2014-08-12 10:20, Brandon Long wrote:
> So, if I have an "email" message, I can no longer just parse it.
> Instead, there are actually two
> types of email messages, and the only way to know how to parse it
> is to know a priori which type it is.
> [...]

Actually the presence of an SMTPUTF8 flag *is* reflected in the
header section of a message. It's in the WITH subfield of a
(likely topmost) Received header field (RFC 6531), as inserted
by your MX or MSA:

   UTF8SMTP	ESMTP with SMTPUTF8
   UTF8SMTPA	ESMTP with SMTPUTF8 and AUTH
   UTF8SMTPS	ESMTP with SMTPUTF8 and STARTTLS
   UTF8SMTPSA	ESMTP with SMTPUTF8 and both STARTTLS and AUTH
   UTF8LMTP	LMTP with SMTPUTF8
   UTF8LMTPA	LMTP with SMTPUTF8 and AUTH
   UTF8LMTPS	LMTP with SMTPUTF8 and STARTTLS
   UTF8LMTPSA	LMTP with SMTPUTF8 and both STARTTLS and AUTH

http://www.iana.org/assignments/mail-parameters/mail-parameters.xhtml#mail-parameters-7

(and equivalently for LMTP)

> Is there a separate three letter filename extension for 6532 messages,
> for example? (for platforms which use such things).

RFC 6532:
   message/global -> .u8msg

RFC 6533:
   message/global-headers -> .u8hdr
   message/global-delivery-status -> .u8dsn
   message/global-disposition-notification -> .u8mdn

Mark

---

From: Ned Freed <[email-masked]>
Date: Tue, 12 Aug 2014 07:54:30 -0700 (PDT)

> Because all systems are "sealed" and there's never any leakage.

Wrong on all counts, I'm afraid. First and foremost, in practice you have never
been able to "just parse" email messages, for the simple reason that too many
message creation agents don't follow the rules and create wildly incompliant
messages. So there's always some heuristics involved, unless of course you're
willing to only accept syntacticaly valid messages, in which case yes, you
can "just parse" messages, including EAI messages.

Where the lines are drawn has always been a tradeoff, and one which has
changed over the years. It used to be the case that a lot more crap was
generally tolerated. The spam problem has led to an overall tightening up
what's tolerated.

Second, because of overall lack of compliance, there have always been
many types of messages.

Third, of course there's leakage.

> on messages for quite a while, and yes, its sometimes broken.

Actually, no, it sucked from the get-go. Entire books were written on how to
deal with the CJK disaster long before any of this was anything even close to
being "global".

This mess is part of what led to the development of MIME. Yes, it really
does go back that far.

> messages needs an explicit external piece of data...

Sorry, I'm not going to revisit or defend past design decisions. My point was
and is that when you said that a piece of information was missing, that
statement was incorrect.

You may not like or agree with how this was done. (For that matter, I never
said I liked or agreed with how it was done.) But for better or worse, there's
now a standard in place, and absent compelling evidence of there being a
problem with implementing that standard - evidence which AFAICT you have not
provided - it's not appropriate to propose competing mechanisms to that
standard.

> example? (for platforms which use such things).

Yes there is: .u8msg. See RFC 6532 section 3.7.

> Or are you saying it would be fine for us to have such a header, but not to

It's one thing to manifest such a marker in your IMAP store as a header. People
have been exposing various sorts of store metadata that way for years, and it
seems to have been Mostly Harmless.

It's quite another to write a specification advocating using such a mechanism
to identify messages. It that's done it will create silly states when EAI
messages are sent with such a marker but not SMTPUTF8 flag and vice versa.

				Ned

---

From: Jan Kundrát <[email-masked]>
Date: Tue, 12 Aug 2014 18:08:36 +0200

> seems to have been Mostly Harmless.

I'm afraid that once these headers show up in GMail, these will become a 
de-facto standard.

I don't have a horse in this race -- but I have seen what happens when a 
major player does something non-standard and people assume that it must've 
been the correct thing.

With kind regards,
Jan

-- 
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/

---

From: Ned Freed <[email-masked]>
Date: Tue, 12 Aug 2014 17:16:56 -0700 (PDT)

> major player does something non-standard and people assume that it must've

Good point. Consider my position changed to "don't do this, period".

As for the with clauses, they provide a "solution" for mail that undergoes
final delivery, but not for IMAP APPEND.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Wed, 13 Aug 2014 01:00:32 -0700

> Third, of course there's leakage.

Ok, let me try to rephrase my point.  Before, there were two types of
messages, well specified / syntactically correct message, and not.  The not
well specified messages amount to a non-trivial number, but best
effort/heuristic results are fine.

If we imagine handling 1B messages/day, 2% being the "not well specified",
we have 20M messages handled by heuristic.  If we have only 1% heuristic
failures, that's 200k bad messages a day.  Eh, maybe ok.  Especially since
in practice, these tend to be spam messages or the 8bit chars in the header
are limited to some "content preview" header or other boneheadedness, or
best case, its only a character or two that's broken.

Adding under-specified 6532 messages to the mix means that we are now
generating messages that can easily slip into the second pool.  We try our
best to only generate well specified / syntactically correct messages.. "be
conservative in what you send" and all, but now we're being forced to take
steps that we know will increase the number of failures.  We're attempting
to adjust our heuristics to compensate, but that seems like a poor response
compared to making the messages be well specified.

Now, one can add some bit outside of the message to say its 6532... and
then modify everything that exchanges or stores messages to also exchange
and store that bit.  Passing messages to procmail?  Guess we need a new env
var.  Maildrop programs used by smtp servers to pass messages to imap
servers.  Mailing list software.  Mailing list archives.  How you call
spamassassin.  The thread about this during the discussion suggested adding
a new field to the From_ mbox separator.  Guess a new attribute field in
the maildir spec?

> and is that when you said that a piece of information was missing, that

I'm saying that requiring the information to be external grossly increases
the development required to support the standard.  I already pointed out
some common tools above.  For us, not having to add a new piece of external
metadata means that all we have to do is upgrade our parser... and validate
how we use addresses and fix hopefully minor issues.  If we have to use
external metadata, then we have 100s of data paths and data stores that
need to be upgraded.

We've also already agreed that leaks happen, which means separating the
metadata from the data itself guarantees that under specified data will
leak.  Making the message itself well specified means that
re-synchronization can take place, that messages can pass through agnostic
mechanisms (whether agnostic by choice or by happenstance) and be
understood on the other side.

> provided - it's not appropriate to propose competing mechanisms to that

I thought implementation feedback was a useful thing, and generally desired
prior to something becoming a standard.  I also thought that internet
standards were generally considered a work in progress... 6532 is standards
track but not a STD.

I also wasn't proposing a competing mechanism, I was proposing that
implementation experience showed, to me, a strong possibility that a
clarification or change to the standard would be beneficial.  Also,
frankly, there is nothing in the standards that say we MUST not do this, so
thanks for the exhortation from authority, I'll take it under advisement.

Frankly, I don't understand this concept of refusing to revisit or defend.
 I tried to find and follow the discussion in the eai wg archives... and to
me, the decision seemed under explored.  The original section spent a lot
of real estate on the importance of such an explicit in message spec, only
to be removed.  I found two threads about it, about equal numbers of people
contributing to either side (like 1-2 on each side), a somewhat heavy
handed dismissal of the need, a call for consensus that passed due mostly
to non-contributors.  Perhaps more took place in meetings, or there were
other discussions that someone could point me to.

Perhaps you can explain to me what 'compelling evidence of there being a
problem implementing the standard' would mean in practice?

Brandon

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 13 Aug 2014 14:08:30 +0200

On Wed 13/Aug/2014 10:00:32 +0200 Brandon Long wrote:
> generating messages that can easily slip into the second pool.  We try
> our best to only generate well specified / syntactically correct
> messages.. "be conservative in what you send" and all, but now we're
> being forced to take steps that we know will increase the number of
> failures.  We're attempting to adjust our heuristics to compensate, but
> that seems like a poor response compared to making the messages be well
> specified.

QUESTION:  UTF8 compliant receivers may want to reject utf8 messages for
certain recipients who cannot upgrade their readers.  But thanks to the
specific reply code, retrying doesn't seem to be a problem if the
message can be downgraded trivially.  So, is it advisable to *always*
use the SMTPUTF8 extension if both peers support it?

That habit would produce valid 6532 messages which are also valid 5322,
hence trivially downgradable.  Section 3.4 of RFC 6531 says:

                                          If the SMTPUTF8-aware SMTP
   client is aware that neither the envelope nor the message being sent
   requires any of the SMTPUTF8 extension capabilities, it SHOULD NOT
   supply the SMTPUTF8 parameter with the MAIL command.

Now, having an external marker would make the client aware, and thus
inhibit homogeneous treatment of messages.  That strategy seems to be
good if only a few servers switch to utf8...  If I were Google, I'd
assume they follow.

Ale

---

From: Daniel Vargha <[email-masked]>
Date: Wed, 13 Aug 2014 13:17:53 +0000

I fully agree with Brandon, the standard SHOULD consider the use case when a
message is transferred from one system to another as a blob (e.g. flat file) and
the only available "metadata" is that the message is in MIME format. Having
some sort of well defined UTF8 indicator in the header section of the message
would make it much simpler to adopt the new standard as it would require
substantially less development effort in most cases.

Regarding Ned's concern about inconsistent states I think it would be a workable
solution to only honour the UTF8 indicator in the headers when the UTF8 flag
is not available from metadata. In a well known UTF8 context where the SMTP
protocol or the message store already "knows" that the message is UTF8 the
indicator in the headers can be ignored.

I think it is generally desirable to reduce (or at least not increase) the amount
of heuristics required to successfully parse a MIME message. We should try to
learn from previous mistakes instead of repeating them.

Daniel

From: Brandon Long <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 13 August 2014 09:00
To: Ned Freed <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [ietf-822] utf8 messages

Ok, let me try to rephrase my point.  Before, there were two types of messages, well specified / syntactically correct message, and not.  The not well specified messages amount to a non-trivial number, but best effort/heuristic results are fine.

If we imagine handling 1B messages/day, 2% being the "not well specified", we have 20M messages handled by heuristic.  If we have only 1% heuristic failures, that's 200k bad messages a day.  Eh, maybe ok.  Especially since in practice, these tend to be spam messages or the 8bit chars in the header are limited to some "content preview" header or other boneheadedness, or best case, its only a character or two that's broken.

Adding under-specified 6532 messages to the mix means that we are now generating messages that can easily slip into the second pool.  We try our best to only generate well specified / syntactically correct messages.. "be conservative in what you send" and all, but now we're being forced to take steps that we know will increase the number of failures.  We're attempting to adjust our heuristics to compensate, but that seems like a poor response compared to making the messages be well specified.

Now, one can add some bit outside of the message to say its 6532... and then modify everything that exchanges or stores messages to also exchange and store that bit.  Passing messages to procmail?  Guess we need a new env var.  Maildrop programs used by smtp servers to pass messages to imap servers.  Mailing list software.  Mailing list archives.  How you call spamassassin.  The thread about this during the discussion suggested adding a new field to the From_ mbox separator.  Guess a new attribute field in the maildir spec?

I'm saying that requiring the information to be external grossly increases the development required to support the standard.  I already pointed out some common tools above.  For us, not having to add a new piece of external metadata means that all we have to do is upgrade our parser... and validate how we use addresses and fix hopefully minor issues.  If we have to use external metadata, then we have 100s of data paths and data stores that need to be upgraded.

We've also already agreed that leaks happen, which means separating the metadata from the data itself guarantees that under specified data will leak.  Making the message itself well specified means that re-synchronization can take place, that messages can pass through agnostic mechanisms (whether agnostic by choice or by happenstance) and be understood on the other side.

I thought implementation feedback was a useful thing, and generally desired prior to something becoming a standard.  I also thought that internet standards were generally considered a work in progress... 6532 is standards track but not a STD.

I also wasn't proposing a competing mechanism, I was proposing that implementation experience showed, to me, a strong possibility that a clarification or change to the standard would be beneficial.  Also, frankly, there is nothing in the standards that say we MUST not do this, so thanks for the exhortation from authority, I'll take it under advisement.

Frankly, I don't understand this concept of refusing to revisit or defend.  I tried to find and follow the discussion in the eai wg archives... and to me, the decision seemed under explored.  The original section spent a lot of real estate on the importance of such an explicit in message spec, only to be removed.  I found two threads about it, about equal numbers of people contributing to either side (like 1-2 on each side), a somewhat heavy handed dismissal of the need, a call for consensus that passed due mostly to non-contributors.  Perhaps more took place in meetings, or there were other discussions that someone could point me to.

Perhaps you can explain to me what 'compelling evidence of there being a problem implementing the standard' would mean in practice?

Brandon

---

From: Mark Martinec <[email-masked]>
Date: Wed, 13 Aug 2014 19:29:54 +0200

> substantially less development effort in most cases.

It is not possible (even in the absence of SMTPUTF8 support) to be
able to transfer e-mail messages with no out-of-band ("metadata" /
envelope) information. The most obvious reason is the list of
recipient addresses, which is not present in a message itself.
Envelope sender may or may not be present in a mail header
(as a Return-Path header field). Other examples are RFC 3461 data
(RET, ENVID, NOTIFY, ORCPT). The SMTPUTF8 flag is just one more of
such out-of-band pieces of information necessary for mail transmission.

   Mark

---

From: Ned Freed <[email-masked]>
Date: Wed, 13 Aug 2014 09:08:51 -0700 (PDT)

> effort/heuristic results are fine.

And now there are still two types types of messages by this criteria, but the
set of syntactically valid messages has increased to include those with utf-8
in certain places.

> best case, its only a character or two that's broken.

Not sure of the relevance, but OK.

> Adding under-specified 6532 messages

What's underspecified about EAI messages? Either a message contains utf-8 in
certain places, making it an EAI message, or it doesn't making it something
else.

Or perhaps more to the point, in making your argument you are
assuming the conclusion you want to reach: That such messages are
in fact "underspecified".

> to the mix means that we are now

I'm having a great deal of difficuly parsing this, but what I think you're
trying to say is that there will be cases where, despite your best efforts and
intentions, you will construct what you intend to be RFC 6532 messages but
those messages will contain 8bit material in headers not in utf-8. And
you are concerned that such messages may be misclassified.

First, you're right to be concerned, but not necessarily for the reasons you
think. I am skeptical that EAI implementations are going take the lax attitude
towards incompliant garbage in messagess they have in the past. (Remember that
the trend is consistently towards less and less tolerance as time goes on.) My
own EAI implementation-in-progress certainly doesn't tolerate such stuff - it
won't accept such messages.

But in this case the label you think you need doesn't mean "EAI message",
rather, it means "this was intended to be an EAI message but may not be".

I seriously question the value of such a field. Or, for that matter,
using the SMTPUTF8 flag in this fashion.

> var.

Or not. procmail is a delivery agent. It can look for a "with SMTPUTF8"
clause in the Received: field.

> Maildrop programs used by smtp servers to pass messages to imap
> servers.

Absolutely have to be modified, because the IMAP server has to have this
information.

And guess what? EAI did exactly this: The SMTPUTF8 extension is valid in LMTP,
the maildrop protocol specified by the IETF.

> Mailing list software.  Mailing list archives.  How you call
> spamassassin.

Mailing lists need access in order to propogate the bit. But they are operating
at the envelope level in order to have access to the list name, so I'm not
seeing the problem.

Archiving and AS/AV software generally operates in the "has to handle anything
tossed at it", so aside from logging the bit or using it as another input, I'm
not sure there's much point. And if there is, compliance archiving typically
comes into play at the SMTP or IMAP APPEND, both of which have this information
readily available. Operational archiving requires access to store metadata for
other reasons. And milter in theory provides access to SMTP extension
information, so that case is also covered.

> The thread about this during the discussion suggested adding

Yes, the back end store format has to store this information for IMAP
to work properly.

> the development required to support the standard.

So does using SMTP and IMAP extensions, as opposed to the just-send-UTF-8 your
proposal is likely to result in an which the WG rejected.  Your point is ...
what, exactly?

> I already pointed out

Quite possible.

> leak.

Again, you're begging the question.

>  Making the message itself well specified means that

> standards were generally considered a work in progress... 6532 is standards

This is implementation feedback? On a list where proposals for header
extensions are typically discussed, rather than either of the two lists for EAI
discussions and feedback?

Sure doesn't sound like it to me. Rather, it sounds like an argument for what
is effectively a competing proposal, and coming from someone at Google, that's
cause for major concern.

> thanks for the exhortation from authority, I'll take it under advisement.

Of course there isn't. There also isn't a statement to the effect that you
should develop your own private, different competing SMTP and IMAP extensions
that use GB18030 instead of utf-8. Rather, it's hoped that people doing such
things will take the possible consequences of their actions into account.

And in this case the consequences could be extremely dire. If people assume
such an indicator in the message body is effectively the same as the SMTPUTF8
extension, things could get very ugly indeed.

> Frankly, I don't understand this concept of refusing to revisit or defend.

Seriously? You tnink every decision we made, including the most fundamental
ones, should be open for review and required to be defended any time anyone
shows up with what they think is a  problem?

You have way more time to spend on this stuff than I do.

> problem implementing the standard' would mean in practice?

For me, compelling evidence would consist of a set of compelling use cases
along with an analysis showing the cost versus benefit. Thus far I've seen no
acknowledgment of the likely costs whatsoever.

				Ned

---

From: Daniel Vargha <[email-masked]>
Date: Wed, 13 Aug 2014 19:07:09 +0000

It is possible and it is happening to millions of messages every day.
A message is sitting in an archive store, the user exports it to an .EML
file and then imports the file into another archive system. The
destination system needs to parse the message for indexing in order
to make it searchable. Quite often a message goes through this
export/import process several times during it's life time.

Another example where it is assumed that a MIME message is self
contained and allows parsing without additional metadata is the
MBOX format (http://en.wikipedia.org/wiki/Mbox)

Daniel

---

From: Brandon Long <[email-masked]>
Date: Wed, 13 Aug 2014 16:21:56 -0700

Let me try one more time, since something isn't making it through.

I have three messages.  One message has an entirely 7bit header with 2047
encoded subject.  Another message is a 6532 message, with the subject in
utf8.  A third message is has a cp-1250 8bit subject.  There are two 8bit
bytes in the subject in both of the last two messages, and in the cp1250
case, those two bytes happen to also be a valid utf8 character.

We want to be able to parse all three of those and do so correctly.  We
know the third type is technically invalid, but we see millions of such
messages every day, dropping all of those would be a dis-service to our
users.  We currently see way more of such messages than we do of 6532
messages... though in practice, the most common charset now is utf-8, so I
guess those are now the same as 6532 messages that have leaked.

An example, we receive the Subject: Zdj\xc4\x99cia.  In UTF8,
that's Zdjęcia, in cp-1250, that's ZdjÄ™cia.

How do I tell which its supposed to be?  Our encoding detector chose
incorrectly.

And my apologies if bringing this to ietf-822 instead of the eai-wg list
was the wrong choice, it wasn't clear to me that the latter was still
active since the completion of the working group, and that with its
completion, there's no longer a "split" between the two, and that a concern
specifically about the format of email messages (which now includes 6532)
would belong on the list about such things.

I also seemed to have triggered some fear of a revolt which I don't
understand.

And I realize that to someone who spent years working on this that being
asked to retread these things is annoying.  Unfortunately, the resulting
RFCs don't include summarized information on why other possibly choices
were considered and rejected.  I'm unclear on how one is supposed to gain
this knowledge short of reading years worth of mailing list archives across
multiple lists... and even that doesn't help about things discussed
off-line or on other lists I don't even know to look for.

Brandon

---

From: Ned Freed <[email-masked]>
Date: Wed, 13 Aug 2014 17:56:01 -0700 (PDT)

> substantially less development effort in most cases.

I'm skeptical of the claim, but if you absolutely have to have something, why
not add a Received: field containing a "with smtputf8" clause, assuming one
isn't there already?

> indicator in the headers can be ignored.

That assumes people will read the standard. It's far more likely that,
given an obvious indicator, they will simply use it.

> learn from previous mistakes instead of repeating them.

That's the absolute worst example you could have picked, because the most
serious design error in MIME is the MIME-Version: field. You know, the field
that tells you whether or not a given message is a MIME message. Sound
familiar?

You might want to jot down the hall and ask Nathaniel Borenstein about it. He's
far more emphatic about how stupid that was than I am.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Wed, 13 Aug 2014 18:34:35 -0700 (PDT)

> guess those are now the same as 6532 messages that have leaked.

I thought I understood the problem you were attempting to solve, but now I'm
totally confused, because this seems to hqve nothing to do with additional
labeling of legitimate EAI messages at all.

You say you have to deal with invalid messages with 8bit in headers. You say
that there's a trend towards these using utf-8 rather than some other charset.
You say that EAI messages are in the distinct minority. And finally, you say
there are issues with your heuristics misidentifying the charset.

Given that EAI messages are currently in the minority, your first order of
business clearly needs to be work on those heuristics. Beyond that, it seems to
me that your focus needs to be on calling out the details nonstandard stuff
you're doing, rather than creating openings for silly states with the
standard stuff.

More specifically, when you receive an invalid message that needs or has
undergone heuristic processing, why not just label it as such? This way
there's a clear indicator that the message has issues and that there
may be problems interpreting it.

This label is actually orthogonal to marking the message as an EAI message.
If your heuristics say, which high probability, that this is an EAI
message, then you probably want to set the EAI bit so that other things
will treat it as such. But the additional label tells you that the EAI
label came about implicitly rather than explicitly.

I also note that existing stock of messages containing invalid 8bit in the
headers are not EAI messages by definition. And you can check this by looking
at the timestamps in the message, message metadata, or both. So the lack of
these labels on old messages is a nonissue.

You can also use the label to write down the heuristics you have, or will
apply. Or whatever other contextual details exist that aren't stored
anywhere else that may assist in the handling of the message.

What am I missing here?

				Ned

---

From: Brandon Long <[email-masked]>
Date: Thu, 14 Aug 2014 17:41:24 -0700

> labeling of legitimate EAI messages at all.

My point is that without a label, I can't tell the difference between the
6532 messages and the illegitimate messages, given just the message.

Brandon

---

From: Ned Freed <[email-masked]>
Date: Thu, 14 Aug 2014 18:42:38 -0700 (PDT)

> 6532 messages and the illegitimate messages, given just the message.

Yes, that's exactly what I thought. But given that, surely it makes more sense
to label the illegimate messages as such, especially since you're going to want
to set the EAI message bit for some of them, making it essentally an orthogonal
setting?

Did you even bother to read the rest of my response?

				Ned

---

From: Jan Kundrát <[email-masked]>
Date: Fri, 15 Aug 2014 13:27:08 +0200

My understanding is that Brandon's system previously required no 
out-of-band label at all. Whenever a message was processed in their 
infrastructure, they would use a single algorithm, and that algorithm 
contained some heuristic for dealing with messages which were using 8bit 
data in headers. Everything worked pretty well for them.

Now, the EAI introduced another syntax, and that syntax breaks their 
heuristics. Ned's suggestion is to make use of the out-of-band labeling and 
extend the heuristic to only apply the old crutches when the label says 
that this isn't an EAI message, while Brandon's complaint is that they use 
no such labeling at this time because it isn't required at all. Brandon 
also suggests that there are many other places which process these messages 
without the external metadata.

By "out-of-band" or "external", I mean what gets set in the ESMTP envelope.

Hope this helps, and please correct me if my understanding of your 
positions is not correct.

>From my point of view, it would be most interesting to see what happens 
when I deploy e.g. a Postfix version which understands this new RFC along 
with an older Dovecot which doesn't do it. Suddenly, an IMAP client is in 
the same position as Brandon describes because there's no way of accessing 
the ESMTP envelope through IMAP.

Cheers,
Jan

---

From: Daniel Vargha <[email-masked]>
Date: Fri, 15 Aug 2014 12:38:58 +0000

>not add a Received: field containing a "with smtputf8" clause, assuming

Received: headers are not very reliable, and the syntax is is not well
defined. 
Successfully parsing a Received: header itself requires a lot of
heuristics. 
To be honest I would not be happy to rely on them. Also, when a  message
is 
transferred between archive stores no new Received: header is normally
added.

>given an obvious indicator, they will simply use it.

Is this a serious argument? Why would you bother writing a standard if you
don't 
expect people to read it?

>that tells you whether or not a given message is a MIME message. Sound

I don¹t understand this comment. What example are you referring to? (Of
course
I am familiar with the MIME-Version: header, I have read the corresponding
RFC 
many times)

Daniel

---

From: Ned Freed <[email-masked]>
Date: Fri, 15 Aug 2014 07:13:16 -0700 (PDT)

> Received: headers are not very reliable, and the syntax is is not well

On the contrary, it's quite well defined. See RFC 5321. The issue isn't that
it's poorly defined, but rather that there are a lot of agents that don't
create it properly.

> Successfully parsing a Received: header itself requires a lot of

A full parse does, and so does looking for IP address information (which
doesn't appear directly as a clause value and whose position was only
standardized late in the game). Looking for a with clause with a particular
value does not. 

> is transferred between archive stores no new Received: header is normally
> added.

Uh huh. And neither is whatever new header is being proposed here. Why
is one preferable to the other?

> don't expect people to read it?

It's deadly serious. There's quite a lot of monkey-see-monkey-do
out there.

Your "why bother" argument is bogus though. Other people do read the standards;
enough to make it worthwhile to develop them. And some finally read them when
they find their hack didn't work.

> I am familiar with the MIME-Version: header, I have read the corresponding

The MIME-Version field has turned into a wart on the protocol. There's no way
to bump the version since too many things are hard-coded to look for the 1.0,
so it's primary purpose of providing a version indicator is gone. We're stuck
at 1.0. You can't even put a comment on the field since some agents are known
to hate that.

And on the other side, a lot of agents will assume MIME even if the if field
isn't present. It also gets attached willy-nilly to a lot of non-MIME messages
because that's easier than checking.

As a result the information value of the field is essentially nonexistent: You
have to attach it to MIME messages but you cannot count on it to tell you
anything.

It's a fully worked example of how redundant indicators turn into warts. And in
the case of MIME-Version, despite being in the standard from day one this
process was more or less complete in a couple of years.

				Ned

---

From: Daniel Vargha <[email-masked]>
Date: Fri, 15 Aug 2014 15:15:43 +0000

On 15/08/2014 15:13, "Ned Freed" <[email-masked]> wrote:

>> >> message is transferred from one system to another as a blob (e.g.
>>flat
>> >>file) and
>> >I'm skeptical of the claim, but if you absolutely have to have
>>something,
>> >why

Maybe because it was defined too late, and not in the right place? (RFC
5321
is about SMTP not about MIME) From the parser's point of view the reason
is 
indifferent, the reality is that it is better not to rely on it. Even RFC
5321 
says: 

"...receiving systems MUST NOT reject mail based on the format of a trace
header field and SHOULD be extremely robust in the light of unexpected
information or formats in those header fields."

>standardized late in the game). Looking for a with clause with a

Looks like we have quite different ideas about reliability and parsing.
I certainly would not consider the partial parsing approach you suggested
as reliable.

>is one preferable to the other?

Because 
1) the Received: header is already used and abused in many ways
2) as you admitted, there are a lot of agents that don't create it properly
3) semantically it does not makes sense to put the charset information in
   the Received: header (it is meant to be a trace field)
4) if we define a new field, we don't need to worry about finding the
newly 
   defined field with bogus syntax in historic emails sent before the
standard 
   was published

>
>> >
>> >> Regarding Ned's concern about inconsistent states I think it would
>>be a
>> >>workable
>> >> solution to only honour the UTF8 indicator in the headers when the
>>UTF8
>> >>flag
>> >> is not available from metadata. In a well known UTF8 context where
>>the
>> >>SMTP
>> >> protocol or the message store already "knows" that the message is
>>UTF8
>> >>the

Is this enough reason for designing the standard for the monkeys?

>
>> >
>> >> I think it is generally desirable to reduce (or at least not
>>increase)
>> >>the amount
>> >> of heuristics required to successfully parse a MIME message. We
>>should
>> >>try to

Thanks for explaining. As I said before, I am familiar with the
MIME-Version: 
header, but I have not used is as an example. (You said it was the absolute
worst example I could have picked)

Daniel

---

From: Brandon Long <[email-masked]>
Date: Fri, 15 Aug 2014 12:44:58 -0700

On Thu, Aug 14, 2014 at 6:42 PM, Ned Freed <[email-masked]> wrote:

> > > > messages... though in practice, the most common charset now is
> utf-8, so
> > > I
> > > I thought I understood the problem you were attempting to solve, but
> now
> > > I'm

Yes, but I didn't understand it until this response.  It is an interesting
alternative.  You still seem to be assuming an external eai bit, but even
without that, yes, theoretically we could mark any non legitimate message
at smtp-in time.  It would be harder to do at imap APPEND time, though,
since I believe clients assume the messages as uploaded are stored as is..
I'd have to test to see if that was a failure.  Potentially, I would have
to be careful with any source of messages to make sure .. and that means we
may need to upgrade our other api's and migration tools to have a bit for
eai.

Brandon

---

From: Ned Freed <[email-masked]>
Date: Fri, 15 Aug 2014 13:06:34 -0700 (PDT)

> the ESMTP envelope through IMAP.

That's an incompliant setup, so what happens doesn't have much to do with the
standards. There's an IMAP extension in addition to  an SMTP extension; if the
IMAP server offers access to the EAI messages without the client having
requested such access (and of course there will be no way to do that in an
unextended IMAP server) you're operating outside the scope of the standard.

I'll also note that the SMTPUTF8 indicator during transport doesn't necessarily
translate to an RFC 6532 message on delivery. The EAI-ness of the message
during transport could be limited to envelope fields that end up not appearing
in the delivered message. Of course you can just blindly set the store-side
indicator on the basis of the SMTPUTF8 indicator if you like.

This stuff is tricky.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 15 Aug 2014 23:52:22 +0200

On Fri, Aug 15, 2014 at 01:27:08PM +0200, Jan Kundrát wrote:
> From my point of view, it would be most interesting to see what
> happens when I deploy e.g. a Postfix version which understands this new RFC
> along with an older Dovecot which doesn't do it.

The two speak LMTP, and the Dovecot rejects EAI messages as described
in 6531.

Arnt

---

From: Mark Martinec <[email-masked]>
Date: Sat, 16 Aug 2014 00:23:13 +0200

> I certainly would not consider the partial parsing approach you suggested

It's only the topmost (last added) Received header field that is
to be considered here. It is added by your own mailer, and you
know exactly what software that is using. The topmost added
Received header field is always in the same format, its contents
is always trustworthy. It is likely RFC 5321 compliant, but even
if it is not, you already know and can rely on its idiosyncrasies.
So there is no good excuse for not fetching the WITH clause
from this field, if other ways of accessing the EAI flag are
not available.

   Mark

---

From: Jan Kundrát <[email-masked]>
Date: Sat, 16 Aug 2014 10:49:43 +0200

> The two speak LMTP, and the Dovecot rejects EAI messages as described

In their current setup at $work, they use procmail. There's plenty of 
HOWTOs on the web which talk about how to set up procmail for delivery, and 
at least some of these do not use the -z flag for activating the LMTP mode. 
Looking at the trace headers from the $work setup, I don't see any LMTP 
trace fields -- in fact, it looks as if their procmail added no trace 
headers at all, and the last one I see is an ESMTP received by the target 
IMAP server from their spam filtering GW.

-> the point is that Dovecot at $work does not even know whether it's an 
EAI message or not.

I have a hunch that there are other companies with a setup similar to this 
one, no matter whether it's compliant, or expected by the EAI documents.

Cheers,
Jan

---

From: Ned Freed <[email-masked]>
Date: Sun, 17 Aug 2014 07:33:07 -0700 (PDT)

> I haven’t been very involved with EAI, so please indulge my attempt to
> restate the problem to see if I understand it correctly: a 6532 message is
> *implicitly* of type message/global, whereas a non-6532 message is implicitly
> of type message/rfc822.  That implicit state is specified is inferred from the
> presence or absence of 8-bit characters.

With the proviso that those characters have to be valid utf-8 and have
to appear in specific parts of a header.

It's also the case that the SMTPUTF8 marker provided by EAI is not a reliable
indicator of a RFC 6532 message: It may be set for a message/rfc822 message
that happens to have had EAI addresses in the envelope at some point, and it
may be clear for a random 8bit message that happens to meet RFC 6532 criteria.
Indeed, the SMTPUTF8 transport flag's purpose is to try and keep EAI material
from getting into places where it won't be understood, not to identify RFC 6532
messages.

It's even possible for nonconformant messages containing non-utf-8 8bit to
arrive with the SMTPUTF8 bit set. For example someone might forward such a p	
message to someone with an EAI address.

>  Brandon and others have problems because there are non-conformant messages
> with 8-bit characters that are not 6532 messages, most often because they
> use Windows-1250 instead of UTF-8.  Is that a correct restatement?

> How does Google (or anyone else) tell that a message is cp-1250 instead of
> UTF-8?  Can we specify a clear algorithm for detection?

I can't speak to how Google does it, but the way we handle it is to apply a set
of heuristics. Obvious ones include a check for utf-8 syntax validity (the
large the amount of 8bit text, the less likely it will meeting utf-8 syntax
rules and end up being something else). Note that isolated 8bit characters like
an accented "e" never qualify as utf-8. Checks for certain escape sequences,
will, if found, identify various iso-2022-? variants with high reliability. And
so on.

But I doubt there's a One True Algorithm for any of this. Moreover, the
heuristics require adjustment as the message population shifts. (What we're
seeing is that utf-8 is slowly taking over.)

> If so, how about automatic translation to UTF-8?

We offer that, but I understand why people are reluctant: It loses information.

> Failing that, how about encapsulating the whole thing in a new content-type
> (perhaps “message/windows-still-sucks”)?  I’d rather come up with a
> recommended best practice for handling the non-conformant messages than try to
> get all the existing conformant implementations to add something new.

That's a variant of what I'm suggesting: Instead of marking the good messages,
mark the bad ones. 

> On the other hand, I think Brandon was merely suggesting a new header field
> that would serve as a way for conformant messages to bypass the heuristics.  I
> would be supportive of such a heuristic if it were better defined than
> MIME-Version was.  But even so, we’d still need the best practice guidelines
> for handling non-UTF-8 8-bit messages, so my inclination would be to start
> there. — Nathaniel

Good point. We recently wrote a document tht discussed how to handle invalid
addresses using heuristics. A document discussing how to handle invalid 8bit
would certainly be possible, and by getting the various issues documented
it's possible best practices would emerge.

The real question is whether there's the interest and energy to do it. Speaking
personally, having just agreed to cochair DMARC, I'm essentially booked.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sun, 17 Aug 2014 12:15:12 -0700 (PDT)

> Maybe because it was defined too late,

The Received: field was initially defined in RFC 821 in August 1982. MIME was
first published as RFCs 1341/1342 in June 1992, almost a decade later.

> and not in the right place? (RFC 5321 is about SMTP not about MIME)

Which is, in fact, the right place, given that it's a transport trace field,
and we're talking about actions taken by the transport.

In fact MIME has almost nothing to do with any of this, other than providing
an example of what not to do.

> From the parser's point of view the reason is
> 5321 says:

> information or formats in those header fields."

Which says nothing about possible dangers of extracting a single bit of
information in a robust way.

> I certainly would not consider the partial parsing approach you suggested

Nonsense. I'm talking about searching for a pair of adjacent tokens in a
string. The parsing risks are false positives - unlikely given the tokens
involved - and false negatives - also unlikely given that anything generating
these tokens is going to be new code written to conform to these
specifications.

And as Arnt has pointed out, you also have control over the agent generating
the line you care about most of the time.

Actually, the risks of parsing issues are dwarfed by the other issues with
using this field: That it is present because of the message envelope, not the
header, that it was added to an incompliant message in order to make it
transport-safe, that the field wasn't added when it should have been, and so
on.

Those are the reasons you're better off examining the message yourself
and not trusting other agents. And all of those reasons apply equally to
any new header field we would defined at this point.

> 1) the Received: header is already used and abused in many ways

You have yet to provide a case for that.

> 2) as you admitted, there are a lot of agents that don't create it properly

See above.

>    the Received: header (it is meant to be a trace field)

We're not talking about charset information.

>    defined field with bogus syntax in historic emails sent before the

You really think there's enough of a chance of "with smtputf8" appearing
the Received: fields of mail created with dates after the publication of
the EAI specifications that this represents a significant risk over and
above the other manifest risks of using the field?

You appear to be operating in a world entirely different from my own. I
therefore have nothing further to say to you about this point.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 18 Aug 2014 05:03:50 +0200

> the line you care about most of the time.

Someone else this time, I think, although I've said the same thing in
the past. (My two cents: Parsing Received is simple and easy, making
sense of the parsed information often is neither.)

There is another aspect of this that I feel hasn't been thoroughly
discussed, namely the likelihood that signalling 6532 compliance in a
different way would be more (or less) reliable than the current way.

What I mean is that it certainly would be possible to say e..g "
"Mime-Version: 1.1 means 6532". But lots of people ignore
Mime-Version, including Gmail. So is there an argument that some other
way to signal 6532 compliance would be more reliable, more likely to
be set and used correctly, than the current way?

Arnt

---

From: Tony Finch <[email-masked]>
Date: Wed, 27 Aug 2014 16:02:40 +0100

> recipient addresses, which is not present in a message itself.

That is what the BCC: header is for (before message submission, if not
at other times).

> (as a Return-Path header field).

Right.

> Other examples are RFC 3461 data (RET, ENVID, NOTIFY, ORCPT).

I think it was a mistake not to specify in-header versions of those
directives.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Trafalgar: Cyclonic in northwest, otherwise mainly northerly or northwesterly
5 or 6. Slight or moderate. Showers in northwest. Good.

---

From: Ned Freed <[email-masked]>
Date: Wed, 27 Aug 2014 10:31:46 -0700 (PDT)

> That is what the BCC: header is for (before message submission, if not

Not really. RFC 5322 is pretty clear about this:

   The "Bcc:" field (where the "Bcc" means "Blind Carbon Copy") contains
   addresses of recipients of the message whose addresses are not to be
   revealed to other recipients of the message.

The problem is using Bcc: this way before submission conflates other cases,
e.g., the expansion of a mailing list maintained by the user agent, with the
one where you *really* don't want the other recipients to know about that
blind-carbon. So when the message actually gets sent, there's no way to
tell from the stored copy whether or not to generate separate copies containing
an indication of their Bcc: status.

Of course a lot of things don't implement proper Bcc: semantics anyway,
so this behavior ends up being "good enough". But let's not pretend it isn't
problematic in some cases, because it isn.

And as for after submission, such usage is problematic in the extreme. It
can easily lead not only to regaruly recipients becoming aware of blind
carbon recipients, it exposes their blind carbon status. It also can
conflict with legitimate uses of Bcc:

And in both cases the field fails to capture other per-recipient envelope
semantics.

> I think it was a mistake not to specify in-header versions of those

Actually, this was discussed at length and after the implications became
clear a decision was made not to do it.

ENVID is the best example. Support you define a field to be used on final
delivery, let's call it final-envelope-id. The problem is that lots of
agents resubmit messages with their headers more or less intact. Now
consider what happens when such an agent picks up such a message and
submits it to infrastructure that doesn't implement NOTARY. The result
can easily be to deliver a message with a completely bogus final-envelope-id
field.

Such leakage is very common with return-path fields, so there's running
code saying it will happen. But this is a tolerable situation with
return-path, where (1) We're not dealing with an identifier and (2) Enough
delivery agents generate the field. 

This is not to say there's no way to log the envelope-id in the header.
The place to do that would be as a new clause in the Received: field.
But the NOTARY work was done long before RFC 2821 came out and clarified
the syntax and extesnion model for Received: fields. And defining such a
model was far beyond the purview of the NOTARY WG.

Of course times have changed and nothing prevents us from defining
envelope-id (and orcpt) clauses for Reeived:.

As for RET and NOTIFY, I don't really think it makes the cost-benefit
cut for addition to Received:. And if you really need to carry the
envelope around at this level of detail elsewhere, that's what BSMTP
is for - in fact one of the reasons for specifying it and requiring
NOTARY support in it was for exactly this application.

				Ned

---

From: Mark Martinec <[email-masked]>
Date: Thu, 28 Aug 2014 10:46:08 +0200

Tony Finch <[email-masked]> wrote:
>> That is what the BCC: header is for (before message submission,
>> if not at other times). [...]
>> I think it was a mistake not to specify in-header versions
>> of those directives.

In addition to what Ned Freed said, in case of mail forwarding
(alias/forward, not resent) where a transition MTA replaces one or more
recipient address with a new set, e.g. final mailbox destinations,
or replacing a role address with final recipients), there is no place
for mucking with Bcc or Cc or To header fields. These must remain
intact, just like with a passage of a message through a mailing list.

   Mark

---

From: Chris Newman <[email-masked]>
Date: Fri, 29 Aug 2014 13:07:38 -0700

> I think it was a mistake not to specify in-header versions of those

In the case of ORCPT, there's an Original-Recipient header defined in RFC 3798:

  http://tools.ietf.org/html/rfc3798.html#section-2.3

		- Chris