Subject: [EAI] I-D Action:draft-ietf-eai-rfc5335bis-05.txt

From: [email-masked]
Date: Fri, 03 Dec 2010 08:45:01 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Email Address Internationalization Working Group of the IETF.

	Title           : Internationalized Email Headers
	Author(s)       :  . Yang, S. Steele
	Filename        : draft-ietf-eai-rfc5335bis-05.txt
	Pages           : 15
	Date            : 2010-12-03

Full internationalization of electronic mail requires not only the
capabilities to transmit non-ASCII content, to encode selected
information in specific header fields, and to use non-ASCII
characters in envelope addresses.  It also requires being able to
express those addresses and the information based on them in mail
header fields.  This document specifies a variant of Internet mail
that permits the use of Unicode encoded in UTF-8, rather than ASCII,
as the base form for Internet email header field.  This form is
permitted in transmission only if authorized by an SMTP extension, as
specified in an associated specification.  This specification updates
Section 6.4 of [RFC2045] to conform with the requirements.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-eai-rfc5335bis-05.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 03 Dec 2010 20:47:12 +0000

1) In Section 4.3:

uQuoted-Pair is defined twice:

 uQuoted-Pair   = ("\" (VCHAR / WSP / UTF8-non-ascii )) / obs-qp

and

 uQuoted-Pair   = ("\" uText) / obs-qp

I think the first version is correct, so the second needs to be deleted. 
However, looking at this made me thing: do we really need 
backslash-escaping of UTF-8? Firstly, in email without EAI a single 
character was always escaped, so I don't know how parsers are going to 
behave for multibyte UTF-8 sequences.
Secondly, do we really need escaping for UTF-8? UTF-8 is already allowed 
elsewhere unescaped. So maybe it would be easier just to forget about 
escaping of UTF-8.

If people agree, then the production uQuoted-Pair needs to be removed, 
together with all productions that reference it directly or indirectly.

2). I had the following comment on -04:

  description  = "Content-Description:" unstructured CRLF

  The <utext> syntax is extended above to allow UTF-8 in all
  <unstructured> header fields.

The problem with this is that <unstructured> is not referencing <text>.
RFC 5322 defines:

  unstructured    =  (*([FWS] VCHAR) *WSP) / obs-unstruct

So should this document also extend <unstructured> to allow UTF-8?

The latest version replaced that with:

  description    = "Content-Description" ":" *uText

First, I think CRLF is missing at the end of the new definition.

But I am also wondering if this document should actually be extending 
<unstructured> anyway, as it is used by many other header field. So:

   unstructured =/ *([FWS] (VCHAR / UTF8-non-ascii) ) *WSP
                           ; extending <unstructured> from RFC 5322, 
Section 3.2.5

?

---

From: John C Klensin <[email-masked]>
Date: Fri, 03 Dec 2010 16:28:32 -0500

--On Friday, December 03, 2010 20:47 +0000 Alexey Melnikov
<[email-masked]> wrote:

(Speaking as someone occasionally considered to be an email and
internationalization expert, not as co-chair)

> I think the first version is correct, so the second needs to
> be deleted. However, looking at this made me thing: do we
> really need backslash-escaping of UTF-8? Firstly, in email
> without EAI a single character was always escaped, so I don't
> know how parsers are going to behave for multibyte UTF-8
> sequences.

Well, the escape mechanism would have to be very carefully
defined to apply to Unicode code points, no matter how
represented in the UTF-8 stream, not subsequent octets.  That
definition is not clear in the existing documents, IMO.  For
most implementation purposes, that definition is equivalent to
"convert to UTF-32 (or to UTF-16 and be _really_ careful about
surrogates), apply escapes, convert back if needed".  That is
actually fairly convenient for BMP/UCS-2 (UTF-16 without
surrogates) or UTF-32 native systems, but isn't going to be easy
to say in the spec.

> Secondly, do we really need escaping for UTF-8? UTF-8 is
> already allowed elsewhere unescaped. So maybe it would be
> easier just to forget about escaping of UTF-8.

Well, I tried to raise a piece of this in my review.   If 

    "foo\ bar"@example.com
is permitted, then one either has to prohibit embedded spaces
(and, more generally, all C0 and C1 characters plus space) or
one has to permit 
    "fú\ bar"@example.com
I don't see any way to take another position that doesn't lead
to a world of pain.

My preference is to ban spaces, C0, and C1 characters from
local-parts, but I think we can't get rid of escapes without
that ban. 
 
>...

Depends on whether we think Content-Description is free text or
a protocol element, doesn't it?  I would normally argue for
"protocol element" but the definition of Content-Description
seems pretty clear that it is just a narrative description and
that argues for UTF-8, probably unrestricted (no escapes either)
and terminated only by the line terminator. 

> First, I think CRLF is missing at the end of the new

yes.  and that becomes very important as we move toward
permitting embedded non-ASCII characters.   Note too that we
need to be _very_ careful in this area to avoid prohibiting
strings that consist of mixed ASCII and non-ASCII characters
because those strings are essential for most
Latin-character-based writing systems and languages.
 
> But I am also wondering if this document should actually be
> extending <unstructured> anyway, as it is used by many other
> header field. So:
> 
>                            ; extending <unstructured> from RFC
> 5322, Section 3.2.5
> 
> ?

    --john