Subject: [EAI] Rechartering for Standards Track

From: Shawn Steele <[email-masked]>
Date: Wed, 13 Jan 2010 20:46:12 +0000

I’d like the WG to recharter for standards track ASAP.  Or charter a second effort for standards track if the WG still feels that experimentation is valuable in some areas.  Lack of a standard is blocking improvements in international email support for our products.

Several of our (Microsoft) product teams are interested in extending international support for email in markets like China.  Exchange and Live would like to develop to the standards, and it’d really help if we had a clearer standards track RFC for guidance.  I think that the results of the experimental phase are fairly clear, at least in broad terms, and support the general UTF-8 approach.  Immediately standardizing UTF8SMTP, for example, would go a long way toward allowing us to enable our products, even if it still took a while to finalized downgrade.

As a secondary benefit, if teams like Exchange start seriously trying to adopt UTF8SMTP, as a standard, then additional details may be revealed that would help the WG’s continued investigation.

It’s hard for me to get my company to invest in this area when there aren’t any standards ☹

- Shawn

SSDE
Microsoft
Windows UX

 
http://blogs.msdn.com/shawnste

To start the discussion, here’s something we could use to update the charter?
Description of Working Group:
Since early in the effort to internationalize domain names, which
resulted in the standards associated with IDNA, it has been
understood that internationalization of email address local parts is
required. At the same time, email address internationalization poses
a series of special problems. Constraints on the interpretation of
local-parts by any system other than the final delivery one make
address encoding nearly impossible. The need to use addresses in both
the email envelope and in header fields, and to do so in ways that are
at least compatible, suggests that this is not a simple and isolated
problem.

This working group’s previous experimental efforts investigated the
use of UTF-8 as a general approach to email internationalization.
That approach is based on the use of an SMTP extension to enable
both the use of UTF-8 in envelope address local-parts and optionally
in domain-parts and the use of UTF-8 in mail headers -- both in
address contexts and wherever encoded-words are permitted today.
Other protocols were similarly extended.

The initial set of experimental RFCs validated the general approach
while revealing weaknesses in some areas such as downgrade.
The investigation has reached the point where the usefulness to
end users is blocked by the lack of adoption of the Experimental
RFCs by major mail vendors.  Further progress requires formal
Standardization of the core concepts.

Several of the WG’s experimental RFCs have been proven
during the experimental phase and now need moved to standards
track, which the WG will now do.  Additionally there are some
areas may need more effort or investigation.  The WG may
create either standards track or additional experimental RFCs
as required for those areas.

Standards Track Documents:

RFCs that have been proven and need moved to standards track:

* Internationalized Email Headers (RFC 5335)
* Internationalized Delivery Status and Disposition Notifications (RFC 5337)
* SMTP Extension for Internationalized Email Addresses (RFC 5336)
* IMAP Support for UTF-8 (draft)
* POP3 Support for UTF-8 (draft)

Downgrading has been tricky, so the updated RFCs should be
revised to allow for, but not depend upon, a future downgrade
standard.  Retransmission by legacy protocols should still be permitted.

Unchanged Document:

Overview and Framework for Internationalized Email (RFC4952)
is informational and the basic concepts still apply.

Additional Documents:

Other documents need further work or different approaches.
The WG may continue experimental work in these areas, use
a different experimental approach, or resolve the issues and
agree to a standard track approach.

* Downgrading Mechanism for Email Address Internationalization (RFC 5504)
* Mailing Lists and Internationalized Email Addresses (draft)
* Displaying Downgraded Messages for Email Address Internationalization (draft)
An additional document may be required clarifying the expectations
around multiple Unicode and ASCII addresses for the same mailbox.

Deliverables

The following deliverables are foreseen in this charter. The
WG chairs may (re)structure the deliverables into specific
documents or document sets as needed. Adding or removing
documents outside of these deliverables will require a charter
update.

* UTF-8 SMTP Support (Standard)
* UTF-8 POP Support (Standard)
* UTF-8 IMAP Support (Standard)
* Header format specification (Standard)
* Downgrade (Exp or Info)

Going forward, it is possible that the downgrading specification
will go for Informational due to the difficulty of fully specifying
all necessary behavior.

Additional possible documents suggested:
* Advice for MUA implementors (Info)
* Multiple Unicode and ASCII addresses for the same mailbox (Info or Standard?)
Goals and Milestones:
Feb 2010

Recharter for standards track

Mar 2010

Headers & DSN (Standards)

Apr 2010

SMTP (Standard)

Apr 2010

Coexistence of Unicode & ASCII addresses (Info)

May 2010

IMAP & POP3 (Standard)

Jun 2010

Downgrade?

?

?

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 18 Jan 2010 13:20:16 +0000

Hi Shawn,
Thank you for posting a new charter proposal. This looks like a good 
starting point. Some comments below:

>       Description of Working Group:
 [...]

> as required for those areas.
The last 2 sentences seem like a bit of handwaving. I think we need to 
be either more specific here, or remove this text, as your proposed 
charter already says "need to recharter to add other documents".

> standard.  Retransmission by legacy protocols should still be permitted.
Can you elaborate on what you meant in the last sentence?

> is informational and the basic concepts still apply.
I think there were some earlier discussions about updating the document, 
for example to talk about UTF-8 username/passwords used in EAI POP/IMAP. 
So I would rather have an option to update this document as needed.

John can comment more on this.

Regards,
Alexey

---

From: Shawn Steele <[email-masked]>
Date: Tue, 19 Jan 2010 01:29:13 +0000

> Thank you for posting a new charter proposal. This looks like a good starting point. Some comments below:

I should point out that I'm hoping someone else'll drive the rechartering :)

>> Several of the WG’s experimental RFCs have been proven during the 
>> experimental phase and now need moved to standards track, which the WG 
>> will now do.  Additionally there are some areas may need more effort 
>> or investigation.  The WG may create either standards track or 
>> additional experimental RFCs as required for those areas.

> The last 2 sentences seem like a bit of handwaving.

They are :).  As noted, I don't feel like I (should) own the rechartering, but I was being wishy washy because I think that downgrade is hard and needs more experimentation.  However, if the WG managed to make some sort of breakthrough, then I was wondering if we'd really need to be blocked by another recharter. 

So mostly I worded it like this as a discussion point, and it worked, we're discussing it:)

>> Downgrading has been tricky, so the updated RFCs should be revised to 
>> allow for, but not depend upon, a future downgrade standard.  
>> Retransmission by legacy protocols should still be permitted.

>Can you elaborate on what you meant in the last sentence?

Again I'm trying to stimulate discussion, this was a draft.

The current language in some of the drafts/RFC's, kind of assume that there'll be a downgrade of some form.  That needs "tweaking" to permit a future downgrade without actually forcing a dependency that we can't provide yet.  Eg: "currently there is no downgrade, but in the future maybe there will be a downgrade."

That said, I didn't want to say "If your message fails with EAI, you're stuck because there's no downgrade."  I think we need to figure out appropriate language to say that it may be resent with pre-EAI protocols.  (Eg: If you happen to have an ASCII address in your address book, you could retry with that using the pre-EAI protocols.)  Perhaps we'd take that for granted, but I thought it should be called out.  I also don't want to accidentally disallow the case where you can resend in ASCII just because we're pointing to a non-existent downgrade document.

>> Overview and Framework for Internationalized Email (RFC4952) is 
>> informational and the basic concepts still apply.

> I think there were some earlier discussions about updating the 
> document, for example to talk about UTF-8 username/passwords
> used in EAI POP/IMAP. 

That's fine, that makes sense.

-Shawn