Subject: Re: Is it good to publish a new RFC on SMTP without internationalisation?

From: Dan Oscarsson <[email-masked]>
Date: Thu, 16 Mar 2000 14:05:30 +0100 (MET)

>> I think you, Matti Aarnio and Perry Metzger all think that I want to
>> start using UTF-8 without any compatibility to the current
>> ASCII only world. I do not!
>
>	You do.  You CAN'T introduce UTF-8 in ADDRESSES.
>	You can (with similar means to 8BITMIME) do it in
>	message BODY (and perhaps in headers).
>
>	SMTP transport on  MAIL FROM / RCPT TO handles about
>	ADDRESSES, those you can't modify to use UTF-8 encoded
>	characters.

Why not? Of course you can have an address that contains non-ASCII.
We are going to have domain names containg non-ASCII. We must
be able to send e-mail to those domains too!

>
>> Doing it this way with ESMTP there is no problem with interoperability.
>> If a MTA using UTF-8 talks to an old MTA not supporting the
>> UTF8 option of ESMTP, it has to encoded domain names, headers etc
>> using ASCII encodings. Just like is done today when a MTA with
>> a n 8 bit body connects to a MTA without 8BITMIME, it has to
>> convert into a 7-bit MIME body.
>
>	Sure, but what is the point in having UTF-8 encoded DOMAINS in
>	use at all, if you have to support both those, and compatibly
>	encoded ones ?  They would mean DIFFERENT lookup keys on the DNS,
>	after all..

Yes, unfortunately DNS have to support a compatibility encoding of
names to handle old software.
But the compatibility encoding is not the future. The future is UTF-8.
It will take a long time, unfortunately, before ASCII compatibility
encoding can be turned off, but UTF-8 can be the majority fairely
soon. IETF has stated that UTF-8 must be supported in protocols.
Even SMTP must change.

   Dan

---

From: Matti Aarnio <[email-masked]>
Date: Thu, 16 Mar 2000 16:57:39 +0200

On Thu, Mar 16, 2000 at 02:05:30PM +0100, Dan Oscarsson wrote:
> We are going to have domain names containg non-ASCII. We must be able
> to send e-mail to those domains too!

	To imitate Senator Gato:
	   Abuse of the DNS as ``The Directory'' must be stopped!

	To have something productive out of this "can do/can't do", please
	make a proposal of a method for handling the compability mode, and
	conversion rules related to it.

	If the method works, an EHLO capability ``UTF8 ADDR'' (to invent
	a name) can be added quite easily.

	Orthogonal to that is  ``UTF8 MIME'', which is analogous to 8BITMIME.
	(or ``UTF8 ADDR MIME'' to cover both in "dense" form.)

	That having said, adding a paragraph to SMTPUPD in sense of:

		While default rules for SMTP command lines allow only
		7-bit characters, it may become necessary in the future
		to have some EHLO capability announced method to carry
		e.g. UTF-8 [RFCnnnn] encoded UNICODE characters.

	might be worth it to make implementors aware that even this
	fundamental looking rules may change.

	For that matter, I think the DRUMS documents need a chapter, or
	perhaps additional INFORMATIONAL document about implementation
	notes, and possible future development directions SMTP/Email
	implementors should be aware of.

-- 
/Matti Aarnio	<[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Thu, 16 Mar 2000 10:09:19 -0500

Dan,

what you propose would break email interoperability for the entire planet.

Keith

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Mar 2000 22:02:44 -0500

> 	 fundamental looking rules may change.

If I recall, there are several statements in 821bis (smtpupd) already whose essence is that standardized and negotiated extension options can alter almost anything.  More could certainly be added if
the WG believes that is important (suggestions as to _exactly_ what should be said and where would be appreciated).  But I suggest that, even after I cleaned some of them out with -11,  one of the
problems with that document is the number of times essentially the same thing is said and, given that, I think we should be careful about adding more of them.

> 	 implementors should be aware of.

My perception is that the WG, and the editors, are exhausted and that we should conclude the existing documents as efficiently as possible without adding additional tasks.   If there is interest and
energy to prepare additional or supplemental notes, I'd recommend putting together a draft, or at least a detailled outline, and then discussing how to get input on it and process it with Chris
and/or the ADs.

   john

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Mar 2000 22:24:49 -0500

> be able to send e-mail to those domains too!

At the risk of stirring up a lot of trouble, I would turn this around and suggest that, if one can't figure out how to reliably and consistently send email to non-ASCII domains and/or addresses, then
there is little point in putting non-ASCII characters in the DNS and some other solution for internationalization should be sought.   

"Reliably and consistently" in this case includes "gets to intended recipient" and "does not get to unintended recipient".  I believe that those criteria imply that the entire range of
canonicalization and comparison problems must be solved and that the community must agree on a solution and mechanism (or, at the risk of massive non-interoperability, but knowing what went wrong,
tag or negotiate the mechanism(s) being used).  This is, I hope obviously, a lot more than just stuffing UTF-8 encoded characters in selected places and getting them through the transport system.

> soon. IETF has stated that UTF-8 must be supported in protocols.

As Dave Crocker, I,  and others have suggested, your estimates of how rapidly this sort of thing can be deployed to all of the MTAs and MUAs in the world are, from our experience, much too
optimistic.  And "the majority" is arguably not nearly enough.  Interestingly, one of the worked examples we have in this area is that it took over a decade to deploy the DNS itself to enough density
of use that we could safely assume that host names didn't have to be available in host tables and that MX records would be examined and processed [at least nearly] correctly.  And that was with an
Internet that was much smaller, much more cooperative, and with a much higher percentage of users and system administrators who could be counted on (or persuaded) to install system upgrades because
protocols had changed.   

And, has Paul Hoffman has pointed out, you (and others) are, in your enthusiasm, fairly severely misreading RFC 2277.  It doesn't say what you seem to think it does.  Even if it did, one could be
fully conforming by inserting language into 821bis that explicitly identifies the components of forward and reverse paths (as well as command verbs, etc.) as protocol elements.  Some people clearly
believe that, and not "ASCII because it is/was good enough", was the original intent of 821 and its predecessors.  That was for many of the same reasons that non-ASCII (or, if you prefer, non-ISO 646
BV, non-IA5, or non-IA4) characters are prohibited in many places in protocols from organizations that are historically and obviously more sensitive to national and regional interests (as a goal more
important than even interoperability) than the IETF has been.

    john

---

From: "D. J. Bernstein" <"djb- "@cr.yp.to>
Date: 18 Mar 2000 07:11:49 -0000

John C Klensin writes:
> the entire range of canonicalization and comparison problems

I once used a quoted triple space in my email address on a business card,
and most people mistyped it as a single space. Disaster! What conclusion
do you draw? Pick one:

   (1) We need some standard ``canonicalization and comparison'' rules,
       requiring MTAs to treat any number of spaces the same way.

   (2) We have to prohibit quoted spaces, because doing #1 correctly is
       obviously too difficult.

   (3) I'm being sarcastic. In reality, people stick to a very limited
       character set on business cards, so this problem doesn't arise.

---Dan