Subject: [EAI] A potential collision free local part ACE ID

From: "Chris Walker" <[email-masked]>
Date: Fri, 16 Nov 2007 04:01:28 -0700

John,

Didn't you write in RFC 3696 the following:

   Without quotes, local-parts may consist of any combination of
   alphabetic characters, digits, or any of the special characters

      ! # $ % & ' * + - / = ?  ^ _ ` . { | } ~

   period (".") may also appear, but may not be used to start or end the
   local part, nor may two or more consecutive periods appear.

_____________________________________________

If this is correct, a local part in the form .joebloggs is not legal, 
and an ACE identifier could use that namespace, as in:

[email-masked]

Where .yn-- is the identifier that indicates an ASCII encoded local part.

Per RFC 3696 [email-masked] can 
not be a valid email address, is this correct?

Chris

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: John C Klensin <[email-masked]>
Date: Fri, 16 Nov 2007 06:11:23 -0500

--On Friday, 16 November, 2007 04:01 -0700 Chris Walker
<[email-masked]> wrote:

>    Without quotes, local-parts may consist of any combination
> of    alphabetic characters, digits, or any of the special
> characters
> 
>    period (".") may also appear, but may not be used to start
> or end the    local part, nor may two or more consecutive
> periods appear.
>...

>> If this is correct, a local part in the form .joebloggs is not
>> legal,  and an ACE identifier could use that namespace, as in:
>> 

Yes, I probably did.  But 3696 is not a standard.  More
important is that "without quotes" part.  While we give mail
system operators the advice to avoid creating local-part
addresses that require quoting (largely because the quotes get
lost or fouled up sometimes), nothing prevents them from doing
so.  

In particular, while I don't know if it still exists in the
wild, I knew a system some years ago that used to embed file
paths in addresses by startng them with periods and considered
the fact that many unknown and potentially hostile systems would
trash such addresses to be an advantage.

Conversely, and perhaps more important, the fact that this form
is treated as invalid makes it inappropriate for use as a
transparent ACE because precisely the legacy systems you are
concerned about will tell to reject it in transit.  Because it
is a syntax error, rejecting it in transit doesn't even violate
the "don't tamper with the local-part" rule.  Other systems,
citing the robustness principle, will accept it anyway and treat
it as an address rather than an ACE.

Finally, I don't want to be the one who tells an end user that,
despite the fact that she thinks her email address is something
intelligible in her normal language and script, her real
address, which she needs to give her friends who don't use that
script, is 
   [email-masked]

I suspect you don't either.

Sorry, but I don't think this works.

    john

---

From: "Chris Walker" <[email-masked]>
Date: Fri, 16 Nov 2007 05:27:17 -0700

The form 

is not for submission to an email system, it is only for use in ASCII 
data proccing systems, it would have to be converted back to UTF8 to be 
submitted to a mail system. The .yn-- signals to the system that is 
transistioning from ASCII to UTF8 that the local part needs to be 
decoded back to UTF8.

if [email-masked] is not 
restrictive enough how about any of the following, none of which should 
be legal.

[email-masked] 
[email-masked] 
[email-masked] 

And as I'm sure you are aware in reference to your comment about quoted 
local parts, ".joebloggs" does not start with a period character, it 
starts with a double quote character. 

Again none of these forms are suitable for submitting to any type of 
mail server. They just represent a internationalized email address when 
the address is considered data in a data processing system. We just need 
an ACE identifiter that can't exist in a real email address, to 
determine that a value needs decoding.

If none of these forms seem safe then finally the form

yn--
JiM5NjA7JiM5NDU7JiM5NjE7JiM5NDA7JiM5NDg7JiM5NDk7JiM5NTM7JiM5NDc7JiM5NTY7J
iM5NDU7QCYjOTYwOyYjOTQ1OyYjOTYxOyYjOTQwOyYjOTQ4OyYjOTQ5OyYjOTUzOyYjOTQ3Oy
YjOTU2OyYjOTQ1Oy4mIzk0ODsmIzk1OTsmIzk1NDsmIzk1MzsmIzk1NjsmIzk0MjsgIA==

Where the ENTIRE email address including the local part, the @ sign, and 
the domain part, with an encoding system that does not use the @ sign in 
the encoded output, such as base64, would result in an encoding that in 
no way could collide with a real email address.

Chris

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 16 Nov 2007 10:15:53 -0500

On Fri, Nov 16, 2007 at 05:27:17AM -0700, Chris Walker wrote:

> submitted to a mail system. 

The way I remember it, xn--CCCCCC wasn't for user presentation, and had to be
converted back to UTF8 to be presented to the user, too.  That worked
out splendidly.

A

-- 
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<[email-masked]>                              M2P 2A8
jabber: [email-masked]                 +1 416 646 3304 x4110

---

From: "Chris Walker" <[email-masked]>
Date: Fri, 16 Nov 2007 13:16:32 -0700

-----Original Message-----
From: Andrew Sullivan <[email-masked]>
To: [email-masked]
Date: Fri, 16 Nov 2007 10:15:53 -0500
Subject: Re: [EAI] A potential collision free local part ACE ID

> converted back to UTF8 to be presented to the user, too.   

The major difference, IDNA was designed and intended to work at the user 
interface level.

ACE encoding of an internationalized email address is not for use at the 
user interface level, and not for use as a form that is accepted by SMTP 
or UTF8SMTP.

>That worked> out splendidly.

Well, it can also be argued that IDNA IS WORKING, TODAY, RIGHT NOW; 
which if every DNS client and DNS itself had to be re-implementeed as 
8bit systems, it wouldn't be. IDNA also transits non DNS systems as 
easily as it transits the DNS system, offering virtually no resistance 
to it's adoption by business.

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 16 Nov 2007 15:34:41 -0500

Dear colleagues,

> The major difference, IDNA was designed and intended to work at the user 

Yes, but it was also intended not to be human-readable.  There are
certainly differences, but I think there's enough analogy that we
ought to take that experience into consideration.
 
> user interface level, and not for use as a form that is accepted by SMTP 

Agreed, but it seems to me that we'll have a hard time communicating
that to people who still think that filtering for four-character TLDs
is ok, or people who think that the solution to English-speaking
"intransigence" over internationalization is to run an alternate root.

My view is that ACE is the newest Internet walled garden: it _will_
leak, and the question is merely how annoying the consequences will
be.  It'd be better not to have it at all, in the long run. ASCII-only
is an unfortunate historical problem to be overcome, not a base state
of encoding to which we must always be compatible.
 
> Well, it can also be argued that IDNA IS WORKING, TODAY, RIGHT NOW; 

Yes-ish.  I can tell you from some bitter personal experience that
this is only for some values of "working".  From the point of view of
many users, it has not worked well.  See John Klensin's comments up
thread.  In particular. . .

> to it's adoption by business.

. . .that "no resistance" is merely "no technical resistance".  As it
turns out, that ease of technical implementation (which I still think
was not a bad decision, please note) has meant all sorts of other
business considerations.  Some of those have been at least awkward to
deal with. (There are other internationalization problems that the
ASCII-only world has had to accept, as well, which may not be
entailments of the ACE approach.  I'm not talking about that right
now.)

I think my remarks are probably becoming repetitive, so this is the last
I'll post in this thread.

A

---

From: Claus Färber <[email-masked]>
Date: Sun, 18 Nov 2007 11:50:46 +0100

Andrew Sullivan schrieb:
> Yes, but it was also intended not to be human-readable.

Well, one of its neglected advantages is that it _is_ human-readable.

Of course, a user can't read "xn--1lqs71d" and know what it means. 
However, it _is_ possible to write down the name or read it over the 
phone. Furthermore, you can easily see that it is different from 
"xn--1lq90i".
The same is not true for "東京" and "北京", especially if the user does 
not even have the correct fonts and the computer displays them as "??" 
and "??".

In other words, the property that an ACE only uses the Latin LDH 
repertoire makes it universally human-readable. Unfortunately, this 
advantageous property of ACEs is neglected by the IDNA specs.

It would make sense for user interfaces to show the ASCII version of a 
label if the computer system or the user lacks the prerequisites to 
handle or understand the Unicode version (i.e. use toASCII or toUnicode 
for display, depending on the character repertoire).

Consider the following situation: Aiko (living in Japan) sends Bob 
(living in the US) a message from her email address "愛子＠東京。日", 
which is her usual email address. Charles (also living in the US) asks 
Bob what Aiko's email address is. Bob has no idea how to read the 
characters. Even if Bob could read them, Charles would still have no 
idea how to type them. However, if Bob's system shows him the address as 
"[email-masked]" (either because it knows Bob can't read 
Kanji or because Bob told the system so in order to give the address to 
Charles), he can tell Bob the address (and it's not even that more 
cryptic than e.g. "[email-masked]").

Claus

---

From: John C Klensin <[email-masked]>
Date: Mon, 19 Nov 2007 07:48:46 -0500

--On Sunday, 18 November, 2007 11:50 +0100 Claus Färber
<[email-masked]> wrote:

> In other words, the property that an ACE only uses the Latin
> LDH repertoire makes it universally human-readable.
> Unfortunately, this advantageous property of ACEs is neglected
> by the IDNA specs.

This has been, as you have probably guessed, generally noticed,
and is discussed to some extent in RFC 4690.

> It would make sense for user interfaces to show the ASCII
> version of a label if the computer system or the user lacks
> the prerequisites to handle or understand the Unicode version
> (i.e. use toASCII or toUnicode for display, depending on the
> character repertoire).

Unfortunately, there are three problems that make this less
useful than it would appear on first glance:

(1) With contemporary operating systems, it is often hard to
know whether or not a character can be displayed. There are
often no system calls to inquire whether a given character will
reach the display or be turned into the dreaded boxes or
question marks.  Even if such calls existed, it is unlikely that
it would be wise to cause the overhead of invoking them.  One
simply sends a string off to the display routine (printf or
equivalent) and hopes for the best.  If one always displayed the
ASCII version when one could not be certain that the native form
would not be messed up, one would basically always display an
ACE except when the application was localized to a particular
language and font set that matched the  script of all of the
characters in the label (note that IE7 applies a variation on
that rule -- I don't like what they do for other reasons, but it
does have some logic).

(2) A large fraction of the argument for IDNs, especially IDNs
at the top level, has come from communities who haven't said "we
prefer to write names in our native script" but from those who
have said "our populations don't read or understand Roman-based
characters".  This is not the right place to argue how often the
latter claim is consistent with the facts: there are places in
the world where it certainly is.  For those populations, an
ASCII-based Internet is essentially inaccessible and forcing
strings to ASCII forms in order to make them more clear to you
or me is a disastrously bad policy.

Those of us who speak and write in languages whose scripts are
of Western European origin need to keep reminding ourselves that
there is nothing that makes those scripts -- particularly the
Roman-based ones-- especially "special" or gives them a right to
primacy. It is the result of a series of historical accidents
that email and domain names weren't defined first in locations
that would have resulted in our now talking about, e.g., KCEs,
TCEs, or HCEs (Kana, Thai, or Hangul-based encodings,
respectively) instead of ACEs (Han-based characters are
deliberately not on that list, although I suppose one could pick
a small subset).

(3) The email problem with ACE-on-the-wire approaches --
approaches that would correspond to the IDNA strategy, but not
necessarily what Chris needs -- is that the rules for what can
appear in an email local-part are so general, and so widely
exploited, that there is no way to design an ACE that would be
automatically recognized as such so it could be decoded without
loss of information.  While one might be able to navigate that
problem, such as by requiring an SMTP extension to say "this is
an ACE and should be interpreted as such" rather than "this
address might be fairly unrestricted UTF-8", we also have a long
history in the ASCII email world of structuring local parts.
The reason for the "no one before the delivery MTA gets to
interpret the local part" rule is because nothing else can tell,
in the general case, whether

[email-masked] is
	* an attempt to specify a routing
	* an attempt to indicate an embedded space in an address
	that is more robust than quoting or
	* just an address.

or whether
/S=bloggs/P=joe/PRMD=moss-covered/ADMD=universitymail
/C=[email-masked]  is
	* a (probably-botched .. my memory is fading) X.400
	address and gateway.
	* a slightly-strange pathname in some operating system
	* an indirect reference to the address associated with
	an X.500 certificate   or
	* just an address

or whether
[email-masked]   is
	* local routing information
	* local classification and filtering information used to
	match the sender on a "match or treat as bad spam" basis
	or
	* just an address

or whether
[email-masked] is
	* just like the above, only with a different delimiter or
	* just an address

or whether
[email-masked]  is
	* routing information that is equivalent to what others
	might write as [email-masked] or
	[email-masked] or even
	"chem fred"@someu.example.edu with, in any case, the
	boundary determined not by delimiter but by fixed-length
	substrings   or
	* just an address

or whether
[email-masked]   is
	* a first-name and last-name combination
	* another attempt at routing
	* the user-friendly form of an X.400 distinguished name  or
	* just an address

Now, if one can't tell by looking at a string (and do so with
complete accuracy), whether it is an ACE or a local part with
internal syntax and semantics, then one has a choice about the
design for non-ASCII local parts:

	* Don't try to play the ACE game.
	
	* Forbid structured addresses, such as those above, if
	there are any non-ASCII characters present.  Or restrict
	the structuring to some fairly simple forms that use
	ASCII delimiters, excluding the substring case (last example
	above) entirely and making things a little complicated
	if the non-ASCII characters are RtoL (see the examples
	and discussion in Martin's recent note).

Offering the user or administrator of a system where non-ASCII
addresses would seem normal a choice between non-ASCII local
parts and functionality seems inappropriate, especially if one
accepts the position that, as we move forward, we should,
insofar as possible, stop treating ASCII as occupying a
privileged position.

> Consider the following situation: Aiko (living in Japan) sends
> Bob (living in the US) a message from her email address
> "愛子＠東京。日", which is her usual email address.
> Charles (also living in the US) asks Bob what Aiko's email
> address is. Bob has no idea how to read the characters. Even
> if Bob could read them, Charles would still have no idea how
> to type them. However, if Bob's system shows him the address
> as "[email-masked]" (either because it knows
> Bob can't read Kanji or because Bob told the system so in
> order to give the address to Charles), he can tell Bob the
> address (and it's not even that more cryptic than e.g.
> "[email-masked]").

For better or worse, if Aiko wants to hear from Charles (and
reads a language that Charles can write), Bob is likely to have
to ask her for an address that Charles can use.  Or Charles is
likely to have to ask her out-of-band.   Remember that you are
assuming that Aiko reads English and is willing to use it to
communicate with Bob and, presumably, Charles.   That is the
hard problem because, if she does not, or is not willing, then
they will either have to decide they don't want to communicate
with her or they will need to either learn Japanese or find
translators.  Conversely, if Aiko does want to carry on
correspondence with people who are ignorant of Japanese, she
would be well-advised to maintain an ASCII address alias.  And,
just to stress again that we are trying to design for a world in
which internationalization means just that, rather than "ASCII
with some tricks for other characters and scripts", if Aiko has
also learned to read and write Hindi and wants to carry on a
correspondence with colleagues in India who don't read or write
English, she would be well-advised to maintain a Devanagari
address alias as well.

If she wants to send the _same_ message to both her Indian
colleague and to Bob and Charles, who are ignorant of each
other's languages, she is likely to rediscover the original
reason why multipart/alternative was invented.  That, however,
is another issue, as is the question of how they respond in a
way that each other can read.

No one has claimed this is going to be easy or completely smooth
when messages (or addresses) cross from one language community
into another.  I think that, for practical reasons having more
to do with content than with addressing, we will actually see
little of that sort of crossover.  People will find languages
that they have in common to communicate (just as they do today),
will maintain mail aliases that match those languages, and will
use aliases that match the content in incoming and outgoing
mail.  Probably, because of the existing prevalence, in
international trade and related communications, of the ASCII
subset of the script used to write English, and because in many
parts of the world, people are better at learning English as a
second language than many native English-speakers are good at
learning any second language (especially ones that use a
different script), a very large fraction of the communications
that cross language community boundaries will be in ASCII, using
ASCII addresses.   But we shouldn't have protocols that make
that choice for them, or make it permanent.

    john

---

From: "Chris Walker" <[email-masked]>
Date: Mon, 19 Nov 2007 09:46:56 -0700

-----Original Message-----
From: John C Klensin <[email-masked]>
To: Claus FÃ¤rber <[email-masked]>, [email-masked]
Cc: 
Date: Mon, 19 Nov 2007 07:48:46 -0500
Subject: Re: [EAI] Re: A potential collision free local part ACE ID
> English, she would be well-advised to maintain a Devanagari

I'm afraid that the concept of maintaining an email address for each 
language or script that one may wish to communicate in the body of a 
message is, how you you say it? A non starter.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 20 Nov 2007 10:58:38 +0900

>message is, how you you say it? A non starter.

How many languages do you speak? How many scripts are used by these
languages? How many languages does the average email user speak and
write? How many scripts are used by these languages?

It turns out that an amazingly high percentage of the world's
population are bi- or multilingual. However, given that it's
mostly scripts, not languages, that count, my estimate is that
on a world-wide average, the number of email addresses needed
would be only two or three. Very far from a non-starter.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Yangwoo Ko <[email-masked]>
Date: Tue, 20 Nov 2007 12:00:51 +0900

> would be only two or three. Very far from a non-starter.

Maybe farther than that.

Though it depends on how successful EAG WG's effort will be, my personal 
experience argues that the average number of "scripts" used in email 
addresses of a person (not the number of email addresses) could be less 
than two. When I was teaching my mother-in-law to use email, one of the 
hardest parts is how to associate capital letters printed on key caps 
with the lowercase letters used in email addresses. Now she always uses 
address books that support Korean alias for email addresses. For such a 
case, one Korean email address is enough. And we have many such cases.

---

From: "Chris Walker" <[email-masked]>
Date: Tue, 20 Nov 2007 04:30:59 -0700

-----Original Message-----
From: Martin Duerst <[email-masked]>
To: "Chris Walker" <[email-masked]>, "John C Klensin" 
<[email-masked]>, "Claus FƒeArber"  <[email-masked]>, 
[email-masked]
Date: Tue, 20 Nov 2007 10:58:38 +0900
Subject: Re: [EAI] Re: A potential collision free local part ACE ID

> would be only two or three. Very far from a non-starter.

While I may have control of the local parts of the email address and 
could have them in many languages, will the company hosting the mail 
server have it registered in all of these languages too?

I think the original example on this thread was Keiko who had a Japanese 
email address UTF8 Japanese @ UTF8 Japanese, I don't understand how the 
suggestion that she has a UTF8 Devangari @ UTF8 Japanese helps anything, 
it actually just makes it more complicated.

Does Keiko have to get an additional email provider to get a UTF8 
Devangari @ UTF8 Devangari address if her existing provider can not help 
in the matter?

Chris