Subject: [idn] Do you want ASCII forever?

From: Dan Oscarsson <[email-masked]>
Date: Sun, 21 Apr 2002 12:09:01 +0200

I do not know if the IDN mailing list is going to shutdown
or if the IDN working group is going to move forward with new things?

Before that I would like to know if most of you want to
have ASCII forever, or if you want to move to UCS as the 
character encoding to be used for interoperability?

I have since the beginning of the IDN working group wanted UCS
to the only character encoding to be used between systems.
But from the discussions on the list I have felt there is a big
group not wanting to leave ASCII.

IDNA will result in increased complexity. I have written software
handing the decoding of MIME and URLs. I very much dislike the
mess where every small part of a text line have to first be parsed
into parts, the each part have to be decoded using different
methods. ACE, %-encoding, quoted-printable,...
The world would have been so much simpler if everybody
had used UCS. Why not at least make that the goal and try
to get rid of the "encode on top of ASCII"?

Looking at some of the more important areas: DNS, SMTP, HTTP
and HTML they could be fixed fairely easy.

- SMTP could add a negotiate in startup switching default
character mode to UCS NFC in UTF-8 (requiring
all headers to be UTF-8 and all default for all text).

- HTTP version 1.2 could require all URLs and headers to
be UCS NFC in UTF-8.

- HTML 4.02 could require all URLs to use the same character
set as the rest of the document or use %-encoded UTF-8.

- DNS can use something built on UDNS, though due to having
IDNA it will need an overhead as it will have to query
the server if it can handle UCS (would not have been needed
if we had started with UDNS).

But it requires people willing to take a big step forward. 
I would very much like to help in defining the standards 
but so far I have felt there is too much "use ASCII forever".
IDNA will not bring non-ASCII to my host names in the near
future. It probably will bring it to my web browser, but it
would have worked fine with UTF-8. And I already see all
the failings where URLs end up being displayed %-encoded
instead of using native characters.

What does IETF want?
I think it is high time for a major step forward in
character set interoperability.

Regards,

   Dan Oscarsson

---

From: Florian Weimer <[email-masked]>
Date: Sun, 21 Apr 2002 15:13:33 +0200

> character encoding to be used for interoperability?

UCS not an encoding.  Even today, it is possible to use the Universal
Character Set in many contexts without sacrificing compatibility with
ASCII transporting agents.

> I think it is high time for a major step forward in character set
> interoperability.

Using UCS is not necessarily a step forward in this direction.

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 21 Apr 2002 18:45:39 -0700

Dan Oscarsson <[email-masked]> wrote:

> I would like to know if most of you want to

Speaking as a relative outsider to the IDN working group and its
charter, I see this as a loaded question that completely misrepresents
the whole debate between ACE and UTF-8.

Neither ACE nor UTF-8 "is" the UCS to the exclusion of the other.  UTF-8
is an official Character Encoding Form for the UCS (Unicode), while the
ACE concept in general and Punycode in particular were designed
specifically for the IDN project.  But that has nothing to do with ACE
proponents "not wanting to leave ASCII."  Anyone who is involved in IDN
is, almost by definition, involved because he or she wants to see a way
to use non-ASCII characters in the DNS.

Both ACE and UTF-8 are designed to represent all the characters of the
UCS, while at the same time maintaining compatibility with some legacy
character set.  In the case of UTF-8, the set with which compatibility
is maintained is the entire 7-bit ASCII code space; in the case of ACE,
it is the subset of ASCII used in the existing DNS.

Remember that the entire purpose for inventing UTF-8 in the first place,
ten years ago, was to allow Unicode to be used in Unix and other
ASCII-based operating systems, given their assumptions and
estrictions  -- not a very different situation from inventing ACE to
allow Unicode to be used in the DNS, given *its* assumptions and
restrictions.

Remember, too, that IDNA requires more than just ACE (or UTF-8).  It
also requires nameprep, which is several times more complex than ACE
and, in any event, refutes the argument that existing UTF-8-aware
browsers and e-mail clients could handle a UTF-8-based IDNA with no
change.

> methods. ACE, %-encoding, quoted-printable,...

Of course IDNA will result in increased complexity.  Did anyone honestly
believe it would be possible to add support for an almost limitless
repertoire of characters without some added complexity?

> to get rid of the "encode on top of ASCII"?

I agree that life would be easier if the rest of the world had
cooperated in advance.  But they didn't (and they still aren't now, at
least not completely), and IDN is intended to work in the world that
actually exists, not the one we wish we lived in.

> if we had started with UDNS).

Look at all the changes you are proposing.  Then look at all the
difficulty e-mail clients and Web browsers are having just keeping up
with the standards that already exist.

Timeliness does matter in this project.  While we are sitting here,
debating (again) whether IDNA should use the "perfect world" approach
that requires all these other upgrades, companies are already selling
"internationalized" Web addresses based on proprietary technologies.
Developing a solution that is philosophically cleaner but requires the
rest of the world to make changes will only slow things down.  It will
not speed them up, and it will not result in a justifiably simpler
solution for end users.

> What does IETF want?

According to the charter, IETF wants "internationalized access to domain
names."  There is no mention of Unicode or UCS to achieve this goal, and
although it would be a bit silly to try to achieve the goal *without*
Unicode, that does not imply in any way that UTF-8 or another standard
Unicode CEF must be used to do so.

The charter also says, "A fundamental requirement in this work is to not
disturb the current use and operation of the domain name system, and for
the DNS to continue to allow any system anywhere to resolve any domain
name."  That pretty much rules out changing the rest of the world to be
UTF-8 conformant.

I, too, look forward to the day when the world uses Unicode as
universally as it uses ASCII today (i.e. when the set of systems that
"do not use Unicode" can be described as simply as we currently say "IBM
mainframes" to describe the systems that "do not use ASCII").  But we
aren't there yet, and IDN cannot be the tool to force everyone else to
get there.

Besides, haven't we already had this debate?

-Doug Ewell
 Fullerton, California

---

From: "Edmon Chung" <[email-masked]>
Date: Sun, 21 Apr 2002 23:10:21 -0400

----- Original Message -----
From: "Doug Ewell" <[email-masked]>
> aren't there yet, and IDN cannot be the tool to force everyone else to

The biggest problem with this "aren't there yet" attitude is not preparing
or provisioning for IDN to "get there".  If we do not start somewhere, we
will never "get there".  That is the point.  Not "we want an ideal world
now", but "lets work on the future AS WELL AS the past".

Edmon

---

From: "Doug Ewell" <[email-masked]>
Date: Mon, 22 Apr 2002 08:23:34 -0700

"tedd" <[email-masked]> wrote:

> Who won?

I thought it had been determined that IDN would go with the ACE-based
approach, and that Punycode would be the specific ACE to be used.  Did I
miss something?

-Doug Ewell
 Fullerton, California

---

From: "Edmon Chung" <[email-masked]>
Date: Mon, 22 Apr 2002 12:11:24 -0400

----- Original Message -----
> approach, and that Punycode would be the specific ACE to be used.  Did I

My impression is that IDNA, Nameprep and Punycode are proposed and tabled to
be accepted, but there is also somewhat interest in moving towards 8+ bit
DNS in the future, and a proposal to extend the UDNS concept would be
helpful in that direction.  So you perhaps did miss something...

(btw, I know I missed out on the discussion of the name punycode, but I
think it really undermines all the effort put into creating it and arrogant
to call it "puny" after two years of discussion by a lot of engineers around
the world.  Perhaps something like "Genicode" might be better, representing,
Genie - small but powerful (more positive than puny), Genius, and also
retains the close sound to unicode.)

Edmon

---

From: "Adam M. Costello" <[email-masked]>
Date: Tue, 23 Apr 2002 00:36:55 +0000

Edmon Chung <[email-masked]> wrote:

> I know I missed out on the discussion of the name punycode, but I
> think it really undermines all the effort put into creating it and
> arrogant to call it "puny" after two years of discussion by a lot of
> engineers around the world.

There were roughly equal numbers in favor and against the name Punycode.
As primary designer of the algorithm, I took it upon myself to break the
tie.  :)

I don't see how it could appear arrogant for a work to call itself
"puny".  If anything, it might appear just the opposite (humble),
although I don't intend for it to appear either arrogant or humble, just
a bit whimsical.

AMC

---

From: "Doug Ewell" <[email-masked]>
Date: Mon, 22 Apr 2002 21:04:17 -0700

For what it's worth, I hate the name Punycode too (for several reasons).
I haven't said much about it, though, because I didn't want debate or
criticism of the name to be mistaken for debate or criticism of the ACE
approach itself or Adam's algorithm.  Besides, it's very likely that
only developers (not the general public) will ever be exposed to the
name.

-Doug Ewell
 Fullerton, California

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 23 Apr 2002 09:08:12 -0400

I have to apologize again not to raise this issue earlier, but I think it is
important.  Not to take anything away from you Adam, but there could be a
better name I think, than punycode.

----- Original Message -----
From: "Adam M. Costello" <[email-masked]>
> As primary designer of the algorithm, I took it upon myself to break the

So now there are two more against it.

> although I don't intend for it to appear either arrogant or humble, just

Puny is arrogant because it is a degrading word and for a good piece of
work.  I really dont mind a bit of humor, but this is not really humorous,
it is a joke and there is a small but distinct difference.

What were the alternatives that were considered? Yes puny means small, but
the name should signify both smallness and greatness.

I think Genicode (read as Genie-code) maintains the shortness, compactness
and the sound to unicode, and also have the genie in a bottle whimsical
character.

It also specifies the Genes of I"DNA"!

thoughts anyone?

Edmon