Subject: Re: [idn] Thoughts on nameprep

From: "Bruce Thomson" <[email-masked]>
Date: Sat, 10 Mar 2001 17:06:32 +0900

> But if two implementations do it differently, they will look up different
> domain names.
> That's the whole reason.

I understand that is the motivation for nameprep to repair "bad" names
into "good" ones. But my point was that at least for Japanese, and
I suspect for other languages as well, that anybody who has gotten
to the point where they can reliably enter a user name and password
in the language will already be able to enter a domain name.

Microsoft has recently been in the business of going beyond the spec
to rewrite domain names until they resolve. Adding left-off www.s
etc. A bit unsettling for purist geeks like me, but useful I suppose.
They also change lower case letters to upper case when I am typing
documents in MS Word, and I abhor that, because when it isn't
what I want, I can't turn it off.

Basically, it seems to me that correcting a user's mistake when he
types an illegal character in a domain is on the same level. A fuzzy,
friendly, nudge on the elbow to get the user a little further along.
But this is basically AI at work. Lots of heuristics, not too
predictable. It certainly isn't the kind of thing where people demand
consistency. If it works differently on a different browser or not at
all, so what?

This is especially important when you notice that people think that
nameprep is so complicated that it might not be possible to implement
it accurately in different resolvers. So there was even a suggestion
to move it into the DNS for for consistency. This is would be reasonable
if nameprep consistency was a high priority, but it isn't. Burdening
the DNS with this is asking for big trouble.

The important thing that needs to be in the idn spec is just the legal
character set. Even this is likely to change after it is in place. There
are just too many countries involved to expect to get it right the
first time. Priority should be on getting this nailed down, at least
temporarily, and getting this show on the road. Nameprep can
wait.

It would be wise, also, to learn from the mistakes made when defining
other recent specifications. Take, for example, the PCMCIA spec
(now called the PC Card spec.) The spec writers came up with
an extremely impressive, flexible spec that allowed interoperability
between a myriad of different cards and card readers. Unfortunately,
though, it was impossible to implement it completely on any given
product. So manufacturers would implement maybe 20% of it
and declare themselves compatible. The result is the computer
I bought yesterday, years after the spec was created, can only
handle a subset of existing cards. Thanks a lot.

Lets create just the minimum, and get maximum benefit.

Bruce

----- Original Message -----
From: "Harald Alvestrand" <[email-masked]>
To: "Bruce Thomson" <[email-masked]>; <[email-masked]>
Sent: Friday, March 09, 2001 11:41 PM
Subject: Re: [idn] Thoughts on nameprep

> At 12:39 09/03/2001 +0900, Bruce Thomson wrote:
> >Nameprep can and probably will be implemented at the
> >application level. Microsoft already prepends www. and
> >appends .com if I leave them off. They can handle this
> >kind of thing. But what is the pressure to put it in the spec?
>
> as you say, all implementations can, and probably will, handle "this kind
> of thing".
> But if two implementations do it differently, they will look up different
> domain names.
> That's the whole reason.
>
> --
> Harald Tveit Alvestrand, [email-masked]
> +47 41 44 29 94
> Personal email: [email-masked]
>
>

---

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 10 Mar 2001 09:09:53 +0000

Bruce Thomson <[email-masked]> wrote:

> into "good" ones.

Actually, I think the primary motivation for nameprep is to prevent
names that look the same but are actually different (to prevent domain
spoofing).

> character set.

That's not enough, the normalization is important.  Consider, for
example, e with acute accent.  There are two ways to represent it: as a
single character, or as an e followed by a combining acute accent.  If
domain names are allowed to use either form, then someone could register
a domain with one form, and I could register it with the other form, and
when you follow a link to one or the other, your browser will display
a Unicode domain name, and you won't be able to tell by looking at it
whether you're at my site or the other site.

We can't forbid the combining acute accent, because some languages need
it, and we don't want to forbid the precomposed characters, because
we want to use efficient representations.  The remaining option is to
require normalization.

We also need case-folding, because domain names are supposed to be
case-insensitive.

We also need to forbid some characters (I think you already saw that
need when you spoke of a "legal character set").

And that's all there is to nameprep; everything it does can be
attributed to case-folding, normalization, or forbidding characters.
It's actually pretty simple; it just uses large tables.

AMC

---

From: "Bruce Thomson" <[email-masked]>
Date: Sat, 10 Mar 2001 19:53:41 +0900

Hi Adam,

Your argument that accents make nameprep necessary are
sort of convincing, but I have some more questions. Please
bear with me. :)

> we want to use efficient representations.  The remaining option is to
Isn't this sort of like the question of ligatures in English? I used to type
a lot on keyboards with ff, ffi, and other fun keys, and I miss them. :P
Isn't e-acute-accent just a ligature? If we were to just make ligatures
illegal in domain names, would the screams be all that loud?

Not being from France or other country where these are used, I
wonder whether these characters aren't causing a variety of
other problems already, because you can't look at them and tell
how they were typed. If we are going to allow Unicode in text
files in general, variables in C programs with accents would be
confusing, because you could have two seemingly identical variable
names that are in fact different. Also with user names, passwords,
etc. Why do we have to solve this problem for idns alone?

> We also need case-folding, because domain names are supposed to be
We could also just forbid upper case, except for ASCII, which would
get through on a grandfather clause. No mistypings are likely here.
People expect case to matter. Forbidding is simpler than mapping.

> It's actually pretty simple; it just uses large tables.
Well, this at least is reassuring. As an app writer, if I can be
sure that I just map using one table and forbid with a second one,
I wouldn't worry too much. I would just download your tables. :)
Are you pretty confident that the tables are reasonably usable now?

Also, it seems like there are a lot of table entries to solve what you
said was the main problem: a couple of accent ligatures that have
to be mapped into their decomposed forms (or is it the other way?).
It just seems a bit over-engineered...

Bruce

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 10 Mar 2001 22:00:59 +0100

>illegal in domain names, would the screams be all that loud?

(a) No, they are not only ligatures
(b) IETF should not even think about creating rules for what is equal 
or not. IETF should reference what other organizations do, and 
current nameprep references what Unicode Consortium have defined.

I.e. this discussion doesn't belong here.

    paf

---

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 10 Mar 2001 23:08:21 +0000

First, a comment on the nameprep spec.  The main steps are:

    map, normalize, prohibit

If the steps were rearranged to:

    normalize, fold, normalize, delete, prohibit

then the nameprep spec would get quite a bit smaller.  All the "Case
map" entries would disappear, because it's enough to reference UTR21 for
the fold step, and all the "Additional folding" entries would disappear,
because they are taken care of by normalizing both before and after
folding.  The spec would need to list only the mapped-out, prohibited,
and unassigned sets.

Please note that I am not proposing any change to the algorithm itself!
I'm merely suggesting a more concise way to define it.

Of course implementations would be free to transform the sequence of
tables into fewer tables, as long as they ultimately produce the same
output given the same input.  I bet in most cases, though, cleverness
would be more trouble than it's worth.

Now to continue the other discussion...

Bruce Thomson <[email-masked]> wrote:

> Isn't e-acute-accent just a ligature?

No, it's a letter with a diacritic.  A ligature is two or more letters
that are connected for purely typographic reasons, while remaining
logically separate letters, like your fi, ff, ffi examples.

But for the purposes of this discussion the distinction between
ligatures and letters with diacritics is not relevant.  In both cases
you have a choice between the precomposed representation and the
decomposed representation.

> If we were to just make ligatures illegal in domain names, would the
> screams be all that loud?

I don't see how that would help.  Depending on how the user types, and
depending on the operating system layers between the keyboard and the
application, the application might read e followed by acute-accent, or
might read a single e-with-acute-accent character.  The application
cannot control what the OS does, and should not try to dictate how
people type a particular letter if there are multiple ways (including
cut-and-paste), so it must be prepared to read either representation,
and convert to a canonical representation.

> I wonder whether these characters aren't causing a variety of other
> problems already, because you can't look at them and tell how they
> were typed.

Why should anyone care how they were typed?

> If we are going to allow Unicode in text files in general, variables
> in C programs with accents would be confusing, because you could
> have two seemingly identical variable names that are in fact
> different. Also with user names, passwords, etc. Why do we have to
> solve this problem for idns alone?

We don't, because the Unicode Consortium has already solved it for us,
by defining normalization and case-folding.  :)

> People expect case to matter.

In domain names?  No they don't, because today case doesn't matter
in domain names.  It's not uncommon to see domain names in uppercase
or mixed case for stylistic reasons.  If someone tries to use an IDN
containing some uppercase characters and gets a host unknown error even
though the lowercase version exists, they're going to be surprised and
angry and rightfully so.

> As an app writer, if I can be sure that I just map using one table and
> forbid with a second one, I wouldn't worry too much.

If nameprep were implemented using a straightforward translation of the
spec, there would be one table for mapping, two tables for normalization
(one for compatible decomposition and one for canonical composition),
and one table for prohibition.  There might be a way to optimize that
down to fewer steps and fewer tables, if you're under tight constraints.

> But when I look at the nameprep mapping table there are LOTS of
> entries....  Japanese alone requires over 100 with the current spec,
> and these are unnecessary.

Do you mean the mapping table given in the nameprep spec, or the tables
needed to implement nameprep?  The mapping table that appears in the
nameprep spec doesn't include any Japanese characters.  The tables
needed for normalization do include Japanese characters, but those
tables are defined by Unicode and implicitly referenced by the nameprep
spec.

Maybe the normalization defined by Unicode is overkill for IDN purposes,
but it's already standardized, and off-the-shelf implementations are
likely to exist.

AMC

---

From: "D. J. Bernstein" <[email-masked]>
Date: 10 Mar 2001 23:24:07 -0000

Adam M. Costello writes:
> prevent names that look the same but are actually different

We're going to have registrars prevent this. We're specifying a set of
good names. Freely available registration software will recognize good
names and reject bad names. The whole point is that two different good
names won't look the same. End of problem.

The big question is whether we need to do more than this. Do we need to
allow, in some contexts, bad names that look the same as good names? Do
we need to convert those bad names to the corresponding good names? Why
is this necessary? How will the bad names appear?

Half-width Japanese kana, for example, are bad. Registrars will reject
names that contain half-width kana. But do we need to convert half-width
kana to full-width kana?

Patrik and Martin have repeatedly told us that the answer is yes. Users
will see Japanese domain names in print and type half-width kana into
the computer, Patrik says. They'll expect to connect to the same site.

But Bruce, writing from Japan, says that the answer is no. Users type
full-width kana. Redeploying network software to convert half-width kana
to full-width kana, as Patrik is insisting that we do, would be idiotic.

> It's actually pretty simple; it just uses large tables.

Tables that aren't in current software. Tables that may change. We're
talking about software upgrades on a massive scale, and perhaps further
upgrades on the same scale when Unicode is modified. I'm astounded that
some people are taking such a careless approach to such a large project.

> We can't forbid the combining acute accent, because some languages need it,

Registrars will forbid strings that have an e followed by a combining
acute accent. They'll allow the precomposed e-with-acute-accent.

Now, are you claiming that the usual keyboard interface in some locale
will produce an e followed by a combining acute accent? Please provide
verifiable details so that we can avoid making a Patrik-style mistake.

If there is a problem, why not fix it in the keyboard interface for that
locale? This is fast nameprep: bad names are converted to good names
before they escape the keyboard interface.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 11 Mar 2001 00:51:48 +0000

"D. J. Bernstein" <[email-masked]> wrote:

> How will the bad names appear?

Hostnames will appear in text files and other electronic documents in
a variety of encodings, not just Unicode.  They may have undergone
arbitrary transcoding in the past.  In order to do a lookup on those
hostnames, they must eventually be transcoded to Unicode.  If that
transcoder is not required to output normalization form KC, then it
could very easily produce a bad name.

Not all the precomposed characters that exist in Unicode 3.0 existed in
earlier versions.  There are probably some transcoders still in use that
cannot produce good names because they were written before the required
code points were assigned.  And those transcoders may be perfectly
adequate for their intended purpose, which may be competely unrelated to
domain names.

Therefore, an application that is concerned with domain names would be
wise not to make assumptions about Unicode text that is handed to it,
but instead to perform its own normalization.

> will produce an e followed by a combining acute accent?

In most locales the keyboard interface doesn't produce Unicode at all,
so we also have to consider how the transcoding is performed.

Keyboard interfaces are used to input text for many many purposes.  I
think it would be very presumptuous to think that one narrow purpose,
domain names, should dictate how keyboard interfaces must work.

And as I mentioned above, keyboards are not the only sources of domain
names.

At the end of the day, whether the IETF requires nameprep or not, they
can't hold a gun to anyone's head; people will still be able to write
applications that neglect to do nameprep.  But I bet those applications
will cause headaches for users when they occasionally fail in very
mysterious ways (because the user cannot distinguish bad names from good
names).

AMC

---

From: Patrik Fältström <[email-masked]>
Date: Sun, 11 Mar 2001 10:26:30 +0100

>to full-width kana, as Patrik is insisting that we do, would be idiotic.

I would appreciate you stopping modify and falsify what pepople talk 
about on this mailing list, and I urge the wg chairs to please see 
that the discussions stays focused on what your agenda is instead of 
being dragged into a rathole full of FUD which we currently is 
swimming around in.

I have been saying that codepoint definitions, and mappings, are not 
to be defined by the IETF, but some other organizations like the 
Unicode Consortium.

   paf

---

From: "James Seng/Personal" <[email-masked]>
Date: Sun, 11 Mar 2001 20:37:04 +0800

Dan,

> to full-width kana, as Patrik is insisting that we do, would be
idiotic.

You are basing your argument on a single non-Japanese based in Japan to
make an assumption that Japanese cannot key in half-width kana?

Why not lets hear it out from the Japanese who is on this working group.
I would appreciate if Yoneya-san could step forward to clarify the
situation.

> some people are taking such a careless approach to such a large
project.

Transcoding between legacy and Unicode is a large table. Transcoding may
change as standard evolves. However, that does not seem to prevent
people from using legacy encodings and UTF-8 interexchangely within
applications.

Table is not an issue. Making these tables up-to-date and freely
available is the concern.

> acute accent. They'll allow the precomposed e-with-acute-accent.

And you assumed only Registrars will put names into DNS entries? I have
not realised that DNS has evolved to a stage where it become a property
of Registries and Registrars only.

> before they escape the keyboard interface.

Why do you think people is going to fit their keyboard for just IDN?

Lets see if I can follow the logic.

We want IDN. We want IDN to do UTF-8 and we want easy way (fast
nameprep). Since some keyboard does not do it 'correctly' aka 'the way
we want it', let's fix all keyboard.

Please explain to me how you going to tell keyboard people that you want
them to change something which has been working all the time to fix a
problem in IDN?

-James Seng

---

From: Eric Brunner-Williams in Portland Maine <[email-masked]>
Date: Sun, 11 Mar 2001 09:24:42 -0500

...
> > Registrars will forbid strings that have [some particular property,
> > the specific instance omitted from the original as irrelevant to the
> > mechanism/policy issue.]
> 

What other mechanism(s) do you have in mind? What scope(s) does these
independant mechanism(s) and policy (policies) have?

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 11 Mar 2001 20:57:38 +0000

Eric Brunner-Williams in Portland Maine <[email-masked]> wrote:

> > And you assumed only Registrars will put names into DNS entries? I
> > have not realised that DNS has evolved to a stage where it become a
> > property of Registries and Registrars only.
>

This may just be a terminology issue.  The nicemice.net domain is for
my personal use, and I operate the primary DNS server for it.  I can
add any names I like under it, but I have no plans to register names at
anyone's request.  Am I a registrar?

AMC

---

From: "James Seng/Personal" <[email-masked]>
Date: Mon, 12 Mar 2001 07:04:32 +0800

*sigh* It is this kind of attitude about the whole DNS which has evolves
around ICANN which make the whole so scarely. The moment anything is
been said about outside registry/registrar, they assumed automatically
we are speaking about alternative root with other mechansim and policy
which may threaten their existence.

Hello, there are other names in DNS tree (note, the whole point of DNS
is its hierarchy) which is *NOT* owned by any registries or registrars.

-James Seng

----- Original Message -----
From: "Eric Brunner-Williams in Portland Maine" <[email-masked]>
To: "James Seng/Personal" <[email-masked]>
Cc: "D. J. Bernstein" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Sunday, March 11, 2001 10:24 PM
Subject: Re: [idn] Thoughts on nameprep

> > And you assumed only Registrars will put names into DNS entries? I
have

---

From: "D. J. Bernstein" <[email-masked]>
Date: 11 Mar 2001 23:47:02 -0000

Adam M. Costello writes:
> If that transcoder is not required to output normalization form KC,
> then it could very easily produce a bad name.

Do you have any real examples of this happening? What did the user type?
What software was involved?

Later you claim that there are ``probably some transcoders still in use
that cannot produce good names.'' If you had any real examples, why
would you say ``probably''? I think you're just spreading FUD.

> domain names, should dictate how keyboard interfaces must work.

Have you been paying any attention to Bruce's messages? His point is
that these problems are _not_ unique to domain names. Half-width kana
don't work when users are typing their login names, for example.

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 12 Mar 2001 00:01:14 -0000

> about on this mailing list,

Patrik and Martin on 2001-01-06, 2001-01-07, and 2001-01-08 specifically
mentioned half-width kana as an example of the importance of converting
bad names to good names.

We're supposed to believe that users will be confused if half-width kana
aren't treated the same way as full-width kana in domain names. ``There
are quite a few cases where the degree of determinism and success can be
significantly improved for the end user by additional foldings,'' Martin
said. ``The best example probably is the full-width/half-width folding
for Japanese and some other contexts.''

It now appears that this ``best example'' is bogus. Users won't be
confused at all.

---Dan

---

From: Eric Brunner-Williams in Portland Maine <[email-masked]>
Date: Sun, 11 Mar 2001 19:15:01 -0500

> anyone's request.  Am I a registrar?

From draft-ietf-provreg-dn-defn-00.txt

   Registrar: A person or entity that, via contract with Domain Holders
   and a Registry, provides front-end domain name registration services
   to registrants, providing a public interface to registry services.

Not in the sense of the draft-ietf-provreg-dn-defn-00.txt. In the foo.bar
SLD you may of course import some of the sense of the above (and related)
terms, modify them to your _local_ operational art, and decide that you
are both the (unique) registrar for foo.bar, and the registry operator of
the foo.bar SLD.

Cheers,
Eric

---

From: Eric Brunner-Williams in Portland Maine <[email-masked]>
Date: Sun, 11 Mar 2001 19:04:43 -0500

Mr. Seng,

Just what is it are you attempting to describe?

There was an assertion about the agency performaning a semantic function,
[registrars and diacriticals].

You made some assertion to some purpose, and I'm attempting to find out
just what that assertion is.

Eric

---

From: "Adam M. Costello" <[email-masked]>
Date: Mon, 12 Mar 2001 02:23:12 +0000

"D. J. Bernstein" <[email-masked]> wrote:

> Do you have any real examples of this happening?

As far as I know, lv is the only Unicode transcoder installed on my
machine.  In JIS X 0208 (the main Japanese character encoding) there is
only one character that looks like a small letter mu, and lv sends it to
U+03BC (Greek small latter mu).  In Latin-1 there is only one character
that looks like a small mu, and lv sends it to U+00B5 (micro sign).
Normalization form KC collapses these two characters into one, but lv
doesn't.

Come to think of it, I seriously doubt whether any transcoder would
output normalization form KC by default; if it's going to output any
normalized form at all, it's probably going to use form C, because
that's a better choice for most purposes (but not for domain names),
because it preserves more information.

There may be tools that output decomposed diacritics.  I don't know
any examples, but the combining diacritics are there in the Unicode
character set, and not deprecated, so I don't see how we could be
justified in assuming that no programs will ever use decomposed forms in
documents that might happen to contain host names.

Here's my new argument:  By Murphy's law, bad names will pop up from
time to time.  Even if applications were not required to try to map
bad names to good names, some of them would, to be helpful.  We could
try to forbid applications from being helpful, but they'd do it anyway
rather than annoy their users.  If not all applications perform mapping,
or they don't all do it the same way, then bad names that work in some
situations will not work in other situations, or will map to different
good names in different situations, and users will be mystified and
frustrated.  Much better that a given bad name yields the same result
(be it success or failure) in all situations.  That requires a standard
mapping algorithm.

AMC

---

From: Jonathan Rosenne <[email-masked]>
Date: Mon, 12 Mar 2001 08:53:52 +0200

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf
> Of D. J. Bernstein
> Sent: Monday, March 12, 2001 1:47 AM
> To: [email-masked]

Login names and passwords are a good example. Sometimes case matters, sometimes
it does not.

With DNS, the convention for ASCII is that case does not matter. The parallels
for other languages are things like half-width Kana, normalization and other
things included in nameprep.

Anyone who thinks the requirements of other languages which are similar to case
insensitivity in English are not justified should demand that DNS be changed to
be case sensitive.

> ---Dan

Jony

---

From: "D. J. Bernstein" <[email-masked]>
Date: 12 Mar 2001 08:29:55 -0000

Adam M. Costello writes:
> In Latin-1 there is only one character

Let me get this straight. You're talking about someone whose keyboard
produces Latin-1: in other words, someone who doesn't have a snowball's
chance in hell of successfully typing most IDNs. You think that he's
going to be a happy IDN user if we have his micro converted to mu?

---Dan

---

From: Yoshiro YONEYA <[email-masked]>
Date: Mon, 12 Mar 2001 21:50:48 +0900 (JST)

Hi,

Sorry for short response.

From: "James Seng/Personal" <[email-masked]>
Subject: Re: [idn] Thoughts on nameprep
Date: Sun, 11 Mar 2001 20:37:04 +0800

> I would appreciate if Yoneya-san could step forward to clarify the

Half-width (or 1byte) kana is still used widely in Japan.  One of the
major reason is for backward compatibility.  Still many applications
require half-width kana for input, therefore, still many IMEs produce
half-width kana.  One possible scenario is:

- A user uses legacy application that requires half-width kana.
- He/she inputs half-width kana by 'intelligent IME'.
- The IME studies users preference.
- He/she changes application to Web browser.
- He/she inputs kana by the same 'intelligent IME'.
- The IME produces half-width kana according to studied preference.

And, the same problem will occur with ASCII alpha-numerics and
full-width (or 2byte) alpha-numerics.  It is very hard for users using
proportional fonts to distinguish displayed alpha-numerics is in ASCII
or full-width even in IME dialogue.  I sometimes receive ASCII and
full-width mixed alpha-numerics in e-mail message.

Half-width kana will be obsolete, but full-width alpha-numeric won't.

-- 
Yoshiro YONEYA <[email-masked]>
           aka <[email-masked]>

---

From: "D. J. Bernstein" <[email-masked]>
Date: 15 Mar 2001 11:01:52 -0000

Yoshiro YONEYA writes:
> 'intelligent IME'

Could you please explain this in more detail? How does the typical
Japanese keyboard interface work?

What exactly do you type if you want to connect by FTP to ftp.aol.com
and log in as anonymous with password [email-masked]? What would you
expect to type if aol and the username and the password were replaced by
Japanese names?

Suppose we settle on fast nameprep: it's the keyboard interface's job to
help you type good domain names, so that other programs don't have to
worry about bad domain names. What changes would you make in the
keyboard interface to support this?

> Half-width kana will be obsolete

So, if we take the slow nameprep approach, then in twenty years we'll
have a bunch of networking programs with the useless skill of converting
half-width kana to full-width kana. Right?

As for numbers, Bruce seemed to say that most applications expect ASCII
digits, and that double-width numbers won't work. If that's correct, why
is anybody using double-width numbers?

---Dan

---

From: Bruce Thomson <[email-masked]>
Date: Thu, 15 Mar 2001 22:43:20 +0900

I can help out with the Japanese keyboard explanation. When typing
in Japanese you have a choice of several keyboard input packages
available, of which IME is the most popular. Without IME or equivalent,
you would be limited what is on the keyboard itself, which is basically
your standard ASCII. With IME, you are able to type a whole range of
hiragana, katakana, Kanji, and special symbols, all by typing alphabetics
and then converting.

The default input mode is hiragana, which lets you type sequences of
hiragana
by typing the equivalent alphabetics. For example, type r and a to get the
single hiragana character pronounced ra. You can then either confirm the
hiragana character, add additional hiragana to the sequence, or press
the space bar to convert it into something else. It converts it into what
it guesses is my most likely choice, and I can press the space bar again
to get the next choice. Pressing space a third time gives me a menu with
all possible choices.  With "ra", I get 13 menu choices: the original
full-width hiragana, 4 Kanji, full-width katakana, half-width katakana,
and the alphabetic sequences RA, Ra, and ra, in both full-width and
half-width alphabetics. Whichever choice I make, IME remembers it as my
preferred choice, bringing it to the top of the list of choices next
time I convert the same characters.

IME can be quite annoying in its attempts to be intelligent, remembering
previous choices which were exceptions, or typed by someone else borrowing
my computer, etc. So it's certainly possible for it to encourage mistakes
by leading you to input something you didn't intend.

At this point user names and passwords are probably without exception
limited to ASCII, but depending on what software you use to enter them
it might be possible to accidentally type Kanji, kana, or full-width
alphanumerics. So people learn to be careful.

As to why people still might use double-width numbers (or alphabetics,
for that matter), some people seem to feel that they look better
when intermixed with Kanji text (which is full width). Personally
I don't use them, but some people do. The only reasonable use I have
seen for half-width kana recently is on cell phones, because of the
limited screen size. These non-standard characters are not used
for non-text input such as user names, passwords, credit card
numbers, telephone numbers, zip codes, etc.

If someone gave me the opportunity to work on the Japanese keyboard spec,
I would like to consider trying to wean people off these old characters
by refusing to convert to them (this could be a checkbox option). You
can always type them if you desperately need them by switching keyboard
modes. Currently in addition to the default mode I described above, there
are modes for typing katakana, ASCII, full-width alphanumerics, and
half-width kana directly without the conversion step. I expect that
creators of proportional fonts would then work to make mixtures of
alphabetics and Kanji blend more smoothly.

Bruce

"D. J. Bernstein" wrote:
> eel

---

From: [email-masked] (J. William Semich)
Date: Thu, 15 Mar 2001 10:23:40 -0500

Don't a large number of computer users in Japan (the majority?) use Mac
computers, which support direct input of the local character set from the
keyboard, without IME?

Tech staff here use an iMac running the Japanese version of Mac OS for
Japanese (multilingual) domain name testing purposes just for that reason. 

Bill Semich

---

From: Alfred Novacek <[email-masked]>
Date: Thu, 15 Mar 2001 20:55:16 +0100

[...]
>keyboard interface to support this?

Forcing the keyboard interface to do "fast nameprep" may be a solution for
japanese (I still doubt), it is definitely no solution for most European
languages, because it significantly changes the perception of the average
"Jo User" how DNS (and his computer) works.

Users have become accustomed to the fact that it does not matter whether
they enter letters in domain names in uppercase or lowercase; yet, most
European languages use some "national special characters" as letters, and
these additional letters come in uppercase and lowercase flavours.

In German (I am using this language because I am most familiar with it; but
I had the opportunity to get to know the respective conventions of other
languages in a recent project) these special characters are 'umlaut a'
(lowercase U+00E4; uppercase U+00C4), 'umlaut o' (lowercase U+00F6,
uppercase U+00D6), 'umlaut u' (lowercase U+00FC, uppercase U+00DC) and the
'sharp s' (lowercase U+00DF; no uppercase equivalent).

The average user has got used to the fact that e.g. the domain names
"www.kraeuterpaul.at" and "WWW.KRAEUTERPAUL.AT" are equivalent, and because
they perceive e.g. 'umlaut a' as another letter, they simply won't
understand that "www.kr[U+00E4|uterpaul.at" and "WWW.KR[U+00C4]UTERPAUL.AT"
are not.

Let us see how a keyboard interface doing "fast nameprep" might fit the bill:
(1) You suggest forcing the users to switch "keyboard mode"; well, it is
not hard to predict that European users will forget to do so - simply
because they didn't need to do it so far, and being forced to change habits
just to make an IDN proposal work probably isn't going to be the big
motivational force.
(2) You might suggest to do the translations all the time when something is
entered via the keyboard; well, not long ago, we had a reform of the
orthography of the German language, and - although the original goals were
very ambitious (to greately simplify German orthography in order to make it
much simpler to get it right), the results where only modest changes
(merely because conservative ministers for education thought those original
rules were of great educational value) - and those results are still
debated. A "just write lowercase" proposal - just to get an IDN proposal
working - definitely won't get much (political) support.
(3) There is some mysterious magic how a keyboard interface determines
automatically whether the user is typing a domain name or not; maybe I am
not very imaginative, but it remains mysterious to me how this "magic"
should work (after all, present computers are not very good in doing
something that is not well specified; if they were, we could tell them to
just do IDN right, close down the IDN WG and thus get rid of all those
tedious discussions that come up again and again although a reasonable
solution has been found already.)

But there is another place where we know that we are in fact dealing with
domain names, and nothing else: anywhere in the DNS resolution process -
and the "slow nameprep" proposal specifies to do nameprep just there.

(It may still be debated whether applications are the best place; other
options are DNS resolver libraries and DNS servers; putting any IDN-related
code into just the applications - or any dynamically linked libraries
called by them - has - IMHO - the big advantage that users may upgrade just
them - being immediately gratified by being able to use IDNs, which will
result in a very fast shift to IDN.)

>is anybody using double-width numbers?

All that nameprep specifies is a generic transformation algorithm, with the
actual transformations defined by a set of tables; once those characters
get really out of use, all that needs to be done to remove this "useless
skill" is to replace those tables - no software upgrade necessary!

Sincerely Yours - Alfred Novacek

------------------------------------------------------------------------
Dipl.-Ing. Alfred Novacek
Institute for Data Processing in Business Administrations,
     Economics and Social Sciences
Johannes Kepler University Linz / Austria
E-Mail: [email-masked]
        [email-masked]
WWW: http://www.idv.uni-linz.ac.at

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 15 Mar 2001 22:44:42 +0000

Alfred Novacek <[email-masked]> wrote:

> It may still be debated whether applications are the best place; other
> options are DNS resolver libraries and DNS servers;

If you change only the resolver and/or DNS server, and make no change to
the application, how is the resolver or DNS server going to know what
encoding the application used?  Heck, how is the application going to
know?  For example, I can (and do) use Netscape to view both Latin-1
pages and Japanese pages, but Netscape currently contains no code to
detect (or allow the user to indicate) whether the text entered into the
Location field is iso-8859-1 or euc-jp (because today it's supposed to
be just us-ascii).

AMC

---

From: Bruce Thomson <[email-masked]>
Date: Fri, 16 Mar 2001 09:44:17 +0900

No, Macs have only a 10%-15% market share in Japan. But
the Mac's Japanese-input routines are less trouble-prone
than IME, in my opinion.

Also, IME comes standard with Windows2000. It's only older
platforms where people are using 3rd-party Japanese input
software.

Bruce