Subject: Re: [idn] IDNA: is the specification proper, adequate, and complete? (was: Re: I-D ACTION:draft-ietf-idn-idna-08.txt)

From: John C Klensin <[email-masked]>
Date: Fri, 21 Jun 2002 15:54:21 -0400

--On Friday, 21 June, 2002 15:03 +0200 Erik Nordmark
<[email-masked]> wrote:

>> Because we have tried that approach, mostly inadvertently,
>> many times before, and it has always gotten us into trouble.
>> "You are not permitted to do this" leaves us in a position
>> where we can later make it permissible and give it an
>> interpretation at the same time.  "It is undefined" seems to
>> invariably create a community of people who are sure what
>> "undefined" means --i.e., who assign a meaning to it-- and
>> who then become a serious, installed-base, impediment to any
>> other interpretation.  And "this applies globally" will be
>> taken as the basis for a claim that we are making a seriously
>> incompatible change if we try to do something else in some
>> other name space.
> 
> John,
> 
> I'm not saying that leaving things undefined is the way
> forward. But you seem to be saying, in addition to wanting
> things to be well defined, that there needs to be an attempt
> to either predict the future or only apply IDNA in the
> smallest possible context.

Let me restate the second alternative.  I believe there are
several different ways to do it, but

	* At this time, IDNA should be applied only to
	well-understood cases
	
	* Both it, and any alternatives, should be prohibited
	elsewhere, leaving future cases for future specification.
	
	* We have a procedure for loosening the rules --
	applying IDNA or other conventions elsewhere -- in the
	future if needed.

is the essence of the suggestion.

> I think there is a 3rd alternative which is to, unless
> explicitly overridden elsewhere in the future, apply it
> everywhere. This can be perfectly well-defined as far as I can
> tell and it means that any changes can be done with the
> well-defined basis as a foundation.

Actually, I suggest we have never been able to do that and that
it is exactly the source of my concern.  We try very hard to
avoid getting into situations in which we standardize something,
let it get deployed, and then say "sorry, we are changing that,
you must do something else in this particular case".  IESG has
almost never approved such things, and IETF action has mostly
failed in the marketplace when we have tried to do so.
Installed base is important, installed base backed by our
standards is irreversible.

>> 	* RFC821 specified Internet email transport in terms of
>> 	ASCII.  It didn't explicitly (enough) prohibit sending
>> 	non-ASCII ("8bit") characters.  [...]
> 
>> 	* RFC 1034 and 1035 define labels in terms of ASCII
>> 	characters, but contains more or less vague language
>> 	about how labels containing octet values that cannot be
>> 	ASCII are to be interpreted.  [...]
> 
> The above two examples seems to me to be cases of loosly
> defined specifications. I agree that we want to avoid that in
> the IDN space to the extent possible.

I agree that they are loosely defined.  And that more precise
definition is better.  But it seems to me that precisely
defining something to cover things that are not invented yet is
not the way to go unless it is necessary and we have a clear
picture.  I.e., I am trying hard to avoid having to predict the
future or to constrain it.

>> 	As just one example of this, suppose we come along later
>> 	and want to put Unicode more directly into the DNS.  We
>> 	know that UTF-8 isn't a particularly efficient encoding
>> 	-- its design is strongly influenced by the need to be
>> 	compatible with octet-based systems.    We also know
>>...
 
> I think the place where this discussion belongs is the now
> expired draft-ietf-dnsext-unknown-rrs-02.txt. But I note that
> it is lacking as well in that it silently seems to assume that
> all the owner names are ASCII text by saying:
>    The owner name is
>    still set to lower case.

Yes, probably, although I haven't looked at that draft.  I'm
happy to have that discussion anywhere you direct.  My concern
is that the IDN WG, as a side-effect of a passing phrase in a
document that does not appear to have been carefully examined in
the DNS community and that isn't supposed to impact the DNS, is
imposing a major constraint on future DNS use and expansion.
That doesn't seem right to me.

> I agree it is highly desirable to be able to define new RR
> types and/or classes without the ASCII case insensitive
> comparison of the owner names. But the feasibility of this is
> a function of what existing DNS implementations do with
> unknown types and classes. If it turns out that this isn't
> possible one can still envision efficient encoding of Unicode
> owner names e.g. by applying the Bootstring algorithm (used by
> punycode) but have the output code set be e.g.
> 0-0x40,0x5b-0xff i.e. avoid using bytes which an ASCII case
> insensitive comparison would trip on.
> 
> The alternative seems to be to embark on discussions whether
> RR type X (for different values of X including e.g. SRV,
> NAPTR) benefits enough from IDNA to make that RR type be
> covered by the IDNA specification. I fail to see how this WG
> can have expertize in determining the usage of all defined RR
> types. Even if the WG had that expertise I still wouldn't be
> suprised there were radically opposing judgements about
> "benefit enough from  IDNA" for some types.

Ah, but you see... that is the point.   With the current
wording, at least as I read it, the WG is trying to do exactly
that, and to do it without looking at the RRs involved.  But I
can live with even that.   Let's assume the text said something
more like "IDNA can be applied to all currently-defined and
registered RRs in Class=IN, but is not to be applied to
subsequent RRs or other Classes unless their definitions
explicitly say so".  I wouldn't be wild about that, but, perhaps
unlike Eric Hall, I'm also not willing to lose sleep over it.

But, if we are ever going to be able to extend the DNS in ways
that impact the name space, we must avoid setting a standard
that applies to those spaces where we haven't ventured.  So, it
seems to me that the right thing to do is to deal with the RRs
we have (if necessary, sweepingly and all at once), and to
prohibit the application of IDNA --or any set of rules more
restrictive than "binary"-- until rules and methods of
interpretation are specified by those new definitions.   They
might well apply IDNA.  I would expect that, at least within
Class=IN, most would.  But let's leave ourselves in a position
to late-bind that decision.

     john

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 21 Jun 2002 22:05:15 +0000

John C Klensin <[email-masked]> wrote:

> But, if we are ever going to be able to extend the DNS in ways that
> impact the name space, we must avoid setting a standard that applies
> to those spaces where we haven't ventured.

Any future spec that defines a new RR type or a new class (or any new
region) can stipulate that IDNA does not apply there.

Yes, people are reluctant to make breaks like that, and they should be.
They should do it only if there is a compelling reason.

If we set the default the other way, so that future RR types and
classes need to explicitly adopt IDNA, we might encourage unnecessary
disharmony.

AMC

---

From: John C Klensin <[email-masked]>
Date: Fri, 21 Jun 2002 20:04:18 -0400

--On Friday, 21 June, 2002 22:05 +0000 "Adam M. Costello"
<[email-masked]> wrote:

>> But, if we are ever going to be able to extend the DNS in
>> ways that impact the name space, we must avoid setting a
>> standard that applies to those spaces where we haven't
>> ventured.
> 
> Any future spec that defines a new RR type or a new class (or
> any new region) can stipulate that IDNA does not apply there.
> 
> Yes, people are reluctant to make breaks like that, and they
> should be. They should do it only if there is a compelling
> reason.
> 
> If we set the default the other way, so that future RR types
> and classes need to explicitly adopt IDNA, we might encourage
> unnecessary disharmony.

Adam,

I think I've just understood the philosophical difference here.
Assuming I've gotten it right now, I apologize for not
understanding and identifying it sooner.

To me, IDNA is a kludge.   If we were designing the DNS (and
Internet applications) for the first time today, and it was
clear to us that "full internationalization" (whatever that
means) was desirable, IDNA is almost certainly not how we would
do it.  It is a really clever kludge, one that responds well to
the constraints we have because we are not starting fresh, but
it is a kludge.  And making a kludge the default for future
cases is just not a good idea.

Please note that I am not proposing the LDH be the default, or
the unrestricted binary be the default, or that "make up your
own prefix and do strange things to yourself" be the default.  I
am suggesting that we should create a "no default" situation, in
which new RRs (and, more important, new Classes) are required to
specify how the octets in their labels are to be interpreted,
and what syntax restrictions might be applied to them.

With regard to "encourag[ing] unnecessary disharmony", this is
an area where I have a lot of confidence in the IESG --not just
this IESG, but predictions I think can safely be made about
future IESGs.  If someone comes along and says, well, I'm Bob,
and I want to introduce a new coding scheme into the DNS called
IDNB (for "Bob"), he isn't going to survive the laugh test.  I
imagine that "this doesn't require internationalization" or "I
need to invent a new scheme" will get really intense scrutiny,
as they should.  But no sweeping defaults.  Especially no
sweeping defaults that apply not only to future classes but to
existing ones where there has been absolutely no review and
discussion by this WG on whether or not there is an adverse
impact.  That isn't  just incorrect from a design standpoint, it
is procedurally incorrect.

       john

---

From: Dave Crocker <[email-masked]>
Date: Fri, 21 Jun 2002 20:14:32 -0400

At 03:54 PM 6/21/2002 -0400, John C Klensin wrote:
>you must do something else in this particular case".

At the end of the movie "War of the Roses", Danny DeVito (playing an 
attorney) says that he is going to offer his potential client some free 
advice, and "when a $1000 per hour attorney offers free advice, you should 
listen."

When John and I agree about an action, the rest of you folks should listen.

To use a whimsical legal phrase, folks are talking about unringing a bell.

The working group needs to focus more narrowly:  we have exactly ONE DNS 
PROBLEM that must have immediate solution:

         Permitting an internationalized form of domain name for business
         cards -- that is, for URLs and email.  These are usually called
         hostnames.

         All other IDN requirements can wait.

So far, the IDN working group has produced nothing that is used.  If the 
working group issues a first, narrow specification, we can make immediate 
progress and offer the community something it desperately needs.  We can 
also learn more about how this all works and then apply it to the next, 
ummmmmmm,,, domain.

>future or to constrain it.

correct on all counts.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 23 Jun 2002 00:38:05 +0900

----- Original Message ----- 
From: "Dave Crocker" <[email-masked]>
> also learn more about how this all works and then apply it to the next, 

Do you suggest that the new narrower specification should be about

  1)  transitional directory solution that may become the precursor to the real IDN with "NEW CLASS" ?  or
  2)  the real IDN limited to hostnames  and its supporting RRs (NS,A,MX,CNAME) ?  
 
If future "new class" should have type A and type NS records like "IN" class,  approach 1) is more 
appropriate in this stage. some commercial IDN TLD registries already implemented this transitional
directory by wildcard label (*) resolutions for native 8bit A-type queries. They redirect such incoming
native 8bit A-type queries into a forwarding webserver which maintains registrants-supplied
ASCII URL address with no ACE hostname. Moreover, the forwarding webserver can extract some default
locale information from the http header information from the brower client and make most correct
guess about what the local charset was and transcode the 8bit hostname into UCS one which is then 
compared against UCS-to-URL databases . I think this directory feature is enough for most desperate
consumers for a while  and IMO, forever.

Soobok Lee

---

From: Dave Crocker <[email-masked]>
Date: Sat, 22 Jun 2002 09:25:41 -0700

>Do you suggest that the new narrower specification should be about

"new" is not a word I would use, right now.  I am talking about cutting 
away the generalities from the current work and leaving the portion that 
applies to usage for the immediate need.

>   2)  the real IDN limited to hostnames  and its supporting RRs 

The idea of defining something that is a transition, without already having 
the the end-point of the transition also defined and in the process of 
being deployed is simply not practical.

When "transition" takes 5 years and more, it is counterproductive to think 
of transition as temporary.

The work done here is to define the IDN and an encoding scheme for 
IDN.  Perhaps there will be more encoding schemes and perhaps there will not.

The only prediction about future IETF work and future Internet development 
that has proved accurate is that predictions prove to be inaccurate.

d/

----------

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 23 Jun 2002 02:43:14 +0900

----- Original Message ----- 
> of transition as temporary.

The directory-for-8bit-query approach for IDN need not be declared as "transitional", 
rather may be independant one which may be later obsoleted smoothly by "new class" solution.
Non-ASCII octets in "IN" class  has no defined comparison rules and that opens 
rooms and rationales for per-registry comparison rules that can be implementable only in
a directory approach.

Each TLD registry can decide which one it deploy among "directory" approach, "new class", or both.
Both can coexist for two differnt audiences with/without "new class" supports,respectively, IMO. 
Please correct me if it is impossible or problematic.

Such directory approach allows each TLD registries to adopt its own comparison rules, thus
non-monolithic normalization rules and works better in dealing with charset interoperability
and look-similar/identical character problems. Such rules or experiences would be implementable
as multiple registrations or normalization rules in "new class" IDN in the future 
when  stringprep,UNICODE and local charset standards become mature and stablized.

Most TLD registries feel strongly the need to add  native labels in *both* UTF8 and local charsets,
because existing applications  blindly  make 8bit queries. If those queries are not served, 
end users experience failures. Current ACE architecture requries every end user to install
IDN-plugins to enjoy IDN  safely, but that is not the one that end users and registries
desperately need "right now". The directory approach can fulfill these needs clealy and safely.

Soobok Lee

---

From: Dave Crocker <[email-masked]>
Date: Sat, 22 Jun 2002 12:39:54 -0700

At 02:43 AM 6/23/2002 +0900, Soobok Lee wrote:
>Each TLD registry can decide which one it deploy among "directory" approach,

A decision to use an 8-bit encoding scheme will be rather useless, since 
there is no standard specification for doing it.  The absence of a standard 
means there is no interoperability.

>Such directory approach allows each TLD registries to adopt its own 
>comparison rules,

Oh?  Different rules for each TLD?  What about different for each level in 
the hierarchy, too?

You appear to be intent on making architecture choices that ensure long 
term non-interoperability.  having many choices creates an architecture 
that does not scale.

>Most TLD registries feel strongly the need to add  native labels in *both* 

"local charsets"?  Whatever does that mean?

And UTF-8 is merely an encoding scheme.  It is not the "native" 
representation of the character set.

For that matter, I suspect most TLD registries neither know nor care about 
the technical details of UTF-8 vs. ACE, or the like.  They merely want a 
functioning, standard, interoperable IDN.

>  The directory approach can fulfill these needs clealy and safely.

By the way, the DNS is not a directory system.  This has been explained 
many times.

d/

----------

---

From: "James Seng" <[email-masked]>
Date: Sun, 23 Jun 2002 08:57:34 +0800

Hmm...it seem to me that Lee is proposing something new, a directory
approach which includes various features etc. Therefore, I suggest Lee write
down his ideas in a new internet draft as a basis for further discussion.

-James Seng

----- Original Message -----
From: "Dave Crocker" <[email-masked]>
To: "Soobok Lee" <[email-masked]>
Cc: "IETF idn working group" <[email-masked]>
Sent: Sunday, June 23, 2002 3:39 AM
Subject: Re: [idn] IDNA: is the specification proper, adequate, and
complete? (was: Re: I-D ACTION:draft-ietf-idn-idna-08.txt)

> >Most TLD registries feel strongly the need to add  native labels in
*both*
> >UTF8 and local charsets,

---

From: Dave Crocker <[email-masked]>
Date: Sat, 22 Jun 2002 18:28:57 -0700

>down his ideas in a new internet draft as a basis for further discussion.

yes, that would be helpful.

d/

----------

---

From: John C Klensin <[email-masked]>
Date: Sun, 23 Jun 2002 13:51:11 -0400

--On Saturday, 22 June, 2002 12:39 -0700 Dave Crocker
<[email-masked]> wrote:

>...
> For that matter, I suspect most TLD registries neither know nor
> care about the technical details of UTF-8 vs. ACE, or the like.
> They merely want a functioning, standard, interoperable IDN.

Not to put too fine a point on it, but most users of applications
and the data associated with TLD registries neither know nor care
whether the get their "functioning, standard, interoperable
'IDN'" in the DNS or somewhere else entirely, just so long as the
mechanism meets they needs and is "functioning, standard, and
interoperable".

    john

---

From: [email-masked] <[email-masked]>
Date: Mon, 24 Jun 2002 18:57:36 +0800

Dear  Soobok:
                     Your good suggestion reflects some exised facts and it
is very interesting.
                     The directory-for-8bit-query approach is currently used
in .CC , .NU, .TW . It is independent
on 8 bit or ASCII . The 8 bit approach is based on UTF-8 resolver  of  Win2k
OS . After the "*"  "match to any others" ,  The simulated  UTF-8 NS server
or a redirected  A RR web server can  redirect the client to get an ACE
name to do regular resolving .  By the PUNY code or RACE code approach , the
nameprep are solved in the simulated DNS server side.  Always, it is out of
scope in this working group to discuss  equivalence comparison of non-ASCII
code point, so it is a local issue based on TLD . This approach only work
for a DNS subtree .
                     Even it is safe without new plug-in to client , but
there are some other problems. One is that  need more effort on web proxy
servers to treat native to UTF-8 conversion . Another  important
interoperability protocol issue is the virtual host of web page , that is
how to pass what kind of identifier (native , utf-8 , ACE or others) to let
a web server to act as the "virtual host" of web page and  will guide the
client  re-direct to the target web page.

L.M.Tseng
> Such directory approach allows each TLD registries to adopt its own
comparison rules, thus

---

From: "Edmon Chung" <[email-masked]>
Date: Mon, 24 Jun 2002 07:27:26 -0400

You could add .bz and .tv to the list of TLDs that allow 8bit query,
although we do not use a redirection service but utilizes a dynamic CNAME
mechanism to provider the requesting application with an ACE domain so the
delegated host can host the domain with only the ACE RR.
Edmon

----- Original Message -----
From: "[email-masked]" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; "Dave Crocker" <[email-masked]>
Cc: "IETF idn working group" <[email-masked]>
Sent: Monday, June 24, 2002 6:57 AM
Subject: Re: [idn] IDNA: is the specification proper, adequate, and

> > as multiple registrations or normalization rules in "new class" IDN in
the
> future
*both*
> > end users experience failures. Current ACE architecture requries every
end
> user to install

---

From: "Soobok Lee" <[email-masked]>
Date: Mon, 24 Jun 2002 21:21:41 +0900

Thanks for your interests.

My directory-for-8bit-query approach is just a short-term kludge and I hope we can
prove that  it  serves well  as a seamless migration path to  NEW CLASS IDN.
I hope we could publish  the directory approach as a small part of the entire picture of
the standardization roadmap to new class IDN which appears to  require
 more years of research and preparations  from our experienecs in this WG.

I am writing  a draft  about the minimal specification of the directory-for-8bit-query approach which
seems simple and straightforward. Most challenging part of my work,  is  how to prove that  the roadmap is correct
and would not place any obstacles in the future new class IDN efforts.

My comments continues..

----- Original Message -----
> there are some other problems.

> One is that  need more effort on web proxy
> servers to treat native to UTF-8 conversion .

the conversion may be done on the central forwarding webserver which can gather
more contexts from the incoming http requests. Some http proxy maybe strip the 8th bit off from
the non-ASCII HOST: header values into ASCII ones.  Even in those cases, the original
hostname often can be restored by the forwarding server with simple  "OR with 0x80" operations.
Moreover, Hostnames may not be in UTF8 and their  legacy encodings may not be explicitly specified
in the http headers.  Such brower behaviors have been well documented grouped by vendors , OSes and versions
which context information is, fortunately , available in the incoming http request headers.

> Another  important

Current HTTP/1.1  is still bound to the restriction imposed by ASCII hostname rules
over Host: header. Therefore, To be faithful to the current standards,
 the forwarding webserver SHOULD accept  ASCII-only URL
which contains no native-encoded IDN and no ACE hostname  in my suggestion.

If we assume future new class IDN will have  "A,MX,NS,CNAME..." like "IN" class,
IDNA and future new class IDN are mutually exclusive options.

Soobok Lee

---

From: [email-masked] <[email-masked]>
Date: Fri, 28 Jun 2002 00:21:09 +0800

Dear Soobok:

From: "Soobok Lee" <[email-masked]>
> and would not place any obstacles in the future new class IDN efforts.
      The new class IDN means an isolated subtree of DNS with the same root
, so you need an interface or some trap scheme to let different class RR can
be treated by the current class with difference data type (for ex. UTF-8 or
legacy encoding). It is like to use "bq--"  as an explicit indicater in
domain  name .
> which context information is, fortunately , available in the incoming http
request headers.
        We have some  8bit pass-thru  squid proxy server in our testing bed
. The name from IE browser  pass to proxy server port is legacy encoding
via it  passed thru to resolver in Win2K is  UTF-8 . Althought IE has one
option to always send with UTF-8 from input address bar , but it can not
pass the internal range check of  local native code in IE browser.  The
current situation is legacy encoding is valid passed thru by proxy port.
Many legacy encoding is not always using leading bit  in each byte  (The
second byte of BIG5 , JIS may be 7 bits).
> IDNA and future new class IDN are mutually exclusive options.
    To be backward compatible in ASCII encoding of web servers, the
directory-for-8 bit-query  will need "resulting  name replacement " property
in client side.
An ASCII URI to replace the 8bit  new class name  is one of the solution,
the other is to use the  ASCII keyword. May be the ACE encoded name is
another solution.
    Draft of UNAME has been described the approach to let the  legacy or
UTF-8 character comparing without to care TC/SC  and will replace 8 bit
domain name  to the ACE (or ASCII) identifier.

L.M.Tseng