Subject: Re: [idn] Determining equivalence in Unicode DNS names

From: liana Ye <[email-masked]>
Date: Sun, 20 Jan 2002 23:01:01 -0800

> So, this wg decided that we will use one and only one matching rule,
> just  like we decided to use only one character set. Both of these 
> (the rule and the charset) are created in the Unicode Consortium.

The decision of use one and only one matching rule is 
at false, because of there is no ONE rule can deal with 
hundreds of different scripts no matter how strong it 
appears that you defend the stand.  

The charset appears in the form of a table, which gives 
us a better view of these chars and we have a base 
to work with. But when you try to use only ONE rule to 
deal with all of the chars, the rule has to be bias for 
only one type of script, currently Latin.  If we let this 
pass from this IDN wg, then all of us here are arguing
for nothing, as the "Prohibit CDN code points" is the 
only way out suggested by Kenny Huang.   

Isn't the time for the WG to wake up from the "one rule 
for all" scope?

The only way out is to use multiple rules to treat multiple 
scripts, where each rule is identified by a tag. 

Liana

---

From: Patrik Fältström <[email-masked]>
Date: Mon, 21 Jan 2002 09:14:11 +0100

> appears that you defend the stand.  

The DNS uses only one rule for matching. One charset and one matching rule.

What you think about is something different than DNS, and that is discussed
in other forum than this working group.

   paf

---

From: John C Klensin <[email-masked]>
Date: Mon, 21 Jan 2002 22:55:34 -0500

Liana (and others),

Let's try to review how we have gotten here, since I partially
disagree with Patrik (the disagreement leads, however, to the
same conclusion, only more strongly).

--On Sunday, 20 January, 2002 23:01 -0800 liana Ye
<[email-masked]> wrote:

>> So, this wg decided that we will use one and only one
>> matching rule, just  like we decided to use only one
>> character set. Both of these  (the rule and the charset) are
>> created in the Unicode Consortium.

Actually, I don't believe this is correct --or that the language
isn't precise enough-- so let me try to restate it.  The fact
that there is only one matching rule is a consequence of the
design of the DNS: largely because of the binary nature of the
underlying  architecture, the only matching rule that is
possible involves a bit-wise comparison on each octet in turn.
Unless the labels are assumed to be "binary" (i.e., the
case-mapping rule does not apply) that bit-wise comparison is
made under a mask that collapses ASCII upper-case characters
onto their lower-case counterparts.

Now, we can map or transform all sorts of things together before
they get injected into the actual DNS.  But, as far as the DNS
is concerned, there is only one mapping rule, or two if "binary
labels" are handled differently from "character labels", but
that is it.

Are there ways around this?  Of course there are.  The WG has
looked at many of them and given up on them long ago.  For
example:

(i) One could introduce a completely new set of query types for
labels that might contain non-ASCII strings, and apply a
different matching rule for them.  Unfortunately, this would be
hugely complex -- we are having quite enough problems with the
three address-type records/queries we have today -- and might
take forever to deploy.

(ii) One could introduce a single new label and query type whose
sole purpose was to provide a new form of alias -- like CNAME
only with Unicode (encoded somehow) in the label.  The WG looked
at variants on that theme early on and decided to not pursue
them.  In hindsight, the approach essentially implies "within
DNS layering" and has all of the disadvantages of an "above DNS"
layering scheme and few of the advantages.

(iii) One could adopt the "new class" model, using that class to
impose a somewhat different set of matching rules by redefining
the query types.  That was, you will recall, proposed and
rejected (or, more accurately, ignored), I think mostly because
people were concerned about the length of time it would take to
deploy.

(iv) One could try to use EDNS to specify alternate mapping
rules or conventions.  Again, the problem is deployment,
aggravated by some rather complex issues involving caching and
secondary servers.  And, again, the WG considered some set of
options in this group and couldn't get consensus around moving
forward with them.

One thing that all of these --except, possibly, the last-- have
in common is that, while they were permit a _different_ matching
rule, none of them permit per-language or per-script rules.  You
still get only one (or two).

> appears that you defend the stand.  

The rule doesn't deal with scripts at all.  It doesn't deal with
languages either.  It deals only with strings of bits and deals
with them in a mathematically simplistic way.

> only one type of script, currently Latin.

No.  The only "bias" is the ability to use a bit masking
algorithm for ASCII upper-lower case matching.  There are severe
and complex problems in dealing with the range of Latin scripts.

>  If we let this 
 
> scripts, where each rule is identified by a tag. 

And where are you going to put the tags?  Again, the WG has been
around this question many times.  There isn't a logical place to
put the tags unless one further shortens the fraction of the
label-length-space available for user-visible information.
There are difficult problems in doing the tagging from a user
interface basis.  Language (or even "script") tagging would
prohibit many combinations of characters for which there is
clear demand, and the WG has been reluctant or unwilling to
start making the policy decisions that "one label, one script"
implies.  

The bottom line is that, if you really think you know how to do
this, we are anxiously awaiting the Internet Draft.  But it
needs to cover all of those complex cases with mixed-language
labels and scripts that can't be neatly mapped onto languages
and vice versa.

My own belief is that many of these things require, not a
complex set of binary matching rules, but enough of a notion of
distance functions to talk, not about "matching" but about
similarities and differences.  Cyrillic, Greek, and Latin "A"
look similar, and are "sort of" the same letter (same origins,
overlapping pronounciations), but don't "match".  Greek
lower-case omega "looks somewhat like" Latin "w", but it isn't
the same letter (even "sort of").  Really getting TC<->SC
mappings right may require context, distance/probability
functions, or both (the latter if there isn't enough context).
We can posit doing those things in other sorts of systems, but
not in the DNS, even with a broader set of matching rules.

Now, I know you don't like that answer, and that saddens me.
But wishing, and even unhappiness, won't change the mathematics
that underlie DNS matching.

     john

---

From: "Michael Froomkin - U.Miami School of Law" <[email-masked]>
Date: Tue, 22 Jan 2002 10:41:37 -0500 (EST)

Perhaps this is the decision -- if conscious decision it was? -- that it
is time to revisit?

I recognize this is not a small or necessarily easy change to make, but
after following this debate for some time, I'm increasingly persuaded that
it does seem to provide the best long-run solution...and doesn't affect
backwards-compatibility either, does it?

On Mon, 21 Jan 2002, John C Klensin wrote:
[...]
-- 
		Please visit http://www.icannwatch.org
A. Michael Froomkin   |    Professor of Law    |   [email-masked]
U. Miami School of Law, P.O. Box 248087, Coral Gables, FL 33124 USA
+1 (305) 284-4285  |  +1 (305) 284-6506 (fax)  |  http://www.law.tm
                        -->It's warm here.<--

---

From: Dave Crocker <[email-masked]>
Date: Tue, 22 Jan 2002 08:21:58 -0800

>that underlie DNS matching.

Folks,

I encourage you to read all of John's note very carefully and more than once.

Many participants in this working group desire to solve some very real and 
very serious problems.  However many of those problems are outside of the 
scope of this working group AND there is no technical solution that will 
work for the Internet.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Jan 2002 16:10:11 -0500

--On Tuesday, 22 January, 2002 10:41 -0500 "Michael Froomkin -
U.Miami School of Law" <[email-masked]> wrote:

> Perhaps this is the decision -- if conscious decision it was?
> -- that it is time to revisit?
> 
> I recognize this is not a small or necessarily easy change to
> make, but after following this debate for some time, I'm
> increasingly persuaded that it does seem to provide the best
> long-run solution...and doesn't affect backwards-compatibility
> either, does it?

Well, I made the "new class" proposal for a reason, and I still
believe in the reasoning (which you have more or less summarized
above).  The argument against it, to me, is the number of
problems it _doesn't_ solve.  While we could make some changes
in matching/ equivalence models via a new Class, it isn't going
to solve problems with near-matches that might be considered
matching and might not.  It isn't going to solve problems that
really require different actions for different languages.  It
isn't going to solve problems for people who really want phrases
and not invented words.  It isn't going to resolve the insane
problem of which "Joe's Pizza" is entitled to exclusive,
worldwide, use of "joespizza.com".  And it isn't going to solve
a very large number of other problems.  Consequently, whatever
we do, we are going to need other systems --whether "above DNS"
layering or otherwise-- to deal with those issues.

In that context, let's horribly oversimplify things and assume
that we can make a three-way choice as far as the DNS is
concerned.

	(i) We can do some fairly fancy mapping and exclusions,
	then encode IDNs into the DNS.  IDNA (including
	nameprep) is clearly an instance of this; all of the
	other proposals are seeming to me to be nearly
	isomorphic to it as far as the major issues and
	capabilities are concerned.
	
	(ii) We can do the new class stuff, cleaning up some of
	the compatibility and matching issues, but making only a
	marginal improvement over IDNA in the grand scheme of
	things.
	
	(iii) We can prohibit _all_ encoding of non-LDH
	characters in the DNS, relying entirely on the layered
	stuff to do the internationalization job.

Now, to me, the questions then become whether IDNA (or the first
option more broadly) is likely to cause enough damage to push us
toward the second or third.  And, if the answer is "yes",
whether the advantages of the hard-to-deploy second alternative
make it more attractive than the third one.

I don't know.  But my instincts right now --reinforced by the
lack of interest in, and willingness of people to work on, "new
class"-- are that the real choice lies between the first and
third of those options.    And, of course, for whatever it is
worth, "postpone/prohibit CDN" is, I think, just a special case
of the third option: while the problems with Chinese are
somewhat different, I'm convinced that we have somewhat parallel
equivalence or look-alike problems with almost any language or
script we might focus on.

    john

----

>> (iii) One could adopt the "new class" model, using that class
>> to impose a somewhat different set of matching rules by
>> redefining the query types.  That was, you will recall,
>> proposed and rejected (or, more accurately, ignored), I think
>> mostly because people were concerned about the length of time
>> it would take to deploy.
> --

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Jan 2002 17:45:08 -0500

--On Tuesday, 22 January, 2002 08:21 -0800 Dave Crocker
<[email-masked]> wrote:

>> But wishing, and even unhappiness, won't change the
>> mathematics that underlie DNS matching.
> 
> Folks,
> 
> Many participants in this working group desire to solve some
> very real and very serious problems.  However many of those
> problems are outside of the scope of this working group AND
> there is no technical solution that will work for the Internet.

I would have said "no technical solution within the context of
the DNS, or at least within Class=IN, that will work for the
Internet".   But I assume and hope that we agree about that.

    john