Subject: Re: [idn] IDNA: is the specification proper, adequate, and complete? (was: Re: I-D ACTION:draft-ietf-idn-idna-08.txt)

From: John C Klensin <[email-masked]>
Date: Tue, 11 Jun 2002 20:41:15 -0400

--On Tuesday, 11 June, 2002 23:04 +0000 "Adam M. Costello"
<[email-masked]> wrote:

> John C Klensin <[email-masked]> wrote:
> 
>> (i) The specification now appears to say that applications can
>> decide to use IDNA or not.  Presumably, they can decide to use
>> something else instead.
> 
> Paul Hoffman / IMC <[email-masked]> replied:
> 
>> If either sentence is at all true, we need to fix it.  I
>> don't see any place that says that an application doesn't
>> need to use IDNA for non-LDH domain names.
> 
> You're both right.  John must be referring to this sentence in
> section 1:
> 
>     This document does not require any applications to conform
> to IDNA,     but applications can elect to use IDNA in order
> to support IDN while     maintaining interoperability with
> existing infrastructure.

yes

> So applications can indeed decide not to use IDNA, in which
> case they can't use non-ASCII characters in domain names.  If
> you want to use non-ASCII characters in domain names, IDNA is
> your only option.

IMO, that needs to be said, and said right there and very, very,
explicitly.

>> And I certainly don't see anything that would indicate that
>> the second sentence is true.
> 
> Agreed.  We can't stop applications from using custom mappings
> from non-ASCII text onto domain names, but the rest of the
> world would never see the non-ASCII characters.  The rest of
> the world would simply see RFC 1035 domain names, which
> contain only ASCII characters (and possibly octets 80..FF,
> which are not defined to represent any characters at all).

But we can say, clearly, that any such applications or
activities are non-conforming to IETF standards.  And I believe
we should do so.
 
> John wrote:
> 
>> It is also not clear whether "an application", as used in the
>> spec, refers to a standard protocol and expectations about how
>> all conforming implementations will behave or to particular
>> implementations and implementation choices.
> 
> Both, I think.  An individual application can unilaterally
> decide to support IDNA, and a spec can opt to require IDNA
> support.

Ok.  I think it should be more clear.  And I need to think
through the cases on this.

>> But, if an application does a DNS lookup for an LDH name
>> (with no prefix) with, say, an MX Qtype, and some of the
>> result data are returned with IDNA-appearing prefixes, then
>> the application needs additional clues as to how to present
>> error messages to the user, how to continue processing in
>> other areas, etc.
> 
> I don't see why.  If the application conforms to IDNA, it will
> work.  If it ignores IDNA and treats the name like any other
> traditional ASCII name, it will work.

True as long as it doesn't support a prefix-based
interpretation, or a different prefix, with a different
interpretation than IDNA.  I took the text to imply that you
were willing to tolerate such things.  If you don't intend to,
the text has to both make clear that they are non-conforming
and, preferably, to damn them roundly and explain why.

>> (ii) The specification seems inconsistent about what it about
>> and who needs to understand it.  It implies in several places
>> that it is not about the DNS and has no impact on the DNS.
> 
> Paul replied:
> 
>> I can't find any place that makes it not about the DNS.
> 
> Perhaps John is referring to this:
> 
>     IDNA does not require any changes to DNS servers,
> resolvers, or     protocol elements

For starters, yes.  I'll try to go back and find the rest of the
text that set me off.  Or have a look at some of Dave's "not
related to the DNS, just 'micro layer'" comments of the last few
days.

> John wrote:
> 
>> But it then makes and imposes normative DNS operational
>> statements. E.g., "Non-ACE labels that begin with the ACE
>> prefix will confuse users and SHOULD NOT be allowed in DNS
>> zones" is certainly a requirement about DNS and DNS zone
>> population.
> 
> You are quite right.  When we wrote "IDNA does not require any
> changes to DNS servers", the word "require" was intended in
> the sense of "depend on".  I have already suggested changing
> it to "depend on" (here and in similar places in the spec).

That would be _much_ better, IMO.

> Although IDNA does impose this new recommendation on DNS
> servers, it would still work if it didn't.
> 
>> (iii) Despite the "applications can choose to do this, or not
>> do it" language, section 6.4 effectively implies a
>> requirement that any application that uses the DNS be
>> upgraded.
> 
> Paul replied:
> 
>> Every application of DNS should be upgraded for every
>> extension or  improvement of the DNS protocols.  Nothing in
>> 6.4 requires this, as  far as I can see.
> 
> I think John must be referring to this:
> 
>     All applications that might show the user a domain name
> obtained     from a domain name slot, such as from
> gethostbyaddr or part of a     mail header, SHOULD be updated
> as soon as possible in order to     prevent users from seeing
> the ACE.

yes
 
> "SHOULD" is fairly strong, perhaps strong enough that this
> paragraph might be considered inconsistent with this one:
> 
>     This document does not require any applications to conform
> to IDNA,     but applications can elect to use IDNA in order
> to support IDN while     maintaining interoperability with
> existing infrastructure.

yes
 
> Maybe it would be less contentious if 6.4 merely stated the
> benefits of upgrading applications, leaving the reader
> conclude that doing so is a good idea.

Personally, I would actually prefer that this whole IDN package
come with a plan about how we are going to deprecate the
non-conforming.  The trick we tried to use with ESMTP was to
invent the notion of a "contemporary implementation" and then
talk about what "contemporary" (or "modern", or...) did and how
they behaved.  "Legacy" (or "older", or ...) implementations
could still be said to conform to the old standards, but the
strong implication was that they were not up-to-date if they
weren't prepared to deal with the new stuff.  

I think something like that might help here although, regardless
of my other points and issues, this whole situation feels more
and more as if we ought to try to get an AS out that addresses
relationships among protocols, expectations, and quality of
implementation issues.  I wouldn't mind just stuffing all of
that into IDNA, but it probably isn't the right thing to do (or
fair to you guys).

>> (iv) Section 7 imposes several normative requirements on name
>> servers and zone populations.  Again, these requirements are
>> buried in a document that elsewhere appears to claim that it
>> doesn't impact the DNS.
> 
> First requirement:
> 
>     Internationalized domain name data in zone files (as
> specified by     section 5 of RFC 1035) MUST be processed with
> ToASCII before it is     entered in the zone files.
> 
> This follows directly from section 3 requirement 2.  It is not
> a special requirement on DNS, it is a general requirement on
> any application that elects to use IDNs.  Section 7 is simply
> telling how it applies to DNS. It's still true that IDNA is
> optional; if you don't want to support it, then don't, but
> then you have no way to enter non-ASCII names into zones.

This is, to put it mildly, not clear.  Or not clear enough.  At
least to someone who hasn't been immersed in the document.

> Second requirement:
> 
>     a primary master name server MUST NOT contain an
> ACE-encoded label     that decodes to an ASCII label.
> 
> This is a vacuous requirement.  There is no such thing as an
> ACE label that decodes to an ASCII label, because of the
> design of the ToASCII operation.  Therefore, this is not
> really requiring anything.  I think I once asked for it's
> removal, but since it's harmless, I didn't press it.

Recommendation: Leave it there, since it is harmless and may be
helpful to someone.  But rephrase it, not as a "MUST" (or any
other form of requirement), but as an observation.  E.g., "note
that, since the design of the ToASCII operation prevents any ACE
label from decoding to an ASCII label (i.e., one without any
non-ASCII characters), a primary master name server will
never...".  Or something like that.

> Third requirement:
> 
>> The third paragraph of section 7 seems to contradict or
>> repeal the statements of RFC 2181 that non-ASCII strings are
>> permitted in labels
> 
> Good catch.  We'll have to do something about that.

Yes.  Or Randy and kre will kill you. :-)
 
>> (v) In particular, several of the statements in the draft go
>> beyond almost anything we have about the DNS in trying to
>> narrowly constrain the behavior of RR labels and data that
>> are not yet defined, even in Classes that are not yet defined.
> 
> You found one such statement, please feel free (and
> encouraged) to enumerate the others.  :)

I'll go back and look.  I think other comments above identify
places where I simply reached different conclusions about what
you meant than you intended.  I think those things are worth
clarifying -- others won't read the thing even as carefully as I
have, must less be as familiar with it as you are.

But I'd still prefer to see either enumerations of where IDNA
can be used, or some much more clear language about its
applicability.  That gets back to the "applications" issue and
the very broad "all labels" and its implications.  See below.

>> IDNA, by specifying nameprep profiling as part of the
>> procedure, violates this principle by effectively requiring a
>> one-off protocol for situations in which nameprep is
>> inappropriate but other IDNA steps would be reasonable.
> 
> Currently, all domain labels are compared the same way.

This is exactly where I get hung up.  Leaving the more subtle
issue that Eric is trying to explore and some of its close
relatives aside for the moment, let's assume that it is rational
to apply IDNA to all domain labels and associated parameters are
all _current_ RRs.  That may be completely reasonable.  But
suppose that, at some point, and just as an example, some
proposal goes forward to use a different Class for some purpose,
or to use the DNS for a different purpose entirely.  Such
transitions involve exactly the sorts of infrastructure pain and
suffering that IDNA was designed to avoid, but it is legitimate
for the community to conclude that they are necessary.  If we do
decide to accept that pain and suffering, then it would be good,
IMO, to be able to use it as an opportunity to clean out, as
much  as possible, things that were done only to preserve
compatibility.  I'd like to give those future designers the
opportunity to make choices about whether to use IDNA (or ACE
generally) to handle internationalization, the ability to think
through whether more of the normalization or matching processes
should be moved to the server, rather than being handled
exclusively in the client, etc.  I don't want (or need) to
predict what they will conclude, but I think it is unnecessary
and dangerous for us, at this stage, to write what seems to be
an "all internationalization uses IDNA, now and forever,
including in RRs and Classes and uses of the DNS we have no way
to anticipate" rule.

So I think we all need to understand the points Eric is trying
to make.  But, even if they are totally irrelevant or we
conclude that they aren't worth the trouble, I believe that we
should confine IDNA application and interpretation to what we
can reasonable claim we know about today and that we should
_explicitly_ leave future use and applicability for decisions to
be made in the future.

> Allowing multiple profiles amounts to allowing multiple
> comparison functions. Eric Hall disputes this, but if you
> stumble across two labels, one containing uppercase Latin
> letter O with umlaut, the other containing lowercase latin
> letter o followed by combining umlaut, and you wonder whether
> they match, what is the answer?  Yes, no, or maybe?  For ASCII
> domain names, there is no maybe.  We deliberately designed
> IDNA so that the same is true for IDNs.

Understood, but not the point.  

What follows is going to be long, and I apologize, but we have
proof that this isn't easy to explain.

I strongly suspect that the problems/ issues that are driving
Eric are different from mine, so please don't take this as
representing (or preempting) his concerns and cases.  At the
risk of exposing another philosophical problem, let me repeat
(or paraphrase) what I said in Minneapolis: for me, the essence
of a good engineering job is to understand _all_ of the
constraints that define the solution space for a problem and
then either find a solution that fits within that solution space
or be very clear to identify solutions that may not quite fit
and it issues that make up the tradeoffs.   Conversely, I
consider that saying "we will define the problem very narrowly
so that we can solve it and then leave the mess that creates for
the rest of the world to work out" is lousy engineering and
worst standardization practice.

Now, that is just my opinion, but it is the foundation for an
explanation, so bear with me.

My guess is that the whole notion of "the registries will have
to figure this out" is going to fail miserably.  I don't think
that is a valid argument for not publishing IDNA (or any of the
rest of these IDN WG things) -- I am speculating, and this is
the reason we have proposed standards and multiple maturity
levels.  But, to the extent to which LDH has advantages, those
advantages have "worked" because they have been enforced by code
in applications, not just by registries.  If we say "open
season, anything can go in, but some names aren't going to
resolve because the registries will prohibit them" we end up in
a much weaker state.  We also end up with increased load on
servers and, unless the state of negative caching improves a
lot, probably disproportionately more load on the root.  So, I
contend there are big advantages to being able to sanity-check
names before going off to the DNS and that the advantages get
larger in the very complex world in which nearly-arbitrary
Unicode strings are permitted as names.

Now, suppose someone comes along and invents a new RR type.
And, in addition to learning from our trying to over-rely on
registres, the nature of that RR type is that, while some
internationalization is appropriate for its labels, performance
--for both successful and unsuccessful queries-- is a really
critical issue, even relative to DNS norms.  And let's suppose
that, after appropriate analysis, the conclusion is that the
characters that should be permitted in labels for that RR type
should be slightly more than the 63 (or 64) of LDH, but many
fewer than "most of Unicode"... say a few hundreds or, at most,
a few thousand, of select characters.  And, whatever characters
are chosen, they are unambiguous, for both technical and
user-perception definitions of ambiguity.

Now, to accomplish that goal (please let's not get started on
whether it is reasonable or not -- this is both future and
hypothetical and I don't know what would cause such an RR to be
created), the best path, given our current framework, would be
to modify stringprep to add a table of the characters permitted
by that RR.  _Then_ the right thing to do is to create a profile
that uses that table instead of some of the maps and
prohibitions of nameprep.  

I am suggesting that (i) it should still be possible to use
IDNA, but with a different profile and (ii) we should not, as a
side effect of the way we specify IDNA today, prevent that RR
from being defined, regardless of whether it can use
IDNA+different profile or whether it needs to use an entirely
different protocol to set up, compare, and map names.

   john

---

From: "Adam M. Costello" <[email-masked]>
Date: Wed, 12 Jun 2002 03:12:34 +0000

Doh, I got bitten by my own Reply-To header, and accidentally advised
John to send his reply to the mailing list (which he did), even though
my message (which he was replying to) did not go to the list.  Paul is
surely laughing at me now.  :)

Sorry for the extra list traffic.  Anyone who wants to move this thread
back off the list should override the Reply-To field.

[For anyone who's curious, I composed a single reply to two messages,
one from the mailing list, and another (a response to that one) from
off the list.  Since the mailing list address initially appeared in the
recipient list, the Reply-To field was auto-generated.  When I manually
removed the mailing list from the recipient list, I forgot to remove the
Reply-To.]

I wrote:

> So applications can indeed decide not to use IDNA, in which case they
> can't use non-ASCII characters in domain names.  If you want to use
> non-ASCII characters in domain names, IDNA is your only option.

To clarify for the list readers:  Applications have always been able
to use 8-bit names, and still can, and IDNA will not change that.
Applications might have their own private interpretation of bytes
80..FF as characters, but there is no standard interpretation of those
bytes as characters; the only characters that can currently be used
unambiguously in domain names are the ASCII characters.  If IDNA (as
currently written) is adopted, it will be the standard definition
of internationalized domain names.  The only way to use non-ASCII
characters in domain names in a standard unambiguous way will be to
observe the rules of IDNA (which do allow for non-ASCII representation
in new protocols).

I wrote:

> We can't stop applications from using custom mappings from non-ASCII
> text onto domain names, but the rest of the world would never see the
> non-ASCII characters.  The rest of the world would simply see RFC 1035
> domain names, which contain only ASCII characters (and possibly octets
> 80..FF, which are not defined to represent any characters at all).

John C Klensin <[email-masked]> responded:

> But we can say, clearly, that any such applications or activities are
> non-conforming to IETF standards.  And I believe we should do so.

I don't think they are non-conforming.  Consider this analogous
situation:  For some reason I want to map geographic locations
onto domain labels.  There is no standard for using domain labels
to represent geographic locations, but I make one up, for example
n037d37m05s-w122d23m14s.  No one else will understand that this is
a geographic location, to them it's just a domain label, but it's a
conformant domain label, and it doesn't hurt anyone.

Now suppose my data type is something that contains non-ASCII
characters, but it would get hopeless damaged by Nameprep, so I cannot
use IDNA.  Just as in the previous case, I can make up my own mapping
and use it without bothering anyone else, they just won't understand it.

I wrote:

> It's still true that IDNA is optional; if you don't want to support
> it, then don't, but then you have no way to enter non-ASCII names into
> zones.

As before, I meant "no standard unambiguous way".

I wrote:

>     a primary master name server MUST NOT contain an ACE-encoded label
>     that decodes to an ASCII label.
>
> This is a vacuous requirement.  There is no such thing as an ACE label
> that decodes to an ASCII label, because of the design of the ToASCII
> operation.  Therefore, this is not really requiring anything.  I think
> I once asked for it's removal, but since it's harmless, I didn't press
> it.

John responded:

> helpful to someone.  But rephrase it, not as a "MUST" (or any other
> form of requirement), but as an observation.  E.g., "note that,
> since the design of the ToASCII operation prevents any ACE label
> from decoding to an ASCII label (i.e., one without any non-ASCII
> characters), a primary master name server will never...".  Or
> something like that.

I like that.  Now that I think about it, I should have said "because of
the design of ToASCII and ToUnicode", not just ToASCII.

> But I'd still prefer to see either enumerations of where IDNA can be
> used, or some much more clear language about its applicability.

First sentence of section 1.1:

    Applications can use IDNA to support internationalized domain names
    anywhere that ASCII domain names are already supported

> I believe that we should confine IDNA application and interpretation
> to what we can reasonable claim we know about today and that we should
> _explicitly_ leave future use and applicability for decisions to be
> made in the future.

I suppose future specs that define new places for domain names to appear
could explicitly say "IDNA shall not apply to domain names appearing in
these places".  In the absence of such a statement, the first sentence
of section 1.1 remains as a simple fact.

> Now, suppose someone comes along and invents a new RR type. ...the
> should be... a few hundreds or, at most, a few thousand, of select
> characters.
>
> Now, to accomplish that goal... the best path, given our current
> framework, would be to modify stringprep to add a table of the
> characters permitted by that RR.  _Then_ the right thing to do is to
> create a profile that uses that table instead of some of the maps and

The specification does not require any particular implementation, it
just defines the right answer.  IDNA requires applications to convert
non-ASCII labels to equivalent ASCII labels in certain situations, and
"equivalent" is defined in terms of ToASCII, which is defined in terms
of Nameprep.  Applications are not prevented from imposing additional
restrictions on labels.  IDNA says that anything that doesn't survive
ToASCII is not allowed, but it doesn't say that anything that does
survive ToASCII must be allowed.

The application is under no obligation to implement ToASCII exactly
as it is shown in the spec, as long as it computes the right answer.
If instead of performing Nameprep and then checking some additional
restrictions, the application can use another Stringprep profile that
always returns the same result as Nameprep-plus-additional-restrictions,
it is welcome to do that.  That would just be an optimization.

AMC

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 13 Jun 2002 12:02:38 +0200 (CEST)

> I'd like to give those future designers the

John,

I'm trying to understand why, if e.g. the community goes down
the path of doing a new class, why such a specification can't update the
IDNA RFC to e.g. say that IDNA now only applies to class=IN
(or whatever is approiate). That specification will need to also
specify how QCLASS=ANY is handled, etc, etc.

If we declare that IDNA needs to restrict itself (e.g. to class=IN)
then it seems like we need to solve the QCLASS=ANY issue now.
This seems suboptimal both because the work might be wasted (if a new
class protocol isn't pursued) but worse, the solution might be broken 
since we have no way of predicting what the new class proposal might need
with respect to QCLASS=ANY.

I think there is a separate issue whether the rules for QNAME and RDATA
domain name slots should be the same, or e.g. whether the RDATA slots
in SOA, CNAME(?), etc should use different nameprep (e.g. to allow
different case, or to allow any codepoints). I think that issue can
be decided without having to be able to predict the future.

> different protocol to set up, compare, and map names.

I don't think there is anything that prevents this, but it might be a bit
cumbersom. For instance, a new definition could say "apply ToAscii with
step 2 replaced by foo" where "foo" is something different
than nameprep. (And  similarely for ToUnicode).

As part of my review I explored if there could be a better separation between
nameprep and the other parts of IDNA, but that seems to result in different
handling of case for all-ASCII labels.
If nameprep was separable from ToASCII it would need to be the first step.
This would mean that all-ASCII labels would be case-folded, which they
are not as ToASCII is specified. Hence an application using IDNA
would potentially display different case for non-IDNs to users.

  Erik

---

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jun 2002 10:56:58 -0400

--On Thursday, 13 June, 2002 12:02 +0200 Erik Nordmark
<[email-masked]> wrote:

>> generally) to handle internationalization, the ability to
>> think through whether more of the normalization or matching
>> processes should be moved to the server, rather than being
>> handled exclusively in the client, etc.  I don't want (or
>> need) to predict what they will conclude, but I think it is
>> unnecessary and dangerous for us, at this stage, to write
>> what seems to be an "all internationalization uses IDNA, now
>> and forever, including in RRs and Classes and uses of the DNS
>> we have no way to anticipate" rule.
> 
> John,
> 
> the path of doing a new class, why such a specification can't
> update the IDNA RFC to e.g. say that IDNA now only applies to
> class=IN (or whatever is approiate). 

Because we have tried that approach, mostly inadvertently, many
times before, and it has always gotten us into trouble.  "You
are not permitted to do this" leaves us in a position where we
can later make it permissible and give it an interpretation at
the same time.  "It is undefined" seems to invariably create a
community of people who are sure what "undefined" means --i.e.,
who assign a meaning to it-- and who then become a serious,
installed-base, impediment to any other interpretation.  And
"this applies globally" will be taken as the basis for a claim
that we are making a seriously incompatible change if we try to
do something else in some other name space.

Some examples, in the event that this still isn't clear:

	* RFC821 specified Internet email transport in terms of
	ASCII.  It didn't explicitly (enough) prohibit sending
	non-ASCII ("8bit") characters.  So, when the desire to
	send, e.g., ISO 8859-1 chacters, arose, a community
	sprung up that, since the specification left 8bit
	characters undefined, they were free to define them as
	8859-1.  Another community sprung up that said that,
	since they were undefined, they were prohibited, and
	zeroed the high-order bits (or rejected the messages) in
	MTAs.  The combination led to the "just send 8" movement
	and huge problems in getting MIME, content character set
	tagging, and the SMTP extension model agreed to and
	deployed.  The aftereffects of those problems persist to
	this day.
	
	* RFC 1034 and 1035 define labels in terms of ASCII
	characters, but contains more or less vague language
	about how labels containing octet values that cannot be
	ASCII are to be interpreted.  That has touched off a
	debate within this WG (and I wouldn't expect the debate
	to be much different within namedroppers, but would
	encourage you to try that experiment) as to whether this
	means "octets above 0x7F are undefined and their use is
	prohibited until they are defined" or "octets above 0x7F
	are to be compared exactly, while _all_ octets of 0x7F
	and below are to be compared case-insensitively
	(assuming ASCII interpretation of those bytes).   We
	have either got to be sure that we can predict the
	future, or we need to explicitly bar the cases we don't
	(and don't need to) understand.  The other cases just
	lead to confusion and non-interoperability... and our
	track record for future-predicting is not good.
	
	* There is actually a substantive (in addition to
	philosophical) reason why I have been pushing back on
	the "you can use a binary label anywhere" interpretation
	of 1035 (as represented in 2181 and elsewhere).  While
	I'm unhappy about treating characters from
	case-differentiating scripts differently depending on
	whether the octets of their codings happen to fall in
	the 0x00-0x7F or 0x8F-0xFF ranges for existing
	character-based RR types, it isn't my main concern.   My
	concern is that this interpretation requires that all
	labels for all RRs, in all Classes, present and future,
	be treated as character strings using those rules.
	
	As just one example of this, suppose we come along later
	and want to put Unicode more directly into the DNS.  We
	know that UTF-8 isn't a particularly efficient encoding
	-- its design is strongly influenced by the need to be
	compatible with octet-based systems.    We also know
	that UCS-4 (UTF-32) isn't very efficient either.  But we
	know a good deal about compression, and could do some
	elegant things about compressing a string written in
	Unicode characters.  But, to do that, we need _binary_
	labels (not just binary octets), i.e., a length-encoded
	bit string of up to 63 octets in length and with no
	variant interpretations based on whether the stored
	value of a given octet falls in the 0x00-0xFF range.
	That is pretty easy to do in the DNS structure, but it
	requires per-RR or per-Class definitions of how the
	label string is interpreted/ matched.  If we read
	1034/1035 as "it is all characters" then either this is
	hopeless or we will need to invent another kludge
	(fortunately, the obvious one is still more efficient
	than UTF-8).  If we read 1034/1035 as "only the existing
	RRs and Classes are specfied, new ones get to specify
	their own rules"  then the range of options remains open.

There is even a question about today's installed base.  Has
anyone who is competent to do so taken a careful look about what
the implications would be if IDNA were applied to Hesiod-Class
data?  I'm not very worried about chaosnet, but we usually try
to avoid blowing away know installed applications without
careful consideration.   Now, one could argue that case would be
covered by the language that says use of IDNA is an
application-choice, but that would call for a piece of a
security considerations section (or elsewhere) that explains
issues an application should consider before deciding to use
IDNA.  And that section isn't present.  It would also, I think,
be hard to write and get right -- why look for trouble by making
sweeping statements about where IDNA applies when they don't
appear necessary.

> That specification will
> need to also specify how QCLASS=ANY is handled, etc, etc.

Maybe.  We have successfully dodged this issue with the three
classes that are in use so far.  And I would argue that we are
more likely to be forced to define it if we make IDNA apply
across everything than if we say "IDNA applies here, to cases we
can list and enumerate, and applying it elsewhere requires
specific action".  

> If we declare that IDNA needs to restrict itself (e.g. to
> class=IN) then it seems like we need to solve the QCLASS=ANY
> issue now. This seems suboptimal both because the work might
> be wasted (if a new class protocol isn't pursued) but worse,
> the solution might be broken  since we have no way of
> predicting what the new class proposal might need with respect
> to QCLASS=ANY.

See above.  It is my belief that we need to address the
QClass=ANY question only when a class comes along that requires
it.  We have gotten away without it for a long time.  My own
hunch is that it doesn't have a solution without imposing the
"look everywhere in case someone has data" property of X.500 DSP
on the DNS, which would be a really bad idea and that, instead,
if someone really needs "look in this Class and then in that
one" capability, they are going to have to invent, e.g., a
"QTYPE=INorUC" arrangement and try to make it work.  I wish them
luck.   

Aside: at a procedural level, I think a persuasive argument can
be made that the majority of the IDN WG participants don't
understand the details of this discussion or why it is
important, even after it is called to their attention.    If
that is true, a document that makes "applies everywhere"
statements about DNS use is inappropriate to be treated as a
product of this WG.

> I think there is a separate issue whether the rules for QNAME
> and RDATA domain name slots should be the same, or e.g.
> whether the RDATA slots in SOA, CNAME(?), etc should use
> different nameprep (e.g. to allow different case, or to allow
> any codepoints). I think that issue can be decided without
> having to be able to predict the future.

Absolutely.  My only concern is that it should be decided
explicitly and not by either handwaving (which the current
document appears to me to do) or the hope that the definitions
will appear in some future document which we have no current
plan to get written (which I think at least some of the authors
might argue was the solution).   If we can't decide, and can't
get that document, then I believe that the uses need to be
prohibited until we have clear definitions.  Otherwise, people
will make up their own answers while pointing to the IDNA
standard, and we will have non-interoperability chaos.

>> IDNA, but with a different profile and (ii) we should not, as
>> a side effect of the way we specify IDNA today, prevent that
>> RR from being defined, regardless of whether it can use
> I don't think there is anything that prevents this, but it
> might be a bit cumbersom. For instance, a new definition could
> say "apply ToAscii with step 2 replaced by foo" where "foo" is
> something different than nameprep. (And  similarely for
> ToUnicode).

In practical terms, if anyone actually decides to do this in an
implementation, the burden of demonstrating how to do it in a
non-cumbersome way lies on them.  I think Eric and I have
described how to _define_ it fairly easily.   I note that, from
a definitional standpoint, the whole notion of Classes and RRs
in the DNS ought to be a matter of tables but that the
predominant implementations have ended up requiring code changes
each time one is added and with a lot of that code being
special-purpose.    The implementer's life is hard sometimes.
But, while I firmly believe in making it easy when possible,
hardness (or cumbersomeness) shouldn't be a bar to our doing the
right thing.

And, again, where Eric and I _may_ differ is that, while I'd
rather see this opened up if we can figure out how to do it (in
the interest of modularity and increased likelihood of future
interoperability), my bottom line is simply that we define IDNA
as applying in areas that we have studied and understand that it
applies reasonably, that we ban it elsewhere and leave those
other cases for the future.

> As part of my review I explored if there could be a better
> separation between nameprep and the other parts of IDNA, but
> that seems to result in different handling of case for
> all-ASCII labels.
> If nameprep was separable from ToASCII it would need to be the
> first step. This would mean that all-ASCII labels would be
> case-folded, which they are not as ToASCII is specified. Hence
> an application using IDNA would potentially display different
> case for non-IDNs to users.

Or nameprep would have to special-case those characters and push
them down a code branch it doesn't now have.  I admit that isn't
attractive, but...

        john

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 21 Jun 2002 15:03:48 +0200 (CEST)

John,

I'm not saying that leaving things undefined is the way forward.
But you seem to be saying, in addition to wanting things to be well defined,
that there needs to be an attempt to either predict the future or
only apply IDNA in the smallest possible context.
I think there is a 3rd alternative which is to, unless explicitly overridden
elsewhere in the future, apply it everywhere. This can be perfectly
well-defined as far as I can tell and it means that any changes can be done
with the well-defined basis as a foundation.

> 	non-ASCII ("8bit") characters.  [...]

> 	ASCII are to be interpreted.  [...]

The above two examples seems to me to be cases of loosly defined
specifications. I agree that we want to avoid that in the IDN space to the
extent possible.

> 	their own rules"  then the range of options remains open.

I think the place where this discussion belongs is the now expired
draft-ietf-dnsext-unknown-rrs-02.txt. But I note that it is lacking
as well in that it silently seems to assume that all the owner names
are ASCII text by saying:
   The owner name is
   still set to lower case.

I agree it is highly desirable to be able to define new RR types and/or classes
without the ASCII case insensitive comparison of the owner names. But the
feasibility of this is a function of what existing DNS implementations do with
unknown types and classes. If it turns out that this isn't possible one can
still envision efficient encoding of Unicode owner names e.g. by applying the
Bootstring algorithm (used by punycode) but have the output code set be e.g.
0-0x40,0x5b-0xff i.e. avoid using bytes which an ASCII case insensitive
comparison would trip on.

The alternative seems to be to embark on discussions whether RR type X
(for different values of X including e.g. SRV, NAPTR) benefits
enough from IDNA to make that RR type be covered by the IDNA specification.
I fail to see how this WG can have expertize in determining the usage of
all defined RR types. Even if the WG had that expertise I still wouldn't be
suprised there were radically opposing judgements about "benefit enough from 
IDNA" for some types.

  Erik