Subject: [idn] One profile for domain names, or many?

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 13 Jun 2002 00:28:10 +0000

Let me try to sum up the issues in the ongoing debate over whether
there should be a single standard Stringprep profile for all IDNs, or
whether different profiles should be allowed for different contexts
(like different DNS resource record types).

[By the way, I think concerns that IDNA's definition of IDNs might be
overly restrictive and might cause problems in the future is very much
on-topic for this working group, and such concerns should be aired.  I
thank Eric for raising this issue and working hard to make us understand
his side of it.]

I'll start by recapping the two models.

In the single-profile model (which is used by the IDNA draft), there is
one standard profile (Nameprep) that is used for all comparisons between
textual domain labels, and for all conversions of textual domain labels
between their ASCII and non-ASCII forms.  A textual domain label is one
composed of specific well-defined characters, not an arbitrary sequence
of bytes.  The profile is intimately involved in the definitions of
which labels are equivalent and which labels are valid.  Additional
restrictions can be imposed (some valid labels might still be disallowed
for particular purposes).  IDNA would govern all comparisons between
textual domain labels, and all conversions of textual domain labels
between ACSII and non-ASCII form, but it would have nothing to say
about conversions between domain labels and other data types; those
conversions are out of scope of IDNA, and would need to be defined
by whatever specs define those other data types.  The 8-bit labels
contemplated by RFC 1035 would also be out of scope of IDNA, because
there is no standard interpretation of the bytes 80..FF as characters.

In the many-profile model (as I understand it), domain names are really
sequences of bytes, and their interpretation as characters is merely
a view created by applications for the benefit of human users.  All
comparisons are really exact comparisons except for ASCII letters; the
illusion of case insensitivity for non-ASCII letters is created by
applications applying case folding when they convert user-entered text
into domain-names-which-are-byte-sequences.  Different text-to-byte
conversions, defined by different Stringprep profiles, can be applied in
different contexts, and this will appear to users as different kinds of
sensitivities.

I think the single-profile model is inspired by the existing model of
ASCII domain names as they appear to users.  I think the many-profile
model is inspired by the existing model of 8-bit domain names as they
appear to DNS implementors.

I've had my doubts about whether the many-profile model could really
work, but Eric is confident that it would, and my doubts have lessened
somewhat.  I have no doubt that the single-profile model would work,
and I don't remember anyone claiming that it wouldn't; Eric has argued
that it would make some things less convenient or less efficient.  Let's
assume for the sake of argument that both models work, and we just have
to decide which one we prefer.

Now let's examine an implication of the many-profile model, using a
concrete example scenario (please bear with me, the punchline will be
interesting).  Suppose there is an XXX resource record type, which
uses xxxprep for the first label of its owner name, and there is a YYY
resource record type, which uses yyyprep for the first label of its
owner name.  One registrant asks the administrator of a zone to create
an XXX record with owner label x, and another registrant asks for a YYY
record with owner label y, in the same zone.

Now let's plug in the details:

    x = fullwidth Latin small letter a
    y = Latin capital letter A

    xxxprep = case-fold, then apply NFC
    yyyprep = apply NFKC, then prohibit all lowercase letters

What does the zone administrator do?  First, verify that the labels are
properly prepared:

    x == xxxprep(x) ?   Yes.
    y == yyyprep(y) ?   Yes.

Next, check for any collisions:

    xxxprep(x) == x == fullwidth Latin small letter a
    yyyprep(y) == y == Latin capital letter A
    xxxprep(y) ==      Latin small letter a
    yyyprep(x) ==      error

That's all combinations, and they yield all different answers, so there
are no collisions here in the prepared space where DNS servers live.
The two domains are created for the two registrants.

What we failed to detect, and what in general could be extremely
difficult to detect (because Stringprep is not reversible), is the
existence of a collision in the unprepared space where users live and
think and type.  Consider:

    z = fullwidth Latin capital letter A

A user could use this same string z to refer to both of these domains.
If the user asks for an XXX record, z will match one domain; if the
user asks for a YYY record, the very same string z will match the other
domain.

In other words, from the point of view of users, there is not a single
domain name space in which all domain names live, there are multiple
domain name spaces (one corresponding to each profile), and the same
name can refer to different domains in different spaces.

This is unlike ASCII domain names, which all occupy a single name space.
An ASCII name always refers to the same domain regardless of the context
in which it is typed.

The single-profile model retains this single name space characteristic,
while the many-profile model introduces a new concept of multiple
name spaces.  In this sense, the many-profile model is a more radical
departure from the existing ASCII domain name model, while the
single-profile model is more conservative.

(By the way, it's not at all clear to me what should happen if the user
asks for ANY records matching z.)

A simple/conservative approach has both advantages and disadvantages
compared to a more complex/flexible approach.  Let's look at a couple of
those.

Suppose an application has some non-domain-label data type that it
wants to map onto domain labels.  It would be convenient if the trivial
mapping could be used (that is, do nothing, just call the thing a
domain label).  This is easier in the many-profile model, because the
application can define a Stringprep profile well suited to its data
type.  In the single-profile model, the trivial mapping works only if
the original data can tolerate being nameprepped.  If it can't, the
application will have to use a non-trivial mapping from its data type
to domain labels.  IDNA is no help there (remember that IDNA applies
only to conversions of domain labels between their ASCII and non-ASCII
forms, not to conversions between domain labels and other data types),
so the application will always have to do the conversion at both ends;
it cannot benefit from any automatic IDNA conversions built into the
infrastructure (like a new resolver interface).

So that's an example of something that's more difficult (but not
impossible) in the single-profile model.

Now suppose an application simply wants to start using IDNA to allow
users to enter and see non-ASCII characters.  In the many-profile model,
you cannot put a user-entered name into a domain name slot unless you
know the profile to be used for that slot.  Eric has been talking about
resource record types, but DNS is not the only protocol with domain name
slots; we also need to assign profiles to domain name slots in other
protocols, and until we do, IDNs cannot be used in those slots.  For
example, before an email application can add IDN support, it needs to
wait for profiles to be assigned to header fields, SMTP commands, POP
commands, and IMAP commands.  Before a web browser can add IDN support,
it needs to wait for profiles to be assigned to the various URL schemes,
cookies, and SSL certificates.  With the single-profile model, there is
no question of which profile to use, so applications can add IDN support
the day after IDNA is published.

So that's an example of something that's less convenient (but not
impossible) in the many-profile model.

In conclusion, there may very well be more than one right way to
define IDNs, with tradeoffs among the options.  Based on my current
understanding of the issues (which I think is better now than ever),
I would still choose the single-profile model.  I can understand that
other people might choose differently.

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Wed, 12 Jun 2002 22:58:19 -0500

on 6/12/2002 7:28 PM Adam M. Costello wrote:

> into domain-names-which-are-byte-sequences.

You are correct. The ASCII part is confusing but you are correct in that
if an application wants to provide case-mapping it will have to do so in
the profile. There is no lowercase storage in the i18n namespace so the
codes must match precisely. This is a direct result of AMC-Z encoding.

One clarification is that it isn't expectated that other profiles will be
used for human input, although it's not a negative expectation either. The
principle intention is to facilitate the storage of application-specific
domain names, that is all. Although I've typed network names into NetBIOS
applications, I've never actually typed a NetBIOS name (including all of
the markers and codes). I've typed SRV owner names into dig for debugging
purposes, but never for user purposes. I've certainly never typed the
domain names needed to tunnel IP through DNS messages [1]. And so forth.
The principle intention is to allow applications to make use of the domain
names they need to function in the namespace, and that these names
probably won't be entered by users directly. Human input must certainly be
considered, but its not the primary expectation.

Moreover, these examples work with the existing STD13 octets. In that
regard, the intention is to facilitate the migration of STD13 octets. This
is especially important if that usage is going to be prohibited.

>     xxxprep(y) ==      Latin small letter a

and then nameprep for feelgood purposes, as stated three times now. But
let's assume they didn't.

> user asks for a YYY record, the very same string z will match the other

Queries contain three elements: qname, qtype and qclass. If all three
match then the query succeeds. If any of them don't match, then the query
will fail.

If the applications (not the user) issue these queries as you have
described them then they will ask for:

     qname=FF21  qtype=XXX  qclass=IN
and
     qname=FF21  qtype=YYY  qclass=IN

Does an owner name exist at FF21?
Does it have the queried RRtype?
Does it have the queried class?

Although I'm more than happy to have duplicates omitted, you can see from
the above why it isn't necessary. The domain name octets have to match AND
there has to be a RRtype as well. If the name is wrong (either in the
query or in the zone) then the names which do match probably don't have
the right RR. If the RR matches then they got what they asked for. If they
got something that they didn't want but some other similar name which also
happens to have the RR bound to it then the profile is flacid and the
developer needs to be shot for writing a profile that allows these errors.

> asks for ANY records matching z.)

They MUST get all of the matches that fit in the message. Note that it is
specifically qtype ALL (not ANY). This may appear semantic but its not.
Also note the opposite for qclass ANY (not ALL).

What the application does with those two RRs is up to the application.
Sendmail uses qtype=* to find all of the RRs associated with a mail
domain. What does it do with the RP and TXT RRs that it gets (hint: what
do those RRs have to do with routing mail?).

Really, this is making mountains from molehills. DNS already works this
way today, and the planet has yet to fall off its axis.

> other people might choose differently.

Where will the existing users of eight-bit domain names go if:

 1) they can't stay in STD13 because it has been prohibited

 2) the profile-centric i18n namespace forbids the applications
    from defining their own profiles?

I got news, one of those two changes will be ignored. Standards are
measured by their use, not their intent.

[1] http://slashdot.org/article.pl?sid=00/09/10/2230242&mode=thread&tid=95

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 13 Jun 2002 05:49:06 +0000

"Eric A. Hall" <[email-masked]> wrote:

> One clarification is that it isn't expectated that other profiles
> will be used for human input...  The principle intention is to allow
> applications to make use of the domain names they need to function in
> the namespace, and that these names probably won't be entered by users
> directly.
>
> regard, the intention is to facilitate the migration of STD13 octets.
> This is especially important if that usage is going to be prohibited.

That's very interesting.  Maybe there's a middle ground here.

Suppose the IETF were to issue a clarification of the DNS spec stating
in no uncertain terms that bytes 80..FF in domain names must match
exactly, and that there is no standard interpretation of those bytes as
characters (not for any existing DNS services anyway).  Then I wouldn't
object if applications that don't like Nameprep were encouraged to use
8-bit names in DNS for non-human-oriented purposes.

IDNA with its single profile would govern the universe of textual domain
labels (labels composed entirely of well-defined characters, not just
opaque bytes).  These labels are intended for human users and are used
in a wide variety of protocols, not just DNS.  Applications that want to
use human-accessible domain names containing non-ASCII characters in a
standard representation would use IDNA (they would have nowhere else to
turn).

Outside the universe of textual domain labels are the STD-13 domain
labels that contain at least one byte in the range 80..FF.  I call such
labels non-textual because there is no standard interpretation of 80..FF
as characters (although applications can locally map them to characters,
at their own risk).  These labels are specifically DNS-things (I've
never heard of them being used in other protocols).  These labels are
already outside the scope of IDNA, because IDNA can only cope with
well-defined textual labels.  Applications that don't like Nameprep and
don't care whether their domain labels are easily accessible to humans
can ignore IDNA and stick with STD-13 and their own local interpretation
of the bytes, and prepare their labels in any way they like.

Eric, would you be satisfied with that?  What do other people think of
that idea?

Note that we can't stop applications from using 8-bit STD-13 domain
names.  The only question is under what circumstances we encourage or
discourage it.

> 1) they can't stay in STD13 because it has been prohibited

We are not going to try to overturn STD-13.  Language in the IDNA spec
that appeared to be doing that was a mistake, and will be fixed.

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 03:13:22 -0500

> Eric, would you be satisfied with that?  What do other people think of

I've given this some thought, and I think it's a creative approach. But I
don't think it's the right approach, since it only ingrains some of the
problems. I know you don't want to hear this but it would actually make
much more sense to do the exact opposite, which is to prohibit eight-bit
for the standard RRs (which is also what John was saying I think) and to
embrace and encourage the use of multiple profiles in the i18n namespace.

Profiles present a great opportunity to enforce strong and consistent
data-typing with the different domain names. Part of the problem with the
legacy domain names is that they are so malleable with different
interpretations depending on the spec you read. Look at STD19, which was
written to provide an encoding for NetBIOS names, but which MS no longer
uses due (partly) to i18n issues. Meanwhile, they use RFC 2181 as
justification for storing UTF-8 characters in A RRs. I'd much rather see
DNSEXT prohibit eight-bit codes in A RRs and encourage MS to use the same
A RRs as everybody else so that subsequent future uses of A RRs were
consistent. But I'd ALSO encourage them to define a structured syntax for
i18n NetBIOS names as well. Leaving the eight-bit codespace there just
means they don't have to do either of those things, meaning that different
networks will look for i18n A RRs in different ways and places. In short,
encouraging the use of profiles for all domain names encourages strong
data-typing across all applications which use all domain names, which
benefits interoperability tremendously.

This favortism towards strong data-typing also benefits applications which
pass i18n domain names as in-band protocol data. Obviously, nameprep
allows an i18n version of Finger (as an example) to know precisely what
the rules are for the domain names that it exchanges, as would an i18n
profile of mailbox names.

Along the same lines, a profile for Kerberos also provides a way for
participating applications to be able to create and verify those domain
names when they need the protocol information, even if this data never
goes into DNS. But if it does go into DNS they can use the profile there
as well. draft-ietf-krb-wg-krb-dns-locate-02.txt described a way to
provide DNS->Realm mappings using TXT RRdata to store the Kerberos realm
name, but a profile would allow a kerberos-specific RR to be creeated for
the same purpose and with greater efficiency and accuracy (note that
Kerberos' use of uppercase for realm names prohibits nameprep).

There is another reason for going this route, which is that the presence
of eight-bit codes in the STD13 namespace makes managing UCS names in
dual-mode servers extremely difficult. It essentially requires that
servers flag eight-bit domain names as NOT UCS so that the names don't get
looked at when an EDNS/deACE'd query comes in. Part of the original
motivation for going to a profile centric world was to resolve this
specific problem. I don't want it back in after all this.

Anyway, those are my reasons for arguing that DNSEXT clarify the
interpretation of RRs in the legacy namespace at the same time the new
namespace is defined. Leaving those in there wouldn't really solve any of
the problems we are trying to fix in the long run.

I will post a discussion of the i18n namespace with multiple profiles
tomorrow which will hopefully remove your remaining doubts about its
viability. It is really fairly simple and straightforward, and it yields
many benefits, some of which are outlined above.

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 13 Jun 2002 12:03:20 +0200 (CEST)

> Does it have the queried class?

Eric,

How does this work for non-terminal labels i.e. when there is a zone cut?
I think DNS server just look at qname (and not qtype) to determine that
they need to do a referral (or perform recursion). Having the interpretation 
of the owner name be type specific provides the opportunity (however 
misguided it might be) to try to delegate the two name/type pairs differently.

As far as I can tell using different profiles for QNAME and RDATA domain
name slots doesn't have this issue.
And in the future there is the option to carefully define a new class
that uses a different profile for QNAME.

   Erik

---

From: tedd <[email-masked]>
Date: Thu, 13 Jun 2002 09:30:43 -0400

Hi:

Any comments about what VeriSign is doing?

<http://www.theregister.co.uk/content/6/25698.html>

tedd

-- 
http://sperling.com

---

From: "Edmon Chung" <[email-masked]>
Date: Thu, 13 Jun 2002 09:59:45 -0400

Perhaps we should preserve case for IDNs during transportation, as we do
today with English only names.  Nameprep can be done at the server as well
as the client, but is optional for the client during request and mandatory
for the client if DNSSEC is required.

That is:

1. Application ToACE multilingual domain name
2. Send ACE domain to server without doing Nameprep
3. If fail go to 5.  If successful go to 4.
4. Authenticate and match response by first Namepreping both and comparing.

5. Send again with Namepreping, if successful go to 4, if fail then NX
domain

This way, case can be preserved during transportation just as it is today.
Also, some browsers do the same with English domain names right now anyway:
first send case preserved name, if fails, lowercase (or uppercase) and send
again.

Edmon

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 11:20:17 -0500

on 6/13/2002 5:03 AM Erik Nordmark wrote:

>> Does an owner name exist at FF21? Does it have the queried RRtype?

> How does this work for non-terminal labels i.e. when there is a zone

If there is a cut, then the zone has delegated away the responsibility for
that portion of its namespace to the servers listed in the NS RRs
associated with the owner name of the new child zone.

If there is another RR associated with the owner name, that RR should be
ignored, since the current zone is not authoritative for that portion of
the namespace anymore (it has delegated the responsibility away).

---

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jun 2002 13:59:03 -0400

Let me add one bit of clarification to this (again, I don't
think Eric and I are in complete agreement, but we seem to see
some of the same requirements and solutions)...

--On Thursday, 13 June, 2002 03:13 -0500 "Eric A. Hall"
<[email-masked]> wrote:

>> Suppose the IETF were to issue a clarification of the DNS
>> spec stating in no uncertain terms that bytes 80..FF in
>> domain names must match exactly, and that there is no
>> standard interpretation of those bytes as characters (not for
>> any existing DNS services anyway).  Then I wouldn't object if
>> applications that don't like Nameprep were encouraged to use
>> Eric, would you be satisfied with that?  What do other people
>> think of that idea?
> 
> I've given this some thought, and I think it's a creative
> approach. But I don't think it's the right approach, since it
> only ingrains some of the problems. I know you don't want to
> hear this but it would actually make much more sense to do the
> exact opposite, which is to prohibit eight-bit for the
> standard RRs (which is also what John was saying I think) and
> to embrace and encourage the use of multiple profiles in the
> i18n namespace.

As I indicated in one of my recent long notes, I'd like to make
sure we can define new stuff (certainly Classes and probably
RRs), with different rules.  Examples:

	-- All upper case, with other characters being
	prohibited.
	
	-- Case-dependent matching of ASCII (i.e., lower and
	upper case are different) as well as everything else.
	
	--  True binary labels, with no per-octet interpretation
	(needed for compressed UCS-4 as well as for numeric
	labels).

Now, I'm not sure that we will ever need or want any of these
things (although some of Eric's examples point strongly in that
direction.  But I think anything that we do now --whether an
overbroad statement about IDNA applicability or the (I believe
misguided) text in 2181-- that permanently eliminates our doing
any of the above except by an incompatible change, has few of no
advantages and many disadvantages.

And, yes, I share Eric's belief that strong data-typing and data
type identification leads to better protocols and more
interoperability when it is most important.

    john

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 13:29:02 -0500

> RRs), with different rules.  Examples:

I have no problem with class-specific rules. Servers reject queries for
unknown classes so there is no conflict in the normal scenario.

I would greatly prefer that new RRs in the IN class only use stringprep
profiles and the i18n namespace if they need eight-bit data, simply
because the data-typing provides benefits. However, if an application
needs to store eight-bit RR-specific data -- and if the application is
specifically willing to forego any ACE conversion -- then I see no reason
to prohibit it.

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 13 Jun 2002 21:09:09 +0000

> 5. Send again with Namepreping, if successful go to 4, if fail then NX

Close, but not quite.  I assume you intend to have the client detect
whether the server performed nameprep on the query, and accept the
response if so, otherwise re-issue the query in nameprepped form.  To
accomplish that, in step 4 you must not nameprep the response, instead
you must compare the response against the nameprepped form of the query.

But I can't imagine why anyone would care whether case is preserved on
the way from the client to the server.  I think it's in names coming the
other direction that people might want to preserve case.

AMC

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 13 Jun 2002 21:53:56 +0000

"Eric A. Hall" <[email-masked]> wrote:

> data-typing with the different domain names.

I can see that, but part of my concern is that you have tended to try
to define the subtypes of domain names in terms of DNS resource record
types.  Domain names get used in protocols other than DNS, where they
are not tagged with RR types.  So the subtypes of domain names would
have to be defined more abstractly, for example:

    host names
    mail domain names
    mailbox names
    service names
    kerberos realm names

The RR types could then refer to these abstract subtypes, but other
protocols could also refer to the abstract subtypes.

The tricky part is that some of these subtypes are already in wide use
in a wide variety of protocols without having ever been formalized.  In
my example list of subtypes, my intent is that the host field of a URI,
the exchangers listed in an MX record, and the domain field of an HTTP
cookie are all of type "host name", but no such connection has never
been formally drawn between these various protocol elements.  Similarly,
my intent is that domain names in email addresses in mail headers, and
domain names in email addresses in SMTP commands, are both of type "mail
domain name", even though RFC 821 places tighter restrictions on the
syntax than RFC 822 does.

I suppose the way to attempt to accomodate your model in IDNA (but I'm
not sure it could be done with sufficient rigor) would be something
like this:  ToASCII and ToUnicode would take a Stringprep profile as
a parameter.  This would be yet another thing that the application
would have to select, along with the AllowUnassigned flag and the
UseSTD3ASCIIRules flag.  The IDNA spec would require that Nameprep and
only Nameprep be used with host name labels and mail domain labels.
Furthermore, applications would be forbidden from using IDNA with any
other label types until profiles for them had been standardized.

> There is another reason for going this route, which is that the
> presence of eight-bit codes in the STD13 namespace makes managing
> UCS names in dual-mode servers extremely difficult.  It essentially
> requires that servers flag eight-bit domain names as NOT UCS so that
> the names don't get looked at when an EDNS/deACE'd query comes in.

I came to that realization this morning.  A hypothetical newDNS protocol
that allowed text labels to be represented using UTF-8 while still
supporting the RFC-1035 sequence-of-bytes labels would need an extra
bit per label indicating whether byte values 80..FF are UTF-8 text, or
opaque bytes like in DNS.  A hypothetical new resolver interface would
likewise need this extra bit per label if it wanted to support both
text-labels and byte-labels.

I don't know if I'd call that "extremely difficult".  The only way to
avoid the extra bit while adding native Unicode support is to drop
support for RFC-1035 sequence-of-bytes labels.

In fact, if I were designing a new resolver interface, I'd want it
to have Nameprep built in (and maybe a few other common profiles, if
there are such things), in which case I'd still need extra per-label
meta-info, so the application could say "Please apply Nameprep for
me" or "This label is already prepared, leave it alone" or maybe even
"Please apply Fooprep for me".  The application might also want to be
able to say "Please return only prepared labels to me" or "Please feel
free to return unprepared labels to me, and I'll be responsible for
preparing them if needed".  The latter option allows for the possibility
of preserving case or non-normalized forms (which might or might not be
possible depending on the underlying protocol).

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 17:39:39 -0500

on 6/13/2002 4:53 PM Adam M. Costello wrote:

> in a wide variety of protocols without having ever been formalized.

> been formally drawn between these various protocol elements.

I think I can speak for John when I say that this is what he and I both
want to see fixed. The vagaries and loose standardization has been more
trouble than it is worth.

If you look at the BNF for most of these protocols they are just LDH
sequences already anyway. Also, the pre-2181 history has it that most
systems only recognize LDH (I've even run across applications that
couldn't parse 3com.com because it wasn't explicitly allowed until RFC
1123). If the i18n namespace is being invented for the sole purpose of
standardized representation and interpretation of i18n domain names,
there's really not any reason to go back and re-re-fix the standard
RRtypes so that they are LDH hostnames.

Finally, declaring a <hostname> profile allows the other protocols to
reuse that BNF in their syntax, rather than requiring them to define it
for their own specific use.

> other label types until profiles for them had been standardized.

Again, I would make the applications do the stingprep management, since
they have to do so anyway for basic security measures. All IDNA needs to
be is a simple codec that converts inputs and outputs (unless I completely
misunderstand its inner workings).

> I don't know if I'd call that "extremely difficult".

The resolver doesn't have to keep track of this (it's just passing around
octet sequences) but servers (including caches) have to. Otherwise an
STD13 octet sequence which has been stored for future reference gets read
as an ISO-8859-1 string and converted.

Detecting the format of an incoming sequence is easy: the label is either
STD13 or EDNS, and if it is STD13 with eight-bit codes then it is an STD13
eight-bit label. That's easy to deal with for comparison purposes at that
particular moment but storing it and preventing comparison later isn't
very simple. Also, somebody might expect the implementor to convert the
eight-bit A RR into ACE or EDNS and scream if the implementor doesn't,
which gets really ugly.

However, if the default policy (and the existing standards-track RRs) is
changed so that eight-bit codes aren't used by default -- and new RRs can
only define STD13 octets if they are willing to forego conversion -- then
storing these RRs means it is a little simpler with the flag because it
becomes a distinct exception with clearcut logic and there can be no
confusion that they are never subject to conversion. It also makes
enforcing the policy change somewhat simple since violators are just
rejected by the compliant server. Fix or die. The only remaining parties
who are affected are those entities who have gone off and done their own
thing, and managing the flag mapping becomes their problem rather than
everybody's problem.

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 17:42:26 -0500

Mozilla has made my typo rate go through the roof for some reason.

on 6/13/2002 5:39 PM Eric A. Hall wrote:

> If the i18n namespace is being invented for the sole purpose of
> there's really not any reason to [NOT] go back and re-re-fix the
> standard RRtypes so that they are LDH hostnames.

---

From: "Edmon Chung" <[email-masked]>
Date: Thu, 13 Jun 2002 19:01:22 -0400

----- Original Message -----
From: "Adam M. Costello" <[email-masked]>
> you must compare the response against the nameprepped form of the query.

That is what I had in mind.

> other direction that people might want to preserve case.

Of course there are lots of reasons why the name should not be tampered with
before being sent out to the wire.  For example to accomodate new codepoints
or to use restricted codepoints in the DNS for internal purposes or
experimentation, such as using the reserved characters for testing.  These
are all good reasons for not having the client blindly do nameprep before
sending the request to the wire.  Since the name is already ACEd there is no
worry it would affect any other systems except for it being NX domain.
Therefore I think it makes a lot of sense to not force nameprep on the
application, but instead make it optional but mandatory on the server end.
Just like what we have nowadays for English.

Edmon

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 13 Jun 2002 23:54:29 +0000

"Eric A. Hall" <[email-masked]> wrote:

> > The tricky part is that some of these subtypes are already in
> > wide use in a wide variety of protocols without having ever been
> > formalized.
> >
> > my intent is that the host field of a URI, the exchangers listed in
> > an MX record, and the domain field of an HTTP cookie are all of type
> > "host name", but no such connection has never been formally drawn
> > between these various protocol elements.
>
> I think I can speak for John when I say that this is what he and I
> both want to see fixed.

It would be great to have that fixed, but when will it happen?  Can
IDNA afford to wait for it?  If not, should IDNA go ahead and limit the
applicability of Nameprep to the still-vague concepts of "host name
label" and "mail domain name label" in the hopes that those terms will
become rigorous?  I pose these questions to the working group--would
this be comforting or disconcerting?  In the past, any time Paul and
Patrik and I have tried to talk about "what is a host name" it's been a
mess.

> Again, I would make the applications do the stringprep management,
> since they have to do so anyway for basic security measures.  All IDNA
> needs to be is a simple codec that converts inputs and outputs (unless
> I completely misunderstand its inner workings).

We bundle Stringprep and Punycode and the prefix into a single operation
for convenience of description, so we can say something simple like
"for any text label X, ToASCII(X) is an equivalent ASCII label".  We
needed ToASCII and ToUnicode to serve as definitions of the correct
results for *any* input, not just already-prepared input.  If you have
already-prepared input, you can optimize the Stringprep step down to
nothing.

IDNA never says that ToASCII needs to be performed all at once by a
single entity.  For example, an IDN-aware interface can require its
non-ASCII input to be already prepared, in which case when it performs
ToASCII the Stringprep step has no effect and can be optimized away.
The thing calling the interface then has the burden of performing
Stringprep, but not the rest of ToASCII.  In this scenario the work of
ToASCII has effectively been split across two entities.

> > A hypothetical newDNS protocol that allowed text labels to be
> > represented using UTF-8 while still supporting the RFC-1035
> > sequence-of-bytes labels would need an extra bit per label
> > indicating whether byte values 80..FF are UTF-8 text, or opaque
> > bytes like in DNS.  A hypothetical new resolver interface would
> The resolver doesn't have to keep track of this

Sure it does.  If I pass an 8-bit label to a new-resolver, the resolver
needs to know whether the 80..FF values are UTF-8 text or opaque bytes,
so that it knows how to set that bit in the newDNS query, or so that it
knows whether to convert to ACE before sending a DNS query.

AMC

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 14 Jun 2002 00:08:22 +0000

Edmon Chung <[email-masked]> wrote:

> I think it makes a lot of sense to not force nameprep on the
> application, but instead make it optional but mandatory on the server
> end.  Just like what we have nowadays for English.

I would have no objection to a newDNS protocol allowing unprepared
non-ASCII text in queries and requiring the server to perform the
preparation.  (Well, I might wonder about the computational load on the
server.)

But I don't like the idea of unprepared ACE forms.  It would mean
disagreement about whether two ACE labels match.  Someone applying the
traditional ASCII comparison might say they don't match, while someone
applying your proposed decode-prepare-compare method might say they do
match.  The conversions were designed so that comparing the ACE forms
always yields the same answer as comparing the non-ASCII forms.  The
conversions are designed so that "unprepared ACE forms" don't exist; if
you think you have one, you will find that it doesn't decode, it's just
an ASCII label that happens to begin with the ACE prefix, but it's not
an ACE.

AMC

---

From: "Edmon Chung" <[email-masked]>
Date: Thu, 13 Jun 2002 20:19:39 -0400

But is it not the same experience for the DNS today?  Case is preserved
during transport and matched by the server.  If authentication is required,
the client will match it correspondingly.
There is no issue about non-resolution because the client should retry if
the unpreped name is not found.  This infact is what some of the current
browsers already do for the casing in English.
Edmon

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 14 Jun 2002 00:44:38 +0000

> But is it not the same experience for the DNS today?  Case is
> preserved during transport and matched by the server.

Yes, but existing servers don't know how to compare non-ASCII text
case-insensitively, so that model doesn't work with the existing
infrastructure.  You are free to use that model in newDNS.

Suppose ToASCII(X) == xx--foo, and ToASCII_without_Nameprep(X) ==
xx--bar.  We are not going to say that xx--foo and xx--bar are
equivalent, because the existing domain name standards already say they
are not equivalent.

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 19:42:07 -0500

on 6/13/2002 6:54 PM Adam M. Costello wrote:

> It would be great to have that fixed, but when will it happen?

WILL it happen is the question since it would have to be done by DNSEXT.
Between John, Erik and myself I think a compelling argument can be made
for it. Other WGs need this too (the CRISP WG needs BNF for the RRs so
that ASN.1 syntaxes can be defined so that WHOIS-like information can be
specified for storage in LDAP, for example).

> Can IDNA afford to wait for it?  If not, should IDNA go ahead and limit
> the applicability of Nameprep to the still-vague concepts of "host name
> become rigorous?

I don't know how to answer that. It would seem that a standalone codec
doesn't need to specify anything other than the inputs and outputs, and
without imposing on other standards and WGs. I would also think that
nameprep can say that it provides one method for mapping UCS character
sequences into strings which are usable as LDH labels (and vice-versa)
without imposing on other standards and WGs. Otherwise I'll leave this to
people who are more knowledgable in these matters. I'll say no more on
this subject.

> ToASCII has effectively been split across two entities.

How about mixing the two models? You can pass a stringprep profile to
ToASCII as a parameter, but an empty parameter value (or some other flag)
means that the string has already been prepared and that steps(N) are to
be skipped. Would that work?

> needs to know whether the 80..FF values are UTF-8 text or opaque bytes

Clarification, the resolver doesn't "need" a flag because it already has
access to a flag. On the application side, this would be the Wide API
versus the legacy API, while on the wire side this would be the STD13
versus EDNS label formats. In either case, if the legacy API/label is used
and the label does not contain zz-- then it is not ACE and it certainly
isn't UTF-8.

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 14 Jun 2002 01:16:51 +0000

"Eric A. Hall" <[email-masked]> wrote:

> You can pass a stringprep profile to ToASCII as a parameter, but an
> empty parameter value (or some other flag) means that the string has
> already been prepared and that steps(N) are to be skipped.  Would that
> work?

That would be a fine way to implement it.  In the specification, it
needs to be clear that "apply ToASCII" means the proper Stringprep
profile must be applied, it is not optional.  Exactly when and by whom
it is applied is an implementation detail.

> Clarification, the resolver doesn't "need" a flag because it already
> has access to a flag. On the application side, this would be the Wide
> API versus the legacy API

That works until you have a name in which some labels are non-ASCII text
and some labels are opaque bytes.  I guess in that case the application
could just use the legacy API and do the IDNA conversions itself.  But
another solution is for the new API to take an extra bit per label as
input.

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 13 Jun 2002 23:40:12 -0500

on 6/13/2002 8:16 PM Adam M. Costello wrote:

> That works until you have a name in which some labels are non-ASCII
> text and some labels are opaque bytes.

I had to think about this some.

If any label is EDNS/UTF-8 then all of them must be flagged that way. So
if any domain names come through the legacy API then all of the labels in
that domain name are treated as STD13.

The entire domain name is what gets cached, and not the individual labels.
Therefore, if any of the STD13 labels contained eight-bit data (regardless
of whether there was ACE in any of the others) the entire domain name
would get flagged. There would not be any expansion of the other labels in
that scenario, and that domain name would always be returned as an STD13
sequence.

In the normal scenario, the zone-specific labels can't have eight-bit
character codes since those labels aren't legitimate for delegation, so
only the labels on the left edge of the domain can possibly contain
eight-bit data. This means that the scheme above won't cause problems with
any "normal" domain names (such as delegations) but would only prevent the
abnormal domain names from undergoing conversion.

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 14 Jun 2002 20:28:42 +0000

"Eric A. Hall" <[email-masked]> wrote:

> So if any domain names come through the legacy API then all of the
> labels in that domain name are treated as STD13.

Right.  Which is why it might be convenient for applications if the new
resolver API takes an extra bit per label as input.  Suppose I want to
look up a STD-13 8-bit label under an internationalized domain.  If I
can only flag the whole domain as STD-13, then I need to perform ToASCII
for the other labels.  If I can flag individual labels, then I can pass
in a name containing one STD-13 8-bit label and several non-ASCII text
labels, and the resolver will know what to do.

On the other hand, if you want to discourage the use of STD-13 8-bit
labels, then requiring the use of the old API for the entire name will
help with the discouraging.  :)

AMC

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 14 Jun 2002 20:48:55 +0000

Here is my attempt at wording to clearly delimit the scope of IDNA and
Nameprep without precluding any future directions:

Applicability of Nameprep:

    When ToASCII or ToUnicode is applied to a label, the proper
    Stringprep profile for that label MUST be used.  Nameprep is the
    proper profile for all types of labels that do not specify a
    profile, which includes all types that predate IDNA (for example,
    host name labels and mail domain labels).  In the future the IETF
    might or might not define new types of labels that use other
    profiles with IDNA.

Applicability of IDNA:

    IDNA MAY be used with all types of textual domain names and textual
    domain labels that do not explicitly forbid the use of IDNA, which
    includes all types that predate IDNA (for example, host names and
    mail domain names).  In the future the IETF might or might not
    define new types of textual domain names/labels that forbid the use
    of IDNA.

    IDNA cannot be used with domain labels that contain non-textual
    data.  For example, in the current DNS protocol [RFC1034] [RFC1035]
    labels can contain octet values greater than 0x7F, but there is no
    defined interpretation of those values as characters.  Therefore
    IDNA cannot be used with DNS labels containing octet values greater
    than 0x7F.

With this policy, I think we could publish IDNA now and defer the
one-vs-many-profiles decision until later.  We can make a more informed
decision after the effort to formalize the label types has succeeded or
failed.

I think this policy would make it clear how to use IDNA today, and would
leave the IETF with three options when people ask to define a new label
that doesn't use Nameprep:

 1) Use a STD-13 binary.
 2) Use a text label that forbids the use of IDNA.
 3) Use a text label that requires fooprep with IDNA.

We could in the future decide that one or two of these options are
undesirable and should never be used, but for the moment we can leave
them all open.

What do people think of this?

AMC