Subject: Re: [idn] The layers and character handling in DNS

From: Dan <[email-masked]>
Date: Sat, 17 Feb 2001 15:27:23 +0100 (MET)

>Dan writes:
>> If I look at how I would like things to work, if could be like this:
>> 
>> user enters name in application using LOCAL character set.
>> ->
>> application calls routnes like: gethostbyname
>> ->
>> gethostbyname converts form LOCAL character set to 
>> normalised (not namepreped) UTF-8 (or other format).
>> it may reject the query if the name contains forbidden characters.
>> and sends the query to DNS server.
>
>How does gethostbyname() know for sure what the character set of its
>argument is?

In Unix from the current locale.

Paul said that this thread was having the mapping by the protocol. This
is not so, the protocol uses UCS, it is the API used by the application that
should map between the protocol format and local format. The APIs for
accessing DNS should not return protocol format, it should return the
native format used in applications.

It is the same for the API for LDAPv3. The LDAP protocol uses UTF-8, but
the strings returned by the calls should do the mapping for the
application.

  Dan

---

From: [email-masked] (David C Lawrence)
Date: Sat, 17 Feb 2001 12:38:37 -0500 (EST)

> In Unix from the current locale.

This is not reliable, certainly not in the already deployed software,
which is what you are trying to accomodate by not changing the API.

How does gethostbyname() *know for sure* what the character set of its
argument is?  Answer: it doesn't, because there is no way that it can.

---

From: Keith Moore <[email-masked]>
Date: Sat, 17 Feb 2001 13:10:06 -0500

> argument is?  Answer: it doesn't, because there is no way that it can.

Let me suggest a different answer:

gethostbyname() knows what the character set of the argument is
because, by definition, it is ASCII.

it's a bad idea to try to change this, because if you do, you break 
one of the assumptions on which code that uses gethostbyname 
was written.  you also create new failure modes which existing
code does not anticipate.

(as another example - when the ctype macros/functions were made 
locale-specific, it broke code that was using them to try to parse 
protocol elements that required ASCII. )

if you want an interface to look up addresses from IDNs - whether it be 
in native format, UTF-8, or whatever - you need a different function.
don't overload gethostbyname.

but don't try to retroactively change an existing interface.

Keith

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 17 Feb 2001 16:35:12 -0800

>because, by definition, it is ASCII.

FYI: In the IDNA algorithm, nameprep and ace encoding is done 
_before_ gethostbyname() is called, for exactly the reasons Keith 
lists.

   paf

---

From: Marc Blanchet <[email-masked]>
Date: Sat, 17 Feb 2001 22:37:23 -0500

At/À 16:35 2001-02-17 -0800, Patrik Fältström you wrote/vous écriviez:
>FYI: In the IDNA algorithm, nameprep and ace encoding is done _before_ 
>gethostbyname() is called, for exactly the reasons Keith lists.

- One thing we also have to consider is the API (gethost*) have been 
redefined for ipv6 support, where the new api support both ipv4 and 
ipv6:  RFC2553.  IDN call should handle both old and new non-idn-aware-api.

Marc.

Marc Blanchet
Viagénie inc.
tel: 418-656-9254
http://www.viagenie.qc.ca

----------------------------------------------------------
Normos (http://www.normos.org): Internet standards portal:
IETF RFC, drafts, IANA, W3C, ATMForum, ISO, ... all in one place.

---

From: "D. J. Bernstein" <[email-masked]>
Date: 18 Feb 2001 17:31:58 -0000

Keith Moore writes:
  [ extending gethostbyname() to allow UTF-8 ]
> new failure modes

Namely? Give us the details, Keith.

> but don't try to retroactively change an existing interface.

Don't be an idiot. There are some interfaces that we _want_ to extend.
Sysadmins want to type non-ASCII characters when they're editing zone
files, for example, and users want to be able to type those non-ASCII
characters on the ssh command line. If we can support these extensions
at low cost by extending some other interfaces too, great!

---Dan

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 20 Feb 2001 11:52:30 +0100 (CET)

> but don't try to retroactively change an existing interface.

I agree fully.

I think some of the experience in dealing with this for IPv6 can
be useful, since it had much of the same issues - unmodified applications
can't in general cope with gethostbyname returning an IPv6
address just as unmodified applications can't cope with gethostbyADDR
returning an IDN (in your favorite character set) name.

*If* (and this isn't obvious to me) there is a need for APIs 
that encapsulte ace encoding (or ace + nameprep) together with
the IP address lookups I think it would make sense to look
at getaddrinfo() and getnameinfo().
Those APIs are recommended for IPv6 address lookups *and* they
have flags fields that it easy to extended the e.g. for IDN purposes.

My 2 cents,
  Erik

---

From: Keith Moore <[email-masked]>
Date: Tue, 20 Feb 2001 09:41:00 -0500

> have flags fields that it easy to extended the e.g. for IDN purposes.

getxxxinfo also avoids a huge design flaw in gethostbyxxx - 
getxxxinfo can indicate temporary failures.

Keith