Subject: [xmpp] common IDN mappings

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 07 Apr 2011 16:28:49 -0600

I've started working on rfc6122bis (yes, already!). I think we need to
define common mappings for internationalized domain names (IDNs) so that
we have consistent treatment across the XMPP network. These mappings
will go beyond what is defined for IDNA2008 (RFC 5895 is optional for
generic IDNA applications but IMHO we want to be more specific for XMPP,
and we might want different mappings anyway).

I am thinking about the following ordered list of operations to be
applied to IDNs:

1. Map uppercase and titlecase characters to their lowercase
equivalents. (I.e., we would not preserve case in XMPP, as is done in
IDNA2008 and domain name processing more generally; this is consistent
with the traditional approach in the XMPP community.)

2. Map all characters using Unicode Normalization Form D (NFD). (By
contrast, RFC 5895 suggests NFC.)

However, I've run into a bit of a snag. It seems that IDN is more
strongly tied to NFC than I had realized:

http://www.alvestrand.no/pipermail/idna-update/2011-April/007042.html

The upshot seems to be that we will need to define XMPP domainparts
without reference to U-labels or even IDN (!) if we want to say that
XMPP uses NFD. In essence we would have 4 different layers of string
processing:

1. Raw string = a UTF-8 encoded string of Unicode code points to which
no case mapping or normalization has been applied. This is the kind of
string that a client might receive from the operating system as a result
of user input or somesuch.

2. Domainpart = a UTF-8 encoded string of Unicode code points that has
been case-mapped to lowercase and to which Normalization Form D has been
applied. A raw string would be converted into a domainpart by
lowercasing and then normalizing via NFD. Only domainparts (not raw
strings, not U-labels, not A-labels) would be communicated over the wire
in XMPP.

3. U-label = a UTF-8 encoded string of Unicode code points that has been
case-mapped to lowercase and to which Normalization Form C has been
applied. A domainpart would be converted into a U-label by performing
the recomposition part of NFC. A U-label might be communicated from an
XMPP application (e.g., client or server) to a DNS API that can take
U-labels as input. However, it's necessary to do this only when the
application needs to resolve a domain name via DNS, not during normal
XMPP processing.

4. A-label = an ASCII string that is generated by applying the Punycode
algorithm (RFC 3492) to a U-label and then adding the ACE prefix
("xn--"). I think the conversion from U-label to A-label could be
performed by an IDN-aware DNS API, but it's possible that an XMPP
application would need to complete this conversion (again, only when the
application needs to resolve a domain name).

IMHO this four-fold taxonomy would minimize the work an XMPP application
would need to do when handling domain names on the XMPP network, while
enabling us to convert domain names into U-labels or A-labels only when
absolutely necessary. However, I might be missing some subtleties here,
so feedback is welcome. One wrinkle is that we'd need to write 6122bis
so that domainparts don't reference U-labels or even IDNs (because IDNs
are defined in terms of either A-labels or U-labels and we don't want a
dependency on NFC if we can avoid it). This might be a challenge for the
spec writer, but shouldn't bother implementers much...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Jehan Pagès <[email-masked]>
Date: Sun, 10 Apr 2011 03:49:04 +0900

Hi,

> with the traditional approach in the XMPP community.)

I agree with this. Though I do like to be precise about cases when I
write, for the Internet in general, one knows people don't really
care. So that would be basically a security risk to difference cases.
Making all cases equivalent prevent people being lured by a user
taking the same nick with different case.

>  [...]

For all the rest, I will have to re-read any necessary RFCs related to
IDNs as I did this some time ago.

Jehan