Subject: Re: Testing the concensus of the krb-wg on IDN

From: Ienup Sung <[email-masked]>
Date: Thu, 23 Oct 2003 23:17:07 -0700 (PDT)

[ Please disregard the previous email from me; it was sent by accident.
  Please use this email instead.]

For both principal and realm names when case conversions are required
also for the IDN domain-style names/components, it would be quite
desireable to have the mappings defined for the case conversions as
definitely as possible in my opinion, i.e., rules and mappings should
accompany or co-exist in the extension for the IDN case mappings.

This is mainly due to that any of existing Stringprep profiles does not
take care of the case conversions (to lower or to upper) but only the
case foldings.

Also, IDN + SASLprep + the extension (in my understanding) brings an
issue of name equivalence in terms of user expectations. For instance,
in terms of German user expectations, should the following be seen as
equivalent/same component of principal in UTF-8 or not:

	strasse.de
	straße.de

and also the following in terms of CJK user expectations:

	test.co.jp
	ｔｅｓｔ。ｃｏ。ｊｐ

where it is not glyph variation but with different code point.

I also think some additional canonicalization is necessary before and
also after the Stringprep processing. 

Also, attached please find issue list on the IDN considerations. I also
tried to add some of my suggestions on what should do on the IDN names
before deriving the realms/principal names.

With regards,

Ienup

] Date: Mon, 20 Oct 2003 12:40:45 -0400
] From: Jeffrey Hutzelman <[email-masked]>
] Subject: Re: Testing the concensus of the krb-wg on IDN
] To: Nicolas Williams <[email-masked]>, Kerberos WG 
<[email-masked]>
] MIME-version: 1.0
] Content-transfer-encoding: 7bit
] 
] 
] 
] On Monday, October 20, 2003 07:43:08 -0700 Nicolas Williams 
] <[email-masked]> wrote:
] 
] > On Sun, Oct 19, 2003 at 03:21:54PM -0700, Nicolas Williams wrote:
] >> I disagree.  The case recommendation for realm names is relevant to
] >> interoperability.
] >
] > To clarifiy this statement: the case recommendations of rfc1510/
] > clarifications are implemented, for the most part, by humans rather than
] > software - so the case recommendations in rfc1510/clarifications aren't
] > that relevant to interop, no.
] >
] > But will users be generally able to manage case in non-ASCII names?
] >
] > What if users cut-n-paste text that uses characters they're not familiar
] > with?  (Perhaps users shouldn't do that, but won't they?)
] 
] 
] I really think we need to be talking about realm and principal names 
] separately here.
] 
] Applications rarely just make up realm names from domain names.  Realm 
] names are either provided by the user (via cut-and-paste or manual entry of 
] something he knows) or obtained from the Kerberos library based on local 
] configuration or cross-realm referral from a KDC (which presumably also has 
] local configuration).  The realm names in local configuration were provided 
] by some administrator, either by cut-and-paste or by manual entry.
] 
] The point is, virually all realm names are obtained either by literal 
] copying of something that is already a realm name, or by manual entry of 
] something known to the user -- not by deriving them from DNS domain names. 
] The only time a realm name should be obtained by derivation from a DNS 
] domain name is when a Kerberos administrator is choosing the name of his 
] realm.  Our document defines a convention for that case (upcase the name of 
] a DNS domain you control), but only for that case.  With regards to 
] determining what realm to use in a Kerberos request, we say:
] 
]    ...and a
]    mapping to a Kerberos realm derived syntactically from the domain
]    part of the specified hostname and information from the local
]    Kerberos realms database. [1.2]
] 
]    Before sending a request to the ticket-granting service, the client
]    MUST determine in which realm the application server is believed to
]    be registered [3.3.1]
] 
]    When establishing a new realm name based on an internet domain
]    name it is recommended by convention that the characters be converted
]    to upper case. [6.1]
] 
] In other words, when creating a realm, you end up deriving the name from 
] DNS without any other information.  But an application looking up the realm 
] name for a service will have access to local configuration information 
] telling it at a minimum what realms exist, and most likely containing a 
] mapping from patterns to match against domain names to realm names.
] 
] I don't think we need to say any more about realm names than clarifications 
] already does.  Does someone disagree?  If so, can we split the discussion 
] as it pertains to realms into a separate thread?
] 
] 
] Principal names, however, are another story.  While principal names are not 
] domain names and should not normally be treated as such, there are many 
] cases in which application protocols specify that the principal name for a 
] service is to be constructed from the name of the server on which the 
] service resides.  We even provide a name type for this, and guidance to the 
] effect that
] 
]    If the first component of a name identifies a service and there is a
]    single component following the service name identifying the instance
]    as the host on which the server is running, then the name type SRV-
]    HST SHOULD be used. [6.2]
] 
] and
] 
]    Where the name of the host is not
]    case sensitive (for example, with Internet domain names) the name of
]    the host MUST be lower case. [6.2.1]
] 
] Clearly this is enough guidance for a pre-IDN, clarifications world, in 
] which both hostnames and Kerberos principal names are restricted to 
] US-ASCII.  But just as clearly, IMHO, it's _not_ enough guidance for an 
] IDN+extensions world, in which both names admit richer character sets. 
] So...
] 
] - What guidance should kerberos-extensions provide to application protocols 
] constructing service principal names from Internet domain names?
] 
] 
] -- Jeff
] 
]

---

From: Jeffrey Altman <[email-masked]>
Date: Fri, 24 Oct 2003 09:05:14 -0400

Ienup:

Thank you for the list you have provided and the deep thought which has 
gone into it.  Since our discussion I have been wrapping my head among 
the issues.  I will be sure to address your concerns in our upcoming work.

Jeffrey Altman

Ienup Sung wrote:

>------------------------------------------------------------------------
>
>								10/23/2003
>							[email-masked]
>
>Internationalized Domain Name considerations at Kerberos extension
>------------------------------------------------------------------
>
>1. Overview
>
>Many existing implementations and system administrators derive Kerberos Realm
>and Principal Names from Internet domain-style domain names [1, 2].
>
>While it is rather and relatively a simple task when the coded character
>set (codeset) is a 7-bit ASCII, when the repertoire of the characters to
>support increases and have to support various scripts and languages at
>the same time, the complexity also increases and there are issues that
>the existing standards have not been anticipated. Furthermore,
>recent standardization of Internationalized Domain Name (IDN) and accompanying
>application layer architecture/model called Internationalized Domain Names in
>Applications (IDNA) [3], further increase the complexity and uncertainty on
>the compatible support of the IDN.
>
>This memo tries to describe what are the surrounding issues when we move to
>the Unicode character set repertoire and also when we have to support IDN
>and also have maximum compatibility with the IDNA implementations.
>
>This memo does not cover or deal with any other internationalization issues
>at the moment. The U+XXXX and U+YYYYYY are the usual Unicode notations where
>the XXXX and the YYYYYY denote code point values in the Unicode coding space.
>
>
>2. Issues and Possible Resolution Suggestions
>
>In my opinion, the following issues should be mentioned at the
>extension and hopefully with normative assertions/guidelines/recommendations
>on what should be done:
>
>- Issue-1: IDNA allows various label separators such as U+002E (full stop),
>	U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), and
>	U+FF61 (halfwidth ideographic full stop).
>- Resolution-1: Before such domain-style names be used at the Kerberos
>	implementations to derive the Realm, Principal Names, or both, such
>	label separators should all be mapped to 7-bit ASCII/UTF-8 full-stop
>	character ('.' which is U+002E). This is only for Internet
>	domain-style names.
>
>- Issue-2: IDNA allows both ASCII Compatible Encoding (ACE) [3, 5] names and
>	IDN names with 8-bit characters as possible user input.
>- Resolution-2: The Kerberos extension will use UTF-8 as the codeset of
>	the choice; thus if user input is in ACE which starts with "xn--"
>	prefix, then, such names must be converted back to non-ACE, UTF-8
>	IDN names through the [3]'s conversion steps before the use. For
>	any IDN names with 8-bit characters with non-UTF-8 codsets/codepages,
>	such names should be must be converted to UTF-8 IDN names before
>	the use if the name is not in the UTF-8.
>
>- Issue-3: Since the Nameprep [4] used as the Stringprep [5] profile at
>	the IDNA does the case-folding [7] mappings before the Normalization
>	Form KC (NFKC) [8], the IDNA effectively allows various equivalent IDN
>	names for a domain name (or, hostname for that matter). On the other
>	hand, the SASLprep [9] that will be used by the extension does
>	not employee the case-folding mapping and, as a consequence, generates
>	different results for various equivalent names.
>- Resolution-3: Before the use, any IDN names with 8-bit characters must be
>	converted to UTF-8 if necessary and then the Nameprep must be applied;
>	this will ensure that the various equivalent names map to a single
>	Realm and Principal names derived.
>
>- Issue-4: SASLprep and Nameprep prohibits Private Use Area (PUA) characters
>	and hence denying characters/character codes of user-defined characters
>	commonly used in Asian countries at the Principal names.
>- Resolution-4: Either accept that as the limitation, update the
>	SASLprep and allow the PUA characters, or come up with
>	a Kerberosprep profile that will allow the PUA characters by
>	omitting the Stringprep Table C.3 from the "Prohibited Output"
>	section.
>
>- Issue-5: The clarification [2] defines Realm and PrincipalName be encoded
>	as KerberosString. The KerberosString type includes ACE string by
>	definition.
>- Resolution-5: To reduce future confusions, the exclusion recommendation of
>	the ACE encoded string at the KerberosString with a rationale is
>	desired preferably at the [2] and also at the extension.
>	If to allow the ACE, then, also ample descriptions on consequences
>	and also additional info should be given at the [2] and the extension.
>
>- Issue-6: Case conversions are required for the Principals and Realms;
>	Unicode introduces complications to case mappings due to 
>	the character set includes characters from various scripts and
>	languages.
>- Resolution-6: If the case conversions are also required for the IDN names,
>	they must be done as normative steps for the Realm and Principal Names.
>	In doing so, it is necessary to provide the rules and mapping tables
>	to have the deterministic case conversions; the rules and mappings
>	should come from [7] and standard Unicode data files. Furthermore,
>	considering normalized names are necessary to have deterministic name
>	matchings in my opinion, the case conversions should not be done alone
>	but should be done before or possibly together with the normalization
>	depending on the normalization form that will be used for the Realm
>	and the Principal names. Also, considering there is no mapping tables
>	defined for the case conversions in any of the existing
>	profiles or the Stringprep itself, it appears that the WG should
>	come up with case conversion mapping tables that will be used at
>	the Stringprep profile's mapping step prior to or along with
>	the normalization step. (More investigation and study would
>	required on this issue by the way.)
>
>- Issue-7: Unlike case-foldings that can be done in *pretty much* locale-
>	independent way, case conversions can yield and also require different
>	per-locale conversion mappings.
>- Resolution-7: Either there should be additional mappings at the mapping
>	step of the Stringprep profile that will be used for the extension
>	so that such locale-specific case conversion differences will be
>	erased (in a sense, a small set of case-folding or semi-case-folding,
>	if you will, on certain characters) or allow such per-locale conversions
>	and treat the converted names not equivalent.
>
>
>3. References
>
>[1] RFC 1510 Kerberos
>[2] Kerberos clarification Internet Draft
>[3] RFC 3490 IDNA
>[4] RFC 3491 Nameprep
>[5] RFC 3492 Punycode
>[6] RFC 3454 Stringprep
>[7] UAX21 Case Mappings for Unicode 3.2
>[8] UAX15 Unicode Normalization Forms for Unicode 3.2
>[9] SASLprep Internet Draft
>  
>

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 24 Oct 2003 08:37:53 -0700

> accompany or co-exist in the extension for the IDN case mappings.

Yes, I've come around to this view also.

I'm also concluding that the fact that private or new naming conventions
can use domain names in ways that existing software don't know about is
not a big deal - users can deal with this, to some extent, and we could
have something to say about how to express such naming conventions so
that new software could automatically do the right thing.

> 	stra?e.de
> 	??????????

This should be handled by nameprep - if it isn't, oh well, it cannot be
our place to address this now.

Lately I've been advocating recommending (requiring) that where one
knows that a principal name component [or a realm] is a domain name (an
IDN) then apply saslprep(nameprep(x)) to the string in question.

Apparently, current practice is to always lower-case domain names as
principal name components, in software, but to leave realm names as
entered (though it is recommended that realm names be created in all
upper case).  We should extend this to IDNs and recommend that non-ASCII
realm names also be case folded in SW.

> also after the Stringprep processing. 

After?  Because of your above examples?  I'm not convinced.

> before deriving the realms/principal names.

Thanks!

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Fri, 24 Oct 2003 14:33:56 -0400

> realm names also be case folded in SW.

This is very nearly a correct description of what happens.  Kerberos 
doesn't downcase anything passed to it _as a principal name_, but it does 
provide tools for applications to use in constructing principal names from 
domain names, and such tools do downcase the domain name.  IMHO it would be 
appropriate for applications to apply nameprep to domain names before using 
them to construct principal names (but the Kerberos spec needs to be 
explicit about whether applications MUST or MUST NOT do this; ambiguity 
will lead to trouble).  I think it's up to the implementors whether 
krb5_sname_to_principal() or its equivalent runs nameprep or expects its 
caller to have already done so; I can see advantages to both approaches.

Also, I believe your description of current practice wrt realm names is 
correct -- Kerberos never upcases a realm name passed to it, though it 
might upcase a domain name being used to construct a realm name, as in the 
last-resort case of finding the realm for a service.  Of course, this 
entire discussion is relevant only for domain-style realm names; we 
certainly wouldn't want to randomly go around applying nameprep or case 
conversion to X.500-style realm names!

I'm beginning to wonder whether "upcase your domain name" is a realistic 
recommendation in an IDN world.  For example, I must assume that 
'strasse.de' and 'straße.de' are in fact not the same domain; however, a 
German realm administrator will upcase them both as 'STRASSE.DE'.  It might 
be better to suggest "upcase your domain name only if it is all ASCII". 
That would be consistent with the text we have regarding principal name 
components, which requires case folding only "Where the name of the host is 
not sensitive" -- a condition which is met for ASCII domain names, but not 
for IDN names.

Comments?

-- Jeff

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 24 Oct 2003 11:59:05 -0700

> conversion to X.500-style realm names!

Heh - X.500-style realm names.  Can we deprecate those?  Does anyone use
them?

> recommendation in an IDN world.  For example, I must assume that 

It might be, but I'd rather downcase, primarily so we can use the case
mapping table of stringprep/nameprep as the reference.

OTOH, I've found the traditional upcasing of relam names useful as it is
a loud reminder that something is a Kerberos V principal name and not
some other type of name that just happens to lookup very similar to
Kerberos V principal names in some cases.

> be better to suggest "upcase your domain name only if it is all ASCII". 

Yes, "upcase your domain name only if it is all ASCII" - there's not
need to say "otherwise downcase it" because nameprep takes care of that :)

> not sensitive" -- a condition which is met for ASCII domain names, but not 

Nameprep downcases non-ASCII names.

> Comments?

See above.

I think we'll want to have a word about naming conventions and the name
type hint.  Specifically it would be good to strongly recommend, or
maybe require, that principal naming conventions involving domain names
be unambiguous with respect to other principal naming conventions such
that: a) such principal naming conventions can be expressed in some
simple language, and b) the intended naming convention for some
principal name can be determined by inspecting the principal name.  The
name type hint could be used to disambiguate between otherwise ambiguous
naming conventions, if provided, but it'd be best if all _new_ naming
conventions were required to be unambiguous wrt to others.

This then would, for example, allow krb5_parse_name() to properly set
the name type hint and properly nameprep() the relevant components of
its inputs, in most cases.

And krb5_parse_name() could even apply ToUnicode() to ACEs as
appropriate.  (Although IDNA does not ban non-ACE names that start with
the ACE prefix - it couldn't because such names might have existed
pre-IDNA, which concern applies here as well; but still, I'm willing to
go with such a ban, at least at the moment.)

And krb5_parse_name() could do all this even if/when new naming
conventions are added later.  Yes, definitiona of the conventions would
have to be distributed, but this is hardly a common event and, in any
case, they could be stored in a distributed directory service.

Cheers,

Nico
--