Subject: Re: [I18n-discuss] draft-freytag-troublesome-characters

From: Bill Jouris <[email-masked]>
Date: Fri, 7 Jul 2017 18:13:35 +0000 (UTC)

It seems that this would be substantially clearer and more usable, not to mention probably shorter, if it started by laying out a few general principles (Examples are taken from Latin Script, as this is the one I am most familiar with):

0. Scripts.  For any given DNS level, only one script MAY be used.
That is, if you have paypal in primarily Latin letters, you MUST NOT use a Cyrillic "a" for the second character.
1. Pre-composed codepoints. If a precomposed codepoint exists in Unicode, you MUST NOT use the basic letter and a combining character to recreate the pre-composed one.
   
Thus, since U+00E9 LATIN SMALL LETTER E WITH ACUTE exists, combining U+0065 LATIN SMALL LETTER E and U+0301 COMBINING ACUTE ACCENT is not allowed. 

2. Overlays. If a precomposed codepoint is used, the combining form of the same diacritic MUST NOT be used as well.
Thus, you cannot combine U+00E9 with U+0031.  Note that, because U+0061 LATIN SMALL LETTER I can be seens as a precomposed codepoint for U+0131 LATIN SMALL LETTER DOTLESS I plus U_0307 COMBINING DOT ABOVE, the specific case of U+0069 plus U+0307 is addressed. 

3. Combinations.  A combining diacritic or other combining codepoint MUST only be used if there is a language (identified by omniglot or other reliable source) which uses that combination. Thus, for example, you can ONLY use U+0071 LATIN SMALL LETTER Q plus U+0304 COMBINING MACRON if you can show that there is a language which uses that combined symbol. 

There remain a few cases which are ambiguous.  For example, those which entail multiple diacritic marks.  Consider the Yuroba letter hen with a high tone.  This can be created either using U+1EB9 plus U+0300 or by using U+00E8 plus U+0323. (My suggestion would be that we mandate using the tonal combining diacritic, while disallowing the other version.) Bill Jouris
Inside Products

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 10 Jul 2017 12:39:33 -0400

Hi,

Thanks for reading this.  I haven't cleared the remarks below with my co-authors.

> That is, if you have paypal in primarily Latin letters, you MUST NOT use a Cyrillic "a" for the second character.

This won't work, because some languages are written in multiple
scripts and in other cases there are doubtless possible reasons to
combine scripts.  The _general_ approach is probably wise for many
close-to-the-root names, but it's not plain that we could make such a
generic rule.  Remember, the point of the registry is not to create
new restrictions on what people can do, but instead to give a helper
tool to people to make it possible to improve policies already
required by IDNA2008.

> Thus, since U+00E9 LATIN SMALL LETTER E WITH ACUTE exists, combining U+0065 LATIN SMALL LETTER E and U+0301 COMBINING ACUTE ACCENT is not allowed. 

Your example fails, at least, because it's fixed by NFC.  In general,
under Unicode rules, there are only a couple possibilities:

    1.  Precomposed and decomposed forms of the "same character" are
    available.  If these are really the same character then
    normalization solves the problem.

    2.  A precomposed form for an already-composable decomposed form
    does not exist.  In this case, a precomposed form will not be
    added because of the stability rules.

    3.  A precomposed character that has strong similarity with a
    composable character is needed.  The characters are not the same
    character, even though they are for historical reasons extremely
    similar.  This case is at the extreme end of confusables
    ("homoglyphs"), and there may also be socio-linguistic reasons why
    the characters are problematic.  These cases _can't_ be caught by
    Unicode, but they're also not "the same character", so rules of
    the sort you're suggesting won't catch them.

(3) is really one of the kinds of cases I think we need the registry
for.

> Thus, you cannot combine U+00E9 with U+0031.  Note that, because U+0061 LATIN SMALL LETTER I can be seens as a precomposed codepoint for U+0131 LATIN SMALL LETTER DOTLESS I plus U_0307 COMBINING DO…

I'm not entirely sure I understand this suggestion, but if I do it
sounds like it would make a lot of Vietnamese problematic.

> 3. Combinations.  A combining diacritic or other combining codepoint MUST only be used if there is a language (identified by omniglot or other reliable source) which uses that combination. Thus, fo…

Why is this a requirement?  DNS labels are not words and they're not in any language.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]