Subject: [I18ndir] draft-faltstrom-unicode11-07

From: "Pete Resnick" <[email-masked]>
Date: Mon, 04 Feb 2019 10:34:58 -0600

Folks,

This is going to be our first "group review" document. Peter and I will 
assign a person to be "pen holder" for the review sometime today. In the 
meanwhile, please read over 
https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/ and be 
prepared to discuss it tout de suite. We want to get this review done in 
the next 2 or 3 weeks.

pr
-- 
Pete Resnick http://www.episteme.net/
All connections to the world are tenuous at best

---

From: "Pete Resnick" <[email-masked]>
Date: Wed, 06 Feb 2019 13:07:13 -0600

On 4 Feb 2019, at 10:34, Pete Resnick wrote:

> Folks,
>
> This is going to be our first "group review" document. Peter and I 
> will assign a person to be "pen holder" for the review sometime today. 
> In the meanwhile, please read over 
> prepared to discuss it tout de suite. We want to get this review done 
> in the next 2 or 3 weeks.

Harald has graciously agreed to be the pen-holder for the review of this 
document. The plan of action is to have an open discussion in the 
directorate to help Harald formulate that review. As with other reviews, 
the outcome is normally one of:

Not Ready
On the Right Track
Almost Ready
Ready with Issues
Ready with Nits
Ready

so think in those terms when you are reviewing the document and make 
comments that help decide what the outcome of the review is.

The IETF Last Call ends March 5, so let's try to get this done in the 
next 3 weeks.

Go forth and discuss.

pr

---

From: "Pete Resnick" <[email-masked]>
Date: Wed, 13 Feb 2019 17:02:08 -0600

Except for John's note, it's been a week with no comments. Please, let's 
get on the ball here and read this document and make any comments that 
you have so that Harald can write up his review.

pr

On 6 Feb 2019, at 13:07, Pete Resnick wrote:

>> will assign a person to be "pen holder" for the review sometime 
>> today. In the meanwhile, please read over 
> Harald has graciously agreed to be the pen-holder for the review of 
> this document. The plan of action is to have an open discussion in the 
> directorate to help Harald formulate that review. As with other 
> reviews, the outcome is normally one of:
>
> Not Ready
> On the Right Track
> Almost Ready
> Ready with Issues
> Ready with Nits
> Ready
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 13 Feb 2019 16:28:27 -0700

Thanks for the poke! I'll read it again this evening.

---

From: Martin J. Dürst <[email-masked]>
Date: Thu, 14 Feb 2019 05:36:53 +0000

Hello Pete,

> you have so that Harald can write up his review.

I have read the document, except for the tables. I hope to have time 
next week to write up my comments (mostly nits). My overall judgement 
would be somewhere between "Ready with Issues" and "Ready with Nits".

The main issue I found is a the end of the Introduction, where it says:

                                                  The current document
    resolves this issue and suggests that the IDNA2008 standard followsO
    the Unicode Standard and not update RFC 5892 [RFC5892] or any other
    IDNA2008 RFCs.

I think it would be good to make clear that this is the recommendation 
at this point in time, i.e. up to and including Unicode 11. This would 
make it clearer that we are not sure what may or may not happen in the 
future. I'd suggest e.g. the following wording:

                                                  The current document
    resolves this issue and suggests that the IDNA2008 standard, for
    the current update to Unicode 11.0.0, follow the Unicode Standard
    and not update RFC 5892 [RFC5892] or any other IDNA2008 RFCs.

I have not in any way checked (nor even printed out) the tables. It says 
that the tables will be used by IANA once the Expert Reviewer approves 
them. But I just checked https://www.iana.org/protocols, and the Expert 
Reviewer is Patrik. I have a lot of trust in Patrik, but if anybody on 
this list has some time to do a cross-check, I think that would be great.

Regards,   Martin.

---

From: "Pete Resnick" <[email-masked]>
Date: Thu, 14 Feb 2019 09:52:25 -0600

> would be somewhere between "Ready with Issues" and "Ready with Nits".

Thanks Martin. I do want to point you and everyone at John's review 
<https://mailarchive.ietf.org/arch/msg/i18ndir/vFddEKMIN0DR55QALal16H_1WYw> 
to see if you have any comments on it. In particular:

	TL;DR summary of this note: It is extremely unwise for the IETF
	to process draft-faltstrom-unicode11-07 before a number of
	other issues (and associated documents) are carefully
	considered and resolved.  The main body of the rest of this
	note is divided into two parts: a summary of the issues and the
	more obvious of the associated documents and a discussion of
	why processing draft-faltstrom-unicode11 without first
	addressing those issues poses significant risks for IDNA, for
	the IETF, and for the Internet. Those parts are followed by a
	third which makes some summary recommendations based on them.
	I'd encourage people to read and try to understand both but
	those who have too little time may want to look at the second
	part first.  The note ends with what I hope are two
	constructive suggestions as to how we might move forward.

I'm sure it would help Harald very much to make sure that we comment on 
eachother's reviews. (I.e., "Oh my, John's right. Ignore that I said 
'Ready'" or "John's mistaken in his review and here's why..." would add 
quite a bit.) Though Harald's a very smart person and I'm sure can 
analyze these things for himself, the reason we planned to have these 
sorts of discussions on the broader i18n docs was to get some good 
analysis done among us all.

pr

---

From: Asmus Freytag <[email-masked]>
Date: Thu, 14 Feb 2019 09:12:54 -0800

> in the next 2 or 3 weeks.

All,

My recommendation is to process this after resolving the minor issues 
mentioned. I see no need to delay this any further, particularly as it 
is already being overtaken by the release of Unicode 12.0 scheduled for 
March 5.

I like section 3. It does need copy editing.

Section 4.1 suffers a bit from mischaracterizing the nature of combining 
marks in Arabic; the marks do not at all combine as deterministically as 
Latin/Greek/Cyrillic diacritics.

The section also cites an internet draft that is not going to be RFCs 
before this document is completed. Perhaps the draft text in question 
could be excerpted in an appendix. (If the other ID moves forward it can 
then cite this RFC instead. Or something).

Section 4.3 claims U+29A1 is PVALID from the beginning, but I find this 
in the IANA IDNA paramater files:

2701-2B4C 	DISALLOWED 	UPPER BLADE SCISSORS..RIGHTWARDS ARROW ABOVE 
REVERSE TILDE OPERATOR

There appears to be a contradiction; it should be reviewed and addressed 
as necessary.

It may be advisable to update this section (and the remainder of the 
draft) to cover Unicode 12.0, which is to be released on March 5, 2019, 
which is before any likely date this RFC can be published. (The data 
files for Unicode 12.0.0 are up and this close to a release should be 
final enough for drafting).

This would pick up 554 new characters and  the following 4 cases of 
change in properties (none problematic):

----------------------------------------------- gc (General_Category)
gc	166D	'Po'	->	'So'	CANADIAN SYLLABICS CHI SIGN
gc	1CF2	'Mc'	->	'Lo'	VEDIC SIGN ARDHAVISARGA
gc	1CF3	'Mc'	->	'Lo'	VEDIC SIGN ROTATED ARDHAVISARGA
gc	A9BD	'Mc'	->	'Mn'	JAVANESE CONSONANT SIGN KERET

----------------------------------------------- bc (Bidi_Class)
bc	A9BD	'L'	->	'NSM'	JAVANESE CONSONANT SIGN KERET
bc	1ED00	'R'	->	'AL'	
bc	1ED3E	'R'	->	'AL'	
bc	1ED3F	'R'	->	'AL'	
bc	1ED40	'R'	->	'AL'	
bc	1ED41	'R'	->	'AL'	
bc	1ED42	'R'	->	'AL'	
bc	1ED43	'R'	->	'AL'	
bc	1ED44	'R'	->	'AL'	
bc	1ED45	'R'	->	'AL'	
bc	1ED46	'R'	->	'AL'	
bc	1ED47	'R'	->	'AL'	
bc	1ED48	'R'	->	'AL'	
bc	1ED49	'R'	->	'AL'	
bc	1ED4A	'R'	->	'AL'	
bc	1ED4B	'R'	->	'AL'	
bc	1ED4C	'R'	->	'AL'	
bc	1ED4D	'R'	->	'AL'	
bc	1ED4E	'R'	->	'AL'	
bc	1ED4F	'R'	->	'AL'	

Note: The range 1ED3F - !ED4F plus 1ED00 reflect a change from default 
value for unassigned code point to actual property for new character, so 
can be ignored.

I like section 5.

A./

PS: note that there will be a Unicode 12.1 shortly due to the pending 
transition in Japan. It will add one code point for a new Japanese Era.

---

From: "Marc Blanchet" <[email-masked]>
Date: Thu, 14 Feb 2019 15:31:36 -0500

Hello,
  read it (did not check the content of Appendix A). comments:
a) it says in the intro:  « It further suggests a path forward for the 
IETF to ensure IDNA2008 follows the evolution of the Unicode 
Standard. ». Unless I skip something, it is far from clear to me that 
when a new version of Unicode is out (as one is expecting pretty soon 
and as Asmus wrote, will likely to happen before the RFC is published), 
what is the exact path forward? Are we going to issue a new RFC every 
time a new Unicode standard is published? My suggestion would be to add 
a section such as: « Guidance to the IANA expert reviewer » and 
write some rules so that we don’t need an RFC or IETF concensus for 
no-issue-unicode-new-version. Some text such as:

<section title=« Guidance to the IANA Expert Reviewer »>
If a new version of Unicode is published, the IANA expert reviewer for 
the IDNA2008 registries should verify that: a) no codepoint in the 
previous version of Unicode changes from PVALID to DISALLOWED as a 
result of applying the rules and properties to the new Unicode version, 
b) no change in normalization of sequences c) …   The result of the 
audit by the expert should be shared to the [email-masked] 
mailing list for other people to confirm. If any of the criteria is not 
met, then the expert should not confirm to IANA to proceed to the 
publication of the new registries of the new Unicode version and report 
to the appropriate Area Director for further instructions. If all 
criteria is met and the audit is confirmed by other people, then the 
expert can confirm to IANA to proceed with the publication of the new 
registries of the new Unicode version.

b) I think it would be appropriate to cite the Root-zone LGR for 
registries to look for advice on how to limit the number of codepoints. 
To my knowledge, this is the only process, for most live scripts, that 
creates such an extensive list, with full review from script 
communities. This could be added at the end of third paragraph of 
section 5, or last paragraph of section 3.2.

c) as proposed by Asmus, I think given timing, we should aim for 
including Unicode 12 in it as it is imminent.

Regards, Marc.

On 4 Feb 2019, at 11:34, Pete Resnick wrote:

> I18ndir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i18ndir

---

From: "Patrik Fältström" <[email-masked]>
Date: Thu, 14 Feb 2019 22:07:53 +0100

On 14 Feb 2019, at 21:31, Marc Blanchet wrote:

> Are we going to issue a new RFC every time a new Unicode standard is published?

Without commenting on your text:

As the expert I have every time "an incompatibility" is detected flagged that for IAB and the IETF (and whatever WG that works with related issues).

What you see in this document is a collection of those "notes". And as you can see, some Unicode versions have been released without any such incompatible changes.

   Patrik

---

From: "Marc Blanchet" <[email-masked]>
Date: Thu, 14 Feb 2019 16:11:31 -0500

On 14 Feb 2019, at 16:07, Patrik Fältström wrote:

> As the expert I have every time "an incompatibility" is detected 
> flagged that for IAB and the IETF (and whatever WG that works with 
> related issues).

I’m sure!

>
> What you see in this document is a collection of those "notes". And as 
> you can see, some Unicode versions have been released without any such 
> incompatible changes.

so we agree then! that there is a need for explicitly state it, in case 
PAF is moving to do farming ;-) and is replaced.

Again, what is missing in the document is clear guidance so we don’t 
ask ourselves what to do and we don’t rely on a single person « good 
behavior ».

Marc.

---

From: "Patrik Fältström" <[email-masked]>
Date: Thu, 14 Feb 2019 22:19:04 +0100

On 14 Feb 2019, at 22:11, Marc Blanchet wrote:

> so we agree then!

I think so, yes.

   paf

---

From: "Patrik Fältström" <[email-masked]>
Date: Thu, 14 Feb 2019 22:33:10 +0100

On 14 Feb 2019, at 18:12, Asmus Freytag wrote:

> Section 4.3 claims U+29A1 is PVALID from the beginning, but I find this in the IANA IDNA paramater files:
>
> 2701-2B4C 	DISALLOWED 	UPPER BLADE SCISSORS..RIGHTWARDS ARROW ABOVE REVERSE TILDE OPERATOR
>
> There appears to be a contradiction; it should be reviewed and addressed as necessary.

Will investigate this more closely. I might have written the wrong code point.

> It may be advisable to update this section (and the remainder of the draft) to cover Unicode 12.0, which is to be released on March 5, 2019, which is before any likely date this RFC can be published. (The data files for Unicode 12.0.0 are up and this close to a release should be final enough for drafting).

In the early version of Unicode 12.0.0 that we have today (unfortunately different version numbers on every file, so there is no firm release version) these are my findings:

-  No change in derived property value from version 11.0.0
   where the property value moves FROM anything else than
   UNASSIGNED

That by itself is for me enough to say, "ok this is good to go".

When looking more closely I see:

- 272 new PVALID
- 282 new DISALLOWED

When looking even more closely I see in Unicode 11.0.0:

02A3;PVALID;I;A;LATIN SMALL LETTER DZ DIGRAPH
0322;PVALID;I;A;COMBINING RETROFLEX HOOK BELOW
AB66;UNASSIGNED;I;J;<reserved>
AB67;UNASSIGNED;I;J;<reserved>

And in Unicode 12.0.0:

02A3;PVALID;I;A;LATIN SMALL LETTER DZ DIGRAPH
0322;PVALID;I;A;COMBINING RETROFLEX HOOK BELOW
AB66;PVALID;I;A;LATIN SMALL LETTER DZ DIGRAPH WITH RETROFLEX HOOK
AB67;PVALID;I;A;LATIN SMALL LETTER TS DIGRAPH WITH RETROFLEX HOOK

Is that a concern to people?

There might be other things in there. This I just saw when browsing differences between 11 and 12.

If IETF "trust" Unicode, and then "trust" registries to implement a good policy, then we are fine.

   Patrik

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Thu, 14 Feb 2019 15:40:29 -0800

> Will investigate this more closely. I might have written the wrong code point.

OK.

> Is that a concern to people?

No, "hooks" are never intended to be "composed" onto characters. These 
are not precomposed versions of combining sequences, just like O-slash 
is not decomposable and never has been, even though there are combining 
slash overlay characters.

This is a longstanding policy by Unicode.

A./

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 14 Feb 2019 19:52:26 -0700

On 2/14/19 8:52 AM, Pete Resnick wrote:

> I do want to point you and everyone at John's review
> https://mailarchive.ietf.org/arch/msg/i18ndir/vFddEKMIN0DR55QALal16H_1WYw to
> see if you have any comments on it. In particular:
> 
> |TL;DR summary of this note: It is extremely unwise for the IETF to
> process draft-faltstrom-unicode11-07 before a number of other issues
> (and associated documents) are carefully considered and resolved.|

I think John raises valid concerns, which I understand as:

1. We really do need to figure out what to do about the issues described
in draft-klensin-idna-5892upd-unicode70, draft-klensin-idna-rfc5891bis,
and draft-freytag-troublesome-characters.

2. Advancing draft-faltstrom-unicode11 as-is could be perceived as
ignoring those issues.

3. We now have a proper venue in which to address those issues, so
perhaps we should give ourselves a chance to succeed.

4. At the least, we might want to include more explicit warnings in
draft-faltstrom-unicode11 about the risks of allowing "troublesome"
characters.

To illustrate why I think John's concerns might be valid, consider this
paragraph in §4.1 of draft-faltstrom-unicode11-07:

   U+08A1 is discussed in draft-freytag-troublesome-characters
   [I-D.freytag-troublesome-characters] and other Internet-Drafts.
   Regardless of whether the discussion of those drafts ends in
   recommendations to include the code point in the repertoire of
   characters permissable for registration or not, it is still
   acceptable to allow the code point to have a derived property value
   of PVALID.

Why would we define a code point as PVALID if we might conclude that it
should not be permitted in registrations?

As always, I stand ready to be corrected in matters of internationalization.

Peter

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 15 Feb 2019 13:32:31 -0800

> draft-faltstrom-unicode11 about the risks of allowing "troublesome"

A strong(er) warning about the risk from simply allowing any combination 
of PVALID code points would be appropriate and it fits into the general 
argument already laid out.

I keep trying to get people to understand that it's not just repertoire 
(which code points) but also the allowed combinations that matters. Here 
is where most peoples (sole) familiarity with the Latin script prevents 
them from easily seeing the issues faced in all complex scripts.

Once we publish LGR-3 later this year, that could be a wake-up call.

> should not be permitted in registrations?

Because the code point that should not be permitted in registrations 
isn't the U+08A1 but the combining mark.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 15 Feb 2019 13:37:35 -0800

I've separated these out, because they are not about the draft, but 
about the process.

> and draft-freytag-troublesome-characters.

Agreed.

These are issues that are worth addressing - if we get the critical mass 
of engagement.

> 2. Advancing draft-faltstrom-unicode11 as-is could be perceived as

Do not agree.  Holding hostage a draft that will fall behind the ever 
more recent Unicode versions if it is held up isn't the way to make 
progress. It's just a way to demonstrate that IETF can't move at all on 
this issue.

A./

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Feb 2019 09:10:01 -0500

(the discussions are drifting in several directions, so I'm
going to try to separate subject lines a bit)

--On Friday, February 15, 2019 13:37 -0800 Asmus Freytag
<[email-masked]> wrote:

>> 2. Advancing draft-faltstrom-unicode11 as-is could be
>> perceived as ignoring those issues.
> 
> Do not agree.  Holding hostage a draft that will fall behind
> the ever more recent Unicode versions if it is held up isn't
> the way to make progress. It's just a way to demonstrate that
> IETF can't move at all on this issue.

I will respond to your slightly earlier and IMO more important
note and to the rest of Peter's fairly soon and then try to get
to a more substantive analysis of the I-D, but let me address
this one issue first.

First, I think describing my concerns as trying to "hold a draft
hostage" is completely inappropriate.  It is pejorative and
tends --whether intentionally or not -- to [further] obscure the
very substantive reasons for not moving forward with this I-D
before other issues are addressed.  Avoiding such language is
especially important because there is, at least in principle, a
way to address most, if not all, of my concerns without holding
the document up until we can resolve the, IMO complex, issues
raised by those other documents.

The core objection to draft-faltstrom-unicode11-07 is that it
essentially says "none of the issues that have been raised are
unimportant with regard to the RFC 5892-required evaluation and
passing them by doesn't set any precedent".  It does not say
that in so many words but, by not addressing those issues (or
even referencing) the rather fundamental issues for which my 6
February note was an attempt at a summary, it blows them off and
puts an unreasonable, if not impossible, burden on trying to
address those issues in the future because this I-D has already
concluded that they are not important enough to hold anything up.

If we (and whatever portion of the IETF cares and has enough
knowledge for their opinions to be competent) think it is
important to advance this document without reaching conclusions
about those issues, than one alternative would be to add a
section to it that identifies those issues (a quick summary and
list of references would do if that were possible), explicitly
explains why and how it is important to move this I-D forward,
and specifies a path forward for doing those evaluations and
fitting their results --whatever they might be-- back into the
specifications and model.   If it really isn't important to
address those issues before this document is published -- if the
document doesn't preempt any of the plausible outcomes of those
analyses and resulting recommendations-- then it should be
rather easy to write that paragraph.   Conversely, if the
paragraph is hard or impossible to write, it is an even stronger
argument for delaying it until the substantive issues can be
addressed.

Finally, whether or not the IETF is actually capable of engaging
on these issues was precisely the subject of the BOF and, at
least IMO, in the experiment represented by this directorate.
If the answer, as you suggest, is that "IETF can't move at all
on this issue", then I suggest it would be useful to agree on
that and then figure out either what to do given that or if we
have any contributions to make to where the work is picked up
and how.  Pretending that the IETF is doing something when it
cannot actually engage thoughtfully and competently and move
forward might cover over a short term problem or two (I continue
to wonder about who or what is creating the sense of urgency)
but is ultimately not helpful for the Internet.

    best,
       john

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Feb 2019 15:51:45 -0500

--On Friday, February 15, 2019 13:32 -0800 Asmus Freytag
<[email-masked]> wrote:

>> 4. At the least, we might want to include more explicit
>> warnings in draft-faltstrom-unicode11 about the risks of
>> allowing "troublesome" characters.

> A strong(er) warning about the risk from simply allowing any
> combination of PVALID code points would be appropriate and it
> fits into the general argument already laid out.

At the same time, both of the suggestions above strengthen the
case for processing draft-faltstrom-unicode11 only in context
with other work.  That warning is precisely the topic of
draft-klensin-idna-rfc5891bis, which clarifies and amplifies on
material that is already in RFC 5890, 5891, and 5894.  Unlike
draft-faltstrom-unicode11-07, which is exclusively about
checking and validation of the combination of Unicode 11.0 (and
7, 8, 9, and 10) against the rule structure of RFC 5892, those
documents are where these warnings should go.   Putting them
into draft-faltstrom-unicode11 would do violence to the
structure of the IDNA2008 documents and would risk introducing
confusion by causing possible confusion between the
presumably-new warning and the existing text, in part because
the warning isn't new.

Now there are questions about whether the warning statements in
draft-klensin-idna-rfc5891bis-01 are adequate, whether they need
to be strengthened, or, in principle, whether the text that is
already present in IDNA2008 is adequate and we just need to get
people to read it.  But those questions are ones we tried to ask
nearly two years ago (significantly longer if one counts from
when preliminary drafts of what became
draft-klensin-idna-rfc5891bis-00 started circulating privately)
and which we haven't been able to get onto a serious IETF
discussion agenda.  To say now that a (stronger) warning about
not just using the PVALID set uncritically is in order is to say
that it is time to either take up that earlier document or to
reinvent it and to do so prior to putting
draft-faltstrom-unicode11 out there by itself and risking
implicitly delivering the message that no (additional) warnings
are needed.

> I keep trying to get people to understand that it's not just
> repertoire (which code points) but also the allowed
> combinations that matters. Here is where most peoples (sole)
> familiarity with the Latin script prevents them from easily
> seeing the issues faced in all complex scripts.

At the same time, when IDNA2008 was completed and approved, we
thought the messages were perfectly clear that it was just an
upper-limit, "don't even think about using labels that do not
conform to these specs", collection but that IDNA would never
work for end users in a satisfactory way unless additional
restrictions were applied to individual proposed labels by those
who actually understood the scripts used in those labels.   One
thing we didn't spend enough time thinking about was how to get
people to actually take that layering model and restriction
seriously enough.  Another is that the requirement was, in
retrospect, was apparently stated in a way that allowed people
to miss it accidentally or to claim ignorance while claiming to
have read and understood the spec (that is the issue that first
motivated draft-klensin-idna-rfc5891bis).  And it is possible
that those who run for-profit registries have sufficient
incentives to ignore the "don't delegate what you don't
understand" rule that nothing the IETF says will have any
effect, in which case we should be asking hard questions about
how much of this work is worth the effort.

I'm also prepared to believe that your work on the "troublesome"
list (see below) and the Generation and Integration Panel work
more generally strongly suggests (maybe even proves) that we
botched the Contextual Rule model and that the relevant
documents should be revised to fix that (if such a fix is even
possible) or at least to contain a strong warning about how few
problematic cases those rules protect against in their current
form.    If others think that is the case, I anxiously and
enthusiastically await an I-D and believe it should be posted
and processed before we issue a document that will be
interpreted (and some obviously intend to be interpreted) as
"everything is just fine".

However, IMO and repeating myself, all of those questions should
be addressed before the IETF pushes out a document whose premise
is that all is ok. 

> Once we publish LGR-3 later this year, that could be a wake-up

Sorry to be pessimistic, but one could have said that about
LGR-2, LGR-1, the efforts (some successful) to get the
equivalent of "don't delegate what you don't understand" into
several generations of ICANN guidelines that precede the VIP
effort, etc.  I certainly hope LGR-3 will be more successful in
raising awareness and compliance, but, noting that it has not
even  been proposed as an IETF standards-track document or gone
through the IETF consensus process, my expectation is that any
attempt to persuade people to apply it to other than the root
will turn into a discussion about contracted and non-contracted
parties, different types of domains, language identification,
etc., most of which are uniquely ICANN problems and most of the
rest of which are a waste of time.  I couldn't make those
discussions up but we've seen all of them before.

I do expect that LGR-3 will be helpful, as several generations
of guidelines and exploration of specific examples have been, to
those registries who actually are trying to confine their
delegations to strings the actually do understand and who are
trying to learn and avoid traps.   But, for them, while all of
this is helpful, none of it (including
draft-faltstrom-unicode11) is necessary, especially when we
remember that the normative requirements of IDNA2008 are tied to
the rules, not a particular list of code points in an IANA
registry.

>> To illustrate why I think John's concerns might be valid,
>> consider this paragraph in §4.1 of
>> draft-faltstrom-unicode11-07:
>> 
>>     U+08A1 is discussed in
>>     [I-D.freytag-troublesome-characters] and other
>>     Internet-Drafts. Regardless of whether the discussion of
>>     those drafts ends in recommendations to include the code
>>     point in the repertoire of characters permissable for
>>     registration or not, it is still acceptable to allow the
>>     code point to have a derived property value of PVALID.
>> 
>> Why would we define a code point as PVALID if we might
>> conclude that it should not be permitted in registrations?

Because of the same reason that, for example, bans archaic
scripts from the root but allows them if an appropriate registry
knows what it is doing and has a legitimate use for that script.
Unless we are prepared to say that the decision that Unicode
made to include that code point was so seriously wrong-headed
that someone writing Fulani / Fula in Arabic script would need
to write the character as a combining sequence, we need a far
stronger reason than distaste for making it, by exception,
DISALLOWED.   From a core IDNA2008 standard point of view, the
difficulty with U+08A1 has always been that it does not
decompose as some reasonable people would expect, not that it is
not a legitimate code point for a letter.  Now, should it be
permitted in registrations in zones that are actually
multilingual and/or multiscript, especially in zones that allow
Arabic-script labels derived from or associated with other
languages that use that script?  Almost certainly not.  But that
is a rather different question and one that IDNA2008 almost
certainly has no practical way to address.

> Because the code point that should not be permitted in
> registrations isn't the U+08A1 but the combining mark.

I am not sure what you mean by this.   Are you proposing to ban
all combining marks? If so, that would do violence to scripts
and new code points that are actually coded the way Unicode
claimed (around the time IDNA2008 was designed) were going to be
coded.  If you think that IDNA2008 should be modified to contain
a list of scripts for which combining sequences are not allowed,
I think that would be a really interesting proposal (one for
which the work of the ICANN processes could provide a great
starting point), but no one has made it yet and it would be yet
another argument for holding off processing
draft-faltstrom-unicode11 until that proposal was ready lest the
number of registered labels that would be invalidated would
increase even more.

Similarly, if you are proposing adding either a contextual rule
for (combining) Hamza Above (U+0654) (and maybe U+0655) or
prohibiting them entirely by exceptions, that might be a
reasonable thing to do but (a) it would be exactly the sort of
thing one would expect to see in draft-faltstrom-unicode11 (if
that can't be done now, it can't be done at all) and (b) I'm
sure someone would stand up and complain that it should not be
allowed as a change to the rules because it might invalidate
currently-delegated labels, the non-binding recommendations to
not use combining characters in the Arabic script
notwithstanding.   I can even guess who would stand up first and
what new "transitional rules" would promptly appear although
that should not be significant to the issue.

As a final issue for this piece of the discussion -- and more
evidence that draft-faltstrom-unicode11 should not be processed
and approved before these other issues and documents are
addressed, others should note that there is a basic program with
normative (or even explanatory) use of
draft-freytag-troublesome-characters.  We've discussed this
before, but one of the fundamental assumptions behind IDNA2008
(and even IDNA2003, although it was a bit less explicit there)
was that, a very small number of obvious issues and code points
notwithstanding, the IETF was going to stay out of the business
of making code point by code point evaluations of Unicode or
signing off on those made by others except as part of The
Unicode Standard.  Until we have consensus for changing that
principle (and probably a plan for how to cross-check your work
and maintain the table beyond your tenure with it, I think
"troublesome" is probably a non-starter as an IETF standards
track document, important reference that is expected to be
useful for a long time, or anything other than "Asmus's personal
list of code points he believes to be troublesome".  That is
clearly a procedural problem rather than a technical one or in
any way a criticism of the list (which I personally believe to
be very valuable), but it is a procedural problem that is hard
to ignore when we propose to reference it from normative
documents.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Feb 2019 16:03:42 -0500

Gentlemen,

In addition to the issues identified below (on which, AFAICT,
only the two of you and Asmus have commented in the week and a
half since it was posted), I note that the Last Call on the I-D
has been issued and discussion (albeit limited) has started on
the IETF list.   I have every confidence in Harald's ability to
write a good summary but, if it written in the usual style of
directorate-as-review-team documents, I'd expect that at least
some IESG members who would prefer to not be bothered by these
issues would more or less ignore it.

I don't have time -- and I assume few others on this list have
time -- to participate in parallel discussions on the two lists
(this one and the IETF one).  I also believe that reasoned
comments on the IETF list that disagree with the conclusions of
some of us should be responded to there so that their authors
(as well as the whole IESG) see the responses and might even be
educated by whatever discussion follows.  So I now have a
dilemma as to whether to keep reading and responding here or to
put this list aside and start paying attention to the IETF one.

Please advise.

   john

--On Thursday, February 14, 2019 19:52 -0700 Peter Saint-Andre
<[email-masked]> wrote:

>> https://mailarchive.ietf.org/arch/msg/i18ndir/vFddEKMIN0DR55Q
>> ALal16H_1WYw to see if you have any comments on it. In
>> particular:
>> 
>> |TL;DR summary of this note: It is extremely unwise for the
>> IETF to process draft-faltstrom-unicode11-07 before a number
>> of other issues (and associated documents) are carefully
>> considered and resolved.|
> 
> 1. We really do need to figure out what to do about the issues
> described in draft-klensin-idna-5892upd-unicode70,
> draft-klensin-idna-rfc5891bis, and
> draft-freytag-troublesome-characters.
> 
> 3. We now have a proper venue in which to address those
> issues, so perhaps we should give ourselves a chance to
> succeed.
>

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Feb 2019 16:12:59 -0500

--On Thursday, February 14, 2019 15:40 -0800 "Asmus Freytag (c)"
<[email-masked]> wrote:

> No, "hooks" are never intended to be "composed" onto
> characters. These are not precomposed versions of combining
> sequences, just like O-slash is not decomposable and never has
> been, even though there are combining slash overlay characters.
> 

Then, just for my education (and that of anyone else who might
be interested), why was U+0322 assigned a code point at all?
Or, alternately, why has it not been deprecated?  If "hooks" are
never intended to be "composed" onto characters, then having a
composing hook available would seem to be useful only for
simulating characters that have not yet been assigned code
points (something I first heard people associated with Unicode
argue was a bad idea during the discussion around the balloting
of ISO 10646 DIS-2) and for causing, however inadvertently,
confusion and/or mischief.

Just wondering.

     john

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Feb 2019 16:32:20 -0500

--On Thursday, February 14, 2019 22:33 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

> If IETF "trust" Unicode, and then "trust" registries to
> implement a good policy, then we are fine.

No comment about the Unicode part at the moment but...

Given history, "Emoji domain names" and even "Unicode domain
names", I suggest that anyone who believes/ trusts those
registries is fairly close to delusional.  Do you disagree and,
if so, on what basis?  

And do remember, as I'm certain you do but comments earlier in
that note do not make it obvious, that IDNA2008 imposes
requirements on lookup applications, not just registries.

   john

---

From: Asmus Freytag <[email-masked]>
Date: Sat, 16 Feb 2019 14:56:07 -0800

> confusion and/or mischief.

John,

this is a useful question and I hope that answering it will be 
productive in not only moving forward with Patrik's draft but also with 
coming to a consensus on the other pieces of the puzzle. Let me give 
some background and more detail:

(1)

Unicode must cater to people writing their language and to scholars 
writing about language. They have different requirements.

Slash overlays, for example, can be used in scientific notation or to 
indicate that a letter or symbol is "cancelled" in some way. That is 
different from the use of "o-slash" in Scandinavian languages, where it 
is a full member of the orthography (sorting, for example, not like an 
"o" with diacritic, but after 'z').

The hook is an entirely parallel case (except perhaps that it is a more 
specialized shape and therefore its use in notations isn't as 
generically common).

(2)

What's missing is a NOT_RECOMMENDED status for the combining slash, the 
combining hook and other characters in the context of identifiers. That 
is what the "troublesome" draft was trying to address in its own way.

 From the Unicode point of view, "deprecated" doesn't enter, because 
those characters do have a function in a universal character encoding - 
even if they should never be used in identifiers (or at least not in 
situations where they can be misused in identifiers).

The latter distinguishes the protocol (IDNA2008) from other layers: many 
code points can be used in mischievous ways if in an under-regulated 
public zone, but can be fine as part of, say subdomain naming under the 
control of a single entity.

(3)

Going back to Latin diacritics that are "attached" or "overlaid": These 
also include at the minimum the various strokes, for example.

For strokes and other overlays, there isn't a generic positioning 
(unlike Dot-above or Dot-below and the other detached ones). Strokes can 
go through any part of a characters glyph shape: at different heights, 
as well as through just one of the "stems" in a character's shape, or 
they can be of different widths. Only characters that are part of an 
orthography would have a settled shape, and in principle, different 
orthographies can come to different conventions even for the same base 
letter. In short, attempting to cover these cases via "glyph 
composition" aren't satisfactory.

For linguistic and other notational uses, a "generic" stroke overlay is 
at once sufficient and necessary: notations are notoriously open-ended; 
in principle any code point might be "annotated" with a generic overlay 
or attached mark. Trying to encode any of those explicitly cannot 
succeed. (During the time DIS-2 was discussed, linguists were 
circulating a "preliminary" list of 2,000 Latin characters with various 
marks. That was in the early '90s. Can you imagine what that list would 
be today?)

(4)

In conclusion: certain combining characters are NOT (and have never 
been) intended for "simulating characters that have not yet been 
assigned code points". UTC is quite consistent with stating that if 
there are such unencoded characters (used in orthographies) that they 
will be encoded if they require the use of combining  marks that UTC 
where UTC does not feel a "decomposition" is canonically equivalent. 
Realistically, up to that point of encoding, one would not expect 
significant use of such orthographies (which tend to be for small 
populations or tend to be "emerging"); so this issue isn't a practical 
problem looking forward, it's allowing unrestricted use of combining 
marks in identifiers that is the problem.

(5)

IDNA2008 should have had a context rule for all combining marks that 
disallowed more than one adjacent instance of the same combining mark. 
At the time, less so now, there existed too many implementations that 
would overprint these. I'm not aware of any examples where such 
duplication is required.

All combining marks should have had a status of "CONTEXT_REQUIRED" 
meaning that including them in any LGR (registry policy) _requires_ the 
use of policy-specific contexts. (This is different from CONTEXTO / 
CONTEXTJ which are context restrictions on the protocol level).

These registry-level required contexts can take one of two forms: one is 
to enumerate all permissible combinations (most orthographies for 
European scripts require only a finite set, most of which are 
precomposed, but a much smaller list requires combination). The other, 
and applicable to complex scripts, is to have generic rules describing 
allowable combinations without enumerating them. Finally, some scripts 
(Arabic and Hebrew) use combining marks in ways that are optional for 
the writing system. In these cases, the easiest "context" is to forbid 
their use.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Sat, 16 Feb 2019 16:29:04 -0800

John,

I would separate out two things:

(1) Getting consensus on how to progress with "rfc5891bis" and 
"troublesome".

(2) Getting the mess around "halting" IDNA parameters cleared up.

Conveniently, the two are on different levels. (2) is about the 
protocol. As written, and as Patrik points out, users aren't required to 
use the published IDNA parameters but can (and do) calculate their own. 
Being "silent" on the protocol end doesn't do anyone any good. 
Implementations are moving forward as if the process hadn't been 
"halted". Therefore, the highest priority is to restart that process.

We do not have consensus that any particular subset of characters 
represents an issue for all uses of the protocol - and even in public 
zones, many can be safely used with proper mitigation (which may require 
context rules rather than repertoire restrictions, or, in some cases, 
some other solution, like variants).

We've also learned that such mitigation is required for many characters 
that were in place from the earliest versions of Unicode. That makes 
changing the protocol retroactively rather more unappealing, while 
raising the urgency to establish best practices at the other layers:

(1) is about getting serious with the notion that the protocol's 
definition of PVALID is only an upper bound and that PVALID and 
"safe-to-use" are quite different concepts.

We should work on getting consensus on:

(a) that these issues must be addressed

(b) a framework on how to address them

(c) how to communicate best practices

With framework, I mean that we need to decide whether we want to stop at 
issuing unspecific warnings and exhortation to "allow only what you 
understand", or whether we can break down the issue for implementers to 
help them and their users understand whether solutions are sufficient.

Communication then includes necessarily a list of specifics: characters, 
their particular issues and some classification and/or default 
mitigation steps appropriate for each.

The process issue is (a), rfc5891bis and troublesome overlap in (b) and 
troublesome (in some incarnation) would cover (c).

Having broken down the issue, I would now recommend Patrik's draft to 
give a warning that is a suitable forward pointer:

Declare that the current draft (Unicode 11 update) addresses how to move 
forward with the protocol, but that the protocol must be applied with 
caution (that foreshadows rfc5891bis) and that some characters may 
require special attention in the context of registry policies (that 
foreshadows "troublesome").

> who actually understood the scripts used in those labels.

This language of "upper limit" is still the language of 'repertoire'; 
the real
restrictions would be those of the 'name space' or 'label space'.

>    One
> motivated draft-klensin-idna-rfc5891bis).

In my answer to your "Hooks" message, I suggest that we should devise
a way to label code points as:

(a) NOT_RECOMMENDED

(b) CONTEXT_REQUIRED

and other classification such as e.g. VARIANT_RECOMMENDED

These would be classifications that are not part of the protocol, but
make clear that certain very specific actions by registries would be
appropriate.

> And it is possible

A conrete, testable framework would be appropriate, because it
allows third parties to evaluate the safety of registry policies.

"Nebulous" and unspecific warnings: "Do only what you understand"
are worse than unhelpful in my view.

> form.

It may seem that way, but I disagree that this can be "fixed" at the 
protocol
level. The reason is that the protocol really can't address issues that are
dependent on language or on that status of current implementations.

For example, for the Root Zone, for Thai, we had to accept context
restrictions that severely impact one of the minority languages. The
reason is that current implementations are biased to the majority
language to such an extent that using the minority languages features
is not safe for identifiers. That's something that does not belong
in a protocol.

The second reason is that many context rules are only approximations
and that conceivably they aren't the "final word". For one, we know
that allowing digits will change the formulation of some of the rules,
but also require additional mitigation (some digits are "variants" of
letter shape - i.e. look identical).

However, as outlined above, I think we need a "framework" of definitions
and concepts that can be applied on the layer above the protocol.

>   If others think that is the case, I anxiously and

The premise is not that "all is OK", but that all issues that are 
specific to
new characters and that MUST be handled on the protocol level are addressed.

Making that more explicit is fine.

> effort, etc.

You misunderstood what I was thinking was specific to LGR-3:

LGR-3 will add 9 South Asian scripts. More so, than the three scripts 
from SEA in LGR-2, these scripts are depending on a combination of 
context rules and variants. They (nearly) complete the inventory of 
complex scripts and therefore LGR-3, for the first time, can serve as 
resource that lets any implementer gauge what might be required outside 
the better understood realm of European + CJK scripts.

> I certainly hope LGR-3 will be more successful in

The proper way to use the information from LGR-3 in an IETF 
standards-track document would be to extract a general framework that 
lists certain types of code points and spells out in general detail what 
sorts of restrictions (or other mitigation) are appropriate for each.

If we restrict this effort (as the Root Zone LGR does) to those 
orthographies in common modern use (for secular and everyday purposes), 
then it becomes feasible to provide not only the framework, but most of 
the data as standards-track.

The reason is that these orthographies are now well understood; new 
characters for them are something that's reasonably rare to very rare 
and updating a data collection is a much more limited effort than 
tracking all of Unicode (as IDNA2008) does.

It would mean to basically declare that anything you do with ancient 
letters for modern scripts or for ancient scripts is your "unsafe" 
business, but it would allow a specification against which it would be 
possible to meaningfully evaluate existing registry policies, IDN tables 
and or LGRs.

> the rules, not a particular list of code points in an IANA

I think we need to give up being comprehensive with characters, but 
standardize a list of code points that fall into certain known 
categories which require (or strongly recommend) certain mitigations.

If we don't ever get around to doing that work for Egyptian 
Hieroglyphics, that's too bad, and if for that reason any registry that 
supports them is seen as "unsafe" that's too bad (actually good).

I'm thinking of a standards-track document that combines a normative 
framework with detailed specifications that are all RECOMMENDED rather 
than MUST. In other words, a strong Best Practices as opposed to 
Mandatory Practices.

But something that does not presume that *all* readers know what they 
don't understand, and that *all* readers are magically able to get 
detailed information about scripts and languages that they don't speak. 
(That's the current state of affairs).

> draft-freytag-troublesome-characters.

The current draft should be seen as a bit of a data collection - I'm 
thinking
that the best way to address the issue may take on a rather different shape.

>   We've discussed this
> Unicode Standard.

I fully agree with this position. The Unicode Standard, being 
"universal" is too open ended to allow anyone else to  do the work of 
"second-guessing" it. For another, at the time new code points are added 
to the Unicode standard, there's limited information about them; you 
will not find online corpora encoded in them that you can study and 
evaluate, because for those to exist, the characters first have to be 
encoded.

None of these restrictions apply to _modern orthographies in 
common-everyday use_. By users, 99%+ are covered, by count of 
orthographies, we are perhaps at 97% and by count of scripts, the 28 
scripts of the root zone probably make up 90% (allowing for a handful of 
smaller scripts used by modern populations, such as First Nations in 
Canada).

Effectively, compared to the moving target of Unicode as a whole, the 
needs of modern orthographies are a *static target.*

That changes the presumptions. With ICANN's root zone project as the 
base, it would be quite feasible to evaluate known issues surrounding 
code points needed for these modern orthographies only.

We are talking about less than 3K code points total (+ 11K Hangul and 
19K Han for which there are existing best practice already).

Even, so, IETF would not have been able to muster the dedicated teams of 
linguists and programmers that did the work for the Root Zone. However, 
once that work is published, it doesn't need to be repeated, but the 
information collected can be used to analyze the code space for modern 
orthographies and to classify it into categories.

Those categories should be a subdivision of PVALID and CONTEXTx code 
points, highlighting those for which special attention is RECOMMENDED.

Unlike the Root Zone LGR, IETF should not specify the details of such 
"special attention", just the general nature of it. That could allow 
such work to reach a standards-track level of consensus, and further 
insulate such a specification from the effects of new facts emerging 
that may effect minor details more than broad categories.

> Until we have consensus for changing that

I think we should not reference the current ID from normative documents, 
but issue a generic description of the kind of problem that registries 
must be prepared to address.

Then follow up with preferably a standards-track document that delivers 
on a framework (and possibly a standardized categorization of modern 
orthographies' characters).

Incidentally, the approach of IDNA2008 isn't dead, even if we go for 
something like that. Because in creating a sub-categorization for 
PVALID, we can use Unicode properties to sift out most of the characters 
that do not require attention.

How so? Ordinary letters and digits as well as, for complex scripts, 
consonants (and independent vowels) normally do not pose issues and do 
not need to get restricted by context rules (the only restriction is the 
modern vs. obsolete/specialized/historic subset).

However, any combining mark, almost invariably needs more attention.

And, like IDNA2008 there are "exceptions": a small list of code points 
that look identical by design which require restriction or treatment as 
variants as appropriate.

You may realize from that, that I'm less enamored of a "database" than 
an actual specific list (which I think will prove stable enough to allow 
maintaining it in RFC form). Anything PVALID outside the list becomes 
"<default> PVALID" and thus "buyers' beware, but legal" - which is the 
current status anyway.

Ergo, adopting a framework plus categorization allows an incremental 
improvement on a layer above the protocol, but is specific enough to 
allow third parties to verify that at least known problem cases were 
handled responsibly. With the current advice "don't and anything you 
don't understand" - such scrutiny is not possible.

>>> To illustrate ....

The remainder of this section of your message is implicitly addressed in 
the thread "Hooks" -- with the additional wrinkle that combining marks 
in Arabic (as in Hebrew) function differently from diacritics despite 
superficial similarities. They are also somewhat optional and more than 
one group defining registry policy decided to opt out on combining marks 
for these scripts altogether - something not possible in other scripts.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 16 Feb 2019 17:47:10 -0800

John,

I did the reverse - please read the other replies first.

> raised by those other documents.

I've addressed my view on how best to treat the interrelatedness in 
another post.

I'm sorry you feel strongly about the language - I do feel strongly 
about tying everything into such an interrelated bundle that nothing can 
move forward. I think that's not productive, and we spend too much time 
on worrying about meta issues.

> concluded that they are not important enough to hold anything up.

As stated elsewhere, my take is Patrik's draft is about  what is 
required on the protocol level. There's a German word that I wish I had 
a fine translation for: "Unbedenklichkeitsbescheinigung". You can see, 
by its length that it covers a full concept. It means something like a 
certification of being unproblematic. That is what Patrik's draft is NOT 
doing; that's what IDNA2008 is NOT doing - it makes a more limited 
declaration of "validity".

> specifications and model.

That's more in line of what I was thinking.

>   If it really isn't important to

If that paragraph is impossible to write, we just pack up and go home.

My thinking (see other posts) is that there is a clear (or at least 
principled) distinction that can be made between protocol and other 
layers and that the protocol layer is by force the most permissive. 
Therefore, nothing in Patrik's draft would detract from being less 
permissive on other layers. (Ultimately with the Root Zone being least 
permissive - I can't see anything in his draft that would undermine that 
work, so why would it undermine anything done on an intermediate or meta 
level - see my other posts).

> on this issue",

No, I'm actually optimistic that IETF can act on the issue, but that the 
core of the "issue" isn't centered on Patrik's draft.

Largely because there's not one issue mentioned in it that doesn't apply 
to the very first version of Unicode covered by IDNA2008. Nothing added 
from Unicode 7.0.0 forward presents a new issue and that's why I so 
strongly object to the idea that holding up this draft is meaningful.

I'm optimistic that once we get Patrik's draft out of the way, we can 
return to rfc5891bis and "troublesome" and refine those. I think that 
refining these requires some new thinking. I've outlined some in my 
other posts.

We don't just need an exhortation for registries, we need to give them a 
"Framework" for doing so and details. (See other posts).

>   then I suggest it would be useful to agree on
---

A./

---

From: Asmus Freytag <[email-masked]>
Date: Sat, 16 Feb 2019 18:14:39 -0800

>> but is ultimately not helpful for the Internet.
The three dashes mean I ran out of steam for the day. :)

Too much discussion.

Can we find a way to structure this better?

A./

---

From: "Patrik Fältström" <[email-masked]>
Date: Sun, 17 Feb 2019 07:54:30 +0100

On 16 Feb 2019, at 23:56, Asmus Freytag wrote:

> What's missing is a NOT_RECOMMENDED status for the combining slash, the combining hook and other characters in the context of identifiers. That is what the "troublesome" draft was trying to address in its own way.

I had a look what PVALID code points have the word "COMBINING" in its name. We talk about 380 of them in Unicode 12.0.0:

U+0300 COMBINING GRAVE ACCENT
U+0301 COMBINING ACUTE ACCENT
U+0302 COMBINING CIRCUMFLEX ACCENT
U+0303 COMBINING TILDE
U+0304 COMBINING MACRON
U+0305 COMBINING OVERLINE
U+0306 COMBINING BREVE
U+0307 COMBINING DOT ABOVE
U+0308 COMBINING DIAERESIS
U+0309 COMBINING HOOK ABOVE
U+030A COMBINING RING ABOVE
U+030B COMBINING DOUBLE ACUTE ACCENT
U+030C COMBINING CARON
U+030D COMBINING VERTICAL LINE ABOVE
U+030E COMBINING DOUBLE VERTICAL LINE ABOVE
U+030F COMBINING DOUBLE GRAVE ACCENT
U+0310 COMBINING CANDRABINDU
U+0311 COMBINING INVERTED BREVE
U+0312 COMBINING TURNED COMMA ABOVE
U+0313 COMBINING COMMA ABOVE
U+0314 COMBINING REVERSED COMMA ABOVE
U+0315 COMBINING COMMA ABOVE RIGHT
U+0316 COMBINING GRAVE ACCENT BELOW
U+0317 COMBINING ACUTE ACCENT BELOW
U+0318 COMBINING LEFT TACK BELOW
U+0319 COMBINING RIGHT TACK BELOW
U+031A COMBINING LEFT ANGLE ABOVE
U+031B COMBINING HORN
U+031C COMBINING LEFT HALF RING BELOW
U+031D COMBINING UP TACK BELOW
U+031E COMBINING DOWN TACK BELOW
U+031F COMBINING PLUS SIGN BELOW
U+0320 COMBINING MINUS SIGN BELOW
U+0321 COMBINING PALATALIZED HOOK BELOW
U+0322 COMBINING RETROFLEX HOOK BELOW
U+0323 COMBINING DOT BELOW
U+0324 COMBINING DIAERESIS BELOW
U+0325 COMBINING RING BELOW
U+0326 COMBINING COMMA BELOW
U+0327 COMBINING CEDILLA
U+0328 COMBINING OGONEK
U+0329 COMBINING VERTICAL LINE BELOW
U+032A COMBINING BRIDGE BELOW
U+032B COMBINING INVERTED DOUBLE ARCH BELOW
U+032C COMBINING CARON BELOW
U+032D COMBINING CIRCUMFLEX ACCENT BELOW
U+032E COMBINING BREVE BELOW
U+032F COMBINING INVERTED BREVE BELOW
U+0330 COMBINING TILDE BELOW
U+0331 COMBINING MACRON BELOW
U+0332 COMBINING LOW LINE
U+0333 COMBINING DOUBLE LOW LINE
U+0334 COMBINING TILDE OVERLAY
U+0335 COMBINING SHORT STROKE OVERLAY
U+0336 COMBINING LONG STROKE OVERLAY
U+0337 COMBINING SHORT SOLIDUS OVERLAY
U+0338 COMBINING LONG SOLIDUS OVERLAY
U+0339 COMBINING RIGHT HALF RING BELOW
U+033A COMBINING INVERTED BRIDGE BELOW
U+033B COMBINING SQUARE BELOW
U+033C COMBINING SEAGULL BELOW
U+033D COMBINING X ABOVE
U+033E COMBINING VERTICAL TILDE
U+033F COMBINING DOUBLE OVERLINE
U+0342 COMBINING GREEK PERISPOMENI
U+0346 COMBINING BRIDGE ABOVE
U+0347 COMBINING EQUALS SIGN BELOW
U+0348 COMBINING DOUBLE VERTICAL LINE BELOW
U+0349 COMBINING LEFT ANGLE BELOW
U+034A COMBINING NOT TILDE ABOVE
U+034B COMBINING HOMOTHETIC ABOVE
U+034C COMBINING ALMOST EQUAL TO ABOVE
U+034D COMBINING LEFT RIGHT ARROW BELOW
U+034E COMBINING UPWARDS ARROW BELOW
U+0350 COMBINING RIGHT ARROWHEAD ABOVE
U+0351 COMBINING LEFT HALF RING ABOVE
U+0352 COMBINING FERMATA
U+0353 COMBINING X BELOW
U+0354 COMBINING LEFT ARROWHEAD BELOW
U+0355 COMBINING RIGHT ARROWHEAD BELOW
U+0356 COMBINING RIGHT ARROWHEAD AND UP ARROWHEAD BELOW
U+0357 COMBINING RIGHT HALF RING ABOVE
U+0358 COMBINING DOT ABOVE RIGHT
U+0359 COMBINING ASTERISK BELOW
U+035A COMBINING DOUBLE RING BELOW
U+035B COMBINING ZIGZAG ABOVE
U+035C COMBINING DOUBLE BREVE BELOW
U+035D COMBINING DOUBLE BREVE
U+035E COMBINING DOUBLE MACRON
U+035F COMBINING DOUBLE MACRON BELOW
U+0360 COMBINING DOUBLE TILDE
U+0361 COMBINING DOUBLE INVERTED BREVE
U+0362 COMBINING DOUBLE RIGHTWARDS ARROW BELOW
U+0363 COMBINING LATIN SMALL LETTER A
U+0364 COMBINING LATIN SMALL LETTER E
U+0365 COMBINING LATIN SMALL LETTER I
U+0366 COMBINING LATIN SMALL LETTER O
U+0367 COMBINING LATIN SMALL LETTER U
U+0368 COMBINING LATIN SMALL LETTER C
U+0369 COMBINING LATIN SMALL LETTER D
U+036A COMBINING LATIN SMALL LETTER H
U+036B COMBINING LATIN SMALL LETTER M
U+036C COMBINING LATIN SMALL LETTER R
U+036D COMBINING LATIN SMALL LETTER T
U+036E COMBINING LATIN SMALL LETTER V
U+036F COMBINING LATIN SMALL LETTER X
U+0483 COMBINING CYRILLIC TITLO
U+0484 COMBINING CYRILLIC PALATALIZATION
U+0485 COMBINING CYRILLIC DASIA PNEUMATA
U+0486 COMBINING CYRILLIC PSILI PNEUMATA
U+0487 COMBINING CYRILLIC POKRYTIE
U+07EB NKO COMBINING SHORT HIGH TONE
U+07EC NKO COMBINING SHORT LOW TONE
U+07ED NKO COMBINING SHORT RISING TONE
U+07EE NKO COMBINING LONG DESCENDING TONE
U+07EF NKO COMBINING LONG HIGH TONE
U+07F0 NKO COMBINING LONG LOW TONE
U+07F1 NKO COMBINING LONG RISING TONE
U+07F2 NKO COMBINING NASALIZATION MARK
U+07F3 NKO COMBINING DOUBLE DOT ABOVE
U+0C00 TELUGU SIGN COMBINING CANDRABINDU ABOVE
U+0C04 TELUGU SIGN COMBINING ANUSVARA ABOVE
U+0D00 MALAYALAM SIGN COMBINING ANUSVARA ABOVE
U+135D ETHIOPIC COMBINING GEMINATION AND VOWEL LENGTH MARK
U+135E ETHIOPIC COMBINING VOWEL LENGTH MARK
U+135F ETHIOPIC COMBINING GEMINATION MARK
U+1A7F TAI THAM COMBINING CRYPTOGRAMMIC DOT
U+1AB0 COMBINING DOUBLED CIRCUMFLEX ACCENT
U+1AB1 COMBINING DIAERESIS-RING
U+1AB2 COMBINING INFINITY
U+1AB3 COMBINING DOWNWARDS ARROW
U+1AB4 COMBINING TRIPLE DOT
U+1AB5 COMBINING X-X BELOW
U+1AB6 COMBINING WIGGLY LINE BELOW
U+1AB7 COMBINING OPEN MARK BELOW
U+1AB8 COMBINING DOUBLE OPEN MARK BELOW
U+1AB9 COMBINING LIGHT CENTRALIZATION STROKE BELOW
U+1ABA COMBINING STRONG CENTRALIZATION STROKE BELOW
U+1ABB COMBINING PARENTHESES ABOVE
U+1ABC COMBINING DOUBLE PARENTHESES ABOVE
U+1ABD COMBINING PARENTHESES BELOW
U+1B6B BALINESE MUSICAL SYMBOL COMBINING TEGEH
U+1B6C BALINESE MUSICAL SYMBOL COMBINING ENDEP
U+1B6D BALINESE MUSICAL SYMBOL COMBINING KEMPUL
U+1B6E BALINESE MUSICAL SYMBOL COMBINING KEMPLI
U+1B6F BALINESE MUSICAL SYMBOL COMBINING JEGOGAN
U+1B70 BALINESE MUSICAL SYMBOL COMBINING KEMPUL WITH JEGOGAN
U+1B71 BALINESE MUSICAL SYMBOL COMBINING KEMPLI WITH JEGOGAN
U+1B72 BALINESE MUSICAL SYMBOL COMBINING BENDE
U+1B73 BALINESE MUSICAL SYMBOL COMBINING GONG
U+1DC0 COMBINING DOTTED GRAVE ACCENT
U+1DC1 COMBINING DOTTED ACUTE ACCENT
U+1DC2 COMBINING SNAKE BELOW
U+1DC3 COMBINING SUSPENSION MARK
U+1DC4 COMBINING MACRON-ACUTE
U+1DC5 COMBINING GRAVE-MACRON
U+1DC6 COMBINING MACRON-GRAVE
U+1DC7 COMBINING ACUTE-MACRON
U+1DC8 COMBINING GRAVE-ACUTE-GRAVE
U+1DC9 COMBINING ACUTE-GRAVE-ACUTE
U+1DCA COMBINING LATIN SMALL LETTER R BELOW
U+1DCB COMBINING BREVE-MACRON
U+1DCC COMBINING MACRON-BREVE
U+1DCD COMBINING DOUBLE CIRCUMFLEX ABOVE
U+1DCE COMBINING OGONEK ABOVE
U+1DCF COMBINING ZIGZAG BELOW
U+1DD0 COMBINING IS BELOW
U+1DD1 COMBINING UR ABOVE
U+1DD2 COMBINING US ABOVE
U+1DD3 COMBINING LATIN SMALL LETTER FLATTENED OPEN A ABOVE
U+1DD4 COMBINING LATIN SMALL LETTER AE
U+1DD5 COMBINING LATIN SMALL LETTER AO
U+1DD6 COMBINING LATIN SMALL LETTER AV
U+1DD7 COMBINING LATIN SMALL LETTER C CEDILLA
U+1DD8 COMBINING LATIN SMALL LETTER INSULAR D
U+1DD9 COMBINING LATIN SMALL LETTER ETH
U+1DDA COMBINING LATIN SMALL LETTER G
U+1DDB COMBINING LATIN LETTER SMALL CAPITAL G
U+1DDC COMBINING LATIN SMALL LETTER K
U+1DDD COMBINING LATIN SMALL LETTER L
U+1DDE COMBINING LATIN LETTER SMALL CAPITAL L
U+1DDF COMBINING LATIN LETTER SMALL CAPITAL M
U+1DE0 COMBINING LATIN SMALL LETTER N
U+1DE1 COMBINING LATIN LETTER SMALL CAPITAL N
U+1DE2 COMBINING LATIN LETTER SMALL CAPITAL R
U+1DE3 COMBINING LATIN SMALL LETTER R ROTUNDA
U+1DE4 COMBINING LATIN SMALL LETTER S
U+1DE5 COMBINING LATIN SMALL LETTER LONG S
U+1DE6 COMBINING LATIN SMALL LETTER Z
U+1DE7 COMBINING LATIN SMALL LETTER ALPHA
U+1DE8 COMBINING LATIN SMALL LETTER B
U+1DE9 COMBINING LATIN SMALL LETTER BETA
U+1DEA COMBINING LATIN SMALL LETTER SCHWA
U+1DEB COMBINING LATIN SMALL LETTER F
U+1DEC COMBINING LATIN SMALL LETTER L WITH DOUBLE MIDDLE TILDE
U+1DED COMBINING LATIN SMALL LETTER O WITH LIGHT CENTRALIZATION STROKE
U+1DEE COMBINING LATIN SMALL LETTER P
U+1DEF COMBINING LATIN SMALL LETTER ESH
U+1DF0 COMBINING LATIN SMALL LETTER U WITH LIGHT CENTRALIZATION STROKE
U+1DF1 COMBINING LATIN SMALL LETTER W
U+1DF2 COMBINING LATIN SMALL LETTER A WITH DIAERESIS
U+1DF3 COMBINING LATIN SMALL LETTER O WITH DIAERESIS
U+1DF4 COMBINING LATIN SMALL LETTER U WITH DIAERESIS
U+1DF5 COMBINING UP TACK ABOVE
U+1DF6 COMBINING KAVYKA ABOVE RIGHT
U+1DF7 COMBINING KAVYKA ABOVE LEFT
U+1DF8 COMBINING DOT ABOVE LEFT
U+1DF9 COMBINING WIDE INVERTED BRIDGE BELOW
U+1DFB COMBINING DELETION MARK
U+1DFC COMBINING DOUBLE INVERTED BREVE BELOW
U+1DFD COMBINING ALMOST EQUAL TO BELOW
U+1DFE COMBINING LEFT ARROWHEAD ABOVE
U+1DFF COMBINING RIGHT ARROWHEAD AND DOWN ARROWHEAD BELOW
U+2CEF COPTIC COMBINING NI ABOVE
U+2CF0 COPTIC COMBINING SPIRITUS ASPER
U+2CF1 COPTIC COMBINING SPIRITUS LENIS
U+2DE0 COMBINING CYRILLIC LETTER BE
U+2DE1 COMBINING CYRILLIC LETTER VE
U+2DE2 COMBINING CYRILLIC LETTER GHE
U+2DE3 COMBINING CYRILLIC LETTER DE
U+2DE4 COMBINING CYRILLIC LETTER ZHE
U+2DE5 COMBINING CYRILLIC LETTER ZE
U+2DE6 COMBINING CYRILLIC LETTER KA
U+2DE7 COMBINING CYRILLIC LETTER EL
U+2DE8 COMBINING CYRILLIC LETTER EM
U+2DE9 COMBINING CYRILLIC LETTER EN
U+2DEA COMBINING CYRILLIC LETTER O
U+2DEB COMBINING CYRILLIC LETTER PE
U+2DEC COMBINING CYRILLIC LETTER ER
U+2DED COMBINING CYRILLIC LETTER ES
U+2DEE COMBINING CYRILLIC LETTER TE
U+2DEF COMBINING CYRILLIC LETTER HA
U+2DF0 COMBINING CYRILLIC LETTER TSE
U+2DF1 COMBINING CYRILLIC LETTER CHE
U+2DF2 COMBINING CYRILLIC LETTER SHA
U+2DF3 COMBINING CYRILLIC LETTER SHCHA
U+2DF4 COMBINING CYRILLIC LETTER FITA
U+2DF5 COMBINING CYRILLIC LETTER ES-TE
U+2DF6 COMBINING CYRILLIC LETTER A
U+2DF7 COMBINING CYRILLIC LETTER IE
U+2DF8 COMBINING CYRILLIC LETTER DJERV
U+2DF9 COMBINING CYRILLIC LETTER MONOGRAPH UK
U+2DFA COMBINING CYRILLIC LETTER YAT
U+2DFB COMBINING CYRILLIC LETTER YU
U+2DFC COMBINING CYRILLIC LETTER IOTIFIED A
U+2DFD COMBINING CYRILLIC LETTER LITTLE YUS
U+2DFE COMBINING CYRILLIC LETTER BIG YUS
U+2DFF COMBINING CYRILLIC LETTER IOTIFIED BIG YUS
U+3099 COMBINING KATAKANA-HIRAGANA VOICED SOUND MARK
U+309A COMBINING KATAKANA-HIRAGANA SEMI-VOICED SOUND MARK
U+A66F COMBINING CYRILLIC VZMET
U+A674 COMBINING CYRILLIC LETTER UKRAINIAN IE
U+A675 COMBINING CYRILLIC LETTER I
U+A676 COMBINING CYRILLIC LETTER YI
U+A677 COMBINING CYRILLIC LETTER U
U+A678 COMBINING CYRILLIC LETTER HARD SIGN
U+A679 COMBINING CYRILLIC LETTER YERU
U+A67A COMBINING CYRILLIC LETTER SOFT SIGN
U+A67B COMBINING CYRILLIC LETTER OMEGA
U+A67C COMBINING CYRILLIC KAVYKA
U+A67D COMBINING CYRILLIC PAYEROK
U+A69E COMBINING CYRILLIC LETTER EF
U+A69F COMBINING CYRILLIC LETTER IOTIFIED E
U+A6F0 BAMUM COMBINING MARK KOQNDON
U+A6F1 BAMUM COMBINING MARK TUKWENTIS
U+A8E0 COMBINING DEVANAGARI DIGIT ZERO
U+A8E1 COMBINING DEVANAGARI DIGIT ONE
U+A8E2 COMBINING DEVANAGARI DIGIT TWO
U+A8E3 COMBINING DEVANAGARI DIGIT THREE
U+A8E4 COMBINING DEVANAGARI DIGIT FOUR
U+A8E5 COMBINING DEVANAGARI DIGIT FIVE
U+A8E6 COMBINING DEVANAGARI DIGIT SIX
U+A8E7 COMBINING DEVANAGARI DIGIT SEVEN
U+A8E8 COMBINING DEVANAGARI DIGIT EIGHT
U+A8E9 COMBINING DEVANAGARI DIGIT NINE
U+A8EA COMBINING DEVANAGARI LETTER A
U+A8EB COMBINING DEVANAGARI LETTER U
U+A8EC COMBINING DEVANAGARI LETTER KA
U+A8ED COMBINING DEVANAGARI LETTER NA
U+A8EE COMBINING DEVANAGARI LETTER PA
U+A8EF COMBINING DEVANAGARI LETTER RA
U+A8F0 COMBINING DEVANAGARI LETTER VI
U+A8F1 COMBINING DEVANAGARI SIGN AVAGRAHA
U+FE20 COMBINING LIGATURE LEFT HALF
U+FE21 COMBINING LIGATURE RIGHT HALF
U+FE22 COMBINING DOUBLE TILDE LEFT HALF
U+FE23 COMBINING DOUBLE TILDE RIGHT HALF
U+FE24 COMBINING MACRON LEFT HALF
U+FE25 COMBINING MACRON RIGHT HALF
U+FE26 COMBINING CONJOINING MACRON
U+FE27 COMBINING LIGATURE LEFT HALF BELOW
U+FE28 COMBINING LIGATURE RIGHT HALF BELOW
U+FE29 COMBINING TILDE LEFT HALF BELOW
U+FE2A COMBINING TILDE RIGHT HALF BELOW
U+FE2B COMBINING MACRON LEFT HALF BELOW
U+FE2C COMBINING MACRON RIGHT HALF BELOW
U+FE2D COMBINING CONJOINING MACRON BELOW
U+FE2E COMBINING CYRILLIC TITLO LEFT HALF
U+FE2F COMBINING CYRILLIC TITLO RIGHT HALF
U+101FD PHAISTOS DISC SIGN COMBINING OBLIQUE STROKE
U+10376 COMBINING OLD PERMIC LETTER AN
U+10377 COMBINING OLD PERMIC LETTER DOI
U+10378 COMBINING OLD PERMIC LETTER ZATA
U+10379 COMBINING OLD PERMIC LETTER NENOE
U+1037A COMBINING OLD PERMIC LETTER SII
U+10F46 SOGDIAN COMBINING DOT BELOW
U+10F47 SOGDIAN COMBINING TWO DOTS BELOW
U+10F48 SOGDIAN COMBINING DOT ABOVE
U+10F49 SOGDIAN COMBINING TWO DOTS ABOVE
U+10F4A SOGDIAN COMBINING CURVE ABOVE
U+10F4B SOGDIAN COMBINING CURVE BELOW
U+10F4C SOGDIAN COMBINING HOOK ABOVE
U+10F4D SOGDIAN COMBINING HOOK BELOW
U+10F4E SOGDIAN COMBINING LONG HOOK BELOW
U+10F4F SOGDIAN COMBINING RESH BELOW
U+10F50 SOGDIAN COMBINING STROKE BELOW
U+11300 GRANTHA SIGN COMBINING ANUSVARA ABOVE
U+1133B COMBINING BINDU BELOW
U+11366 COMBINING GRANTHA DIGIT ZERO
U+11367 COMBINING GRANTHA DIGIT ONE
U+11368 COMBINING GRANTHA DIGIT TWO
U+11369 COMBINING GRANTHA DIGIT THREE
U+1136A COMBINING GRANTHA DIGIT FOUR
U+1136B COMBINING GRANTHA DIGIT FIVE
U+1136C COMBINING GRANTHA DIGIT SIX
U+11370 COMBINING GRANTHA LETTER A
U+11371 COMBINING GRANTHA LETTER KA
U+11372 COMBINING GRANTHA LETTER NA
U+11373 COMBINING GRANTHA LETTER VI
U+11374 COMBINING GRANTHA LETTER PA
U+16AF0 BASSA VAH COMBINING HIGH TONE
U+16AF1 BASSA VAH COMBINING LOW TONE
U+16AF2 BASSA VAH COMBINING MID TONE
U+16AF3 BASSA VAH COMBINING LOW-MID TONE
U+16AF4 BASSA VAH COMBINING HIGH-LOW TONE
U+1E000 COMBINING GLAGOLITIC LETTER AZU
U+1E001 COMBINING GLAGOLITIC LETTER BUKY
U+1E002 COMBINING GLAGOLITIC LETTER VEDE
U+1E003 COMBINING GLAGOLITIC LETTER GLAGOLI
U+1E004 COMBINING GLAGOLITIC LETTER DOBRO
U+1E005 COMBINING GLAGOLITIC LETTER YESTU
U+1E006 COMBINING GLAGOLITIC LETTER ZHIVETE
U+1E008 COMBINING GLAGOLITIC LETTER ZEMLJA
U+1E009 COMBINING GLAGOLITIC LETTER IZHE
U+1E00A COMBINING GLAGOLITIC LETTER INITIAL IZHE
U+1E00B COMBINING GLAGOLITIC LETTER I
U+1E00C COMBINING GLAGOLITIC LETTER DJERVI
U+1E00D COMBINING GLAGOLITIC LETTER KAKO
U+1E00E COMBINING GLAGOLITIC LETTER LJUDIJE
U+1E00F COMBINING GLAGOLITIC LETTER MYSLITE
U+1E010 COMBINING GLAGOLITIC LETTER NASHI
U+1E011 COMBINING GLAGOLITIC LETTER ONU
U+1E012 COMBINING GLAGOLITIC LETTER POKOJI
U+1E013 COMBINING GLAGOLITIC LETTER RITSI
U+1E014 COMBINING GLAGOLITIC LETTER SLOVO
U+1E015 COMBINING GLAGOLITIC LETTER TVRIDO
U+1E016 COMBINING GLAGOLITIC LETTER UKU
U+1E017 COMBINING GLAGOLITIC LETTER FRITU
U+1E018 COMBINING GLAGOLITIC LETTER HERU
U+1E01B COMBINING GLAGOLITIC LETTER SHTA
U+1E01C COMBINING GLAGOLITIC LETTER TSI
U+1E01D COMBINING GLAGOLITIC LETTER CHRIVI
U+1E01E COMBINING GLAGOLITIC LETTER SHA
U+1E01F COMBINING GLAGOLITIC LETTER YERU
U+1E020 COMBINING GLAGOLITIC LETTER YERI
U+1E021 COMBINING GLAGOLITIC LETTER YATI
U+1E023 COMBINING GLAGOLITIC LETTER YU
U+1E024 COMBINING GLAGOLITIC LETTER SMALL YUS
U+1E026 COMBINING GLAGOLITIC LETTER YO
U+1E027 COMBINING GLAGOLITIC LETTER IOTATED SMALL YUS
U+1E028 COMBINING GLAGOLITIC LETTER BIG YUS
U+1E029 COMBINING GLAGOLITIC LETTER IOTATED BIG YUS
U+1E02A COMBINING GLAGOLITIC LETTER FITA
U+1E8D0 MENDE KIKAKUI COMBINING NUMBER TEENS
U+1E8D1 MENDE KIKAKUI COMBINING NUMBER TENS
U+1E8D2 MENDE KIKAKUI COMBINING NUMBER HUNDREDS
U+1E8D3 MENDE KIKAKUI COMBINING NUMBER THOUSANDS
U+1E8D4 MENDE KIKAKUI COMBINING NUMBER TEN THOUSANDS
U+1E8D5 MENDE KIKAKUI COMBINING NUMBER HUNDRED THOUSANDS
U+1E8D6 MENDE KIKAKUI COMBINING NUMBER MILLIONS

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 16 Feb 2019 23:36:47 -0800

>> What's missing is a NOT_RECOMMENDED status for the combining slash, the combining hook and other characters in the context of identifiers. That is what the "troublesome" draft was trying to addres…

Patrik,

of your list below only these twenty (20) are in MSR-4, as possibilities 
(subject to further investigation by script-panels). Many of them are in 
that list only because they do occur in some decompositions and 
therefore "if" a combination was needed, that combination by force would 
be (and would remain) unencoded. (The Ethiopic ones may have been 
included on the basis that we were simply deferring to the experts in 
that script and didn't want to make a ruling on three code points).

*U+0300* 	*Inherited* 	*COMBINING GRAVE ACCENT*
*U+0301* 	*Inherited* 	*COMBINING ACUTE ACCENT*
U+0302 	Inherited 	COMBINING CIRCUMFLEX ACCENT
*U+0303* 	*Inherited* 	*COMBINING TILDE*
*U+0304* 	*Inherited* 	*COMBINING MACRON*
U+0306 	Inherited 	COMBINING BREVE
U+0307 	Inherited 	COMBINING DOT ABOVE
*U+0308* 	*Inherited* 	*COMBINING DIAERESIS*
U+0309 	Inherited 	COMBINING HOOK ABOVE
U+030A 	Inherited 	COMBINING RING ABOVE
U+030B 	Inherited 	COMBINING DOUBLE ACUTE ACCENT
U+030C 	Inherited 	COMBINING CARON
U+031B 	Inherited 	COMBINING HORN
U+0323 	Inherited 	COMBINING DOT BELOW
U+0326 	Inherited 	COMBINING COMMA BELOW
*U+0327* 	*Inherited* 	*COMBINING CEDILLA*
U+0328 	Inherited 	COMBINING OGONEK
U+0329 	Inherited 	COMBINING VERTICAL LINE BELOW
*U+0331* 	*Inherited* 	*COMBINING MACRON BELOW*
U+0358 	Inherited 	COMBINING DOT ABOVE RIGHT

U+135D 	Ethiopic 	ETHIOPIC COMBINING GEMINATION AND VOWEL LENGTH MARK
U+135E 	Ethiopic 	ETHIOPIC COMBINING VOWEL LENGTH MARK
U+135F 	Ethiopic 	ETHIOPIC COMBINING GEMINATION MARK
U+1DCA
	Inherited
	COMBINING LATIN SMALL LETTER R BELOW

And only the seven (7) bold ones are going to be in any Root Zone LGR 
(based on the latest drafts).

300, 301, 303, 304, 308, 327 and 331.

None of the other ones is needed to represent any character that's used 
in an actual, modern orthography that is in everyday common use.

The set above (whether the 20 or the 7) could be categorized as 
"ONLY_RECOMMENDED_IN_ENUMERATED_COMBINATIONS" which is what we are doing 
for the Root: each script will enumerate the specific combinations used.

All others on your list could be labeled NOT_RECOMMENDED (where we then 
need to define what they are not recommended for, whether we call it 
"unrestricted use" or "general use" or whether we want to give this a 
positive spin like "safe profile"). However, upshot is, nobody *needs* 
those except to make mischief.

Now, complex scripts contain many, many more combining marks, but they 
are not named "COMBINING". You can tell them by their property (both 
general category and indic syllable category - the latter property not 
available at the time of IDNA2008).

For complex script the recommendation would be : 
SUBJECT_TO_SCRIPT_SPECIFIC_CONTEXT_RESTRICTION (or something like that). 
I would not envision IETF to spell out acceptable restrictions in detail 
for each code point, but to point to available data on the structure of 
these scripts and the Root Zone as a starting point (free, but 
non-binding example).

The point behind my suggestion is that we should have a framework that 
makes very clear where the deep water starts, so that third parties can 
look at registry policies and have a chance to understand whether they 
are sticking to the shallow end of the pool, or are reckless.

By not making any of this mandatory, individual exceptions aren't a 
problem for compliance. However, wholesale departure would throw up a 
red flag.

If people want to use cuneiform or Egyptian Hieroglyphics and or random 
combining marks with IDNs, then they are still free to do so, but if 
they do that as part of ccTLDs or gTLDs or other public zones, third 
parties can step in and warn their users.

The worst issues are usually not the exotic ancient scripts but some 
scripts that look close to modern ones, or obsolete code points in 
modern scripts used for ancient of recently extinct languages. These 
have high potential for malicious use such as being mistaken for a 
similar modern code point.

So, in a nutshell, the proposal would be to categorize the high use 
cases and propose typical mitigation strategies for any subset that have 
issues. Many characters are simply required, even if they cause issues, 
but pretty good mitigation is available.

Anything outside that subset would then be "BUYER BEWARE".

A./

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Feb 2019 02:24:53 -0500

Asmus and Patrik,

I think we are in danger of confusing several issues and, in the
process, losing an important option that could actually help
move us forward.  Deliberately not quoting from either of your
recent notes (people who haven't read them should go back and do
so)...

(1) Patrik (mostly), especially in the light of the text in The
Unicode Standard that suggests that no more code points for
precomposed characters would be added when appropriate base
characters and combining ones exist (you will recall that text,
between versions 3.x and 7.0 at least, was part of the key to
the original frustration with U+08A1; it appears to now be
gone), we know now, just as we knew in the analysis that led to
IDNA2008, that we couldn't get rid of combining characters
entirely.  So a list of how many of them there are (either by
name or some more accurate measure) is interesting, but not
particularly useful for this discussion.

(2) Everyone: If I correctly understand what Asmus is
suggesting, it would imply changing the model of IDNA2008 (or
adding other standards to it) to shift from a binary "not
permitted" (DISALLOWED or CONTEXTx and failure of those tests)
or "allowed at the discretion of the registry (PVALID or
CONTEXTx and passing the tests) to include at least an
additional "Not Recommended" category.  That would either be a
substantive modification to IDNA2008 or a requirement for a new
layering within the model.  It would raise some important
questions about the role of the current I-D, but I think we
could fairly easily modify it (or an updated version of one of
the other pending I-Ds) to allow a forward reference.   Anyone
want to start making the argument for going there?  As an aside,
I can't remember whether and how much it made it onto the WG's
radar (although I'm confident Patrik and Harald will remember
it), but the original proposal for what became IDNA included
additional "Probably yes" and "Probably no" categories that
could be reasonably described as "Recommended only if you really
need it" and "Not Recommended unless you are really, really,
sure you know what you are doing", so this is not a radical idea
and we may actually have some text lying around.   Are we ready
to try to start that effort and do we have a theory about where
we find WG leadership, participants, and reviewers?

(3) Asmus (mostly), in the context of a discussion of an update
or revision of IDNA2008 (or a decision that none is needed), I
do not see reasoning based on "actual, modern orthography that
is in everyday common use", what is going to be allowed in the
root zone, or assorted MSR and LGR versions as being helpful.
Speaking for myself, I find what I know about what is going on
with the development of those tables and recommendations very
interesting and have learned from it even while I recognize some
of the implications of the difference between, e.g., "everyday
common use" and "visibility and activity in the ICANN
community".  Because, at least as I understand it, "orthography"
is strongly connected to (and probably meaningless without)
strong connection to a language, making global decisions based
on it is even more suspect, at least unless one can identify
every language that has ever been used (or might be used in the
future) with each relevant script and is prepared to exclude
many labels from the DNS that are not based on "words" or how
words are constructed in those languages.   However, while, as
we have discussed before, I strongly favor being very
restrictive about what is allowed in the root zone, for the rest
of the DNS (all the way down the tree, not just at the second or
possibly third level), the repertoire allowed by IDNA2008 is
supposed to be an absolute upper limit, not just a set of
recommendations.  As I think we've also discussed, one of the
things that was pointed out very forcefully to us during
IDNA2008's development was the complete legitimacy of IDNs with
labels in selected archaic scripts -- scripts that were the very
opposite of "modern" or "everyday common use" in zones (or
levels) subsidiary to specialized domain (whether top-level or
not).  Similar comments apply to restrictions based on
orthographic assumptions, noting that, even in the root, there
are several labels that are orthographically impossible as
words.  FWIW, some of the strongest arguments for allowing those
archaic script labels came from ICANN contracted parties.
Independent of the issues with trying to reverse the IETF
decision to allow them, I'd predict that any attempt to
discourage their use in appropriately-specialized domains would
result in a very unpleasant and widely-supported backlash from
those registry operators, possibly including litigation against
ICANN for setting up mechanisms for restricting explicit
authority given those registries by contract.    If you, or
anyone else in this group, believe that we should be restricting
IDNA2008 to eliminate archaic scripts, or other scripts that are
not in modern and common use, please speak up because it would
affect a lot of our discussions and decisions going forward (and
make an extremely strong argument for a WG to reopen and revise
IDNA2008 itself).   Note too that "appropriately-specialized
zones" has nothing to do with the usual interpretation of
"public zones": while none of us care very much (or have much
influence over) zones that are not generally accessible from the
public Internet (see RFC 6055), the "public zone" or "public
domain" definition used around ICANN is a term related to name
sales and not technical issues. It would be easy to imagine an
appropriately-specialized domain in which names are also for
sale on a non-discriminatory basis wrt the would-be registrants.

(4) Asmus (mostly) FWIW, my recollection of the VIP discussions
that preceded and led to the LGR effort were that allowing
combining characters of any type in the root, except insofar as
they were needed to write a country name in its conventional
spelling, was a bad idea.  If the current effort has decided to
reverse that decision, I hope the original reasoning is getting
careful attention and that, e.g., SSAC is being consulted.
However, for the reasons above, that is just not an IETF problem
regardless of what decision(s) are reached.

(5) Asmus (mostly, again)  Categories for code points like "Not
Recommended"  (much less suggestions like
"SUBJECT_TO_SCRIPT_SPECIFIC_CONTEXT_RESTRICTION") are not
allowed in IDNA2008, at least as it is now defined (see (2)
above).  Any recommendations to registries are part of other
work and we have tried so far to maintain a careful separation
between the upper-bound requirements of IDNA2008 and those
recommendations.  We could change that, but, again, I hope that
those favoring doing so would speak up so, if those ideas have
consensus, we can start shaping them into a WG proposal (and
consider all of these documents in that light).   There are lots
of things I would not recommend for general use that I think
would be really problematic to classify as "buyer beware",
"reckless", or "probably inherently evil".   If you believe that
IETF should go into the Applicability Statement business for
IDNA (taking us straight toward "Not Recommended"), make a
proposal but note that, unless there is a turnaround, that might
encounter as many or more issues with adequate review as we have
had with base IDNA and other i18n documents.

(6) (Patrik mostly)  Asmus mentions a new indic syllable
category.  Have you made a determination of whether it would
make sense to modify RFC 5892 to reflect that category (and any
other new ones) within the rules?   While I don't recall our
explicitly discussing Unicode categories introduced subsequent
to around TUS 5.0, it would seem to me that such categories fall
into the range of things that the community should be made aware
of and consider as part of a new version review.   That alone
would see to require additional text in the I-D.

(7) Everyone: Given Asmus's explanation and comments, it seems
to me that there is significant justification for adding the
combining hook and (retroactively) the two combining slashes
(U+0337 and U+0338) (and "other characters") to the exception
list as DISALLOWED because they are not intended for use with
letters (despite the same General Category values as combining
marks that are so intended) but for other purposes.  The
question makes the analysis associated with the current I-D
incomplete and appears to me to justify either making a revision
that would require restarting the IETF Last Call or withdrawing
and rethinking the I-D itself.  The alternative, it seems to me,
would be to explicitly dismiss that explanation and reasoning.

(8) Everyone (again): It seems to me that the review called for
in Section 5.1 of RFC 5892 can be interpreted either narrowly or
broadly.  The narrow interpretation would say that we run the
rules, look for incompatible changes to already-assigned code
points, and then stop because there is no other basis for saying
"wait, this isn't right and might require either an additional
Category F exception or possibly even an additional rule", at
least short of initiating an IDNAter effort.  Personally, I
don't think that was ever the intent, even when we thought we
were more in synch with Unicode's model for evaluating and
assigning new code points than we turned out to be.  From that
broader point of view, which is reinforced by both the language
of RFC 6452 and the observation that the issues that started
being identified when Unicode 7.0 introduced U+08A1 were not
dismissed out of hand as our tough luck, Patrik's calculations
are a starting point for a review of new features and changes,
not an almost-completely-mechanical conclusion to the process.
If one takes the broader view and factors in this recent
discussion between Patrik and Asmus and the comments above in
this note, then draft-faltstrom-unicode11-07 is not ready for
publication independent of the issues about related, and
possibly prerequisite, document identified in my note of 6
February.

   best,
    john

p.s. One odd bit of good news in the above is that anything that
would require revising Section 2.6 of RFC 5892, adding
additional rules to Section 2, or even adding a forward pointer
to work on recommendations about characters, eliminates the
question of the document category for whatever documents makes
those changes.  One small issue we can eliminate while we get on
to substantive questions :-(.

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Feb 2019 02:31:24 -0500

Hi.

I deliberately did not include a specific response to this note
in the one I just sent, only partially because that was getting
too long.

FWIW, I think Asmus's recommendations below, especially the one
about overlaid combining marks and CONTEXTO, are entirely
reasonable and appropriate.  If others agree, they would not be
hard to write up, either as a new section of
draft-faltstrom-unicode11 (and obviously changing the main
thrust of that document from "no changes needed, just calculate
the tables") or as a new I-D (which, unfortunately,
draft-faltstrom-unicode11 would need to normatively reference to
avoid published a set of tables in which we no longer believed
and creating massive confusion about what the IETF was
recommending/ requiring.

    best,
      john

--On Saturday, February 16, 2019 14:56 -0800 Asmus Freytag
<[email-masked]> wrote:

>> --On Thursday, February 14, 2019 15:40 -0800 "Asmus Freytag
>> (c)" <[email-masked]> wrote:
>> 
>>> sequences, just like O-slash is not decomposable and never
>>> has been, even though there are combining slash overlay
>>> characters.
>>> 
>> Or, alternately, why has it not been deprecated?  If "hooks"
>> are never intended to be "composed" onto characters, then
>> having a composing hook available would seem to be useful
>> only for simulating characters that have not yet been
>> assigned code points (something I first heard people
>> associated with Unicode argue was a bad idea during the
>> discussion around the balloting of ISO 10646 DIS-2) and for
>> causing, however inadvertently, confusion and/or mischief.
> 
> John,
> 
> productive in not only moving forward with Patrik's draft but
> also with coming to a consensus on the other pieces of the
> puzzle. Let me give some background and more detail:
> 
> (1)
> 
> Unicode must cater to people writing their language and to
> scholars writing about language. They have different
> requirements.
> 
> Slash overlays, for example, can be used in scientific
> notation or to indicate that a letter or symbol is "cancelled"
> in some way. That is different from the use of "o-slash" in
> Scandinavian languages, where it is a full member of the
> orthography (sorting, for example, not like an "o" with
> diacritic, but after 'z').
> 
> The hook is an entirely parallel case (except perhaps that it
> is a more specialized shape and therefore its use in notations
> isn't as generically common).
> 
> (2)
> 
> What's missing is a NOT_RECOMMENDED status for the combining
> slash, the combining hook and other characters in the context
> of identifiers. That is what the "troublesome" draft was
> trying to address in its own way.
> 
>  From the Unicode point of view, "deprecated" doesn't enter,
> because those characters do have a function in a universal
> character encoding - even if they should never be used in
> identifiers (or at least not in situations where they can be
> misused in identifiers).
> 
> The latter distinguishes the protocol (IDNA2008) from other
> layers: many code points can be used in mischievous ways if in
> an under-regulated public zone, but can be fine as part of,
> say subdomain naming under the control of a single entity.
> 
> (3)
> 
> Going back to Latin diacritics that are "attached" or
> "overlaid": These also include at the minimum the various
> strokes, for example.
> 
> For strokes and other overlays, there isn't a generic
> positioning (unlike Dot-above or Dot-below and the other
> detached ones). Strokes can go through any part of a
> characters glyph shape: at different heights, as well as
> through just one of the "stems" in a character's shape, or
> they can be of different widths. Only characters that are part
> of an orthography would have a settled shape, and in
> principle, different orthographies can come to different
> conventions even for the same base letter. In short,
> attempting to cover these cases via "glyph composition" aren't
> satisfactory.
> 
> For linguistic and other notational uses, a "generic" stroke
> overlay is at once sufficient and necessary: notations are
> notoriously open-ended; in principle any code point might be
> "annotated" with a generic overlay or attached mark. Trying to
> encode any of those explicitly cannot succeed. (During the
> time DIS-2 was discussed, linguists were circulating a
> "preliminary" list of 2,000 Latin characters with various
> marks. That was in the early '90s. Can you imagine what that
> list would be today?)
> 
> (4)
> 
> In conclusion: certain combining characters are NOT (and have
> never been) intended for "simulating characters that have not
> yet been assigned code points". UTC is quite consistent with
> stating that if there are such unencoded characters (used in
> orthographies) that they will be encoded if they require the
> use of combining  marks that UTC where UTC does not feel a
> "decomposition" is canonically equivalent. Realistically, up
> to that point of encoding, one would not expect significant
> use of such orthographies (which tend to be for small
> populations or tend to be "emerging"); so this issue isn't a
> practical problem looking forward, it's allowing unrestricted
> use of combining marks in identifiers that is the problem.
> 
> (5)
> 
> IDNA2008 should have had a context rule for all combining
> marks that disallowed more than one adjacent instance of the
> same combining mark. At the time, less so now, there existed
> too many implementations that would overprint these. I'm not
> aware of any examples where such duplication is required.
> 
> All combining marks should have had a status of
> "CONTEXT_REQUIRED" meaning that including them in any LGR
> (registry policy) _requires_ the use of policy-specific
> contexts. (This is different from CONTEXTO / CONTEXTJ which
> are context restrictions on the protocol level).
> 
> These registry-level required contexts can take one of two
> forms: one is to enumerate all permissible combinations (most
> orthographies for European scripts require only a finite set,
> most of which are precomposed, but a much smaller list
> requires combination). The other, and applicable to complex
> scripts, is to have generic rules describing allowable
> combinations without enumerating them. Finally, some scripts
> (Arabic and Hebrew) use combining marks in ways that are
> optional for the writing system. In these cases, the easiest
> "context" is to forbid their use.
> 
> A./
>

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 18 Feb 2019 00:47:59 -0800

John,

thanks for attempting to organize things a bit.

I'll respond point by point for now.

A./

> particularly useful for this discussion.

"Combining" characters are not all the same.

If a mark is part of an existing decomposition, there's a chance that other
characters are needed that use the same mark with a different base
character, but no precomposed letter will be added because of normalization
stability.

If a mark is not used in decomposition, but a letter exists that has the 
feature,
(hook, slash, stroke, etc) then it's an indication that Unicode will not 
consider
any sequence of letter+mark canonically equivalent to a precomposed
"letter with mark".

Beyond these two subsets there are a large number of diacritic like general
combining marks for which the current or future membership in either of 
these
sets is undecided.

Beyond the universe of "diacritics" in the general sense are many characters
that are full "letters" in the context of complex scripts, but may otherwise
function like combining marks. Except for some unusual circumstances (like
double vowels) the question of decomposition doesn't enter for these.

"Combining" characters are not all the same.

> we find WG leadership, participants, and reviewers?

I'm glad it's not a "radical idea", but in dredging up some precursors, you
are losing the essence of my proposal. I find statements like "being sure
you really really know what you are doing" worse than useless.

What I am recommending is a combination of:

(1) identifying a "core" set of characters that are widely needed, widely
used, and therefore well-understood by non-specialist users.

(2) implicitly identifying the inverse set as "not well understood" and in
need of some explanation if you intend to use a few of them extend
the core set. (No explanation would imply a lack of due diligence).

(3) creating subcategories in the "core" set that reflect well-understood
issues with certain code points:

     (a) the code point is known to have duplicates (U+0259 and U+01DD)

     (b) the code point is known to require generic context (which the
           registry would be expected to apply with some flexibility based
           on the target languages, etc., but where the total absence of
           meaningful generic context rules in an LGR would signal a lack
           of due diligence.)

     (c) the code point should by default only occur in enumerated
          combinations. (Unrestricted support would imply a lack of due
          diligence).

The task of (3) is impossible if applied to "all of Unicode" - we simply
don't understand most scripts (and code points) that are in limited
use well enough to make a meaningful determination.

> words are constructed in those languages.

Almost. There's no need to be open ended for future languages if you
write registry policies. And the classifications I am thinking of should aid
in devising registry policies (or guidelines for such) but not change the
protocol.

We have in fact looked at "every" _existing language_ that is customarily
written with a given script and which is not moribund or extinct.

For that Latin script some several hundred languages were researched,
for the Arabic and Cyrillic scripts a few dozen each. For most other
scripts the number of languages of interest is between one (1) and
about a dozen.

The scope has been such that we are confident that we cover anything
that is in use by ordinary (non-scholarly / non-specialist) users.

The beauty with making recommendations is that we can focus on
this "core" set, which impacts the cases of practical importance without
making Egyptian Hieroglyphics illegal. We'd simply advise that outside
a domain for Egyptologists they are probably not advisable as identifiers,

AND

if you do want to make them available you are on your own, because
there's not enough experience with them in identifiers.

About Non-words: in some complex scripts it can be very difficult to express
things that aren't in syllable form. That rules out many arbitrary
sequences of code points but it is not the same as applying a spelling
dictionary: in fact for the Root Zone we tended to "disapprove" of rules
that appeared to be "spelling rules" instead of "deep structure".

In most other scripts there is more freedom for "random" strings,
but even there, things work best for base letters (not random sequences
of diacritics).

>   However, while, as

My proposal, in a way, calls their bluff: none of them "undertstand" any
of these character collections, so by definition, they can't satisfy the
requirement of only delegating what they understand.

The recommendations should lead to "conservative" but "practical and
useful" label generation rules.

And one would expect that the vast majority of actually delegated labels
are compatible with them (after accounting for malicious registrations).

> authority given those registries by contract.

Describing a framework for well-behaved registry policies is one thing. If
ICANN wants to require contracted parties to use such a framework that
is not an IETF issue.

It should not prevent us from creating and describing such a framework.

>     If you, or

I don't think we need to attempt to prevent the sale of domains to
Egyptologists, or ones that allow the full range of Latin phonetic 
characters.

But such zones are inherently less safe wrt spoofing, and it's time we had
a framework that allowed to address this issue in quality.

This has nothing to do with restricting IDNA2008.

> regardless of what decision(s) are reached.

That's a silly restriction and makes no sense.

The current root zone LGR effort is based on the "Procedure" document,
which does not contain this particular constraint.

The result of the effort is a set of restrictions that work with all 
words that
require combining marks, not just "country names" but disallowing things
that are structurally unsound.

> above).

Well aware of that.

Proposal is for a new layer. We would need a good name for it.

> Any recommendations to registries are part of other

We may be able to leverage existing work to do a classification;
what is possible will become more apparent as we delve into an actual
proposal.

The review effort isn't that different from trying to come to grips with
"troublesome"; the idea that we could sidestep things by merely
populating a registry is not working out as well as we had hoped.

> would see to require additional text in the I-D.

Advise against.

There are tons of new properties. And tons of existing properties that could
have been considered for IDNA2008 but were not.

And they are not applicable to most "archaic" scripts.

My suggestion was aiming at using some of these to see if they flag
the type of code points that registries need to give "special attention"
to for certain complex scripts.

Not relevant for the protocol level.

> would be to explicitly dismiss that explanation and reasoning.

Nah, that's the same kind of meaningless focus on individual characters
as the IAB "emergency brake" was based on. Only about 20 (!) characters
with the word "COMBINING" in their name are potentially useful for
making "words" or "near-words". The rest is for phonetic annotations
and similar specialist use.

Which makes them legitimate for a protocol that decides to allow domain
names that cater to specialists.

But NOT_RECOMMENDED for anything aimed at ordinary users.

Not to forget that characters that look like they might be decomposable
with U+0337 or U+0338 go back to the original IDNA2008 (or even IDNA
2003). "Fixing" that now on the protocol level would be really weird.

If you want to DISALLOW combining marks: there are some that are used
with "scripts" that aren't letters (like byzantine musical notation).

We have a list of those in MSR-4 where we point out cases where
we think the application of Unicode properties dragged in some
small  umber of truly unintended code points.

> not an almost-completely-mechanical conclusion to the process.

I think that exception lists need to be looked at in  light of what the
current list say about the focus and intent of the protocol.

I'm strongly not in favor of trying to "solve" certain issues on the
protocol level, especially not issues of long standing.

That said, I don't see in the diffs anything that's a new concern of note
and think Patrik's got it right.

> possibly prerequisite, document identified in my note of 6

Totally disagree with this conclusion.

A./

---

From: "Patrik Fältström" <[email-masked]>
Date: Mon, 18 Feb 2019 09:52:56 +0100

> Not relevant for the protocol level.

Are you saying that there is NO category or combination of categories that can be used for "advice against use by registries while being PVALID"?

I am allergic to "picking code point by code point". And want that to be such an extreme exception it can be. IETF is not good at picking code points.

   Patrik

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Mon, 18 Feb 2019 01:08:50 -0800

> Are you saying that there is NO category or combination of categories that can be used for "advice against use by registries while being PVALID"?

No, advise against overloading this issue into your current draft.

> I am allergic to "picking code point by code point". And want that to be such an extreme exception it can be. IETF is not good at picking code points.

There are historic reasons why looking at blank slate trying to qualify 
130,000 code points individually was a bad idea.

The situation has changed.

There's been a multi-year, multi-team effort to do just this work (with 
proper linguistic input) for the "high-value" subset of code points that 
are of interest to ordinary, non-specialized users in practically all 
languages that have educational (or better) instutional support.

But, to come back to these properties.

For example, Indic Syllable Categories identifies "Nukta". Generally, 
Nuktas are used with very specific base letters (they make "new" 
letters). Therefore, generically, you might start giving them a 
category: RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY

For ISC gives a category "Consonant". Those generally do not need any 
context rules.

However, category Virama, and category Vowel_Dependent do need context 
rules (otherwise you get structurally unsound labels).

So, you could give them a a category: RESTRICTED_CONTEXT_RECOMMENDED

and likewise for all the other ISC values.

This works as long as your categories are reasonably generic and do not 
try to delve into "which context rules" would be recommended, because 
that's highly script dependent. We just know that you shouldn't allow 
them unrestricted by default.

Does that answer?

A./

---

From: Patrik Fältström <[email-masked]>
Date: Mon, 18 Feb 2019 10:24:12 +0100

> On 18 Feb 2019, at 10:08, Asmus Freytag (c) <[email-masked]> wrote:
> 
My draft is updating RFC5892. Sure, a separate discussion is whether all changes to 5892 that we are coming up with should be in my draft or not, but let’s ignore that now.

So let me be more specific, is there a category that can be used to find what code points should get this new by you invented property value, or added to one of the context rules?
> There are historic reasons why looking at blank slate trying to qualify 130,000 code points individually was a bad idea.
> 
Well, once again we must separate the different issues from each other or we will never be done. I see now, I guess:

1. Should all updates to 5892 be in draft-faltstrom-unicode11?
2. Can new derived property value and changes be done via categories decided by UTC?
3. Should IETF start pick individual code points?
> There's been a multi-year, multi-team effort to do just this work (with proper linguistic input) for the "high-value" subset of code points that are of interest to ordinary, non-specialized users in practically all languages that have educational (or better) instutional support.
> 
> For example, Indic Syllable Categories identifies "Nukta". Generally, Nuktas are used with very specific base letters (they make "new" letters). Therefore, generically, you might start giving them a category: RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY
> 
What you say is that:

- Code points with Indic Syllable Category Nukta should have derived property value RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY

Correct?
> For ISC gives a category "Consonant".
> 
What you mean is “Indic Syllable Category Consonant is to be treated as RFC 5892”?
> Those generally do not need any context rules.
> 
Generally? :-)

I want rules, I am an engineer... Black/white etc :-D
> However, category Virama, and category Vowel_Dependent do need context rules (otherwise you get structurally unsound labels).
> 
What you mean is that there should be a rule like (you mix category and derived property value):

- ISC category Nukta should get derived property value RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY
- ISC category Consonant should be treated like RFC 5892 (no special handling)
- Other ISC categories should get derived property value RESTRICTED_CONTEXT_RECOMMENDED

Is this correct?

What should then the action be for these two new derived property values?

   Patrik

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Mon, 18 Feb 2019 01:59:21 -0800

On 2/18/2019 1:24 AM, Patrik Fältström wrote:
>
>
> On 18 Feb 2019, at 10:08, Asmus Freytag (c) <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> My draft is updating RFC5892. Sure, a separate discussion is whether 
> all changes to 5892 that we are coming up with should be in my draft 
> or not, but let’s ignore that now.

OK.

>
> So let me be more specific, is there a category that can be used to 
> find what code points should get this new by you invented property 
> value, or added to one of the context rules?

I think so - but that needs to be verified in more detail (not something 
that gets settled in the span of one email)

>> There are historic reasons why looking at blank slate trying to 
>> qualify 130,000 code points individually was a bad idea.
>>
> Well, once again we must separate the different issues from each other 
> or we will never be done. I see now, I guess:
>

I'm generally sympathetic to IETF experts not wanting to be "pickers" 
based on limited direct expertise.

Also sympathetic to not duplicating unnecessarily any work that's 
already been condensed into properties - provided they fit.

>> There's been a multi-year, multi-team effort to do just this work 
>> (with proper linguistic input) for the "high-value" subset of code 
>> points that are of interest to ordinary, non-specialized users in 
>> practically all languages that have educational (or better) 
>> instutional support.
>>
> - Code points with Indic Syllable Category Nukta should have derived 
> property value RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY
>
> Correct?

As an example. In my understanding these tend to work that way in all 
the modern scripts that have them

> What you mean is “Indic Syllable Category Consonant is to be treated 

In essence.

> I want rules, I am an engineer... Black/white etc :-D

Well, you allowed for exceptions.  :)

>> However, category Virama, and category Vowel_Dependent do need 
>> context rules (otherwise you get structurally unsound labels).
>>
> - ISC category Nukta should get derived property value 
> RECOMMENDED_AS_PART_OF_ENUMERATED_SEQUENCES_ONLY

Yes, note that these are examples - there are more ISC values, but these 
three are good to get the discussion started.

> What should then the action be for these two new derived property values?

In existing RFC5892 one would have implemented 
RESTRICTED_CONTEXT_RECOMMENDED as CONTEXTO with actual context rules.

This does not scale, because one would need to duplicate the entire work 
of the Root Zone LGRs for these scripts and then the result would not be 
unique, because there are multiple ways of achieving the same end with 
context rules.

So a better "action" would be: "registry, provide some context rules for 
these code points - they can be script specific, or language-specific 
depending on the zone - but if you don't provide special handling for 
some characters  we expect an explanation, and if you don't provide 
special handling for all of these code points, we suspect that you 
didn't do your due diligence".

Actual drafting would be required to make that a useful statement.

Again, this is something that doesn't scale perfectly for 
lesser-understood scripts (e.g. archaic).

That's why I would leverage something like MSR-4 (where someone NOT_IETF 
did the picking) to first divide the worlds into the "knowable" and the 
"unknowable". You could still use the properties to assign categories 
and actions, but the conclusions would not be as definite - for some 
scripts, nobody fully knows how they work.

A./

---

From: "Pete Resnick" <[email-masked]>
Date: Mon, 18 Feb 2019 11:37:19 -0600

The ART ADs (who are monitoring this list and will hopefully notice 
this) are in a better position to answer this query, but from my 
perspective:

This list is likely to have a more in-depth discussion of the document 
than any more general LC discussion on the main IETF list. I think it is 
more valuable to have your input here.

I also expect the ART ADs at the very least to take the input of this 
review very seriously, and I expect the sponsoring AD of this document 
would be unlikely to let issues raised by this review to go unaddressed.

pr

---

From: "Alexey Melnikov" <[email-masked]>
Date: Mon, 18 Feb 2019 13:37:10 -0500

John/Pete,

Agreed.

> would be unlikely to let issues raised by this review to go unaddressed.

Indeed, ART ADs will be watching both mailing lists. At the moment I don't see any email on [email-masked] other than the one from Suzanne Woolf.

Best Regards,
Alexey

---

From: Ben Campbell <[email-masked]>
Date: Mon, 18 Feb 2019 12:52:13 -0600

Do I understand correctly that Harald plans to send a summary of any conclusions?

Thanks,

Ben.

> On Feb 18, 2019, at 12:37 PM, Alexey Melnikov <[email-masked]> wrote:
> 
> John/Pete,
>

---

From: "Pete Resnick" <[email-masked]>
Date: Mon, 18 Feb 2019 12:53:14 -0600

Yes. Harald is the assigned "reviewer" in the directorate tool, and 
therefore the penholder on the output of this discussion.

pr

On 18 Feb 2019, at 12:52, Ben Campbell wrote:

>> Indeed, ART ADs will be watching both mailing lists. At the moment I 
>> don't see any email on [email-masked] other than the one from Suzanne 
>> Woolf.
>>
>> Best Regards,
>> Alexey
>>
>>> pr
>>>

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Feb 2019 17:28:08 -0500

Patrik,

I may come back to the rest of your discussion with Asmus later,
but want to see if I can explain where I'm coming from on what,
to me, is the critical path aspect of it...

--On Monday, February 18, 2019 10:24 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

>...
>>> I am allergic to "picking code point by code point". And
>>> want that to be such an extreme exception it can be. IETF is
>>> not good at picking code points

Agreed.  Especially at the "not good at" part.

>> There are historic reasons why looking at blank slate trying
>> to qualify 130,000 code points individually was a bad idea.
>> 

Although I agree with you about separation of issues, I think
Asmus and I (and possibly you) are still not communicating about
a key part of that situation.  I still believe the differences
are mostly about miscommunication rather than disagreements
after full understanding, so want to try to explain from yet
another perspective.  I know you know all of this, but some
people on this list may not and a review may be helpful in any
event.

Understanding an important part of the history of IDNA requires
remembering the evolving relationship with UTC (or at least
people claiming to speak for it).   We got a good deal of advice
from them when IDNA2003 was being prepared.  That advice
included basing everything on tables of code points, advice
about mappings generally and NFKC and case folding in
particular, and advice about code points that could just be
ignored.  From observation of the IDNA2003 specifications,
specifically Stringprep and Nameprep, the authors and IETF
consensus took most if not all of that advice.  It didn't work
out well, leading to RFC 4690 and ultimately to IDNA2008.  I am
convinced that the problems were not due to any malice but
instead to lack of understanding, or disagreement about the
priorities of, the constraints of the DNS and considerations
about Internet community use of that type of identifier.  So we
moved on to what became IDNA2008.  Interestingly, the same
actors in the Unicode Consortium had apparently concluded that
IDNA2003 was not working well either because, well before
serious work on IDNA2008 started, they went to ICANN with a
proposal that (without knowing exactly what the Unicode folks
intended) ICANN leadership interpreted as "ignore those IETF
folks, they are incompetent in this area, just turn all of this
over to us".  

So we started on what became IDNA2008.  We again got a good deal
of advice from the direction of UTC.  In addition to the advice
that would have amounted to repeating decisions in IDNA2003 we
had concluded were mistakes (again, different assumptions about
design, use context, constraints, and maintenance), we were
advised to not use properties and categories (because, in the
last analysis, decisions were complicated and subjective and
that the real information could only be in code point by code
point analysis and tables), were advised to drop "Probably yes"
and "Probably no" as unnecessary and making things too
complicated, and were even advised to drop the new version
reviews and, in particular, the possibility of adding to the
exceptions list because they were going to be completely
unnecessary.  We were also advised to retain mandatory mappings
similar to those of IDNA2003 even if it meant giving up the
no-information-loss relationship between what came to be called
A-labels and U-labels and to adopt an absolute stability policy
such that any code point or label that was ever permitted to be
created under IDNA2003 would remain valid (and with the same
interpretation) forever.   We took some of that advice but,
after much wider consultation than IDNA2003 or anything since
2010 has gotten, rejected most of it.  

One result was UTR46.  Another was an additional approach to
ICANN suggesting that they just reject the IETF's outputs and
let the Unicode Consortium take over.  And now we are told that
another set of people have come together under an ICANN
umbrella; with what may be a different agenda, priorities, and
symptoms; and that they have done a lot of work, thereby
changing the picture.  The work is (at least mostly and AFAICT)
of very high quality, just as (at least most of) the Unicode
Consortium work has been.  But it is clear that it has been
structured under different priorities and assumptions,
assumptions that originate with the requirements of the root and
seeing how that can be abstracted to other zones.  There appear
to also have been other assumptions or actions that have been
driven by the needs of name-sellers, similarities and
differences between contracted and non-contracted parties,
possibly excessive influence from those who can afford to play
the ICANN system, and public domains (in the "for sale or
otherwise available to unaffiliated parties") sense.   Much of
it is driven by ideas about strong recommendations rather than
boundary requirements, a model the IETF explicitly decided to
stay out of with both IDNA2003 and IDNA2008.  Some of the advice
is not even consistent with advice emerging from other
ICANN-sponsored entities.    Maybe consistency is in the near
future and all of that is applicable to the entire DNS, but we
have to see that as unproven at best.  If we are going to use
the information that is emerging from ICANN work in the IETF,
its consistency (both internal and with whole-Internet and
whole-DNS requirements) has to be demonstrated through IETF
discussion and evidence of informed consensus, not assertions
that, because a good deal of work has been done, "the situation
has changed".  The alternate hypothesis has to be that another
set of assumptions, by yet another cast of characters, about
what is important led to conclusions that may not be appropriate
for the global Internet and the entire DNS hierarchy.

> Well, once again we must separate the different issues from
> each other or we will never be done. I see now, I guess:
> 

Let me state that differently, in a way that might result in
less delay in publication of that document relative to other
ways to reduce risk.  Like it or not, we are in a situation in
which people consider the IANA tables and not the rules to be
definitive (for this example, it actually makes little
difference, but is easier to explain).  We should also remember
the amount of pushback we got about declaring some
previously-valid code points or strings to be invalid or
interpreting them differently (whether it could be shown that
such strings existed in the DNS as more than demonstrations or
not) and that we had an informal treaty with a number of
registries in which they agreed (or we agreed with them) that
some changes were important but that we weren't going to make
similar changes again.  Even if we need to invalidate something
that was valid through the time RFC 6452 was published or when
the initial IAB statement about U+08A1 noted that there were
some fundamental questions, we should do it now (or at least
explain the issues) rather than publishing tables that look
complete and then coming back in a fairly short time and saying
"whoops".  

That, to me, requires examining the documents and questions in
the queue to see whether anything in them might change our minds
about the validity of code points under the 2010 RFC 5892 rules
and, if so, what needs to be done to mitigate that.  In part
because it isn't an example either you or I developed, the
question of whether combining hooks and slashes should be added
to the exceptions list in Section 2.6 and DISALLOWED (and what
other code points deserve the same treatment) should be
addressed now, not at some vague point in the future when the
IETF gets a transfusion of energy in this area.

So my question is what we need to do with
draft-faltstrom-unicode11 that would hold the risk of publishing
tables with code points shown as PVALID but which we later feel
a need to change our minds about to an absolute minimum (I don't
like the idea of changes from DISALLOWED to PVALID either, but
agree they are slightly less problematic).  That includes being
sure that we are ok with the rules and structure of IDNA2008
after we have learned whatever it is we have, or should have,
learned since late 2011.  That means that
draft-faltstrom-unicode11 or something that will be processed
and approved concurrently must address the outstanding errata on
5892 and either fix them with document updates or decide they
are basically irrelevant.   If we are going to continue to rely
on registries to be responsible for a larger number of decisions
and more finesse than we have seen in practice in many cases,
then we need to review the documents that bear on that question
and either publish them or make note of the issues in some
effective way in draft-faltstrom-unicode11.  And, if there is
any chance that we are going to add exceptions or adjust
IDNA2008's rules -- in RFC 5982, RFC 5891, or elsewhere, I think
we need to resolve that before draft-faltstrom-unicode11 is
published if only because such changes might also change the
contents of the tables.

If we publish draft-faltstrom-unicode11 with the general tone of
"all the tests pass, everything is fine, and we should just move
forward", we are likely to either preempt any future reviews or
changes or find ourselves facing outrage from the community if
we do later conclude changes are needed.

> 2. Can new derived property value and changes be done via
> categories decided by UTC? 3. Should IETF start pick
> individual code points?

If the answer to the first question is "no" and/or the second is
"yes", IDNA2008 is in need of at least some redesign and
possible replacement.  Fortunately, I don't believe that is the
case even while I think that some of the property values and
categories might need reassessment.  But they are exactly the
questions I think we need to be asking.

Finally, picking up one comment from Asmus's later note because
it adds context to what I write above...

--On Monday, February 18, 2019 01:59 -0800 "Asmus Freytag (c)"
<[email-masked]> wrote:

>...
>> So let me be more specific, is there a category that can be
>> used to  find what code points should get this new by you
>> invented property  value, or added to one of the context
>> rules?
> 
> I think so - but that needs to be verified in more detail (not
> something that gets settled in the span of one email)

So, suppose we publish draft-faltstrom-unicode11-07 without any
significant changes as, e.g., RFCnnnn.  And then suppose we
decide to invent a new property, let's call it ComplexScript1,
and insert a new rule as Section 2.11 of RFC 5892 (with the
appropriate change to Section 3) and publish the corresponding
RFC5892 update as RFCnnnx.  Suppose there are at least some code
points that are PVALID under RFCnnnn but DISALLOWED under
RFCnnnx or DISALLOWED under RFCnnnn but PVALID under RFCnnnx (or
that they are associated with contextual rules that change their
validity in some circumstances but not in others).  I suggest
that would be Really Bad.  It would put us into the position of
making incompatible changes that we should have anticipated or
at least warned people might be coming because the need for them
would not be 2019 or 2020 discoveries but issues we know about
now (and, in most cases, have known about for years).   We would
invite a new version of UTR#46 that attempted to specify (or
advise people to avoid) a transition between
IDNA2008/IDNA2008-nnnn and IDNA2008-nnnx.   And the end result
would be that the too-large number of implementations and
behaviors that are described, or try to pose, as "IDNA" would
increase.

Conclusion (again): rather than publishing
draft-faltstrom-unicode11 in -07 form and substantive content,
we need to review all of the issues that might affect either the
design of IDNA2008 or what we tell people about how to apply it
and then either whip any relevant documents into shape and
publish them concurrently (or before) a revised
draft-faltstrom-unicode11 or revise draft-faltstrom-unicode11 to
include sufficient warnings, placeholders, and/or
forward-pointing references to be sure that we do not tell
readers that some code points or actions that we can anticipate
possibly changing are ok (or not) and then later have to tell
them otherwise.

    john

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Feb 2019 18:56:08 -0500

--On Monday, February 18, 2019 01:08 -0800 "Asmus Freytag (c)"
<[email-masked]> wrote:

> This works as long as your categories are reasonably generic
> and do not try to delve into "which context rules" would be
> recommended, because that's highly script dependent. We just
> know that you shouldn't allow them unrestricted by default.

But there are no code points, at least no code points outside
the ASCII repertoire, that are "unrestricted by default".
Every code point classified as PVALID and every label containing
code points that are valid by application of appropriate
contextual rules, is _still_ restricted by the rule that
registries (at all levels of the DNS) are required to understand
what they are doing and to be responsible about it.  If we want
to give (or point to) additional advice and can figure out how
to do that, fine, but where the specification is today isn't
"unrestricted by default" and setting that up as a strawman does
not, IMO, help us make progress.

If this were the only issue and we concluded IDNA2008 should not
go further in the direction of advice than it does today, then
the best way to get draft-faltstrom-unicode11 published as
quickly as possible (see below), might include:

* Get draft-klensin-idna-rfc5891bis into shape and ready for
publication.  I think it would require a bit of work to be used
this way.  In particular, make sure it says what we want it to
say about "troublesome".

* Persuade ICANN to release "troublesome" as part of general
advice to registries, especially at the second level but for the
use of anyone who felt like using it.   ICANN's doing that would
avoid IETF constraints about informed community consensus, code
point by code point analysis and listings, and long-term
concerns about maintenance... and would almost certainly be a
lot faster than trying to develop IETF consensus about those
issues.

* Include text in draft-faltstrom-unicode11 that stresses the
upper limit nature of IDNA2008, mentions the requirement that
registries impose their own restrictions and be careful about
it, and point to draft-klensin-idna-rfc5891bis for further
discussion.   If there is really urgency about
draft-faltstrom-unicode11 (see below) I can see ways to make
that reference non-normative so it would not hold up publication
_if_ we were quite confident we were actually going to finish
and publish draft-klensin-idna-rfc5891bis.

  --or-- 

we could try to figure out if the IETF wants to go into the
recommendations business, make a recommendation about a WG or
other mechanism to accomplish that, etc.  However a good idea
that might be, nothing that has occurred in either the last
several months (before and during IETF 102) or the last four and
a half years (since the first posted version of
draft-klensin-idna-5892upd-unicode70) has given me much hope
that it would be quick.

_However_

* I still haven't seen an explanation of why publication of
draft-faltstrom-unicode11 is urgent or who is pushing for it
other than the equivalent of "we've screwed around long enough".
That isn't much of an explanation because we have screwed around
long enough with a fairly long list of i18n documents, none of
which have been singled out in this way.  Some of them haven't
even been mentioned in this discussion including the early 2014
attempt to revise the IETF character set and languages policy
document (draft-sullivan-rfc2277-bis-00).

* The above does not address issues raised and unresolved from
that list of other documents, just the relationship between the
present I-D, "troublesome", and registry responsibility.

best,
   john

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Mon, 18 Feb 2019 16:25:22 -0800

> the ASCII repertoire, that are "unrestricted by default".

By "unrestricted by default" I mean an IDN table where this code point 
simply
appears in the list of allowed code points for the registry without any
LGR WLE or context rules or without being limited to a sequence.

I'm sorry if you misconstrued this.

> what they are doing and to be responsible about it.

Yes, no argument.

I was addressing the case where registries simply do not add any
restrictions of their own, even though they most probably should have.

>    If we want
> to do that, fine,

good (both alternatives)

> this way.  In particular, make sure it says what we want it to

Or about "a framework for advice to registries on label generation rules 
". (*)

That's how I am thinking about this.

(*) it's not just character selection.

> lot faster than trying to develop IETF consensus about those

I think we should still have a bit more of a discussion of what I call the
"framework" -- this has more to do with the types of actions that would
be appropriate and the types of character categories.

Also, about the way all of this can be related to other Unicode properties.

Even if we end up pushing this as a request to ICANN, it would help if we
got a better idea of what we would like to see. It can't just be to ask 
ICANN
to "publish advice to registries".

> and publish draft-klensin-idna-rfc5891bis.

That would work for me.

> draft-klensin-idna-5892upd-unicode70) has given me much hope

Understand your reasoning for being cautious - but I thought with the
i18ndir we have at least the start of an attempt for IETF to become serious
about issues like this.

A./

PS: about "urgency" - as it stands 5892 applies to all of Unicode. With each
version, the "automatic" propagation of properties will become more of
the standard way to handle Unicode and the IANA parameters will become
less relevant.

You could say, as long as Patrik's draft doesn't contain any new exceptions
we don't need it. Not sure I buy that.

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 18 Feb 2019 16:48:06 -0800

On 2/18/2019 2:28 PM, John C Klensin wrote:

(skipping long digressions about the history of IDNA - I wasn't there 
and can't evaluate any claims)
> possibly changing are ok (or not) and then later have to tell

My take would be that I see IDNA2008 as a protocol setting an upper limit.

I do not see a realistic way to change the protocol or to set a 
different (lower) upper limit that is mandatory for all users of the 
protocol. I regard any attempt in that direction as a complete 
non-starter. For similar reasons, I see no utility in trying to add any 
of the new code points to exception lists (because we would then need to 
perhaps existing ones to exception lists and that's a non-starter).

I think that "advice to registries" and the reminder to registries about 
the due diligence aspect of the protocol are different animals, and 
progress might well be possible (especially as ICANN is actively trying 
to engage in that field).

Therefore, our task boils down to making sure we issue draft-klensin 
with the appropriate warnings and exhortation in short order, perhaps on 
the heels of draft-faltstrom.

Based on that, if we can pull this off, we should see whether we can 
provide a generalized "framework" that registries can use to evaluate 
their due diligence and that provides some context and recommendations.

To the extent that it makes recommendations about existing code points 
these should ideally be based on external references, not IETF 
decisions. Those references could be combinations of Unicode properties, 
or pointers to properties derived by other external entities.

Also, I'm thinking any recommendations in the framework should be 
generic: Patrik and I discussed some examples; these include binary 
divisions of "need attention" / "do not need attention" and "is well 
understood (because in modern use by ordinary users)" / "in not well 
understood (because not in such use)" and sub-divisions like "generally 
needs a context rule" / "not" rather than giving specific context rules.

Patrik dislikes when I write "generally" but what that means is nothing 
more than the shorthand for "Registries should expect that this code 
point must be evaluated for appropriate context rules, which may be 
somewhat specific to the code point, the script and / or the language 
for which the zone is intended, or depend on the presence or not of 
certain other code points in a zone's repertoire; in some instances, 
such an evaluation may conclude that a certain combination of code point 
script and language, plus presence or absence of other code points in 
the zone repertoire, mean that no such context is required after all."

A "framework" document would explain these concepts and / or link such 
categories to (combinations of) certain Unicode properties or properties 
developed by other parties. Then we can go further and suggest that 
ICANN or whoever go and make more detailed advice building on such a 
framework.

A./