Subject: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and Unicode 11.0.0) to Informational RFC

From: "Paul Hoffman" <[email-masked]>
Date: Mon, 03 Dec 2018 15:34:56 -0800

Forwarded message:

> From: The IESG <[email-masked]>
> To: IETF-Announce <[email-masked]>
> Cc: [email-masked], [email-masked], 
> [email-masked], Russ Housley 
> <[email-masked]>
> Subject: Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and 
> Unicode 11.0.0) to Informational RFC
> Date: Mon, 03 Dec 2018 07:33:18 -0800
>
> The IESG has received a request from an individual submitter to 
> consider the
> following document: - 'IDNA2008 and Unicode 11.0.0'
>   <draft-faltstrom-unicode11-05.txt> as Informational RFC
>
> The IESG plans to make a decision in the next few weeks, and solicits 
> final
> comments on this action. Please send substantive comments to the
> [email-masked] mailing lists by 2018-12-31. Exceptionally, comments may 
> be
> sent to [email-masked] instead. In either case, please retain the 
> beginning of
> the Subject line to allow automated sorting.
>
> Abstract
>
>
>    This document describes changes between Unicode 6.3.0 and Unicode
>    11.0.0 in the context of IDNA2008.  It further suggests for the 
> IETF
>    a path forward regarding ensuring IDNA2008 follows the evolution of
>    the Unicode Standard.
>
>    In a few cases changes have been made in the Unicode Standard 
> related
>    to the algorithm IDNA2008 specifies.  IDNA2008 do give the ability 
> to
>    add exceptions for backward compatibility to the algorithm but the
>    conclusions provided in this document suggests no such changes.
>
>    Thus this document requests that IANA update the tables to Unicode
>    11.
>
>    In addition, all registries should continue the practice of
>    calculating a repertoire using conservatism and inclusion 
> principles.
>
>
>
>
> The file can be obtained via
> https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/
>
> IESG discussion can be tracked via
> https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/ballot/
>
>
> No IPR declarations have been submitted directly on this I-D.

---

From: "Paul Hoffman" <[email-masked]>
Date: Mon, 03 Dec 2018 16:02:01 -0800

Before I go to the [email-masked] mailing list with my concerns about 
this draft, I hope it is OK to bounce them off people here in case I'm 
wildly off track.

=====

In Section 1:
    Specifically, the Internet Architecture Board did issue a statement
    [IAB] which requested IETF to resolve the issues related to the code
    point ARABIC LETTER BEH WITH HAMZA ABOVE (U+08A1), introduced in
    Unicode 7.0.0 [Unicode-7.0.0].  This document resolves this issue 
and
    suggests IDNA2008 standard is to follow the Unicode Standard and not
    update RFC 5892 [RFC5892] or any other IDNA2008 RFCs.

In Section 4.1:
    The discussion in the IETF concluded that although it is possible to
    create "the same" character in multiple ways, the issue with U+08A1
    is not unique.  In the case of U+08A1, it can be represented with 
the
    sequence ARABIC LETTER BEH (U+0628) and ARABIC HAMZA ABOVE (U+0654).
    Just like LATIN SMALL LETTER A WITH DIAERESIS (U+00E4) can be
    represented via the sequence LATIN SMALL LETTER A (U+0061), and
    COMBINING DIAERESIS (U+0308).  One difference between these 
sequences
    is how they are treated in the normalization forms specified by the
    Unicode Consortium.

This sounds like the IETF is saying that if the Unicode Consortium 
changes how a character appears in a normalization form other than for 
case folding (Section 2.2 of RFC 5892), that change does not affect the 
tables for IDNA2008. Is that correct?

=====

In Section 4.1:
    As U+08A1 is discussed in draft-freytag-troublesome-characters
    [I-D.freytag-troublesome-characters] and elsewhere.  Regardless of
    whether those discussions ends in recommending including the code
    point in the repertoire of characters permissable for registration 
or
    not, it is acceptable to allow the code point to have a derived
    property value of PVALID.

This sounds like it is saying that even though 
draft-freytag-troublesome-characters is meant for standards track, 
because it is not yet finished, this document (which is informational) 
can ignore the other document and make changes to the IANA registry. If 
that's correct, it concerns me because it could make the IANA registry 
unstable for characters that we know about and are actively discussing. 
If I'm not correct, I'd like to hear why so that maybe this document can 
be reworded.

--Paul Hoffman

---

From: John C Klensin <[email-masked]>
Date: Mon, 03 Dec 2018 20:21:33 -0500

Paul,

This is a very difficult situation and I want to be clear that I
understand Patrik has done what he has been asked to do, that I
am not blaming him for accepting those requests, and that I
always (this case included) believe we are better off with a
document on the table to which people can react than by trying
to make progress by collective handwaving.

Let me state a variation on your two concerns much more broadly
and add a third...

(1) When the IAB issued its statement in January 2015, the
problem was believed to be about U+08A1.   At about that time or
soon thereafter, we learned more about the issues and started
posting versions of draft-klensin-idna-5892upd-unicode70 that
explored the generalization of that problem into what some of us
have come to call the "non-decomposing code point" problem and a
few related issues.  That I-D, which is not referenced, does not
actually propose solutions although it goes much further than
the IAB Statement in exploring and explaining the issues.  The
IAB issued a subsequent statement on the subject (also not
referenced) that can be interpreted in a variety of ways, one of
which is fairly close to "the IETF isn't engaging on this, so
never mind".  Describing this document as having resolved the
issues is true iff "resolved" in the IETF now means "blown off
without any meaningful solution" or something close to it.

In addition, I personally read "suggests IDNA2008 standard is to
follow the Unicode Standard and not update RFC 5892 [RFC5892] or
any other IDNA2008 RFCs" as effectively eliminating the
version-by-version review process called for in Section 5.1 of
RFC 5892 (and followed by Patrik and yourself to produce RFC
6452) if favor of uncritically accepting changes in The Unicode
Standard.  Insofar as that reading is correct, this is
necessarily a Standards Track document that updates RFC 5892 and
makes what I believe is a very substantive change to it.

(2) Your question about normalization is a bit different.  At
least AFAICT, the question of UTC changing the normalization
properties of a given code point is a non-issue because those
properties are covered by stability rules and won't change.  If
they were to change in a way that altered whether a putative
label was NFC-compliant, that would certainty be reflected in
IDNA2008 processing because that processing requires strings in
NFC form.  However, we made assumptions that were reflected in
the design of IDNA2008 about there being no code points such
that one form could be constructed by a combining sequence of
two or more other and both that form and the combining sequence
be NFC-valid.  See draft-klensin-idna-5892upd-unicode70 for more
discussion on that point and its implications.

(3) The entanglements between this document and several others
(including draft-klensin-idna-5892upd-unicode70,
draft-freytag-troublesome-characters (which is either the basis
for modifying 5892 as I think your comments suggest or which
complements the clarification about registry responsibilities in
draft-klensin-idna-rfc5891bis)), as well as the IETF's inability
to act on any of them, were the motivation of the BOF at IETF
102 that led to this mailing list.  I am told, although I wasn't
able to find it quickly, that there is now a notation from the
AD in the file that this document must be reviewed by the
directorate that BOF decided should be created (and with which
the AD(s) agreed).  I believe that directorate should be
launched and its charter and scope be made clear and that it
should then look at this document in context with the others,
examining the questions you have raised and others, and then
make recommendations as to how to proceed.  I believe that the
plan that came out of the BOF was completely consistent with
that view.  Only after the directorate's recommendations are
available to the community for review and consideration do I
believe that an IETF Last Call on this document is appropriate.

And, yes, I have expressed that view to Alexey (in his
Responsible AD role).

best,
    john

--On Monday, December 3, 2018 16:02 -0800 Paul Hoffman
<[email-masked]> wrote:

> Before I go to the [email-masked] mailing list with my concerns
> about this draft, I hope it is OK to bounce them off people
> here in case I'm wildly off track.
> 
> =====
> 
> In Section 1:
>     Unicode 7.0.0 [Unicode-7.0.0].  This document resolves
> this issue and
>     is not unique.  In the case of U+08A1, it can be
> represented with the
>     COMBINING DIAERESIS (U+0308).  One difference between
> these sequences
> This sounds like the IETF is saying that if the Unicode
> Consortium changes how a character appears in a normalization
> form other than for case folding (Section 2.2 of RFC 5892),
> that change does not affect the tables for IDNA2008. Is that
> correct?
> 
> =====
> 
> In Section 4.1:
>     point in the repertoire of characters permissable for
> registration or
> draft-freytag-troublesome-characters is meant for standards
> track, because it is not yet finished, this document (which is
> informational) can ignore the other document and make changes
> to the IANA registry. If that's correct, it concerns me
> because it could make the IANA registry unstable for
> characters that we know about and are actively discussing. If
> I'm not correct, I'd like to hear why so that maybe this
> document can be reworded.
> 
> --Paul Hoffman
> 
> _______________________________________________
> i18nRP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i18nrp

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 3 Dec 2018 17:33:20 -0800

On 12/3/2018 4:02 PM, Paul Hoffman wrote:
>    Unicode 7.0.0 [Unicode-7.0.0].  This document resolves this issue and
>    is not unique.  In the case of U+08A1, it can be represented with the
>    COMBINING DIAERESIS (U+0308).  One difference between these sequences
> case folding (Section 2.2 of RFC 5892), that change does not affect 
> the tables for IDNA2008. Is that correct?

It's actually not correct that these are necessarily the "same" 
characters, if by "same" you mean _identical_ glyph outlines in 
high-resolution/high-quality layout.

While U+08A1 contains a graphical element that looks like the letter BEH 
and another that looks like a HAMZA, the relative placement of these two 
are not necessarily the same as when both BEH and HAMZA are used 
independently. (Effectively reflecting that the combination is an 
independent letter, not an alternate encoding for the sequence).

Unlike the case of Latin Ä, where high-resolution/high-quality rendering 
ideally results in the exact same appearance, reflecting the fact that 
precomposed code point and sequence are alternating encodings of the 
same letter).

Unicode normalization has never been a full "glyph folding"; it was 
never intended to do that: it's design point is to fold alternate 
encodings for the same abstract entity. This is not the same as 
preventing exact lookalikes, because there are some entities that are 
quite obviously logically distinct, yet are lookalikes.

The case of Latin/Greek capitals (not so relevant for IDNs) and 
Latin/Cyrillic (both Capitals and lowercase, therefore more relevant for 
IDNs) are well known. Everybody accepts that once a letter is part of  a 
different script, it is different and cannot be normalized.

Another distinction that cannot be normalized is that between letters 
and digits. Yet there are quite a few scripts where native digits 
sometimes have the same shape as a native letter. (In Latin we have 0 
and O; in our typographic tradition most, but not all, fonts will strive 
to make these distinct - that is not so in other scripts, where the 
shapes are often identical). Again, there's a shared understanding that 
normalizing digits to letters causes more problems than it solves and 
therefore the two are treated as distinct.

What I see Patrik's document reflecting is the inherent limitation of 
the normalization algorithm: it folds alternate encodings of the same 
abstract entity, but does not purport to be a universal glyph folding.

Given that the set of PVALID code points has long included both code 
points and sequences that have identical glyphs, it was surprising that 
the process was stopped for something that was merely a set of close 
lookalikes.

Finally, there's nothing here about a "change" in normalization. Whether 
any code point that is contemplated for addition should have a 
decomposition or not is something that depends on the identity of the 
abstract entity that it represents, not simply on the shape, and 
certainly not on an approximate shape - or a shape oriented character name.

Now, once a decomposition is found to be appropriate, the stability 
rules actually stipulate that the code point should not be added, so the 
only cases that are at all permissible are code points that represent a 
distinct entity that is not otherwise encoded. And, to continue that 
line of thought, the fact that some entities share a shape (or that some 
sequences could result in a more or less identical shape) is not 
something that effects this determination. Unicode is fundamentally not 
a lego set for creating glyph shapes; what is encoded is abstract 
characters (even if there's the historical baggage of precomposed 
characters in the face of open-ended use of diacritics that has lead to 
normalization in the Latin/Greek and Cyrillic scripts).

>    point in the repertoire of characters permissable for registration or
> can ignore the other document and make changes to the IANA registry. 
> If that's correct, it concerns me because it could make the IANA 
> registry unstable for characters that we know about and are actively 
> discussing. If I'm not correct, I'd like to hear why so that maybe 
> this document can be reworded.

The set of code points that are PVALID are overbroad; they represent an 
outer envelope of what is allowed under the protocol for ANY given zone, 
but that set is much wider than what should be RECOMMENDED for inclusion 
in ALL zones (or, setting aside the special restrictions for the Root, 
recommended for all public zones).

The draft ID cited in Section 4.1 is clear that it does not intend to 
change the set of PVALID code points; there are some zones where 
allowing any PVALID code point may well not cause issues. However, in 
widely shared zone, particularly those with users from different 
languages and scripts, a well-chosen subset of the PVALID code points 
would reduce security issues and improve usability.

The details of the best approach in these cases are not always obvious. 
In Arabic, for example, it is the combining HAMZA that would become NOT 
RECOMMENDED (because unlike the letter U+08A1 and similar) it is not 
needed to form "useful mnemonics" in Arabic or any of the other 
languages using the Arabic script.

PVALID code points include those that look like (!) and (') - these are 
also not to be recommended for general use.

In the Indic scripts there are many examples of compound glyphs that can 
be achieved by more than one sequence of code points. In these 
instances, the Unicode standard has gone on record identifying the 
sequences that MUST NOT be used. (As only one of the alternate sequences 
is legitimate, there is no normalization -- because normalization would 
have implied both sequences are equivalent).

Unfortunately, IDNA2008 lacks a scalable mechanism to DISALLOW sequences 
of code points. (The only available formalism, the use of CONTEXTO rules 
is not scalable beyond a few cases, as it is not presented in a machine 
readable format -- unlike for example RFC 7940).

The draft ID cited in Section 4.1 may well RECOMMEND that registries not 
allow such sequences (and that they use machine-readable specifications 
based on RFC 7940 to do so). However, that entire discussion is 
unaffected by Patrik's document and unaffected by IANA proceeding in the 
context of the existing parameters for IDNA 2008.

A deeper linguistic reason for keeping the domains of these ID's 
separate is in the fact that best practices for support of complex 
scripts in public zones would implement policies that have context rules 
that go beyond preventing specific alternate sequences. In many cases, 
the latter fall out from applying more general restrictions that 
reflect, for example, the syllable structure that is a common feature of 
complex scripts. Such general restrictions are sensitive not only to the 
script, but in some cases, to the language(s) to be supported. A 
one-size-fits-all approach (as would be required on the protocol level) 
is therefore inappropriate.

Perhaps it might be possible to edit the text of Section 4.1 to make 
sure that the "repertoire" mentioned is to be understood as the 
repertoire for "any particular zone" as opposed to something like the 
set of PVALID code points.

With that clarification, I see no issue in that section.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 3 Dec 2018 18:01:16 -0800

On 12/3/2018 5:21 PM, John C Klensin wrote:
> This document resolves

I think this is perhaps not worded in a not very felicitous way.

The intent, I believe, is not a general policy statement, but simply the 
summary conclusion that, based on the expert's review, the changes from 
Unicode 6.3.0 to Unicode 11.0.0 do not rise to the level where a change 
/ update to RFC 5892 is required.

I therefore see no reason to consider the status of this draft changed 
in any way.

A./

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 07:59:00 +0100

Good morning!

I see there are a few comments on this draft, I will try to respond to each one of them.

On 4 Dec 2018, at 1:02, Paul Hoffman wrote:

> Before I go to the [email-masked] mailing list with my concerns about this draft, I hope it is OK to bounce them off people here in case I'm wildly off track.
>
> =====
>
> In Section 1:
> This sounds like the IETF is saying that if the Unicode Consortium changes how a character appears in a normalization form other than for case folding (Section 2.2 of RFC 5892), that change does not affect the tables for IDNA2008. Is that correct?

First of all, this document evaluates the individual changes made up until and including Unicode 11. Sure, one could say this has implications on the IETF view of existence of normalization rules (or not) but that is not the intention here. The result of this review should neither be extrapolated to future versions of Unicode nor to future evolutions of normalizations.

> This sounds like it is saying that even though draft-freytag-troublesome-characters is meant for standards track, because it is not yet finished, this document (which is informational) can ignore the other document and make changes to the IANA registry. If that's correct, it concerns me because it could make the IANA registry unstable for characters that we know about and are actively discussing. If I'm not correct, I'd like to hear why so that maybe this document can be reworded.

First of all, this document is (as it seems to me now) to be Standards Track. So that issue is taken care of.

Regarding the reference to the troublesome characters, I also have referenced a document by Klensin. I do not mind referencing other documents as well regarding U+08A1, but I do NOT want to explain everything in this document (again), i.e. no repeat please.

The klensin document unfortunately have expired but look at diff between -04 and -05 and you see the reference. This is also why I in this version wrote "and elsewhere".

If you have better more references, send them in my direction.

   paf

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 08:07:49 +0100

On 4 Dec 2018, at 2:33, Asmus Freytag wrote:

> What I see Patrik's document reflecting is the inherent limitation of the normalization algorithm: it folds alternate encodings of the same abstract entity, but does not purport to be a universal glyph folding.

This is why I continue to believe IETF must investigate _every_ individual code point where decisions could be made to include exceptions in 5892.

> Given that the set of PVALID code points has long included both code points and sequences that have identical glyphs, it was surprising that the process was stopped for something that was merely a set of close lookalikes.

I do not understand what you mean by "sequence" because a sequence can never be PVALID, only individual code points. If you imply "sequence of code points, each having derived property value PVALID" then I understand.

The reason is simple, pure coincidence. If I exaggerate. And I am sure we will see more things like these.

And for IETF it does not matter as we have to evaluate code points manually. Most of these I catch when I evaluate differences when caluclating derived property values from new versions of unicode. But sometimes the catch is when people just look at the newly added/changed code points.

Detail: no incompatible changes for Unicode 12.0.0 from Unicode 11.0.0, at least not at the current beta of Unicode 12.0.0.

> Perhaps it might be possible to edit the text of Section 4.1 to make sure that the "repertoire" mentioned is to be understood as the repertoire for "any particular zone" as opposed to something like the set of PVALID code points.

Please send text :-)

Yes, the goal of this document is to emphasize what have already been said many many times, that it is not recommended to allow all PVALID code points (in any combination) in the registration policy in _any_ zone. Instead a subset is to be chosen using the conservatism principle explained by IAB, by SSAC and in IDNA2008.

   Patrik

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 08:16:52 +0100

On 4 Dec 2018, at 2:21, John C Klensin wrote:

> In addition, I personally read "suggests IDNA2008 standard is to follow the Unicode Standard and not update RFC 5892 [RFC5892] or any other IDNA2008 RFCs" as effectively eliminating the
> version-by-version review process called for in Section 5.1 of RFC 5892 (and followed by Patrik and yourself to produce RFC 6452) if favor of uncritically accepting changes in The Unicode Standard.  Insofar as that reading is correct, this is
> necessarily a Standards Track document that updates RFC 5892 and makes what I believe is a very substantive change to it.

The goal is to say in this document that it is suggested for changes up until and including Unicode 11.0.0 IETF accept the changes in Unicode Standard.

What to do with future versions is something IETF have to evaluate for every version that is released.

The process I have used for each version of Unicode is the following:

1. Unicode releases beta versions of Unicode

2. I calculate the derived property values and do a diff

3. I report the result to IAB and whatever I feel is the most recent IDNA-related mailing list in the IETF

3.1. If there are no incompatible changes, I say so and simply suggest I will report to IANA "to move on" -- and if I do not hear anything within say two weeks, I simply do so

3.2. If there are incompatible changes, I say so to and suggest an I-D that include my and others review of the situation

Step 3.2. have so far resulted in two things:

- RFC 6452
- This document, where I basically suggest same action as in RFC 6452

> I am told, although I wasn't able to find it quickly, that there is now a notation from the AD in the file that this document must be reviewed by the
> directorate that BOF decided should be created (and with which the AD(s) agreed).

See <https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/history/>

2018-12-03 05 Alexey Melnikov This document needs to be reviewed by the to-be-created-soon Internationalization Directorate.

   Patrik

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 08:17:31 +0100

On 4 Dec 2018, at 3:01, Asmus Freytag wrote:

> The intent, I believe, is not a general policy statement, but simply the summary conclusion that, based on the expert's review, the changes from Unicode 6.3.0 to Unicode 11.0.0 do not rise to the level where a change / update to RFC 5892 is required.

Correct.

Only up to and including Unicode 11.

   paf

---

From: John C Klensin <[email-masked]>
Date: Tue, 04 Dec 2018 05:33:16 -0500

--On Tuesday, December 4, 2018 08:16 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

>...
> 3. I report the result to IAB and whatever I feel is the most
> recent IDNA-related mailing list in the IETF
> 
> 3.1. If there are no incompatible changes, I say so and simply
> suggest I will report to IANA "to move on" -- and if I do not
> hear anything within say two weeks, I simply do so
> 
> 3.2. If there are incompatible changes, I say so to and
> suggest an I-D that include my and others review of the
> situation

Patrik,

I think these are the right steps.  As far as they go, they are
certainly consistent with what the specs require.   There are,
IMO, two weaknesses which might interact with Paul's concerns
(or not):

(a) You are talking about a relatively mechanical process.  The
review anticipated by IDNA2008 is supposed to also identify new
code points that, because it is possible that the needs of IDNA
may differ from the criteria used by UTC in assigning new code
points and corresponding properties, may need to be identified
as exceptions or assigned to one of the CONTEXTx categories.  No
calculation check such as the ones you describe is going to
detect those cases; detecting them requires manual checking of
the newly-assigned code points and whatever rationale Unicode
provides for them.  It may be worth noting that, whatever the
merits of the observations that caused the original IAB
statement and the non-deoomposing character discussion, that
case was detected by precisely such a manual check, not the
procedure outlined above.  We need both.

Incidentally, if Asmus's comments about the insufficiency of the
CONTEXTo model for a range of problems is correct, we should be
looking at whether IDNA2008 is in need of changes/ updating.
Those changes might turn out to be substantive or as minor as
making it much more clear that what IDNA2008 permits is a
superset of what should be allowed in a particular zone and who
has responsibility (and accountability) for selecting a
zone-specific subset (aided by whatever hints we can provide).
It seems clear to me that we should not process this document
until we have answers to, or at least much more clarity about,
that issue.  YMMD.

(b) With this list supposedly dedicated to procedural
discussions (at least until the ADs or Directorate change that)
and the IDANbis on focused on the protocols and almost
completely dormant, the only place in the above procedure in
which the manual checks can be expected to be performed (or at
least assured) are as part of  "report... to the IAB".  However,
if the IAB has higher priorities and has opted out, lacks the
time and energy, lacks the expertise and does not have a Program
with the expertise, the odds of that review getting done (and
producing a report that the community can review) are probably
slight to none.   My hope going into the BOF was that a model
would emerge that would be able to support, manage, or actually
perform those reviews.  Because the Directorate has not been
established and we don't know either its formal charter or its
membership, much less its priorities, it is impossible to know
whether it will be able to fulfill that role.  However,
advancing this document before either  there is evidence that
the manual reviews have been carefully and competently performed
would seems to me to violate the intent of the standard.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 04 Dec 2018 05:40:24 -0500

--On Tuesday, December 4, 2018 07:59 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

>...
> The klensin document unfortunately have expired but look at
> diff between -04 and -05 and you see the reference. This is
> also why I in this version wrote "and elsewhere".
>...

For the record (because you, Patrik, know this already), "the
klensin document" (an ambiguous reference because there are a
few that may be relevant --see my response to Paul) was allowed
to expire because there appeared to be no way to process it. In
at least one case, I believe I was advised by a relevant AD to
let it expire.  The request for the IETF 102 BOF was the result
of that situation.  Those documents will be updated and reposted
as soon as the Directorate (or the ART ADs) tell me to do so.

    john

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 11:40:44 +0100

On 4 Dec 2018, at 11:33, John C Klensin wrote:

 I think these are the right steps.  As far as they go, they are certainly consistent with what the specs require.
> There are, IMO, two weaknesses which might interact with Paul's concerns (or not):
>
> (a) You are talking about a relatively mechanical process.

Yes, indeed.

What I did not mention is that I also keep track of, to my best ability, comments on various unicode mailinglists, the main IETF mailing list and others where I feel Unicode and IDNA issues might come up. And also of course I collect comments sent to me directly. This as my responsibility as a liaison (etc).

Yes, this is ad-hoc, but as far as I know the best we can do at the moment.

Now when (if? ;-) ) we get this i18nrp group going it will make this ad-hoc process easier and a bit less ad-hoc, but trust me, I will not stop doing what I am already doing regarding this ad-hoc thing. It will just make my life easier (even though it might be more work) and definitely easier for others to track what I am doing as I will know exactly where to report things (and where to catch things).

So, even if it is ad-hoc, we should not ignore this, and I should have had it in my list.

And no, it does not replace anything you and/or Paul ask for, but at least >0.

   Patrik

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 04 Dec 2018 11:42:00 +0100

On 4 Dec 2018, at 11:40, John C Klensin wrote:

> For the record (because you, Patrik, know this already), "the klensin document" (an ambiguous reference because there are a few that may be relevant --see my response to Paul) was allowed to expire because there appeared to be no way to process it. In at least one case, I believe I was advised by a relevant AD to let it expire.  The request for the IETF 102 BOF was the result of that situation.  Those documents will be updated and reposted as soon as the Directorate (or the ART ADs) tell me to do so.

Correct, I knew this.

I just wanted to explain why the reference was removed from my draft.

   paf

---

From: "Paul Hoffman" <[email-masked]>
Date: Tue, 04 Dec 2018 06:59:19 -0800

On 3 Dec 2018, at 22:59, Patrik Fältström wrote:

>> this draft, I hope it is OK to bounce them off people here in case 
>> I'm wildly off track.
>>
>> =====
>>
>> In Section 1:
>> changes how a character appears in a normalization form other than 
>> for case folding (Section 2.2 of RFC 5892), that change does not 
>> affect the tables for IDNA2008. Is that correct?
>
> First of all, this document evaluates the individual changes made up 
> until and including Unicode 11. Sure, one could say this has 
> implications on the IETF view of existence of normalization rules (or 
> not) but that is not the intention here. The result of this review 
> should neither be extrapolated to future versions of Unicode nor to 
> future evolutions of normalizations.

This is good to know. In that case, could you either remove the "One 
difference between these sequences..." sentences, or add a sentence in 
the Introduction section that says "The result of this review should 
neither be extrapolated to future versions of Unicode." Either action 
would clear up this confusion.

>> because it is not yet finished, this document (which is 
>> informational) can ignore the other document and make changes to the 
>> IANA registry. If that's correct, it concerns me because it could 
>> make the IANA registry unstable for characters that we know about and 
>> are actively discussing. If I'm not correct, I'd like to hear why so 
>> that maybe this document can be reworded.
>

Procedurally, it is not, I believe. A new draft needs to be issued, and 
the IETF Last Call has to start again. Fortunately, this IETF Last Call 
is only a few days old, so this should not delay anything much.

> Regarding the reference to the troublesome characters, I also have 
> referenced a document by Klensin. I do not mind referencing other 
> documents as well regarding U+08A1, but I do NOT want to explain 
> everything in this document (again), i.e. no repeat please.

Agreed.

> The klensin document unfortunately have expired but look at diff 
> between -04 and -05 and you see the reference. This is also why I in 
> this version wrote "and elsewhere".
>

This misses my concern. There is an active draft 
(draft-freytag-troublesome-characters) that seems to want to change the 
IANA registry. Your draft (draft-faltstrom-unicode11) also wants to 
change the registry, but in a different way. My question is whether we 
should be making the registry unstable in this way.

--Paul Hoffman

---

From: Asmus Freytag <[email-masked]>
Date: Tue, 4 Dec 2018 10:04:05 -0800

On 12/4/2018 6:59 AM, Paul Hoffman wrote:
> (draft-freytag-troublesome-characters) that seems to want to change 
> the IANA registry.

Where do you get that?

The draft in question suggests setting up an additional registry with 
information allowing domain name registries to set more restrictive 
policies. It says NOTHING about the status of code points under IDNA 2008.

> Your draft (draft-faltstrom-unicode11) also wants to change the 
> registry, but in a different way. My question is whether we should be 
> making the registry unstable in this way. 

There is no conflict between these drafts.

A./

---

From: "Paul Hoffman" <[email-masked]>
Date: Tue, 04 Dec 2018 10:21:30 -0800

> policies. It says NOTHING about the status of code points under IDNA 

This is where there is concern. In a few places, 
draft-faltstrom-unicode11 makes suggestions to registries about their 
policies. draft-freytag-troublesome-characters does the same, but with 
different results. If draft-faltstrom-unicode11 is only about which 
characters are legal for IDNA2008, it should say that explicitly, and 
say that it is not delving into registry policy. However, I think Patrik 
actively wants to have his draft suggest registry policies that match 
IDNS2008.

--Paul Hoffman

---

From: Asmus Freytag <[email-masked]>
Date: Tue, 4 Dec 2018 10:48:45 -0800

> until we have answers to, or at least much more clarity about,

Complex scripts (such as Indic and related scripts) are conceptually 
written as a stream of syllables. However, what is encoded is the set of 
elements that make up syllables. This leads to a well-formedness 
problem: it is possible to type sequences that are not well-formed 
syllables.

There are two kinds: syllables that "can't happen" given the phonetics 
of given language, and  syllables that are structurally unsound (violate 
the principles of the writing system).

Both can be problematic. The latter, because unsound sequences can look 
exactly like correct sequences, or, alternatively, because unsound 
sequences are not supported by renderers and fonts and lead to 
unpredictable display.

The "can't happen" sequences may cause issues to readers; the best 
analogy is that they are mentally processed like a nonsense letter (such 
as the ones invented by Dr. Seuss in "On Beyond Zebra"), instead of like 
a nonsense "word" as an arbitrary sequence of Latin letters would.

Structurally unsound sequences should be prohibited by policy, while 
some or all of the phonetically impossible sequences may be worth 
prohibiting for added security, but the latter are language dependent. 
(Some languages may differ in what constitutes a structurally sound 
syllable as well, an issue we encountered in the Thai script, where 
renderers designed for the Thai language will not handle some of the 
other languages).

Because of the (potential/actual) language dependency much of these 
constraints cannot be embedded in the protocol layer like IDNA 2008.

There are subset of unsound sequences that Unicode calls out as having a 
"do not use" status independent of language (See chapter 12 of the 
standard for examples). These could be captured with CONTEXTO (or 
similar) constraints in an updated IDNA2008.

However, they are always a subset of the "real" set of unsound 
sequences; any registry doing a good job for complex scripts would 
therefore find that implementing the CONTEXTO constraints becomes 
redundant. (Also, unlike RFC7940, these rules are not machine readable 
and therefore not trivial to implement and verify).

Fundamentally, IDNA2008 primarily focuses on repertoire, the few 
CONTEXTx rules notwithstanding. Changing that model would be a major 
undertaking and it is questionable that the results would be beneficial.

Perhaps some other tack could be taken: in your draft, it could be 
pointed out that Unicode assigns "do not use" to some sequences (which 
means that there is no text, identifier or not, that should contain 
them). Preventing these from registrations could be declared part of the 
responsibility of registries under RFC 5891.

This approach is more flexible because it allows registries to adopt 
context rules that are a superset of the rules needed to exclude these 
particular sequences. Which is precisely what we are doing in the Root 
Zone. If you go to http://icann.org/idn and look for the set of RZ-LGR 
proposals, you will find that the context rules for the various Indic 
scripts do not explicitly call out these "do not use" sequences, but 
that they adopt context rules on some of the constituent code points 
that also disallow those sequences, while implementing a more general 
constraint appropriate to the writing system.

For an example, see section 7 of 
https://www.icann.org/en/system/files/files/proposal-devanagari-lgr-27jul18-en.pdf

This shows the more generalized rules for Devanagari, but also where 
they had to be tweaked to allow for both Hindi and other languages, e.g. 
Santali.

For comparison, check for example, Table 12-1 in 
http://www.unicode.org/versions/latest/ch12.pdf

You will find that Table 12-1 is covered by the rule:

     3. M: must be preceded by C or CN

     (Read: "Dependent vowel signs (matras) must follow a consonant
      with or without an optional Nukta").

This rule prevents dependent vowel signs from being combined with 
independent vowel letters, which is what the list in table 12-1 
enumerates. However, the RZ-LGR rule also covers many other contexts 
that are inappropriate (unsound) for matras; some of the latter 
constraints may not be as universal, and, unlike the prohibition of 
Table 12-1 would be out of scope for a protocol.

The Root Zone LGR uses RFC 7940 to translate this rule into a single 
line of XML, which can be machine processed.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Tue, 4 Dec 2018 12:33:30 -0800

On 12/4/2018 10:21 AM, Paul Hoffman wrote:
> different results.

With compatible results. That is, my draft only ever suggests to use 
stricter subsets.

That is always compatible - if it were otherwise, efforts like the Root 
Zone LGRs would somehow be in conflict with IDNA2008.

> If draft-faltstrom-unicode11 is only about which characters are legal 
> for IDNA2008, it should say that explicitly, and say that it is not 
> delving into registry policy. However, I think Patrik actively wants 
> to have his draft suggest registry policies that match IDNS2008.

I read the draft as being about the former.

None of the text passages that you quoted seem to indicate otherwise? If 
you have an example of a text passage, could you cite it here?

A./

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 05 Dec 2018 08:56:27 +0100

On 4 Dec 2018, at 19:21, Paul Hoffman wrote:

> However, I think Patrik actively wants to have his draft suggest registry policies that match IDNS2008.

There are three things I want to do:

1. Say that for Unicode version up to and including 11.0.0, IETF should follow the Unicode Standard

2. Registries should really implement IDNA2008 and nothing else

3. Registries should not "just" allow all PVALID (etc) code points, but also include a conservative registration policy

I do though not see (3) is in violation with other documents. It just emphasizes the same thing already said elsewhere. For example in the "troublesome characters" draft there is a suggestion on a method to use when developing one such policy.

   paf

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Wed, 5 Dec 2018 10:03:53 -0800

> 2. Registries should really implement IDNA2008 and nothing else

"Registries should implement IDNA2008 and nothing that is incompatible 
with it" would be a better statement as the "nothing else" can be read 
to contradict your item 3.

> 3. Registries should not "just" allow all PVALID (etc) code points, but also include a conservative registration policy

add:

"as mandated in RFCs <get the IDNA RFC citations from John Klensin's 
draft here>."

> I do though not see (3) is in violation with other documents. It just emphasizes the same thing already said elsewhere. For example in the "troublesome characters" draft there is a suggestion on a…

Correct. The mandate exists and should just be cited.

And I agree with your characterization of the intent of the "troublesome 
characters" draft.

A./

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Dec 2018 16:46:45 -0500

--On Wednesday, December 5, 2018 08:56 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

> 3. Registries should not "just" allow all PVALID (etc) code
> points, but also include a conservative registration policy
> 
> I do though not see (3) is in violation with other documents.
> It just emphasizes the same thing already said elsewhere. For
> example in the "troublesome characters" draft there is a
> suggestion on a method to use when developing one such policy.

It is also an existing requirement.  See discussion and
references in draft-klensin-idna-rfc5891bis but that I-D doesn't
actually change anything other than clarifying the existing
rules.   So, whether (3) should be said here may be
controversial unless it is clearly stated as restating rules
established elsewhere (a reference to
draft-klensin-idna-rfc5891bis would help with that, but
certainly is not essential).  On the other hand, if you were to
say anything about registry policies that changed that rule or
encouraged a belief that anything authorized by the 5892 rules
was ok, that would not only clearly update several other
documents but be very controversial.

     john

p.s. addressing another issue, I don't believe we've ever made a
document Historic that is part of the record of evolution of a
Standards Track specification that is not itself considered
Historic.   Even if we had, I think it would be really
unfortunate in this case.

---

From: "Patrik Fältström" <[email-masked]>
Date: Thu, 06 Dec 2018 11:43:39 +0100

On 5 Dec 2018, at 19:03, Asmus Freytag (c) wrote:

> "as mandated in RFCs <get the IDNA RFC citations from John Klensin's draft here>."

Correct, I blame my Swenglish dialect for the mistake in wording I did.

   paf

---

From: Larry Masinter <[email-masked]>
Date: Thu, 6 Dec 2018 16:59:06 -0800

Registries should be conservative in the sense of advising their clients
about the likely usability of the registered name among their target
audience.
I don't think the IDNA RFCs currently are conservative enough. As I've noted
before, usability includes "re-enterable" (including "retypable") .

> -----Original Message-----
> From: i18nRP <[email-masked]> On Behalf Of Patrik Fältström
> Sent: Thursday, December 6, 2018 2:44 AM
> To: Asmus Freytag <[email-masked]>
> Cc: [email-masked]; Paul Hoffman <[email-masked]>
> Subject: Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt>
(IDNA2008
> and Unicode 11.0.0) to Informational RFC
> 
> >>     3. Registries should not "just" allow all PVALID (etc.) code
points, but also
> include a conservative registration policy
> >>
> >
> > add:
> >
draft
> here>."

---

From: "Patrik Fältström" <[email-masked]>
Date: Fri, 07 Dec 2018 05:43:22 +0100

On 7 Dec 2018, at 1:59, Larry Masinter wrote:

> Registries should be conservative in the sense of advising their clients about the likely usability of the registered name among their target audience.
> I don't think the IDNA RFCs currently are conservative enough.

Agree. Specifically, as so many are misunderstanding what the RFCs are saying, and even if also others (like SSAC) request conservatism when coming up with a subset of PVALID (etc) repertoire, I felt it can never be said too many times this is a requirement.

> As I've noted before, usability includes "re-enterable" (including "retypable") .

I might not agree with this, as for example very few characters in arabic script are "retypable" for me. So this is at least a separate issue. But what the registry chooses, I feel is up to them, as long as they do come up with a good repertoire.

   Patrik

---

From: Larry Masinter <[email-masked]>
Date: Thu, 6 Dec 2018 21:34:22 -0800

In reply to my note of 7 Dec 2018, at 1:59

Patrik replied:

> I might not agree with this, as for example very few characters in arabic
script are "retypable" for me.

That is important if you are in the registry's client's target audience. The
interesting point is that a "retypable" requirement "covers" many other
issues, including normalization, joiners, confusables, font choice. It's
plausible to restrict strings used in visual inspection to be retypable.

Usability may vary over time (hopefully no usable strings would become
unusable), depending on deployment.

 > So this is at least a separate issue. But what the
> registry chooses, I feel is up to them, as long as they do come up with a
good
> repertoire.

I think it would be reasonable to supply guidelines that are concrete and
value-based, even if difficult to measure (using Captcha?)

Larry
--
https://LarryMasinter.net

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Dec 2018 00:39:26 -0500

Larry,

The theory about registries in the current IDNA RFCs was to
impose a strong requirement that they be conservative including,
but not limited to, understanding the scripts they are
permitting.  That requirement is, I believe, consistent with
both what you are saying and what Asmus appears to be saying on
that subject.   The details about how to best accomplish that--
to convince them to actually do so-- seem to be a more
complicated matter.  

As I've said in other, probably too-long, notes, most registries
who manage zones within an enterprise or other organization have
been fairly good about it; many of the second-level registries
and others whose business and operational model involves selling
names to unrelated parties have tended to adopt the least
restrictive policies they believe they can get away with.

Part of the problem with the IDNA RFCs is that the requirement
for registry conservatism and responsibility is too scattered
through several documents to come across as important as I think
we all believe it is.   draft-klensin-idna-rfc5891bis (now
expired, but waiting in the wings) was intended to address that
issue.  I haven't looked at the text for a while but strongly
suspect it does not identify and stress labels being
"re-enterable" and "retypable" as you suggest.   It should.  If
you don't mind taking a look at it, I'd appreciate recommended
text.   Or, if that isn't convenient, I'll try to remember to
say something when I next open the document.

thanks,
    john

--On Thursday, December 6, 2018 16:59 -0800 Larry Masinter
<[email-masked]> wrote:

> Registries should be conservative in the sense of advising
> their clients about the likely usability of the registered
> name among their target audience.
>> From: i18nRP <[email-masked]> On Behalf Of Patrik
>> Fältström Sent: Thursday, December 6, 2018 2:44 AM

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Dec 2018 01:12:47 -0500

--On Friday, December 7, 2018 05:43 +0100 Patrik Fältström
<paf=[email-masked]> wrote:

> I might not agree with this, as for example very few
> characters in arabic script are "retypable" for me. So this is
> at least a separate issue. But what the registry chooses, I
> feel is up to them, as long as they do come up with a good
> repertoire.

Patrik,

I think your second point ("up to them" as long as it is "good"
--including sufficiently conservative) is the important one and
the one on which we can all agree.  But I still believe that,
with appropriate qualification, Larry is still correct.   The
very decision to move in the direction of IDNs meant that there
would be labels that would be much easier to read (and enter)
for some populations and other labels that would be easier for a
different population.  It seems to me that we just have to
accept that.  One think it implies is that, if a label is
written in a script one doesn't use frequently, "typing" may
require a character-picker.  Character-pickers are better than
nothing but no fun at all, especially if one does not easily
recognize the characters involved and differences between then
(and, to save Asmus pointing it out, complex scripts may
complicate both the recognition problem and the picking one).
We also need to remember, as Ted Hardie and others have reminded
us multiple times, that there are increasing moves toward voice
input, a trend that will de-emphasize typing and, most likely,
make the lives of those of us who can speak a given language
only with horrible accents much more "interesting".

However, to the extent to which one can sensibly use (including
simple reading of the characters) a label at all at all, I think
ability to re-enter it, to be assured that, insofar as possible,
there are no other labels in the zone that can easily be
mistaken for it, strings that are likely to be entered where it
is is intended, etc., are as important to a conservative
approach as things we've talked about more such as ensuring that
strings are well-formed for the script and that interactions
between RtoL characters and characters with neutral or LtoR
directionality don't cause, e.g., labels to appear in orders
that others might not predict.

I hope Larry has a better (and shorter) way to formulate that
than I do, which is why I'm hoping for text.

best,
    john

---

From: "Patrik Fältström" <[email-masked]>
Date: Fri, 07 Dec 2018 07:16:03 +0100

On 7 Dec 2018, at 7:12, John C Klensin wrote:

> But I still believe that, with appropriate qualification, Larry is still correct.

I think we dive into the details here.

1. I think the ability (for example) copy and paste, respond to, click on, save in bookmarks etc is the most important global requirement

2. The ability to type, to enter via a keyboard, read on a bus, etc is secondary and have localization issues so very much different than (1)

My point was that because of this, I think retypable is very very very much different than other, global, requirements, and the latter should not (I think) muddy the waters of the first requirement I list above.

   paf

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Dec 2018 01:33:49 -0500

--On Friday, December 7, 2018 07:16 +0100 Patrik Fältström
<[email-masked]> wrote:

> 1. I think the ability (for example) copy and paste, respond
> to, click on, save in bookmarks etc is the most important
> global requirement
> 
> 2. The ability to type, to enter via a keyboard, read on a
> bus, etc is secondary and have localization issues so very
> much different than (1)
> 
> My point was that because of this, I think retypable is very
> very very much different than other, global, requirements, and
> the latter should not (I think) muddy the waters of the first
> requirement I list above.

Agreed.  However, I still have this strange idea (very clear in
the now-ancient RFC 1591) that domains are expected to serve a
community that if often not global, a concept that is consistent
with Larry's more recent comment about "target audience".
Within such an audience, the sort of criteria Larry is talking
about are almost certainly helpful... again, as long as they
don't interfere with the sort of global ones you identify above.
Larry's criteria also make a strong argument for a clear target
audience for a given zone or hierarchy.  That is a problem for
registries that want to be able to register labels from every
possible script even if they put some restrictions on those
labels, but we have known about that problem since the very
first ICANN IDN study effort if not earlier.

So I think there are global criteria (I like your list but
imagine that we could improve it somewhat with more thought)
that every label and FQDN should support and that there are
relatively more local criteria that have to be evaluated against
a target audience.   I don't know if "primary" or "secondary"
are quite the right category names, but I think we agree about
the principle.   It is also clear to me that, independent of
anything that is done globally, someone is more likely to get
themselves into trouble (or be tricked into it whether the
tricks are intentional or not) with labels in a script that they
find incomprehensible that with a more familiar one... and that
is something we better just get used to because, beyond some
point, technology won't help.

best,
   john

---

From: Nico Williams <[email-masked]>
Date: Fri, 7 Dec 2018 00:41:47 -0600

On Fri, Dec 07, 2018 at 07:16:03AM +0100, Patrik Fältström wrote:
> 1. I think the ability (for example) copy and paste, respond to, click
> on, save in bookmarks etc is the most important global requirement

+1.

> 2. The ability to type, to enter via a keyboard, read on a bus, etc is
> secondary and have localization issues so very much different than (1)

The ability to type in domainnames you see in ads is not that
interesting for the obvious reason that advertisers would not
choose to use domainnames that their targets cannot type in!

As to domainnames seen elsewhere than in ads... typing them in is still
not that interesting: there's always OCR (just point your smartphone at
it and click) and QR codes (same thing).

Besides, not concerning ourselves so much with the ability to type in a
domainname is a very good simplification to make.  In general most of us
cannot type in more than a very small number of scripts -- the ability
to type in any domainname is basically an oxymoron.  We couldn't even
require that users only ever see A-labels, which most users (perhaps!)
could type in, because that would utterly defeat the whole point of
IDNA.

> My point was that because of this, I think retypable is very very very
> much different than other, global, requirements, and the latter should
> not (I think) muddy the waters of the first requirement I list above.

+1.

Nico
--

---

From: Vint Cerf <[email-masked]>
Date: Fri, 7 Dec 2018 01:45:02 -0500

VGC: I am not so sure advertisers are that smart...

v

On Fri, Dec 7, 2018 at 1:42 AM Nico Williams <[email-masked]> wrote:

> IDNA-UPDATE mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idna-update
>

-- 
New postal address:
Google
1875 Explorer Street, 10th Floor
Reston, VA 20190

---

From: Nico Williams <[email-masked]>
Date: Fri, 7 Dec 2018 00:50:08 -0600

> VGC: I am not so sure advertisers are that smart...

I am quite certain that they are, or if not, will be.  They spend lots
of money, and they hate to spend it badly.  They even spend lots of
money on how to spend lots more money well.

Nico
--

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Dec 2018 02:26:55 -0500

--On Friday, December 7, 2018 00:50 -0600 Nico Williams
<[email-masked]> wrote:

>> > The ability to type in domainnames you see in ads is not
>> > that interesting for the obvious reason that advertisers
>> > would not choose to use domainnames that their targets
>> > cannot type in!
>> 
> I am quite certain that they are, or if not, will be.  They
> spend lots of money, and they hate to spend it badly.  They
> even spend lots of money on how to spend lots more money well.

Nico,

While I tend to agree with Vint -- not because of smartness or
the lack thereof but because many advertisers have, empirically,
found it much cheaper to register a lot of domain names than to
spend time and energy trying to protect trademarks, worrying
about competitors trying to capture or squat on names, etc.--
this is not a problem we need to solve.  There are many reasons
why we should encourage both registrants and registries to be
conservative, whether the advice we give about conservatism
involves only the global criteria that Patrik believes (and I
agree) should be primary or we expand that advice to include
some or all of the considerations Larry, Asmus, and I have been
advocating.  If the labels that are actually registered (whether
actively used or not) are a subset of what our conservative
guidance would suggest, that is great and no harm has been done.

On the other hand, we've seen names registered, and registered
as SLDs as well as further down in the tree, that clearly won't
meet even global criteria for conservatism and responsible
behavior. much less guidance focused on relevant target
communities.  Some of those registrations have clearly been the
result of a shortage of thinking or an excess of cleverness or
cuteness, not just the result of various forms of malice.  There
has been a lot of the latter too and we can at least hope that
more conservative registry policies would stop some of it.   So
I don't think an argument that we don't need to say something
(and say it very clearly) about the importance of conservative
behavior holds up ... and I hope you were not intending to make
that argument.

best,
    john

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 00:15:54 -0800

> I might not agree with this, as for example very few characters in arabic script are "retypable" for me. So this is at least a separate issue. But what the registry chooses, I feel is up to them, a…

And there you are already limiting yourself to a tiny corner of the problem.

The list of code points - in isolation - is the smallest issue.

For most scripts (by number), the question of context rules for the 
entire label (built from context rules for syllabic context of certain 
categories of code points) are what distinguishes gibberish (stuff that 
neither renderers nor human readers can process) from structurally sound 
labels representing words or the kind of non-word mnemonics applicable 
to that script.

For some scripts, a registry policy that does not establish variants, is 
either not conservative or does not match deep-seated user expectations. 
For zones that support multiple scripts, the experience of the Root Zone 
is showing, almost all scripts need robustly defined cross-script 
variants to make the zone minimally robust.

Defining context rules and variants is the meat of the problem of 
defining conservative, secures and usable registration policies.

All the talk about "repertoires" hides this fundamental truth - which is 
the more misleading as registries for some well-known European languages 
may survive without the other two pillars or LGR design - as long as 
they support a single language (or a few closely related ones) only. 
These languages being familiar to many in this discussion may reinforce 
the misunderstanding that repertoire selection is primary.

It is not - it just is the first step.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 00:17:08 -0800

> about the likely usability of the registered name among their target

They simply should prohibit unusable sequences.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 00:41:48 -0800

> say something when I next open the document.

To "typable", you might add "processable" and "renderable", not to 
mention "human-readable". (Where multiple languages/scripts are 
supported, "typable" should obviously mean"typable by a native user of 
that language/script").

Allowing certain specialized or historical forms that are not part of 
the modern orthographic repertoire would allow characters that are 
unlikely to be supported by keyboards (or even fonts). (Some such 
characters are effectively not recognized by modern readers, or simply 
mistaken for more familiar characters).

Allowing "do not use" sequences renders labels not processable by 
processes that do expect data to not contain effectively deprecated 
content.  (They are also structurally unsound, see next).

Allowing structurally unsound sequences for any of the fifteen or so 
complex scripts risks data not being renderable, as neither layout 
systems or fonts are (universally) expecting to deal with them (or do it 
differently in unpredictable ways). T

Allowing arbitrary sequences can lead to strings that human readers are 
not prepared to recognize because they violate deep-seated assumption of 
what is possible. Linearly progressing, non-connecting scripts tend to 
not have that issue; ASCII certainly doesn't, but already with Latin 
combining marks you enter territory where fully random sequences can't 
be supported.

Then you get into the issue of substitutable code points/labels that 
should not be simultaneously allowed (aka "blocked" variants) -- the 
other type of variants ("allocatable") is more of a usability issue, 
which for some scripts can be critical but is not strictly required 
under the heading of "Conservatism".

Allowing two labels that even reasonably observant readers will 
substitute for one another, because different abstract text elements in 
Unicode may have identical (or very near identical) appearance when 
presented in typical user interfaces, adds no value, only security 
risks. (Yes this is a subset of the wider issue of labels that are 
merely similar, but that's not an excuse to take well-known an 
unambiguous cases off the table at the policy level).

Spelling out these considerations would go beyond what is currently 
scattered through the IDNA RFCs, but would be a first step to putting 
the Registries on notice what their responsibility actually entails.

Now that we have the Root Zone LGR giving practical examples (and the 
emerging reference LGRs doing the same for the second level), there's 
less excuse for registries not doing the right thing - good approximate 
solutions exist.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 00:49:04 -0800

> 2. The ability to type, to enter via a keyboard, read on a bus, etc is secondary and have localization issues so very much different than (1)

A longer reply to this in my prev. note, but I emphatically disagree.

If you are dealing with a foreign language / script to you, then you 
will be reduced to selecting or perhaps copying; but for your own 
language and script (and for too many of the modern scripts) there are 
many possible sequences that so deeply violate the reader's expectation 
(or the systems' expectations) that they can effectively not be 
understood (read and remembered) by the users or displayed or processed 
by the system (or not reliably so across systems).

For many scripts, allowing too much will make labels every bit as 
unusable as allowing too little.

> My point was that because of this, I think retypable is very very very much different than other, global, requirements, and the latter should not (I think) muddy the waters of the first requirement…

Your item (1) addresses much more the level of implementation of the 
user presentation of IDNs, and less what should be allowed in IDN 
labels. Apples and oranges, it seems to me.

Item 1 is what the UASG should be concerned with, Item 2 is what IDNA 
should be concerned with (at the guidance level).

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 00:52:20 -0800

> money on how to spend lots more money well.

It may not be advertisers that use such 'broken' labels. All sorts of 
mischief can be created with stuff that your system won't let you type 
(even if you are a native user on a "native" system).

A./

---

From: Nico Williams <[email-masked]>
Date: Fri, 7 Dec 2018 10:42:07 -0600

> about competitors trying to capture or squat on names, etc.--

But what has that got to do with whether advertisers are smart enough to
use domainnames in advertisements that users can type in?

> this is not a problem we need to solve.  There are many reasons

Good.  Not having to solve it is a big win.

> conservative, whether the advice we give about conservatism

Indeed.  I've long been saying that registries should produce policies
appropriate to their domains.

> guidance would suggest, that is great and no harm has been done.

Yes.  Registrants might not agree, but they can always negotiate with
the registries or pick a registry that works for them.

> behavior holds up ... and I hope you were not intending to make

I only said that we shouldn't have to address the side-of-the-bus
problem.  You seem to have read into that more than I wrote, or perhaps
you're remembering things I've said before?

Registry policies that apply all the way down the tree would be an
innovation.  I'm not sure how I feel about that yet.  I do recognize the
problems with confusables, but as I've said before, that feels like a
losing battle.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Fri, 7 Dec 2018 10:45:47 -0600

On Fri, Dec 07, 2018 at 12:52:20AM -0800, Asmus Freytag (c) wrote:
> >I am quite certain that they are, or if not, will be.  They spend
> >lots of money, and they hate to spend it badly.  They even spend lots
> >of money on how to spend lots more money well.
> 

Anyone who wants their target audience to be able to type in a label
will pick labels wisely.

Anyone who does not care whether their target audience is able to type
in a label will be free to construct more interesting labels.

I see nothing wrong with that.

Of course, we weren't discussing malicious use of confusables.  That's
another story.

Nico
--

---

From: Shawn Steele <[email-masked]>
Date: Fri, 7 Dec 2018 18:19:50 +0000

> I might not agree with this, as for example very few characters in 
> arabic script are "retypable" for me. So this is at least a separate 
> issue. But what the registry chooses, I feel is up to them, as long as 
> they do come up with a good repertoire.

IMO that's not a requirement for non-language-speakers.  The intent of IDN is to enable language communities to use their own services, not necessarily that all resources be accessible to users of any other language.

Resources that are keen to provide content more broadly would, I think, best be served by using multiple names, with localized ones for native speaker usability and perhaps an ASCII one for the international audience.

I could even imagine something like a company with a Mongolian URL for Mongolian speakers, a Simplified Chinese URL for the rest of their Chinese customers, and an ASCII URL for their international partners.

-Shawn

---

From: Larry Masinter <[email-masked]>
Date: Fri, 7 Dec 2018 10:23:53 -0800

Look at RFC1736 , Functional Recommendations for Internet Resource Locators”, section 4.6: Locators are human transcribable

https://tools.ietf.org/html/rfc1736#section-4.6

   Users can copy Internet locators from one medium to another (such as

   voice to paper, or paper to keyboard) without loss or corruption of

   information.  This process is not required to be comfortable.

Changing Internet locators to allow for Internationalization need not necessarily eliminate this valuable property.

Larry

From: Asmus Freytag (c) <[email-masked]> 
Sent: Friday, December 7, 2018 12:49 AM
To: Patrik Fältström <[email-masked]>; John C Klensin <[email-masked]>
Cc: Larry Masinter <[email-masked]>; [email-masked]; [email-masked]; Paul Hoffman <[email-masked]>
Subject: Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and Unicode 11.0.0) to Informational RFC

 

If you are dealing with a foreign language / script to you, then you will be reduced to selecting or perhaps copying; but for your own language and script (and for too many of the modern scripts) there are many possible sequences that so deeply violate the reader's expectation (or the systems' expectations) that they can effectively not be understood (read and remembered) by the users or displayed or processed by the system (or not reliably so across systems).

For many scripts, allowing too much will make labels every bit as unusable as allowing too little.

Your item (1) addresses much more the level of implementation of the user presentation of IDNs, and less what should be allowed in IDN labels. Apples and oranges, it seems to me.

Item 1 is what the UASG should be concerned with, Item 2 is what IDNA should be concerned with (at the guidance level).

A./

---

From: Larry Masinter <[email-masked]>
Date: Fri, 7 Dec 2018 11:28:26 -0800

I meant “usability” in the software engineering sense in https://en.wikipedia.org/wiki/Usability

Usability isn’t a binary “usable” vs. “unusable”. Rather, usability can vary by the task at hand and the user performing the task.

At least, that’s why I suggested “likely usability” and “among their target audience”.

The reason for emphasizing transcription is not that there aren’t other operations that are possibly more frequent (copy/paste a URL from one context to another, remember on a bookmark list) but rather that transcription is the most stringent requirement – if a user can transcribe a name resulting in the same sequence of Unicode codepoints, then they can display the name, distinguish the name from other (transcribable) names. 

From: Asmus Freytag (c) <[email-masked]> 
Sent: Friday, December 7, 2018 12:17 AM
To: Larry Masinter <[email-masked]>; 'Patrik Fältström' <paf=[email-masked]>
Cc: [email-masked]; 'Paul Hoffman' <[email-masked]>
Subject: Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and Unicode 11.0.0) to Informational RFC

A./

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Dec 2018 15:02:44 -0500

--On Friday, December 7, 2018 11:28 -0800 Larry Masinter
<[email-masked]> wrote:

>...
> The reason for emphasizing transcription is not that there
> aren't other operations that are possibly more frequent
> (copy/paste a URL from one context to another, remember on a
> bookmark list) but rather that transcription is the most
> stringent requirement – if a user can transcribe a name
> resulting in the same sequence of Unicode codepoints, then
> they can display the name, distinguish the name from other
> (transcribable) names. 

Larry, I hope Asmus will respond further to this because he is
far more expert on relevant issues across a very wide range of
scripts and writing systems than I am, but I don't believe what
you write above is true.  Some scripts --of which the Roman
Script (Basic Latin of some millennia ago) is a good example--
simply have characters that are more easily distinguishable from
either other by people who cannot actually read the script or
associated language(s) that others.  Put a hypothetical person
who has never seen either before in front of a short string of
characters in that script and in front of a script written with
connected characters, complex use of ligatures, and subtle
distinctions among characters (look at Arabic digits for two and
three for an example of what I mean by "subtle" ... or look at
"O" and, in many contemporary type designs, "Q").  Then supply
that person with character pickers for the two scripts.  You
would almost certainly get accurate transcription of the Roman
characters and probably would not get it with the other script.
In neither case would be ability to transcribe imply the ability
to render and display (I'd encourage observation of five and six
year olds learning to write), nor would it imply that ability to
accurately copy and paste.

So, ability to transcribe may be a useful goal, but it isn't,
IMO, close to Patrik's global requirements list.  

best,
   john

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 7 Dec 2018 12:36:02 -0800

On 12/7/2018 12:02 PM, John C Klensin wrote:
> you write above is true.

Globalization is often misunderstood - it's neither the magic bullet that
makes everything equally accessible to everyone, nor is it exclusively
about presenting everything to every user in their native context.

Instead, it's an interesting mixture of both.

For IDNs the goal is to allow full native support of mnemonic identifiers
in the context of the language/script native to the user - for all such
combinations of languages and scripts so that, in principle all users
can be supported.

But it's also about enabling the accessing of resources outside your
native bubble - Unicode is a big step in that direction, because before it
you were often locked into a native character set baked into your system.

However, a label in a foreign script will not be "mnemonic" for you,
and there's nothing we can do about that in this context. However,
we should expect that labels in your script (and where relevant your
language) are indeed able to be mnemonic.

Otherwise, why not use digits like telephone numbers?

I think that is ultimately what Larry has in mind and what I cover by
"typable". You should be able to read a label, understand it, and
reproduce it by typing, or be able to tell someone about it.

For the Latin script (because of the features that you are attempting
to outline below) even arbitrary sequences of code points do not
generally compromise the ability for a label to be mnemonic and to
support dealing with it other than by clicking/pasting.
(The above is strictly true only for ASCII, once you add the full range
including combining marks a bit more care must be taken - as will
be evident when the draft for the Latin Root Zone LGR becomes
available, until then, I'll spare myself the digression into details)

For other scripts, there's a wide variety of issues.

One example from Arabic: Quite a few letters in Arabic share positional
forms, that is, they are only distinct if, for example, at the end of a 
word.

Also, some languages using the Arabic script support keyboards that
only have one of these letters, but not the "standard" Arabic-language
one. As a result, while users can "read" an Arabic-language name,
a geographical name, for example, they could not type that name.
If they did, it may even look right, but be different.

Therefore, a registry that attempts to support both languages in
the Arabic script, but does not support variants, will have a fraction
of its labels that are de-facto unusable for some users.

Understood in that sense, "typable for a native user" is a valid goal
that informs necessary features of label generation rules (registry
policies) that enable the use of labels as mnemonic, in the same
way we take for granted for non-IDN labels.

A./

>   Some scripts --of which the Roman

---

From: Larry Masinter <[email-masked]>
Date: Sat, 8 Dec 2018 00:00:33 -0800

We were discussing Patrik’s document and what advice or rules to give to Registrars about being conservative,

And some questions about the motivation or intelligence of Registrars and their Clients.

It would help if the Registrars could pass off responsibility to their Clients; the only reason not to do so would be that the clients shouldn’t be expected to know the complex rules.

The transcription problem is relatively easy to explain and understand, and covers most other ways in which a name could be “bad”. 

These names don’t fall from the sky. Someone chooses them. Give them a clear motivation.

From: i18nRP <[email-masked]> On Behalf Of Asmus Freytag (c)
Sent: Friday, December 7, 2018 12:36 PM
To: John C Klensin <[email-masked]>; Larry Masinter <[email-masked]>; 'Patrik Fältström' <paf=[email-masked]>
Cc: [email-masked]; 'Paul Hoffman' <[email-masked]>
Subject: Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and Unicode 11.0.0) to Informational RFC

 
--On Friday, December 7, 2018 11:28 -0800 Larry Masinter
 <mailto:[email-masked]> <[email-masked]> wrote:

...

One example from Arabic: Quite a few letters in Arabic share positional
forms, that is, they are only distinct if, for example, at the end of a word.

A./

 
best,
   john

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 8 Dec 2018 13:12:34 -0800

On 12/8/2018 12:00 AM, Larry Masinter wrote:
> It would help if the Registrars could pass off responsibility to their 
> Clients; the only reason not to do so would be that the clients 
> shouldn’t be expected to know the complex rules.
>
> The transcription problem is relatively easy to explain and 
> understand, and covers most other ways in which a name could be “bad”.
>

The rules are complex even if the basic requirements can be stated simply.

The process of teasing them out for the 28 scripts that make up all but 
a miniscule fraction of modern use is progressing nicely, but including 
preparatory phases will probably have taken most of a decade before 
completion. (I'm referring to the Root Zone LGR program).

The good news is that the result of this effort will be well documented 
(linking the final design to the underlying details of the scripts and 
language use) and that it will also be available in a machine-readable 
format.

That means that for the first time, there would be a well-researched 
(and well-documented) starting point for anyone trying to create and 
enforce registry policies for pretty much any writing system used in 
daily transactions.

Policies for the second level would have to extend this template to 
allow digits and hyphen - and perhaps relax some other restrictions that 
are seen as relevant only to the root. This is a much smaller task than 
starting from scratch, but for some scripts, even that task is not 
entirely trivial.

Common to the rules for most of the scripts is that they define variant 
labels and the restrictions applicable to them (never allocatable to 
unrelated entities and most commonly the first allocated blocks all 
subsequent registrations of variants). Carefully designed to make 
indistinguishable or semantically equivalent labels mutually exclusive 
or reserved for a single applicant, these policies require access to a 
registry to implement. Only when you know what is allocated already can 
you evaluate whether a new label is a blocked variant.

Being machine readable should make it easy to integrate these rules into 
registry operations. Therefore, I'm a bit confused as to what would be 
gained by somehow distributing this responsibility down the chain - 
other than surfacing restrictions built into the registry to applicants.

A./

---

From: Larry Masinter <[email-masked]>
Date: Sun, 9 Dec 2018 13:55:28 -0800

I think we’re looking at the problem from different ends of the telescope.

I’m concerned with giving guidance to the use of Unicode in other naming contexts, including URLs.

I think we should extend the advice to anyone assigning a “Unicode name” e.g., email addresses social media handles. 

In particular, the  IRI document RFC3987 needs update to align its recommendations with IDNA and the WHATWG URL “living standard” (which mainly neglects giving any advice at all the those choosing URLs.)    

Larry

--

 

From: i18nRP <[email-masked]> On Behalf Of Asmus Freytag (c)
Sent: Saturday, December 8, 2018 1:13 PM
To: Larry Masinter <[email-masked]>; 'John C Klensin' <[email-masked]>; 'Patrik Fältström' <paf=[email-masked]>
Cc: [email-masked]; 'Paul Hoffman' <[email-masked]>
Subject: Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> (IDNA2008 and Unicode 11.0.0) to Informational RFC

The process of teasing them out for the 28 scripts that make up all but a miniscule fraction of modern use is progressing nicely, but including preparatory phases will probably have taken most of a decade before completion. (I'm referring to the Root Zone LGR program).

The good news is that the result of this effort will be well documented (linking the final design to the underlying details of the scripts and language use) and that it will also be available in a machine-readable format.

That means that for the first time, there would be a well-researched (and well-documented) starting point for anyone trying to create and enforce registry policies for pretty much any writing system used in daily transactions.

Policies for the second level would have to extend this template to allow digits and hyphen - and perhaps relax some other restrictions that are seen as relevant only to the root. This is a much smaller task than starting from scratch, but for some scripts, even that task is not entirely trivial.

Common to the rules for most of the scripts is that they define variant labels and the restrictions applicable to them (never allocatable to unrelated entities and most commonly the first allocated blocks all subsequent registrations of variants). Carefully designed to make indistinguishable or semantically equivalent labels mutually exclusive or reserved for a single applicant, these policies require access to a registry to implement. Only when you know what is allocated already can you evaluate whether a new label is a blocked variant.

Being machine readable should make it easy to integrate these rules into registry operations. Therefore, I'm a bit confused as to what would be gained by somehow distributing this responsibility down the chain - other than surfacing restrictions built into the registry to applicants.

A./

---

From: John C Klensin <[email-masked]>
Date: Sun, 09 Dec 2018 18:00:49 -0500

Larry,

Not such where you are headed.  First of all, "Unicode name" has
been the source of a huge amount of confusion caused in part
because many of those who use that and similar terms seem to
think "Unicode" is an appropriate synonym for "non-ASCII".
Second, the rules for email addresses are very clear in the
SMTPUTF8 specs.  Probably wrong (and I hope to get to that as
soon as we get the issues with draft-faltstrom-unicode11 and the
Directorate questions nailed down), but very clear.  There is
also existing IETF guidance for other sorts of name-like things
in the PRECIS Identifier class and for another, possibly
overlapping, group of identifiers in UAX#31.

Asmus has identified the work that is going on to provide at
least a framework for narrowing choices beyond that list,
especially when one can identify a particular target audience.

As I'm certain you remember, the IETF had a WG that was, among
other things, supposed to update RFC 3987 but that essentially
could not reach consensus and gave up after identifying multiple
problems with that spec.   Perhaps we should have moved it to
Historic or placed it in a Not Recommended category as a result
of that -- certainly the language of RFC 2026 would strongly
support such an action when there are known technical defects
and we don't have a plan for moving forward, but there doesn't
seem to have been sufficient energy and consensus to take that
action either.

So, other than second-guessing work that is supposedly already
complete, what do you have in mind?  And when do you expect to
get I-Ds posted?

 best,
    john

--On Sunday, December 9, 2018 13:55 -0800 Larry Masinter
<[email-masked]> wrote:

> I think we're looking at the problem from different ends of
> the telescope.
> 
>  
> 
> I'm concerned with giving guidance to the use of Unicode in
> other naming contexts, including URLs.
> 
> I think we should extend the advice to anyone assigning a
> "Unicode name" e.g., email addresses social media handles. 
> 
>  
> 
> In particular, the  IRI document RFC3987 needs update to align
> its recommendations with IDNA and the WHATWG URL "living
> standard" (which mainly neglects giving any advice at all
> the those choosing URLs.)    
> 
>

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sun, 9 Dec 2018 15:38:56 -0800

> I think we’re looking at the problem from different ends of the telescope.
Not necessarily.

> naming contexts, including URLs.
I'm interested in that aspect as well, even though my current projects 
is focused on IDN labels for domains.

Many of the considerations I mentioned apply - they are based on the 
nature of the writing systems more than on particularities specific to 
domain names.

> I think we should extend the advice to anyone assigning a “Unicode 
> name” e.g., email addresses social media handles.
>

Agreed.

Here are some potential distinctions between different types of systems 
that handle Unicode names.

1) public vs. single entity

That is, who can create names. If any user can create names, then you'll 
want some protection against malicious registrations (or some other 
means of handling duplicate handles)

For singly entity controlled naming systems, malicious use and 
inadvertent duplication would not be a concern (as long as names are 
truly assigned by that entity and not via pass-through from other data - 
like document titles).

2) registry-based vs. open naming

If there's a list of names maintained in a registry (or database of some 
sort), it's possible to query it and prevent some duplicate names from 
being added -- that allows the addition of script or writing-system 
dependent rules to manage collisions.

Without such a database, inadvertent or malicious duplicates are hard to 
police.

3) exact or permissive lookup

Some name systems perform folding (like punctuation removal for e-mail 
names). Such lookup systems could directly support linguistic and visual 
variants by mapping all of them to a "preferred" form of the label.

Exact lookup systems must mimic that with a support for variant labels. 
Unlike a label folding scheme, the variant mechanism allows more 
fine-grained control over which form(s) of the label will succeed in 
lookup (and registration).

> In particular, the  IRI document RFC3987 needs update to align its 
> recommendations with IDNA and the WHATWG URL “living standard” (which 
> mainly neglects giving any advice at all the those choosing URLs.)
>

OK; looks like there's work to do.

A./

> *From:*i18nRP <[email-masked]> *On Behalf Of *Asmus Freytag (c)
> *Sent:* Saturday, December 8, 2018 1:13 PM
> *To:* Larry Masinter <[email-masked]>; 'John C Klensin' 
> <[email-masked]>; 'Patrik Fältström' <paf=[email-masked]>
> *Cc:* [email-masked]; 'Paul Hoffman' <[email-masked]>
> *Subject:* Re: [I18nrp] Last Call: <draft-faltstrom-unicode11-05.txt> 
> (IDNA2008 and Unicode 11.0.0) to Informational RFC
>
>     We were discussing Patrik’s document and what advice or rules to
>     give to Registrars about being conservative,
>
>     It would help if the Registrars could pass off responsibility to
>     their Clients; the only reason not to do so would be that the
>     clients shouldn’t be expected to know the complex rules.
>
> The process of teasing them out for the 28 scripts that make up all 
> but a miniscule fraction of modern use is progressing nicely, but 
> including preparatory phases will probably have taken most of a decade 
> before completion. (I'm referring to the Root Zone LGR program).
>
> The good news is that the result of this effort will be well 
> documented (linking the final design to the underlying details of the 
> scripts and language use) and that it will also be available in a 
> machine-readable format.
>
> allow digits and hyphen - and perhaps relax some other restrictions 
> that are seen as relevant only to the root. This is a much smaller 
> task than starting from scratch, but for some scripts, even that task 
> is not entirely trivial.
>
> Common to the rules for most of the scripts is that they define 
> variant labels and the restrictions applicable to them (never 
> allocatable to unrelated entities and most commonly the first 
> allocated blocks all subsequent registrations of variants). Carefully 
> designed to make indistinguishable or semantically equivalent labels 
> mutually exclusive or reserved for a single applicant, these policies 
> require access to a registry to implement. Only when you know what is 
> allocated already can you evaluate whether a new label is a blocked 
> variant.
>
> Being machine readable should make it easy to integrate these rules 
> into registry operations. Therefore, I'm a bit confused as to what 
> would be gained by somehow distributing this responsibility down the 
> chain - other than surfacing restrictions built into the registry to 
> applicants.
>
> A./
>

---

From: Larry Masinter <[email-masked]>
Date: Sun, 9 Dec 2018 15:45:08 -0800

The IRI working group, which I take responsibility for starting, never had
any disagreements, it just suffered that there was not enough activity.

> Not such where you are headed.  First of all, "Unicode name" has been the
> source of a huge amount of confusion caused in part because many of those
> who use that and similar terms seem to think "Unicode" is an appropriate
> synonym for "non-ASCII".

Sure, but you knew what I meant.

> SMTPUTF8 specs.  Probably wrong (and I hope to get to that as soon as we
> get the issues with draft-faltstrom-unicode11 and the Directorate
questions
> nailed down), but very clear.  There is also existing IETF guidance for
other
> sorts of name-like things in the PRECIS Identifier class and for another,
> possibly overlapping, group of identifiers in UAX#31.

All of these disjoint rules are likely to get out of sync. E.g. the  rules
for email addresses don't match the rules for mailto: URLs. 
For not very good reasons.

> Asmus has identified the work that is going on to provide at least a
> framework for narrowing choices beyond that list, especially when one can
> identify a particular target audience.

Yes, the work is fine, important, and valuable. But the applicability needs
to be extended. From the general perspective of choosing domain names to use
in email addresses and URLs, the criteria can be expressed operationally,
and simply as "don't choose names that can't be transcribed easily by your
target audience" and then point to Asmus work for a detailed analysis of
reasons why some sequences might have some difficulty.

But it is easy transcription that was the original requirement for URLs, and
based on several use cases. If you want IRIs to support those use cases,
you'll need a spec that defines the rules for the URLs what will survive
transcription.

> As I'm certain you remember, the IETF had a WG that was, among other
> things, supposed to update RFC 3987 but that essentially could not reach
> consensus and gave up after identifying multiple
> Historic or placed it in a Not Recommended category as a result of that --
> certainly the language of RFC 2026 would strongly support such an action
> when there are known technical defects and we don't have a plan for moving
> forward, but there doesn't seem to have been sufficient energy and
> consensus to take that action either.
> 
> So, other than second-guessing work that is supposedly already complete,
> what do you have in mind?  And when do you expect to get I-Ds posted?
> 
>  best,
>     john
> 
> 
> --On Sunday, December 9, 2018 13:55 -0800 Larry Masinter <[email-masked]>
> wrote:
> 
> > I think we're looking at the problem from different ends of the
> > telescope.
> >
> >
> >
> > I'm concerned with giving guidance to the use of Unicode in other
> > naming contexts, including URLs.
> >
> > I think we should extend the advice to anyone assigning a "Unicode
> > name" e.g., email addresses social media handles.
> >
> >
> >
> > recommendations with IDNA and the WHATWG URL "living standard"
> (which

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 10 Dec 2018 08:41:17 +0000

> Anyone who wants their target audience to be able to type in a label

Yes. My way to say this is that not all advertisers are that smart, but 
those that aren't will sooner or later disappear (*). Similar for 
non-advertisers: If they come up with a good domain name, or (part of) 
an URI, they'll be remembered and found easily, if they don't, they'll 
just be ignored.

I don't think the IETF should be doing the advertisers' work, and even 
if we would, I feel very strongly that none of this would belong into 
Patrik's draft, which is about the update of the repertoire.

> I see nothing wrong with that.

Me neither.

> Of course, we weren't discussing malicious use of confusables.  That's

Yes indeed.

Regards,   Martin.

(*) Of course, because advertising is a rather vague and fuzzy business, 
the elimination doesn't work very quickly :-(.

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 10 Dec 2018 09:09:57 +0000

> Changing Internet locators to allow for Internationalization need not necessarily eliminate this valuable property.

And it doesn't. People get confused because they think differently about 
scripts they are not familiar with (where they often assume the worst, 
and are explicit about it) than about scripts they are familiar with 
(where they make the right assumptions implicitly, often without even 
being able to say what these assumptions are).

I have just searched through all of RFC 3986, but I have nowhere found 
anything telling people not familiar with ASCII that they should be 
careful not to create domain labels such as "L0CAT0R". That's because 
writers of RFC 3986 assumed that it would be quite obvious that letters 
and (more-or-less) lookalike digits shouldn't be mixed needlessly. And 
even more, nowhere in RFC 3986 does it say to be careful not to confuse 
'b', 'd', 'p', and 'q', even if these characters look quite the same up 
to only a rotation/reflection. (I remembered being confused about these 
for a while in primary school.)

The same is true for other scripts. Users of Japanese know to not mix up 
Kanji, Hiragana, and Katakana needlessly (*). Users of any script know 
where the potential confusables are, and how to deal with them in daily 
life. There is no need to tell them about the details.

What is desirable are restrictions such as those that Asmus has 
described, on the registry level, to help avoid inadvertent or malicious 
collisions, and to buffer for cases where repertoire-only limitations 
are not enough.

But these are separate from the draft under discussion now.

Regards,   Martin.

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 10 Dec 2018 09:11:01 +0000

On 2018/12/10 08:45, Larry Masinter wrote:
> .

-- 
Prof. Dr.sc. Martin J. Dürst
Department of Intelligent Information Technology
College of Science and Engineering
Aoyama Gakuin University
Fuchinobe 5-1-10, Chuo-ku, Sagamihara
252-5258 Japan

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 10 Dec 2018 09:27:00 +0000

[Sorry, that mail went off before it was written. Here's the actually 
intended content.]

>> any disagreements, it just suffered that there was not enough activity.

For the record, I totally agree with Larry's statement that the reason 
for the WG to not advance was a lack of activity. However, "never had 
any disagreements" is probably an understatement.

>> For not very good reasons.

If you know about any specific example where this is actually the case, 
please tell us. As coauthors of the current mailto: RFC 
(https://tools.ietf.org/html/rfc6068), we should be able to fix this. 
The usual fix would be to say in the mailto: spec that the actual 
address has to confirm to the limits of the EAI specs.

Please note that I plan to reissue RFC 6068 when we can officially 
submit drafts including non-ASCII characters. The purpose will be two-fold.

First, to have the examples with the real characters, so that we can 
eliminate one level of escaping (we already have to deal with escaping 
rules in email addresses and in IRIs/URIs, and so it's very helpful if 
we can discard the additional escaping needed to represent non-ASCII in 
ASCII-only RFCs).

Second, we actually currently don't refer to EAI. Percent-encoding in 
the local part is currently only constrained as follows: "When a
<local-part> containing non-ASCII characters will be used to compose a 
message, the <local-part> MUST be transformed to conform to whatever 
encoding may be defined in a future specification for the 
internationalization of email addresses.". That future has already 
arrived (spec-wise) and is arriving implementation-wise, so I plan to 
change this oblique reference to an actual reference.

All of the above work is independent of the draft under discussion 
currently, and so Patrik's draft should proceed, with wording 
improvements where necessary.

Regards,   Martin.