Subject: [I18ndir] Finding a way to conclude the review of draft-faltstrom

From: Harald Alvestrand <[email-masked]>
Date: Mon, 18 Feb 2019 22:34:08 +0100

I've been spending some large chunk of today trying to catch up on these
mailing lists (i18ndir and i18nrp). I am not yet caught up.

The conclusions I have so far include:

- For the most part, the action - on - characters in Patrik's draft is
uncontroversial (as soon as he fixes the erroneous assigment of
SPHERICAL ANGLE OPENING UP as PVALID - that's not a character I want to
see in a domain name label!)

- There's a great deal of material on what we can and cannot assume
about the behavior of other actors going forward (much experience has
now been had about the behavior of the actors over the last 10 years).
This material doesn't fit within a document on a normative track (BCP,
standard).

- There's a huge gulf between "the protocol machinery should prevent
this domain from ever being used" and "this domain is a reasonable thing
to register and use", and a need to have at least some mapping of that
gulf for the community's education. However, getting agreement on any
single statement in that map seems harder than agreeing that the
technical content of Patrik's draft is largely uncontroversial.

- I wonder how I ever volunteered for this :-)

It's not clear to me how this all feeds into a single review document.
Expecting it all to fit may raise false expectations.

-- 
Surveillance is pervasive. Go Dark.

---

From: John C Klensin <[email-masked]>
Date: Mon, 18 Feb 2019 23:34:46 -0500

Harald,

Thanks for taking this on: I don't envy you the job but can't
imagine a better person to do it.

However, I think your conclusions completely miss/omit the main
point I've been trying to make starting on this list with my
note of 6 February and, at least IMO, reinforced by much of the
discussion since.   That position is that there have been a
rather long series of issues raised, in multiple documents,
about IDNA2008 and the appropriateness of uncritically applying
its procedures to new versions of Unicode.  If those issues are
not addressed in some reasonable way but draft-faltstrom is
published instead, it is likely (given some other discussions, I
might say "nearly certain") to be taken as an IETF consensus
statement that things are just fine and none of the issued
identified in those other documents are relevant or in need of
being reflected in IDNA2008.  That is bad for two reasons:

	* It is likely to provide an extra impediment should we
	decide to take those issues up (and extra inertia if we
	can't find the energy to do so).
	
	* Should we take them up and conclude that something
	needs to be done, it would mean we'd have to face the
	problems with incompatible changes.

To me, that leads to the conclusion that the character list is
not uncontroversial at all (but, instead, are or should be very
controversial) because there are a number of characters that are
shown, uncritically, as PVALID that would not be if we looked
carefully at those other issues, the discussion of the last
several days, or both.  Of that list of issues, the one that
Asmus, Patrik, and I have been discussing for the last few days
-- reflected in your last two bullets but ultimately about what
the IETF can draw definitive boundaries around, what we can
expect of registries, and what specific advice should be given
to the latter and who should give it -- is the least complex of
those issues because, as I indicated in a note earlier today, we
could publish a revised version of
draft-klensin-idna-rfc5891bis, refer to it with a bit of text in
draft-faltstrom, and basically say "someone else's problem".
The other issues, most of which actually affect code point
classifications, are harder.

My conclusion from the above takes me back to where I started on
the 6th (and earlier, but not on this list), which is that it is
inappropriate to publish draft-faltstrom (or even complete the
IETF Last Call) without taking enough of a look at those other
issues to be sure that publication of draft-faltstrom doesn't
constrain possible solutions and that any text needed to prevent
long-term constraints or complications is inserted in it.

        john

--On Monday, February 18, 2019 22:34 +0100 Harald Alvestrand
<[email-masked]> wrote:

> I've been spending some large chunk of today trying to catch
> up on these mailing lists (i18ndir and i18nrp). I am not yet
> caught up.
> 
> - For the most part, the action - on - characters in Patrik's
> draft is uncontroversial (as soon as he fixes the erroneous
> assigment of SPHERICAL ANGLE OPENING UP as PVALID - that's not
> a character I want to see in a domain name label!)
> 
> - There's a great deal of material on what we can and cannot
> assume about the behavior of other actors going forward (much
> experience has now been had about the behavior of the actors
> over the last 10 years). This material doesn't fit within a
> document on a normative track (BCP, standard).
> 
> - There's a huge gulf between "the protocol machinery should
> prevent this domain from ever being used" and "this domain is
> a reasonable thing to register and use", and a need to have at
> least some mapping of that gulf for the community's education.
> However, getting agreement on any single statement in that map
> seems harder than agreeing that the technical content of
> Patrik's draft is largely uncontroversial.
> 
> It's not clear to me how this all feeds into a single review
> document. Expecting it all to fit may raise false expectations.
> 
> --

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 18 Feb 2019 23:47:21 -0800

> Expecting it all to fit may raise false expectations.
Looks like a fine summary.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Tue, 19 Feb 2019 09:38:37 -0800

On 2/18/2019 8:34 PM, John C Klensin wrote:
> its procedures to new versions of Unicode.

And some of the many positions were concerned with the fact that the 
other ones were focused on the wrong problem.

Given the design of the protocol, moving forward is the appropriate option.

Harald nailed it with:

- There's a huge gulf between "the protocol machinery should prevent
this domain from ever being used" and "this domain is a reasonable thing
to register and use", ...

It is the former that we are concerned with as far as Patrik's draft is 
concerned and nothing since Unicode 6.3.0 is so dramatically at odds 
with what has always been part of the protocol that there's grounds to 
call for declaring a national emergency over it.

The protocol, by itself, does not deliver the latter guarantee (as much 
as some people might want it to do), and there is a substantial gulf. 
I'm in the camp that firmly believes that narrowing this gulf by trying 
to do after-the-fact tweaking of the protocol should be off the table - 
and firmly so.

And that means, the issues that comprise that "gulf" do not get 
addressed in substance in the protocol or Patrik's update.

> If those issues are

We should (among the assembled experts) have, by now, a loose agreement 
that the guarantee of "this domain is a reasonable thing to register and 
use" is something that gets to be addressed via the registries' 
responsibility for due diligence and cautious action.

The lack of activity in that arena, and we should agree on that one as 
well, may be partially based on business decisions, but is also based on 
total lack of understanding of what makes up that gulf. The former we 
may never be able to address short of having a cudgel that would force 
bad actors to stop bad behavior. The latter, we can most assuredly 
address, and we do not need to hold Patrik's paper for that - because we 
don't anticipate changes to the protocol.

Technically, John's draft is an update to the RFCs describing the 
protocol, but it's not a technical change; rather setting a needed 
emphasis and reiterating the foundation for us to address this "gulf". 
Our energies should be focused on getting that moving forward, not on 
holding up Patrik's draft.

> Asmus, Patrik, and I have been discussing for the last few days

While the issues are real, they all fall into the realm of making 
guarantees that the _protocol as such_ was either never intended to 
make, or has, for so long, not made, that retrofitting them is 
tantamount to doing a new protocol (IDNA2020), which, after the last 
transition, is something I wouldn't wish on anybody. (UTS # 146 anyone?).

> draft-faltstrom, and basically say "someone else's problem".

That's what we should do, but we should use the clarity of Harald's 
language, i.e. explaining the "gulf" in something like his terms.

Whether we do that explanation in Patrik's draft or in rfc5891bis is 
largely a detail. I'm partial to a simple forward pointer and then, 
having discharged ourselves of the 11.0 or now 12.0 update, focus on 
getting the description of that "gulf" and the registries' more specific 
responsibilities done by finishing draft-klensin...

> classifications, are harder.

I would put those details in an information RFC that is carefully 
constructed to limit the needs for updates. I have some ideas that I am 
hoping I will find time to write down so that "engineers" like Patrik 
can reassure themselves that this is something more like a specification 
and not just pure handwaving.

However, as long as I have to continue writing this much e-mail in the 
form of arguments, no draft will be forthcoming. (To me this feels like 
"treading water" so let's find a way to conclude this!)

> long-term constraints or complications is inserted in it.

And I come to the opposite conclusion, i.e. that there's no conceivable 
change to Patrik's document or to the existing IDNA 5892 parameters that 
are worth attempting, because the protocol's stability (including the 
stability of its warts) is more important.

And that we can conclude already now, that we will not touch it, but 
look for finding other ways to make the "guarantee" cited above.

Therefore, there is no junctim and we can get this draft out of the 
perma-waiting loop.

A./

---

From: John C Klensin <[email-masked]>
Date: Tue, 19 Feb 2019 16:46:36 -0500

--On Tuesday, February 19, 2019 09:38 -0800 Asmus Freytag
<[email-masked]> wrote:

>...
>> That position is that there
>> have been a rather long series of issues raised, in multiple
>> documents, about IDNA2008 and the appropriateness of
>> uncritically applying its procedures to new versions of
>> Unicode.
> 
> And some of the many positions were concerned with the fact
> that the other ones were focused on the wrong problem.

Perhaps.   Or perhaps that is part of the key problem.

> Given the design of the protocol, moving forward is the

Ok.  While some of the differences in positions and perspective
are, IMO, important and should be explored further, many were,
as I suggested was possible early on, were just differences in
vocabulary and focus.   I think the difference of opinion
between the two of us comes down to two issues.  As an example
of the vocabulary issue, I have heard no arguments against
"moving forward".  The question is whether the best way to move
forward is to publish draft-faltstrom now, more or less as it
appears in -07, and then deal with the other stuff [a] or
whether there is other stuff that should be addressed and
possibly resolved before draft-faltstrom is published (with
draft-faltstrom reflecting that work if appropriate.

I think that choice comes down to two questions, the first tied
closely to "the design of the protocol"

Background: Many details aside, the basic design target for what
IDNA2008 was intended to allow was, quite consciously, the same
as the target for pre-DNS host names, the syntax allowed in the
host-part of domain-part by SMTP, and the "preferred syntax" of
RFC 1034, i.e., letters [b], digits, and as little punctuation
as feasible.  From that very high conceptual level, the only
difference between the target criteria of IDNA2008 and those
predecessors is expansion from the ASCII repertoire to the range
of abstract characters supported by Unicode.  The question of
how that conceptual definition is turned into rules, and
ultimately into lists of code points, is an implementation
detail.  All of the more specific issues, including whether
labels are conceivably words in some languages (or obey other
orthographic conventions), whether mixed-script label are
allowed, are left to the registry and registries are expected to
behave responsibly (and have been since at least RFC 1591 --
that wasn't an IDN innovation either).

Now if, for example, some combining marks were never intended
for use as letters (or, if one prefers, parts of words) and were
assigned code points for other purposes or notations, then that
letter-digit model dictates that they should not be permitted in
labels.  If a particular version of the specification allows
them, it is what is called a "mistake", arguably an IDNA
protocol mistake due to Unicode's not making a distinction
between non-spacing marks used with letters (even sometimes,
which might have been a case for more CONTEXTO classifications)
and non-spacing marks used only in non-letter contexts.  Even
the non-decomposing character issue is arguably a mistake
because inferences were made about what normalization would
accomplish that turned out to be incorrect (and, incidentally,
the slash characters you mentioned were found in the subsequent
investigation and are called out in
draft-klensin-idna-5892upd-unicode70).

(1) The first question is tied critically to "the design of the
protocol" and what to do about mistakes.  The tradition in a
number of communities is to view them as irreversible and just
try to warn about them or work around them.   Unicode's view of
stability of code point assignments and a number of properties
is consistent with that model (maybe even an extreme version of
it) but they are by no means the only one.   The Internet
(including IETF) tradition has been to fix mistakes but to be
very pragmatic about it (see the second question).  I observe
that no one is running IPv1 these days.  For the specific case
of IDNA2008, the observation that the review and update
procedure specifically allows additions to the exception list
and context rules strongly suggests that the design of the
protocol contemplated fixing mistakes, not just digging in and
moving forward because nothing about the rules could be changed.
In any event, the question of whether and how to fix mistakes
gives us three general options:

(1.1) For anything that appears to be a mistake, create
exceptions or new rules for the "new" code points affected _and_
for all (or at least some) code points that were assigned
earlier that are shown to share the same problems.

(1.2) For anything that appears to be a mistake, create
exceptions or new rules for the "new" code points affected, but
let the old ones go in the interest of stability, not
invalidating existing labels, etc.  IMO, if we are smart and
don't want to invite confusion, we explain the situation --and
why categories for older code points are not-- changed
somewhere, but that is a separate judgment call.

(1.3) We take the position that old decisions are immutable and
that the best thing to do with new code points that may
duplicate older mistakes (or omissions from the rules, or
whatever more kind terminology we choose) is to be
mistake-consistent.  Perhaps we say, somewhere, "well this is a
problem but it is better to make it a problem for the registries
and count on their good behavior rather than making a change to
the protocol".

Of those three choices, the first two seem to me to require
delaying processing and publication of draft-faltstrom until we
can devise the right fixes and language to describe them ... or
at least until we can devise a strategy and language that avoids
having draft-faltstrom say "this code point is PVALID" only to
have some subsequent, mistake-fixing, document come along in (I
would hope) a short time and say "no it isn't".  For the third,
I'd still avoid waiting to process and publish draft-faltstrom
until we can explain what the problem was and why we decided to
ignore it, push it off, or otherwise refrain from changing RFC
5892.  However, there are other ways to handle that and they
depend considerably on the second question.

(2) How critical is it that we get draft-faltstrom out in a
hurry?  Who needs it, for what, and why should the IETF make
whatever compromises are necessary to prioritize getting it out?
I've been asking those questions on and off for some time and
still don't think I've seen an answer.   However, assuming it
really is critical (in ways that the IETF recognizes), then I
think we should look for ways to get on with publication while
compromising our ability to make the decisions under (1) as
little as possible.  That does not imply rushing the document to
publication in a way that effectively preempts that discussion,
leaving us with either 1.3 or trying to deceive the community
about the timing associated with 1.2 (although those are
options) because it would be awkward, but not difficult, to
insert text into draft-faltstrom explaining the loose ends,
indicating that the IETF was going to take the other issues up
as soon as possible, and that people should treat all (or at
least some if we can describe them without normative references
[c]) of the newly-added code points with special care until
those issues are sorted out.   I assume Patrik could write such
a paragraph quickly; if not, I'll volunteer.   But, again, if
getting the document out is not critical enough to allow careful
consideration of what might actually be a mistake that should be
considered as input to a protocol update, I think prudence and
the design of the protocol require that we consider the above
issues (and, ideally, the others in the queue) carefully before
moving forward with publication.

best,
   john

[a] I am deliberately using obviously vague terminology because
I do not believe there is demonstrated consensus about what the
list of other stuff is (although there may be consensus about
some of the items on that list), nor demonstrated consensus
about what work is the IETF's responsibility and what should be
done elsewhere if at all, nor demonstrated consensus about what
conclusions the IETF should accept uncritically from other
bodies (and which bodies).  But, as long as the list of "other
stuff" is non-null, the immediate question about publication of
draft-faltstrom does not requiring getting agreement on any of
that.

[b] I hope we have general agreement on what that term means and
that it is closely bound to abstract characters used to write
the words (or other conceptual units) of human languages.  For
example, mathematical symbols are not "letters".

[c] That comment hides an interesting procedural issue.  Assume
some of the possible changes mentioned above result in text for
draft-faltstrom that includes normative forward references to
documents we aren't ready to process and approve yet.  That
would put draft-faltstrom into the RFC Editor queue with a
normative reference hold so that it would not actually be
published until the other document(s) were ready.  Under normal
circumstances, that would be more than adequate -- while it
would be unusual, draft-faltstrom could be pulled out of the
queue and revised if that later work actually required that.
However, it isn't clear, absent other instructions
--instructions that we have not discussed-- whether IANA would
act to update its tables at the time of IESG approval of the
document or whether they would wait for publication.  If the
former, we would have exactly the same issues with defining a
code point as PVALID and then changing that which are part of
the reason to delay processing and approval.