Subject: [I18nrp] draft-faltstrom-unicode11-04.txt

From: "Patrik Fältström" <[email-masked]>
Date: Sun, 07 Oct 2018 07:53:51 +0200

All,

Version -04 of this draft was just posted.

Background: As we all know IDNA2008 is an algorithm that applied to Unicode gives a subset of code points which are permissible for in turn registries and others to create subsets from to be used in their respective policies.

Being the appointed expert for the IANA registry I have flagged for IAB changes made to Unicode that I feel must be discussed in IETF, and here is the document with my proposal (details are in the draft).

In short the draft proposes (just like RFC 6452) IETF continue to follow Unicode Standard without adding exceptions to the IDNA2008 algorithm, EVEN IF changes are made that are incompatible to earlier versions of Unicode.

The choices for IETF when things like this happens are:

1. Keep IDNA2008 with no exceptions

2. Keep IDNA2008 with exceptions

3. Stop referring (directly) to Unicode as it is not stable enough

Probably more choices than these...

My proposal is [1], together with a more forceful push to strict IDNA2008 adoption. No IDNA2003, no UTS#46, no homebrew mixes. Including that registries really do a careful conservative selection of code points to be used in whatever context it is to be used.

The problem though is that this is *my* view. Sure, a few others have randomly and "by pure luck" told me "this is ok".

But is this a stable informed decision by IETF?

This very fast turn into a process issue where IAB have asked for progress, and IETF is to deliver.

And this is where I take a step back, and want IESG and IAB to make up their mind. This group was created for I18N issues, we have a charter, but then what? Is this where drafts like these should be discussed?

Maybe...

   Patrik

---

From: Asmus Freytag <[email-masked]>
Date: Sun, 7 Oct 2018 01:33:19 -0700

On 10/6/2018 10:53 PM, Patrik Fältström wrote:
> The problem though is that this is*my*  view. Sure, a few others have randomly and "by pure luck" told me "this is ok".

I'm happy to tell you so and not randomly at all, and by replying to you 
here, it's not a reply received "by chance" either, but in what arguably 
is the proper channel for this feedback. (Option 1 in your numbering).

> But is this a stable informed decision by IETF?

I think it should be.

> This very fast turn into a process issue where IAB have asked for progress, and IETF is to deliver.

Agreed; and insofar as Unicode delivers a new version every year now, 
there's not a very big window for this draft; right now, it's up-to-date 
including Unicode 11.0.0 and that's fine as it goes (as its conclusion 
is that further process should revert to normal tracking of new Unicode 
versions - presumably unless some specific issue is found). However, 
publishing something that's a version behind the then latest Unicode 
version will be less satisfactory.

> And this is where I take a step back, and want IESG and IAB to make up their mind. This group was created for I18N issues, we have a charter, but then what? Is this where drafts like these should b…

I would say, yes. Because the nobody benefits from spreading IETFs 
already limited i18n resources any thinner.

It means that the group will perforce have a dual charter, both 
providing i18n input to all specifications as well as discussing and 
finding consensus on core i18n issues central to IETF. IDNA is one of 
those core issues.

A./

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 8 Oct 2018 15:28:32 -0600

> The problem though is that this is *my* view. Sure, a few others have randomly and "by pure luck" told me "this is ok".

That seems like the best path forward.

> But is this a stable informed decision by IETF?

Not yet.

> And this is where I take a step back, and want IESG and IAB to make up their mind. This group was created for I18N issues, we have a charter, but then what? Is this where drafts like these should b…

[repost from ART list]

The ADs wanted more detailed plans for (a) recruiting and training of
directorate members and (b) day-to-day operations. I asked the ADs for
clarification about what level of detail they wanted, but I could have
just proposed a plan to see if it was detailed enough for them. I'll
reprioritize to get this done more quickly.

Peter

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 10 Oct 2018 11:12:54 +0000

> -----Original Message-----
> From: i18nRP <[email-masked]> On Behalf Of Patrik Fältström
> Sent: Sunday, October 07, 2018 1:54 AM
> To: [email-masked]; [email-masked]
> Subject: [EXTERNAL] [I18nrp] draft-faltstrom-unicode11-04.txt
>
> All,
>
> Background: As we all know IDNA2008 is an algorithm that applied to
> Unicode gives a subset of code points which are permissible for in turn
> registries and others to create subsets from to be used in their
> respective policies.
>
> Being the appointed expert for the IANA registry I have flagged for IAB
> changes made to Unicode that I feel must be discussed in IETF, and here is
> the document with my proposal (details are in the draft).
>
> In short the draft proposes (just like RFC 6452) IETF continue to follow
> Unicode Standard without adding exceptions to the IDNA2008 algorithm, EVEN
> IF changes are made that are incompatible to earlier versions of Unicode.
>
> My proposal is [1], together with a more forceful push to strict IDNA2008
> adoption. No IDNA2003, no UTS#46, no homebrew mixes. Including that
> registries really do a careful conservative selection of code points to be
> used in whatever context it is to be used.

I tend to agree. It's more stable for registry operators.

Scott

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Oct 2018 16:17:37 -0500

On Sun, Oct 07, 2018 at 07:53:51AM +0200, Patrik Fältström wrote:
> Unicode gives a subset of code points which are permissible for in
> turn registries and others to create subsets from to be used in their
> respective policies.
> 
> Being the appointed expert for the IANA registry I have flagged for
> IAB changes made to Unicode that I feel must be discussed in IETF, and
> here is the document with my proposal (details are in the draft).
> 
> In short the draft proposes (just like RFC 6452) IETF continue to
> follow Unicode Standard without adding exceptions to the IDNA2008
> algorithm, EVEN IF changes are made that are incompatible to earlier
> versions of Unicode.

+1

We're talking specifically about future incompatible changes in
either case mappings or normalization.  Correct?

And to confirm the IDNA2008 / UTS#46 context:

 - IDNA2008 is hear-no-evil-see-no-evil, says to do no case mappings,
   relies on users entering domainnames in the correct case whatever
   that is

 - UTS#46 says to lower-case first, then apply the rest of IDNA2008

Correct?

> 3. Stop referring (directly) to Unicode as it is not stable enough

(3) has got to be out as it amounts to hard-forking Unicode.  A fork of
Unicode would be momentous and extraordinary, and not to be undertaken
lightly!  It might be useful to think about how we'd go about forking
Unicode process-wise, but I think any analysis will convince us that we
should not actually do it.

(2) is a light fork of Unicode.

I understand that there have been a few backwards-incompatible changes
in Unicode, but that's just life, and it is their problem (sure, our
users' too, but forking Unicode is not going to be satisfying at the
very very least).

We could do a flavor of (2) that I describe below, but I'd rather not.

For me that leaves (1), which is what I'm for.

We should, however, use our liason to communicate to the UC our
displeasure as to backwards-incompatible changes, and that we'd like the
UC to at least consider adding a stability attribute to every codepoint.
If there were a stability attribute, then we could exclude codepoints
whose stability the UC will not guarantee.

If there were codepoint stability attributes, then filtering out
unstable codepoints would be (2') and likely desirable.

Future codepoint assignments could all begin life as unstable and gain
stability with experience.

We could, perhaps, implement codepoint stability attributes at the IETF
alone if the UC balks.  I would be OK with that provided it was simple
and simple to automate -- I don't think we want to be in the business of
inspecting every new codepoint assignment and guessing a stability level
for it, so nothing much more complicated than time-since-assignment
should be used.

> My proposal is [1], together with a more forceful push to strict
> IDNA2008 adoption. No IDNA2003, no UTS#46, no homebrew mixes.

I would much prefer that we adopt UTS#46.

> Including that registries really do a careful conservative selection
> of code points to be used in whatever context it is to be used.

Yes.

> The problem though is that this is *my* view. Sure, a few others have
> randomly and "by pure luck" told me "this is ok".
> 

It should be after the standard IETF sturm und drang :)

> And this is where I take a step back, and want IESG and IAB to make up
> their mind. This group was created for I18N issues, we have a charter,
> but then what? Is this where drafts like these should be discussed?

I believe we need IETF consensus on these matters, but the IAB must be
involved.  Any notion of forking Unicode should require buy-in from the
IAB.

Nico
--

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 12 Oct 2018 00:48:08 +0200

> On 11 Oct 2018, at 23:17, Nico Williams <[email-masked]> wrote:
> 

No, we talk about current incompatible changes to the Unicode Standard between version 6.4 and 11.0. Changes that moves code points from DISALLOWED to PVALID.

>   relies on users entering domainnames in the correct case whatever

No, not at all. IDNA2008 do say that normalization, case mappings and such things are to happen.

> - UTS#46 says to lower-case first, then apply the rest of IDNA2008

UTS#46 is something different, and is not what IETF is dealing with.

IDNA2008 is IDNA2008.

> For me that leaves (1), which is what I'm for.

Thanks for the information.

> whose stability the UC will not guarantee.

Noted.

Yes.

> I would much prefer that we adopt UTS#46.

There are so many incompatibilities between IDNA2008 and UTS#46 that it is not possible.

What implementors can do is to implement a mapping scheme and normalization that might be close to what UTS#46 is doing, but what has been discovered in the ICANN context is that what mapping you use is in fact dependent on context so much that it is impossible in reality to specify just one such algorithm. Instead, the important issue is that whenever that initial transformation is done (which the application is responsible for), then strict IDNA2008 is a necessity. Or else the uncertainty is too large which in turn makes it impossible for the application to make a good context dependent mapping.

   Patrik

---

From: Asmus Freytag <[email-masked]>
Date: Thu, 11 Oct 2018 22:57:48 -0700

> stability with experience.

 From Unicode's perspective, the code point assignments are not unstable 
(what is encoded and where) is not changed. What changes occasionally is 
the understanding of character properties that relate to how a code 
point is used (including in certain software).

While identifiers are a critical use of code points, they are by far not 
the most important one. Therefore, usage information (character 
properties) cannot be arbitrarily frozen.

In some cases, property data does have stability, because reliable 
information is available at design time (case, for example). However, 
whether something is better seen as a combining mark or a punctuation is 
subject to change - unfortunately that is a property picked up by IDNA2008.

Especially where this happens with characters for obscure scripts or 
languages, there's not a lot of benefit of trying to address that with 
an exception.

The real issue is that obscure scripts and languages (i.e. dead ones) 
have no business being supported in a public zone.

That's a different kettle of fish.

A./