Subject: [I18ndir] I18ndir last call review of draft-ietf-regext-dnrd-objects-mapping-06

From: Marc Blanchet via Datatracker <[email-masked]>
Date: Thu, 05 Mar 2020 11:06:41 -0800

Reviewer: Marc Blanchet
Review result: Ready with Issues

I was assigned by the Internationalization Directorate to do a review of this
document with a specific eye on internationalization and also a specific
request from AD to look at section 10.

I would like to point out that in some cases, the spec seem to provide a choice
for the implementor/deposit provider to use something else than UTF-8 for the
non-ascii encoding. For example, section 4.6.2.1. provides a choice of encoding
for csv files: "encoding  Defines the encoding of the CSV file with the default
encoding of "UTF-8". Moreover, section 10 talks about UTF-8 and UTF-16 and
recommends UTF-8 instead of making it mandatory. At the same time, there are
multiple fields in this spec that are defined as UTF-8. Therefore, it would be
appropriate and much less prone to interoperability problems to make UTF-8 the
only encoding possible, specially given that most protocols, data payloads and
software librairies are using UTF-8 encoding. If the authors agree, then
section 10 and 4.6.2.1 could be revised, and probably adding a paragraph in
section 1 or 4 that states the only possible encoding is UTF-8 for both CSV and
XML files.

Section 9.14 schema has a comment on ACE name field. Wonder if A-label would be
more appropriate.

Section 5.6.2.1.1. While in other parts of the spec, the encoding was clearly
identified as UTF-8, the definition of "<rdeCsv:fUName>  Name of the NNDN in
Unicode character set for the <csvNNDN:fAName> field element." does not state
any. Might want to say it clearly as UTF-8 like others.

---

From: Barry Leiba <[email-masked]>
Date: Thu, 5 Mar 2020 15:16:12 -0500

Thanks for the quick review, Marc.

Barry

---

From: John C Klensin <[email-masked]>
Date: Thu, 05 Mar 2020 15:47:32 -0500

(All list-type other than i18ndir removed)

General observations about substance:

(1) I completely agree with Marc about the UTF-8 issue.  Unless
there is a really good reason for allowing either other
normalization forms of Unicode or any other CCS, new protocols
and IETF-specified formats should be restricted to UTF-8.  I
would go a half-step further and suggest that any deviations
from that principle should be explicitly discussed and justified
in an Internationalization Considerations section, i.e., that
section should not be just a description of what XML allows.

(2) I have not studied this spec (or RFC 5732 carefully enough
to be sure, but it appears to me that, with four normalization
forms specified in The Unicode Standard and periodic disputes
about preferences between NF[K]C and NF[K]D, "normalizedString"
may be under-specified.  Clearly actual IDN labels should
(SHOULD?) be in NFC form, but most other fields are less
obvious.  In addition, given increasing trend on the web 9at
least) to do exactly what TUS says to do, which is to normalize
only at comparison time rather than trying to carry strings
around in normalized form, the application of that attribute all
almost all text-type values may be inappropriate. I can find no
evidence in the I-D that those issues were considered; the
document should not progress until they are.

Directorate procedural issue:

I want to thank Marc for copying the i18ndir list on this review.

Given our low, and perhaps waning, levels of activity and the
"no one is expert on all i18n issues" principle described in
draft-klensin-unicode-review, I think it would be helpful if all
requests for reviews of i18n-related documents and review
assignments were posted to the i18ndir list.  That would reduce
the chances of duplicated effort and increase the odds of enough
cross-checking (however slightly) to be sure that directorate
opinions were at least consistent with each other.    Without
that sort of communication, we are not a directorate; we would
be, at best, a loosely-structured review team.

With the understanding that it might not accomplish much of
anything if we continue to be as inactive as we have been in
recent months, is there any reason to not expose review requests
and assignments to this list?

thanks,
   john

--On Thursday, March 5, 2020 11:06 -0800 Marc Blanchet via
Datatracker <[email-masked]> wrote:

> I was assigned by the Internationalization Directorate to do a
> review of this document with a specific eye on
> internationalization and also a specific request from AD to
> look at section 10.
> 
> I would like to point out that in some cases, the spec seem to
> provide a choice for the implementor/deposit provider to use
> something else than UTF-8 for the non-ascii encoding. For
> example, section 4.6.2.1. provides a choice of encoding for
> csv files: "encoding  Defines the encoding of the CSV file
> with the default encoding of "UTF-8". Moreover, section 10
> talks about UTF-8 and UTF-16 and recommends UTF-8 instead of
> making it mandatory. At the same time, there are multiple
> fields in this spec that are defined as UTF-8. Therefore, it
> would be appropriate and much less prone to interoperability
> problems to make UTF-8 the only encoding possible, specially
> given that most protocols, data payloads and software
> librairies are using UTF-8 encoding. If the authors agree, then
> section 10 and 4.6.2.1 could be revised, and probably adding a
> paragraph in section 1 or 4 that states the only possible
> encoding is UTF-8 for both CSV and XML files.
> 
> Section 9.14 schema has a comment on ACE name field. Wonder if
> A-label would be more appropriate.
> 
> Section 5.6.2.1.1. While in other parts of the spec, the
> encoding was clearly identified as UTF-8, the definition of
> "<rdeCsv:fUName>  Name of the NNDN in Unicode character set
> for the <csvNNDN:fAName> field element." does not state any.
> Might want to say it clearly as UTF-8 like others.

---

From: "Marc Blanchet" <[email-masked]>
Date: Thu, 05 Mar 2020 15:54:27 -0500

> I want to thank Marc for copying the i18ndir list on this review.

I have not. see below.

> be, at best, a loosely-structured review team.

I was assigned by Pete by way of a specific i18n review request in the 
datatracker. When the reviewer submits the review, the text is 
automatically broadcasted to the wg mailing list, drafts authors and 
i18ndir mailing lists.

Marc.

---

From: John C Klensin <[email-masked]>
Date: Thu, 05 Mar 2020 17:46:23 -0500

>> draft-klensin-unicode-review, I think it would be helpful if
>> all requests for reviews of i18n-related documents and review
>> assignments were posted to the i18ndir list.  That would
>> reduce the chances of duplicated effort and increase the odds
>> of enough cross-checking (however slightly) to be sure that
>> directorate opinions were at least consistent with each
>> other.    Without that sort of communication, we are not a
>> directorate; we would be, at best, a loosely-structured
>> review team.
> 
> I was assigned by Pete by way of a specific i18n review
> request in the datatracker. When the reviewer submits the
> review, the text is automatically broadcasted to the wg
> mailing list, drafts authors and i18ndir mailing lists.

Aha.  Well, thanks for taking it on anyway,  And, Pete, please
see above.

   john

---

From: Asmus Freytag <[email-masked]>
Date: Thu, 5 Mar 2020 16:14:43 -0800

> section should not be just a description of what XML allows.

+1

(In fact, the i18n considerations in the draft were defective in not
discussing the CSV mode at all.)

About the your suggested "half-step further": do we need to write
a short RFC to explicitly put that on the record? Or how would you
think such a mandate would be expressed?

> obvious.

I would say: IDN labels MUST be in NFC.

There's no reason for formal records of IDN U-labels not to be
in the form specified by IDNA.

> In addition, given increasing trend on the web 9at

Right, there are many other contexts where it makes sense to
keep the data as submitted and not to force normalization.

However, this does not appear to be one of them.

We may need a statement specific to IETF that captures the
suggested policy.

> assignments were posted to the i18ndir list.

Seems fine. Appointing a lead reviewer shouldn't make the
that information private.

> That would reduce

We would simply be a review pool (like a typing pool of old).

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 00:46:48 -0500

Asmus,

--On Thursday, March 5, 2020 16:14 -0800 Asmus Freytag
<[email-masked]> wrote:

>> (1) I completely agree with Marc about the UTF-8 issue.
>> Unless there is a really good reason for allowing either other
>> from that principle should be explicitly discussed and
>> justified in an Internationalization Considerations section,
>> i.e., that section should not be just a description of what
>> XML allows.
> 
> +1
 
> discussing the CSV mode at all.)

An issue I did not catch.

> think such a mandate would be expressed?

I'll turn that one back over to Barry.  We could certainly
produce an Internet Draft that said:

	(i) IETF specifications that use Unicode SHOULD use
	UTF-8, be explicit about that, and not suggest the use
	of other encoding forms.
	
	(ii) If there are reasons why a particular specification
	must deviate from that requirement, it MUST be very
	specific about which forms are allowed and MUST explain,
	in an Internationalization Considerations section,
	exactly why  allowing alternate encoding forms is
	necessary.

But, given the fate of i18n documents in the last year or so, I
wouldn't be optimistic about getting it processed, so maybe
there is a more efficient way to proceed.

If we do need such an I-D, I'm willing to volunteer to do a
first draft, but I think starting on it is pointless without a
clear plan about how it will be progressed and progressed
without, e.g., a lengthy discussion about why we don't want to
allow, for example, UTF-7 or UTF-5.

>> about preferences between NF[K]C and NF[K]D,
>> "normalizedString" may be under-specified.  Clearly actual
>> IDN labels should (SHOULD?) be in NFC form, but most other
>> fields are less obvious.
> 

Absolutely.   My concern was that
draft-ietf-regext-dnrd-objects-mapping-06 specifies a large
number of textual fields and, as far as I could tell from fast
skimming, most or all are specified as "normalizedString"  and
very few of them are IDN labels.

> in the form specified by IDNA.

Of course.

>> least) to do exactly what TUS says to do, which is to
>> normalize only at comparison time rather than trying to carry
>> strings around in normalized form, the application of that
>> attribute all almost all text-type values may be
>> inappropriate. I can find no evidence in the I-D that those
>> issues were considered; the document should not progress
>> until they are.
> 
 
> However, this does not appear to be one of them.

For the many fields that do not appear to be IDN labels, why
not?  From skimming the document, I'd assume that the IDN labels
should be required to be in NFC but that a normalization
requirement is probably inappropriate for most other fields,
especially those fields that appear to be free text
descriptions.  Which ones probably requires a field-by-field
analysis.

> We may need a statement specific to IETF that captures the

Indeed.

>...

best,
   john

---

From: "Patrik Fältström" <[email-masked]>
Date: Fri, 06 Mar 2020 08:17:04 +0100

On 6 Mar 2020, at 6:46, John C Klensin wrote:

> But, given the fate of i18n documents in the last year or so, I wouldn't be optimistic about getting it processed, so maybe

Lets start by trying to get drafts be passed by this list, and then send a canned response. When the response start to look the same, then there is time to actually write an RFC that we instead reference instead of copy-paste the same text every time.

   Patrik

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 6 Mar 2020 00:40:27 -0800

>> We may need a statement specific to IETF that captures the
I did not do a full review, just checking some of the points that
others had noted against the original.

That means I have not looked into the details of all the text
fields. As a matter of policy, if we were to develop one to help
us guide in our review, we may perhaps partition:

(1) IDN U-labels

(2) Free text (description, comment)

(3) Other, protocol relevant

It makes sense to require (1) to be in the format specified by
IDNA (NFC and lowercase), and likewise it does not make sense
for (2) to be normalized.

String that may be relevant as part of some protocol, but are not
IDN U-labels, would be RECOMMENDED to be in whatever
format the protocol sets out for them. (There's a small
benefit to be able to use non-protocol aware tools to do
things like search data sets, and therefore it's preferable to
have the data in the expected form for easy comparison,
even if the protocol describes its own preprocessing step.

To give an example: Unicode allows for character and property
names to elide spaces with or without camel case or to use -
or _ as separators. However, in the context of the standard
as much as possible only one of these formats will be used
(single case with space as separator for character names and
_ separator for properties).

While everybody knows how to compare the values, it helps
the reader to know that the text of the standard follows a
uniform convention.

If a protocol allows mulitple forms, as Unicode does for its
identifiers, it would still be good form to pick a canonical
representation for such protocol-relevant items and as
reviewers we should have a policy on that.

A./

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 06 Mar 2020 07:55:52 -0500

> reviewers we should have a policy on that.

For protocol identifiers, there is already a framework to manage that: Precis.
We don’t need to reinvent the wheel in this directorate.

Marc.

> -- 
> I18ndir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i18ndir

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 06 Mar 2020 08:03:42 -0500

> descriptions.  Which ones probably requires a field-by-field

My reading of the draft is that many fields were just of « documentary 
nature » and no clear need (well, at least it was not specified 
anywhere in the draft) for sorting, searching or comparing, hence no 
real requirement for normalization. We don’t want to impose stuff 
which is not needed.

We have to remember the intent of the draft: this is essentially 
documenting an archive file format. Generated by registrars and 
registries to be sent to a custodian in case of a need to restart the 
service elsewhere. This is really not a protocol per se, it is an 
archive file format.

Marc.

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 09:33:55 -0500

Marc,

--On Friday, March 6, 2020 08:03 -0500 Marc Blanchet
<[email-masked]> wrote:

>>>> normalize only at comparison time rather than trying to
>>>> carry strings around in normalized form, the application of
>>>> that attribute all almost all text-type values may be
>> not?  From skimming the document, I'd assume that the IDN
>> labels should be required to be in NFC but that a
>> normalization requirement is probably inappropriate for most
>> other fields, especially those fields that appear to be free
>> text descriptions.  Which ones probably requires a
>> field-by-field analysis.
> 
> My reading of the draft is that many fields were just of
> « documentary nature » and no clear need (well, at least
> it was not specified anywhere in the draft) for sorting,
> searching or comparing, hence no real requirement for
> normalization. We don't want to impose stuff which is not
> needed.

Exactly.   But, unless I read the review too quickly, you didn't
call that issue out and ask the IESG to push back on the
document until the WG has done the work.

> We have to remember the intent of the draft: this is
> essentially documenting an archive file format. Generated by
> registrars and registries to be sent to a custodian in case of
> a need to restart the service elsewhere. This is really not a
> protocol per se, it is an archive file format.

Understood.  But your comment above still applies: it is not
useful to impose requirements that are not needed.  

I'm pushing on this for an additional reason: for what I assume
are a small (but not zero) number of cases, there is actually a
disadvantage to normalizing unless there is a clear need to do
so.  For those who haven't already seen the discussion and
examples, the general problem is that, when assembling a
character grapheme (i.e., as seen by the reader) from a base
character and more than one combining character, the order of
code points that is optimal for accurate display and the order
of those code points determined by normalization might be
different.  This has been documented wrt Arabic script (someone
may have the reference more handy than I do) but one has to
assume there are other cases out there.  The bottom line is that
any of Unicode's mechanical transformations -- case mapping or
folding, canonical and compatibility mappings -- are sometimes
going to lose information.  Whether that information is
significant or not is likely to have only script-by-script,
possibly language-by-language, and case-by-case answers.  

But the bottom line is that not only is imposing unnecessary
requirements a bad idea but discarding information without
knowing whether it would be useful down the line is too.

Procedural question for you, Pete, and Barry:  would it be
better for you to wait another 24 hours or so (to be sure that
this discussion has stabilized) and then submit a revised review
or for one of the others of us (I fear I'm holding the short
straw, but would be delighted if someone else wants to pick it
up) to submit a separate review?

    best,
    john

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 10:45:30 -0500

Asmus,

Sorry -- I read and responded to Marc's note before seeing yours
and Patrik's.   I think we are in complete agreement, but see
comments inline below.

> us guide in our review, we may perhaps partition:

I think not just our reviews, but the advice we give to WGs
developing protocols and data formats.  That, of course, gets
back to the question as to whether (and when) it will be
appropriate to take the next steps beyond statements like "just
use UTF-8" and "just normalize everything" [1] and toward "don't
normalize (or tamper with case) unless you are actually about to
compare or there is a particular protocol-specific reason to do
so" and "don't use or allow anything other than UTF-8 unless
there is good reason to do so and you can document that reason"

> for (2) to be normalized.

Right.   See also my note to Marc.

To try to keep our language precise, for IDN U-labels, the
format is not only a matter of making sense.  By the definition
-- which was written very carefully and intentionally -- if a
string is not NFC and lowercase (and conformant to most other
IDNA requirements) it is simply not a U-label.  That is the
reason why parts of the IDNA2008 specs are sprinkled with terms
like "putative U-label".

But, as far as I can tell from another quick pass through this
document, your case (1) does not exist at all or may be
violated: domain objects, as described in Section 5.1.1.1, use
A-labels and not U-labels; "U-label" does not appear in the
document at all.  However, NNDN elements (there and elsewhere)
are Unicode strings and, after reading the definition of NNDN in
Section 1 a couple of times, I can't tell whether their
components are expected to be IDNA2008-valid U-labels or not.
IMO, it is another deficiency in this I-D that this is not clear
and that either U-labels must be required or there must be more
explanation why not.

> even if the protocol describes its own preprocessing step.

Right.  However, there is a separate recommendation for the
protocols themselves.  We should we try to minimize the number
of different preferred forms floating around.  For your case (2)
there is no preferred form.  But for the third case and
especially because most of the people designing protocols and
implementing libraries that are not i18n-specific cannot be
expected to be i18n or Unicode experts and because we have a
great deal of experience with strings "leaking" between protocol
contexts, having many different preferred formats out there is
an invitation to confusion and bugs.  The tradeoffs may need to
be evaluated carefully on a case-by-case basis, but simplicity
in the number of options and formats may trump optimality for
the needs of a particular protocol.

> _ separator for properties).

Hmm.  Thanks, I just learned something because I had not studied
(or had forgotten) that set of rules.  [2]

> reviewers we should have a policy on that.

Yes.  And I think it is more than "as reviewers".  As discussed
in earlier notes, we may be getting to the point that it is time
for us to recommend, and the IETF to establish, some clear
guidance on these subjects so that we minimize the number of
situations in which WGs develop work and then we come along
during IETF Last Call and tell them they messed up.  Patrik may
be right that we need more reviews and responses before we are
ready for that, but my sense is that we are fairly close.

best,
   john

[1]  I am not familiar enough with the Regext set of documents
to know if "normalizedString" is defined somewhere.  Certainly,
draft-ietf-regext-dnrd-objects-mapping-06 uses it without
defining it locally or providing a specific normative reference
to such definitions.  At minimum, this suggests another problem
with the I-D that may need fixing.  But, if there is not a clear
definition -- if the instruction for what should go into those
fields is essentially "just normalize", than that is obviously
bad news.  It is bad news even for IDN U-labels because a string
in, e.g., NFC would not conform to the requirements for
U-labels.  Normal IETF and RFC practice requires a statement,
probably in the Terminology section and probably with a
normative reference, to where definitions of that and other
attributes can be found.

[2] It might be of interest that there are two reasons we don't
have "_" in LDH-style domain names because they were prohibited
in the original host name syntax.  And they were prohibited
there precisely because of a human factors argument: when
reading the handwriting of others, people cannot reliably
distinguish between "-" and "_", so it is best to allow one and
not the other.  And the hyphen was chosen in part because the
most common grapheme for the BCD character that evolved into "_"
in ASCII (and ECBDIC) was a left-pointing arrow, not a
horizontal bar at the baseline.  So the ARPANET decided to go
with a single form at the codepoint level while the Unicode
decision was to allow different forms and canonicalize later.
Of course, we have a different variation on the Unicode decision
with "FWS" and "CWFS" constructions in some well-known protocols.

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 6 Mar 2020 15:55:17 +0000

Note below on the draft's reference to the XML Schema normalizedString data type:

> -----Original Message-----
> From: I18ndir <[email-masked]> On Behalf Of John C Klensin
> Sent: Friday, March 6, 2020 10:46 AM
> To: Asmus Freytag (c) <[email-masked]>; [email-masked]
> Subject: [EXTERNAL] Re: [I18ndir] I18ndir last call review of draft-ietf-regext-
> dnrd-objects-mapping-06
> 

[snip]

> [1]  I am not familiar enough with the Regext set of documents to know if
> "normalizedString" is defined somewhere.  Certainly,
> draft-ietf-regext-dnrd-objects-mapping-06 uses it without defining it locally
> or providing a specific normative reference to such definitions.  At minimum,
> this suggests another problem with the I-D that may need fixing.  But, if
> there is not a clear definition -- if the instruction for what should go into
> those fields is essentially "just normalize", than that is obviously bad news.  It
> is bad news even for IDN U-labels because a string in, e.g., NFC would not
> conform to the requirements for U-labels.  Normal IETF and RFC practice
> requires a statement, probably in the Terminology section and probably with
> a normative reference, to where definitions of that and other attributes can
> be found.

Reference here:

https://www.w3.org/TR/xmlschema-2/#normalizedString

The draft should include a normative reference to the W3C XML Schema specification:

http://www.w3.org/TR/2004/REC-xmlschema-2-20041028

Scott

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 11:09:02 -0500

--On Friday, March 6, 2020 07:55 -0500 Marc Blanchet
<[email-masked]> wrote:

> For protocol identifiers, there is already a framework to
> manage that: Precis. We don't need to reinvent the wheel in
> this directorate.

Marc,

Agree completely.  However, my interpretation of Asmus's
suggestion and my variation on it is that the requirement could
be satisfied by a pointer to PRECIS (and, presumably, a
particular profile of PRECIS, which allows many options).   In
that sense, this is just like the "support UTF-8 and nothing
else unless you have a clear reason and explanation" policy that
all three of us are suggesting.  "Use and conform to PRECIS and
specify what profile is being used unless you have a good reason
to do something different and, if you do have such a reason, you
MUST describe and explain it" would be a perfectly good rule.  

That said, and noting that Section 5.1 of RFC 8264 (which is a
clear statement of the "too many profiles or canonical forms are
a bad idea" principle that Asmus and I have suggested in this
discussion), I note that RFC 8265 requires NFC and RFC 8266
requires NFKC.   So, if the "don't normalize, leave that for
comparison time" is appropriate for some strings, especially
those in Asmus's second category, than a PRECIS reference would
presumably need an additional profile.

best,
   john

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 6 Mar 2020 08:42:49 -0800

> with "FWS" and "CWFS" constructions in some well-known protocols.

Unicode's "loose matching" rules are designed to help these
identifiers to be embedded as needed in other protocols, where
there are native restrictions. Unicode does provide an exhaustive
list of what those other protocols might be, but it's been clear
that they include, for examples, programming languages.

Bu making sure identifiers stay unique without requiring literal
hyphens, spaces or underlines, this can be accommodated.

I was simply speculating that for any 'new' protocol that
collects data that have a defined format in other protocols,
it may make sense to not force these data to be normalized
or canonicalized different from what the external protocol expects.

If external protocols allow multiple formats, then it may make
sense to pick one canonical form for the 'new' protocol, just
so data in the new protocol are simplified.

It depends a bit on the use case. A protocol that's a transient
container of data transmitted between implementations
of an external protocol is a different matter from a database
that collects information from potentially different sources,
but could be expected to have some internal consistency.

A./

ps: if/when we get around to writing this down, the language
will need to become much more precise than these musings.

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 6 Mar 2020 08:51:40 -0800

As I read the W#C document, "NormalizedString" is not an actual format, 
other than that it disallows all line endings and tab.

There are parameters for how to normalize remaining whitespace, but 
without specifying which parameters are set. It's like calling a string 
"Unicode normalized" and not knowing from that term whether it's in NFC, 
NFD, NFKC or NFKD.

In the W3C these are enumerated as part of the "constraining facets" 
listed in 3.3.1.1

For an archival format, then, either each field has attributes that give 
these facets, as applicable, or the format itself needs to require 
particular settings.

None of that, by the way, speaks in any way about casefolding or Unicode 
normalization, the kinds of operations that are critical in identifier 
contexts.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 6 Mar 2020 09:00:38 -0800

>> protocol per se, it is an archive file format.

Wouldn't that use case make it effectively a transport protocol for
the entire dataset defining the registry service?

Archive formats have fields with data intended to be consumed by
some other protocol(s) during this "restart a service elsewhere"
process.

It's "easier" to create such an archive if you can just "dump" the 
originating
data "as is" but it may be easier to restore the data if they have some
dependable consistency.

For free text, we know that some scripts prefer non-normalized
data for ordinary text (as normalization either breaks apart some
atomic character or reorders them in ways not all rendering engine/fonts
handle predictably - Bengali is an example of the former).

However, for anything that's a protocol parameter or identifier,
different assumptions should apply.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 6 Mar 2020 09:03:17 -0800

> We don’t need to reinvent the wheel in this directorate.

Not arguing we should invent a new scheme, just that we insist that any
protocol we review make such a selection in whatever framework we agree
on as relevant.

A./

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 6 Mar 2020 17:05:30 +0000

It is what it is, Asmus. I’m not so sure that it is, in fact, critical. ICANN has supposedly been using this archive format for years with no interoperational issues.

Scott

From: I18ndir <[email-masked]> On Behalf Of Asmus Freytag
Sent: Friday, March 6, 2020 11:52 AM
To: [email-masked]
Subject: [EXTERNAL] Re: [I18ndir] I18ndir last call review of draft-ietf-regext-dnrd-objects-mapping-06

As I read the W#C document, "NormalizedString" is not an actual format, other than that it disallows all line endings and tab.

There are parameters for how to normalize remaining whitespace, but without specifying which parameters are set. It's like calling a string "Unicode normalized" and not knowing from that term whether it's in NFC, NFD, NFKC or NFKD.

In the W3C these are enumerated as part of the "constraining facets" listed in 3.3.1.1

For an archival format, then, either each field has attributes that give these facets, as applicable, or the format itself needs to require particular settings.

None of that, by the way, speaks in any way about casefolding or Unicode normalization, the kinds of operations that are critical in identifier contexts.

A./

      -----Original Message-----
      From: I18ndir <[email-masked]><mailto:[email-masked]> On Behalf Of John C Klensin
      Sent: Friday, March 6, 2020 10:46 AM
      To: Asmus Freytag (c) <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
      Subject: [EXTERNAL] Re: [I18ndir] I18ndir last call review of draft-ietf-regext-
      dnrd-objects-mapping-06

   [snip]

   Reference here:

   Scott

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 12:06:12 -0500

Scott,

More generally, the terminology section of the document should
make it clear where to find definitions for all of the attribute
values it uses.  If the normative reference to
http://www.w3.org/TR/2004/REC-xmlschema-2-20041028 takes care of
all of that, so much the better.

The importance of having a clear reference to that definition is
made clear because my/our assumption that it had anything at all
to do with normalization in the Unicode sense has led to an (at
least IMO) interesting, informative, and helpful conversation
that has nothing to do with
draft-ietf-regext-dnrd-objects-mapping-06.  In fact, it has
nothing to do with Unicode normalization forms but only
restricts the use of ASCII CR, LF, and HT.  There is probably a
bug in that definition as well: <string>, the base type for
normalizedString, appears to allow any Unicode character unless
it is restricted in some way.  But, once one escapes the ASCII
repertoire, there are a number of white space or control
characters that should almost certainly be excluded as well.
(U+21B4) is rhe most obvious example but VY (U+000B), REVERSE
LINE FEED (U+008D), NEXT LINE (U+0085), and the partial line
controls (U+008B, U+008C) may belong there as well.

I have not studied this and its implications, but my guess is
that it would have been better for the definition of
normalizedString would be better written in terms of what is
permitted rather than what is prohibited, e.g., by prohibiting
all control characters other than SP (U+0020).  And, regardless
of what W3C chooses to do and when, if the WG wants an archival
form for these databases, you probably want a "no white space or
line/carriage movement characters other than SP" rule, not
normalizedString as now defined.

best,
   john

--On Friday, March 6, 2020 15:55 +0000 "Hollenbeck, Scott"
<[email-masked]> wrote:

>> From: I18ndir <[email-masked]> On Behalf Of John C
>> Klensin Sent: Friday, March 6, 2020 10:46 AM
>> Subject: [EXTERNAL] Re: [I18ndir] I18ndir last call review of
>> draft-ietf-regext- dnrd-objects-mapping-06
>> 
> 
> [snip]
> 
>> [1]  I am not familiar enough with the Regext set of
>> documents to know if "normalizedString" is defined somewhere.
>> Certainly, draft-ietf-regext-dnrd-objects-mapping-06 uses it
>> without defining it locally or providing a specific normative
>> reference to such definitions.  At minimum, this suggests
>> another problem with the I-D that may need fixing.  But, if
>> there is not a clear definition -- if the instruction for
>> what should go into those fields is essentially "just
>> normalize", than that is obviously bad news.  It is bad news
>> even for IDN U-labels because a string in, e.g., NFC would not
>> conform to the requirements for U-labels.  Normal IETF and
>> RFC practice requires a statement, probably in the
>> Terminology section and probably with a normative reference,
>> to where definitions of that and other attributes can be
>> found.
> 
> Reference here:
>

---

From: "Pete Resnick" <[email-masked]>
Date: Fri, 06 Mar 2020 11:27:22 -0600

> assignments were posted to the i18ndir list.

That was actually my intent. I thought the assignment system did that 
automatically, but obviously it doesn't. I will make sure the 
announcement happens in the future.

The request for this review came directly from Barry, but I've got a 
little more time now to do some triaging, so I'll start trying to make 
assignments for documents that smell like they've got i18n involvement.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 6 Mar 2020 09:44:45 -0800

On 3/6/2020 9:05 AM, Hollenbeck, Scott wrote:
>
> It is what it is, Asmus. I’m not so sure that it is, in fact, 
> critical. ICANN has supposedly been using this archive format for 
> years with no interoperational issues.
>
Good to know. If it's been in use, it may indeed have its own backwards 
compatibility constraints,

A./

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 12:45:13 -0500

Scott,

But that takes us full circle and into a set of problems that
are not I18N issues are all.

Given the number of times we have actually had:
 (1) A registry go out of business or been forced out by
	ICANN, and
 (2) The registry replaced by another one, and
 (3) These files used to populate the files for the new
	registry.

... It is hard to argue that there has been extensive
operational experience (and so I appreciate your use of
"supposedly".

And, while ICANN has run some tests to simulate such actions,
there have been, as you presumably, considerable suspicion that
there tests have been somewhat perfunctory.  Again, "supposedly".

In addition, if the only entities who care about this data
format and supporting protocols are ICANN, ICANN Contracted
Parties, and other ICANN-dependent entities, it takes us back to
my long-term concern about the REGEXT work and its boundaries:
if the real criteria for efforts like this document are that it
works for ICANN (however ICANN defines "works") then what is the
value of having the IETF involved?  Clearly, the work gets
better, and broader, review than ICANN could get without
spending money on expert reviewers and probably authors.  But,
despite occasional claims to the contrary, ICANN is not in a
state of poverty in which it could not afford those efforts if
they considered them important.   

>From an IETF standpoint, volunteer time that goes into this sort
of effort is time that is taken away from things that are
arguably more critical.  As a very specific pair of examples,
this directorate has a core IDNA document in AUTH48,
draft-klensin-idna-unicode-review, that I haven't been able to
spend time on in the last few days because Marc's review had to
be prioritized so that any necessary revisions could get into
the LC process before the, IIR, 9 March cutoff.  not clear to me
that is the right optimization for IETF or the Internet.

best,
   john

--On Friday, March 6, 2020 17:05 +0000 "Hollenbeck, Scott"
<shollenbeck=[email-masked]> wrote:

> It is what it is, Asmus. I'm not so sure that it is, in
> fact, critical. ICANN has supposedly been using this archive
> format for years with no interoperational issues.
> 
> 
> 
> Scott
> 
> 
> 
> From: I18ndir <[email-masked]> On Behalf Of Asmus
> Freytag Sent: Friday, March 6, 2020 11:52 AM
> To: [email-masked]
> draft-ietf-regext-dnrd-objects-mapping-06
> 
> 
> 
> As I read the W#C document, "NormalizedString" is not an
> actual format, other than that it disallows all line endings
> and tab.
> 
> 
> 
> There are parameters for how to normalize remaining
> whitespace, but without specifying which parameters are set.
> It's like calling a string "Unicode normalized" and not
> knowing from that term whether it's in NFC, NFD, NFKC or NFKD.
> 
> 
> 
> For an archival format, then, either each field has attributes
> that give these facets, as applicable, or the format itself
> needs to require particular settings.
> 
> 
> 
> None of that, by the way, speaks in any way about casefolding
> or Unicode normalization, the kinds of operations that are
> critical in identifier contexts.
> 
> 
> 
> A./
> 
> 
> 
> 
> 
>       From: I18ndir
> <[email-masked]><mailto:[email-masked]> On
> Behalf Of John C Klensin       Sent: Friday, March 6, 2020
> 10:46 AM
>       To: Asmus Freytag (c)
> <[email-masked]><mailto:[email-masked]>;
> [email-masked]<mailto:[email-masked]>       Subject:
> [EXTERNAL] Re: [I18ndir] I18ndir last call review of
> documents to know if       "normalizedString" is defined
> somewhere.  Certainly,
> defining it locally       or providing a specific normative
> reference to such definitions.  At minimum,       this
> suggests another problem with the I-D that may need fixing.
> But, if       there is not a clear definition -- if the
> instruction for what should go into       those fields is
> essentially "just normalize", than that is obviously bad news.
> It       is bad news even for IDN U-labels because a string
> U-labels.  Normal IETF and RFC practice       requires a
> statement, probably in the Terminology section and probably
> with       a normative reference, to where definitions of that
> and other attributes can       be found.
> 
> 
>    Reference here:
>

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 6 Mar 2020 09:53:41 -0800

On 3/6/2020 9:06 AM, John C Klensin wrote:
> restricts the use of ASCII CR, LF, and HT.

"normalized string" is a poor choice of name in any context
where strings can be/are in Unicode. However, it is what it is,
but because misleading, it really needs to be called out.

> There is probably a

Didn't XML used to exclude control characters generically?

I seem to remember discussions. If so, a holdover?

21B4 is a graphic character that may indicate a linefeed, just like
the pilcrow sign indicates a paragraph. But it's not a control.

> normalizedString as now defined.

NormalizedString allows additional constraints, such as a precise
description whitespace handling. (Preserver, collapse etc).

Not sure whether the format carries these constraints in metadata/
attributes or not. And whether an archive format should or
should not support all possible values of these constraints.

A./

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 6 Mar 2020 18:02:03 +0000

> From: John C Klensin <[email-masked]>
> Sent: Friday, March 6, 2020 12:45 PM
> To: Hollenbeck, Scott <[email-masked]>
> Cc: [email-masked]; [email-masked]
> But that takes us full circle and into a set of problems that are not I18N issues
> are all.
> 
> .... It is hard to argue that there has been extensive operational experience
> (and so I appreciate your use of "supposedly".
> 
> And, while ICANN has run some tests to simulate such actions, there have
> been, as you presumably, considerable suspicion that there tests have been
> somewhat perfunctory.  Again, "supposedly".
> 
> In addition, if the only entities who care about this data format and
> supporting protocols are ICANN, ICANN Contracted Parties, and other
> ICANN-dependent entities, it takes us back to my long-term concern about
> the REGEXT work and its boundaries:
> if the real criteria for efforts like this document are that it works for ICANN
> (however ICANN defines "works") then what is the value of having the IETF
> involved?  Clearly, the work gets better, and broader, review than ICANN
> could get without spending money on expert reviewers and probably
> authors.  But, despite occasional claims to the contrary, ICANN is not in a
> of effort is time that is taken away from things that are arguably more critical.
> As a very specific pair of examples, this directorate has a core IDNA
> document in AUTH48, draft-klensin-idna-unicode-review, that I haven't been
> able to spend time on in the last few days because Marc's review had to be
> prioritized so that any necessary revisions could get into the LC process
> before the, IIR, 9 March cutoff.  not clear to me that is the right optimization
> for IETF or the Internet.

I can't disagree with anything you wrote above, John. From a practical perspective, we should wrap up the review that Marc was requested to provide and move on to more important topics.

Scott

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 13:39:11 -0500

--On Friday, March 6, 2020 18:02 +0000 "Hollenbeck, Scott"
<shollenbeck=[email-masked]> wrote:

>> .... It is hard to argue that there has been extensive
>> there have been, as you presumably, considerable suspicion
>> that there tests have been somewhat perfunctory.  Again,
>> "supposedly".
>> 
>> Parties, and other ICANN-dependent entities, it takes us back
>> to my long-term concern about the REGEXT work and its
>> boundaries:
>> if the real criteria for efforts like this document are that
>> it works for ICANN (however ICANN defines "works") then what
>> is the value of having the IETF involved?  Clearly, the work
>> gets better, and broader, review than ICANN could get without
>> spending money on expert reviewers and probably authors.
>> But, despite occasional claims to the contrary, ICANN is not
>> in a state of poverty in which it could not afford those
>> efforts if they considered them important.
>> 
>> draft-klensin-idna-unicode-review, that I haven't been able
>> to spend time on in the last few days because Marc's review
>> had to be prioritized so that any necessary revisions could
>> get into the LC process before the, IIR, 9 March cutoff.  not
>> clear to me that is the right optimization for IETF or the
>> Internet.
> 
> I can't disagree with anything you wrote above, John. From a
> practical perspective, we should wrap up the review that Marc
> was requested to provide and move on to more important topics.

Scott,

Agreed.  I do note Asmus's recent analysis, which went well
beyond mine. IMO, the issues he raises about what is specified
via additional constraints are imposed are things that ought to
be pinned down and specified, even if the conclusion is that all
elements that now have normalizedString attributes deserve the
same treatment and that there should not be additional
constraints or qualifiers, before the document goes forward.

To paraphrase part of what he said, the choice of
"normalizedString" to describe that particular concept was a
terrible decision but it is what it is.  On the other hand, if
puts some burden on this I-D to be clear about what is going on,
perhaps even summarizing what that rule is about rather than
expecting them to actually track down the reference rather than
making the assumption about Unicode normalization that Asmus and
I did.

The question he asked about metadata is that, if such metadata
exist and are relevant, they either need to be part of the
specification or we are relying on the kind of "everyone
involved knows how this is supposed to work" oral tradition,
which goes back to my mini-rant about "supposedly used by ICANN".

So it seems clear to me, if only from this issue and the UTF-8
one, that the document needs work and that it is possible that
some of that work will be significant enough that another Last
Call will be needed.  In principle at least, that is a decision
you (as Document Shepherd) and Barry (as Responsible AD) could
make today by pulling the draft and taking it off the IESG
agenda.  Not only would that be the fastest way to "move on"
(not even requiring someone to consolidate all these comments
back into the review), but it would probably be more efficient:
I don't know the authors of this document but, as someone who
has written a few RFCs, I'm much prefer to see a situation like
this turned into a dialogue about how to get things right than
to get a review, try to guess what text would satisfy the issues
it raises, and then iterate.

And, Barry, because the discussion of the last day or so
illustrates that REXEXT work, at least if this document is taken
as an example, is making work for people who have not opted to
be part of the WG and hence is delaying other work, I think a
review of the scope and purpose of that WG may be in order.
Declaring success in getting all of the basic framework in plane
and then saying "your problem" to ICANN should be one possible
outcome.  That would, of course, be a way to dispose of the
other REGEXT-related draft we have discussed recently.

best,
  john

---

From: "Pete Resnick" <[email-masked]>
Date: Fri, 06 Mar 2020 13:14:46 -0600

On 6 Mar 2020, at 12:39, John C Klensin wrote:

> Call will be needed.

Please let's keep comments about how an AD or Chair should run the IETF 
process off of this list and instead take it to private email (if at 
all). This discussion should be limited to the technical comments on 
problems in documents. The only process discussion on this list should 
be about how the directorate do things.

Thanks,

pr

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 14:21:07 -0500

--On Friday, March 6, 2020 08:42 -0800 "Asmus Freytag (c)"
<[email-masked]> wrote:

>> [2] It might be of interest that there are two reasons we
>> don't have "_" in LDH-style domain names because they were
>> prohibited in the original host name syntax.  And they were
>> prohibited there precisely because of a human factors
>> argument: when reading the handwriting of others, people
>> cannot reliably distinguish between "-" and "_", so it is
>> best to allow one and not the other.  And the hyphen was
>> chosen in part because the most common grapheme for the BCD
>> character that evolved into "_" in ASCII (and ECBDIC) was a
>> left-pointing arrow, not a horizontal bar at the baseline.
>> So the ARPANET decided to go with a single form at the
>> codepoint level while the Unicode decision was to allow
>> different forms and canonicalize later. Of course, we have a
>> different variation on the Unicode decision with "FWS" and
>> "CWFS" constructions in some well-known protocols.
> 

Interesting and helpful.  Of course, the tradeoffs between
defining a canonical form and using and carrying it around as
much as possible (certainly between systems and in archival
formats) versus allowing local forms and converting to canonical
ones only when necessary is an ancient one that won't get
settled today.  In Multics, canonicalization was expected to
occur in the terminal drivers, so it was hard to even enter a
non-canonical form string. 
 
> but could be expected to have some internal consistency.

Right.  There is another issue which might be relevant to a
record-keeping (or archival) storage format like this one and
would be different from our "on the wire" expectations.   It
takes us back to the reason the Punycode algorithm was
specified.   If one expects most of all of the code points one
will have to deal with will come out of the BMP, UTF-16 may
actually make more sense that UTF-8.  Part of the motivation for
the Punycode work is that UTF-8 takes up more space for East
Asian scripts than it does for ASCII.    For a file or stream
that is mostly ASCII but with some other characters, UTF-8 is
usually still a win.  But, for a file that is mostly in, say,
Han characters, UTF-16 will typically be much more compact and
will eliminate most or all of the decoding problems associated
with UTF-8.  I still don't think that says "use UTF-8 or
whatever you like" as the I-D appears to say, but it may be a
tad less obvious that UTF-8 is the right answer then for data
that are expected to be transferred regularly over the network.

> will need to become much more precise than these musings.

Of course.  Always.

best,
  john

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 06 Mar 2020 16:30:20 -0500

> operational experience (and so I appreciate your use of

I think this discussion is out of scope of this document and its review. 
This document describes an archival format of data. It does not 
describes what the custodian should do with it in case of its need. In 
that situation of doing something with the data, this is not a protocol 
but a best-practice-document-to-read-archival-files, which is to me not 
in the IETF realm. That best-practice-document-to-read-archival-files 
could include information about normalization, case folding, and other 
nice stuff we could think about. But again that 
best-practice-document-to-read-archival-files is in fact in ICANN realm 
not IETF’s. This archival format document do not necessarily need to 
talk about those subjects.

> Parties, and other ICANN-dependent entities,

well, ccTLDs are often following what ICANN is doing (they are not 
obliged to, but often have many advantages to reuse).
So overall, that is pretty much a big set of Internet infrastructure 
components that cares about this stuff.

And we have seen other protocols that you could claim to be ICANN’s 
such as RDAP that included the RIR needs since the beginning. Nowadays, 
there are other organizations and protocols considering RDAP for other 
uses. So it seems to me that this « this-work-is-only-for-ICANN » a 
bit reductive to its value.  I could think of many IETF protocols that 
have a much more narrow scope than protocols/documents related to ICANN.

So I agree with you that, taking my example above, the 
best-practice-document-to-read-archival-files-to-be should be written by 
ICANN community, not here.

Regards, Marc.

> it takes us back to

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 16:56:03 -0500

--On Friday, March 6, 2020 13:14 -0600 Pete Resnick
<[email-masked]> wrote:

> Please let's keep comments about how an AD or Chair should run
> the IETF process off of this list and instead take it to
> private email (if at all). This discussion should be limited
> to the technical comments on problems in documents. The only
> process discussion on this list should be about how the
> directorate do things.

Sorry, Pete, even if its form was not ideal from your point of
view, that was a suggestion about how the directorate does
things.  The directorate is supposed to give advice to the ART
ADs on how to make progress on i18n topics.  The advice I am
giving (or, if you prefer and are actually going to organize and
moderate discussions as I understood you were tasked to do and
have promised to do, a suggestion I am making to i18ndir for a
recommendation to the AD(s) is:

(1) This document, in its current form, is not ready for a
document action and publication as a Proposed Standard.  That
recommendation is perfectly appropriate as part of a document
review.  If it is not, I've lost sight of what IETF Last Calls
are supposed to be about. 

(2) The work required to fix it will probably result in
sufficiently substantive changes that it will be necessary for
the WG to be consulted -or- will require the WG to make some
substantive decisions.  For example, while my opinion (and that
of Scott and Asmus) that some additional references and
explanation of what was intended to appear in various strings
may fall into the range of fixes that are often specified by the
IESG for the authors and/or RFC Editor to sort out, a change to
require UTF-8 is substantive enough that anyone who thinks
UTF-16 (or for that matter, UTF-32 or UTF-7) is an important
enough option to justify retaining should have a chance to weigh
in on that decision.  Alerting the AD and IESG to that seems
completely orderly and a reasonable part of a Last Call review,
whether by the directorate or by an individual.

(3) When a Last Call review raises substantive issues that
involve specialized knowledge, some complexity, or both, there
are two ways to write that review and proceed.  One is to
explain the problem, the specialized knowledge, and the
alternatives in sufficient detail that authors or a WG can go
off and do the right thing.  This stuff is not obvious: I've
learned things from Asmus, Marc, and Scott today.  I would like
to at least hope that they have learned a bit from me and that
those who have been lurking have learned from all of us.  The
second option is to explain the problem sufficiently to make it
clear to the WG and the IESG that there _is_ a problem, then
volunteer to the WG to work with them to develop a solution.
Usually the latter is likely to be more efficient and is
appreciated.  I can remember reviews and discussions over the
years in which that approach has been taken and WG Chairs and
ADs have thanked me for being willing to step in and volunteer.
My memory might be playing tricks, but I vaguely remember having
just such interactions with some Apps AD in the 2011-2015 period
whom you presumably know fairly well.    So I suggest that part
of that advice was just a variation on "not ready for
publication" and the other part was suggesting a way to move
forward.  Either would be appropriate in a review; either would
be appropriate as advice for the directorate to give the AD(s).

(4) At least from my perspective, looking at this document (and
several others, but we haven't talked about them this week) has
suggested that it may be time to update the advice the IETF
gives document authors about i18n issues.  That might possibly
including revising/ updating BCP 18/ RFC 2277 to make a clear
recommendation to use UTF-8 and to reinforce the RFC 2277
requirement that UTF-8 is  mandatory to implement, to justify
any other options, and to make some suggestions about when
normalization (in the Unicode sense) is appropriate and/or to
discuss the tradeoffs.  The days when it was desirable to
encourage individual clients and servers to make their own
choices of CCS or charset, including as possibilities selected
parts of ISO 8859, assorted proprietary code pages (with or
without ISO 2022-based switching), and selected national
standards is long past. I note on rereading 2277 that use of any
other encoding scheme instead of UTF-8 is a sufficiently big
deal as to require a variance procedure, a policy that has not
been followed but which would lay a foundation for sorting some
of these issues out by appeals if we cannot devise a more
appropriate and orderly mechanism.   If it is now time, that is
appropriate advice to get the ADs.  If it is not time but we see
the time coming soon, _that_ is appropriate advice  to give the
ADs.  So the only problem I can see with that part of the
discussion is if you believe that the directorate is only
allowed to examine topics that you assign to it.  If you do,
then I think we are overdue for a discussion about how, in your
words, the directorate does things.

So, can you better explain what problem you see with the
discussion and recommendations -either to the AD or to the
directorate to recommend to the AD- so I can better understand
what I'm forbidden to say and why?

thanks,
   john

---

From: "Pete Resnick" <[email-masked]>
Date: Fri, 06 Mar 2020 16:34:50 -0600

On 6 Mar 2020, at 15:56, John C Klensin wrote:

> things.

Just to be clear, in case it wasn't before, it was only the part of your 
message that I quoted that I had any issue with whatsoever. And up to 
including where it says "the document needs work", I have no issue. It's 
what follows regarding "another Last Call" that that I was commenting 
on.

> The directorate is supposed to give advice to the ART
> review.

You bet. Saying "not ready" is absolutely one of the appropriate 
responses.

> substantive decisions.

It depends on what you mean. My assumption is that once we give a review 
and advice, the WG is responsible to come to us with questions about 
what needs to be done. The review should already lay out what work will 
be involved to make the document "ready" (either in general terms or 
specific suggested changes), and if the WG needs more advice on how (or 
if) to make particular changes, we are happy to advise. Insofar as we 
give general terms of changes that need to be made, we should quantify 
the extent of those changes as much as we can so the AD and the WG 
understand what's involved. And one or more of us might offer to work 
directly with the WG on the issues. (Effectively, what you describe in 
#3.)

However, we don't have to (and shouldn't) tell the AD, "The WG is going 
to have to consult us further". That's presumptuous. The AD might decide 
that the document should be dropped in its entirety. They might decide 
that the WG needs to be rechartered in some way. We have a 
responsibility to tell the AD about the i18n work that needs to be done, 
and how we can help get it accomplished, but not how to do their job.

> (3) ...part of that advice was just a variation on "not ready for

Assuming we're talking about the same part, yes, I agree.

> gives document authors about i18n issues.

I agree. In addition to updates to the documents you talked about, I 
think updating the assorted wiki "cheat-sheets" that I've seen over the 
years into something more comprehensive and helpful would also be great.

> So the only problem I can see with that part of the
> allowed to examine topics that you assign to it.

Nope, not a belief of mine at all.

> what I'm forbidden to say and why?

I think "forbidden" is a bit dramatic. All that I was suggesting is that 
we (as a directorate) not try to tell ADs how they ought to run the Last 
Call process, or any other aspect of the process, unless that advice is 
solicited. (We are not an exceedingly shy bunch about privately telling 
ADs how to do their job, and I would not presume to tell anyone what 
they should or should not say to an AD in private, at least not in my 
capacity as cat-herder of this directorate.)

pr

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 6 Mar 2020 15:28:20 -0800

On 3/6/2020 2:34 PM, Pete Resnick wrote:
> .... All that I was suggesting is that we (as a directorate) not try 
> to tell ADs how they ought to run the Last Call process, or any other 
> aspect of the process, unless that advice is solicited. (We are not an 
> exceedingly shy bunch about privately telling ADs how to do their job, 
> and I would not presume to tell anyone what they should or should not 
> say to an AD in private, at least not in my capacity as cat-herder of 
> this directorate.) 

I welcome this suggestion as a clarification of our mandate.

A./

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2020 18:48:58 -0500

Pete,

Skipping to the bottom...

--On Friday, March 6, 2020 16:34 -0600 Pete Resnick
<[email-masked]> wrote:

>...
> I think "forbidden" is a bit dramatic. All that I was
> suggesting is that we (as a directorate) not try to tell ADs
> how they ought to run the Last Call process, or any other
> aspect of the process, unless that advice is solicited. (We
> are not an exceedingly shy bunch about privately telling ADs
> how to do their job, and I would not presume to tell anyone
> what they should or should not say to an AD in private, at
> least not in my capacity as cat-herder of this directorate.)

I don't think I was telling Barry, or Scott, or anyone else, how
to do their jobs.   I was making a suggestion about how, in my
personal but probably not adequately humble opinion, how things
could be expedited with this document. Barry is free to do with
that advice whatever he likes, a range of options that certainly
includes either ignoring it or telling me in vivid detail where
I would put it.  I have more than enough confidence in him to be
sure that he is aware of both opinions and hope that his
relationship with me is such that he knows that I would not be
particularly offended if he selected the second. 

And, in that context, _if_ he decides he wants to move forward
by bouncing the document back to the WG, "the WG is going to
have to consult us further" might be presumptuous but it is also
a statement of fact.  Unless you make the distinction between
"us" as the directorate and "us" as a bunch of individuals, the
reality is that substantially all of the expertise on these
matters, at a level deep enough to have participated usefully in
the discussion of the last 24 hours, and among people active in
the IETF ("active" as measured by participation in the WGs
concerned with these topics) are on this mailing list (or were a
year ago).  If you have other candidates, why haven't we
recruited them?  As you know, I did have a few from outside who
were ready to be brought in, but you know what happened to that
idea.  So, assuming the WG doesn't have the needed expertise and
perspective internally (if they did, we wouldn't be finding
these issues after IETF LC started) where else are you (or
Barry) going to send them?  I assume the answer isn't to someone
who thinks that all of the issues come down to the implications
of diacritical marks in two closely-related languages that both
use Latin script   The answer feels to me a whole lot like "us".

best,
   john

---

From: "Pete Resnick" <[email-masked]>
Date: Sat, 07 Mar 2020 10:34:42 -0600

I will repeat: What motivated my message was the public comment on this 
list about the document needing another Last Call, which I don't think 
is appropriate output for this group. Just a few more comments below.

On 6 Mar 2020, at 17:48, John C Klensin wrote:

>  Barry is free to do with
> I would put it.

I prefer to have us stick to our knitting and not put Barry in the 
position of giving us colorful feedback. Again, if folks want to make 
comments on other-than-i18n issues, they are welcome to do that outside 
of the context of this discussion and get whatever colorful feedback 
they generate.

> a statement of fact.

No, really, it isn't. The WG could decide to punt on the document 
entirely, or that it will remove the troublesome bits and move on. Or 
that the proposed solution in the review is exactly right and plunk it 
in wholesale. And if the review has the appropriate amount of 
explanation, it should be painfully clear that some of those solutions 
are going to require further consultation with an i18n expert. There is 
a difference between saying, "Doing it this way is tricky and the 
directorate is happy to help if the WG wants to go down this path" and 
"You are required to tell the WG to consult us further if they go down 
this path." Our reviews better sound like the former, not the latter.

> If you have other candidates, why haven't we recruited them?

If you have commentary on my performance (or lack thereof) in this role, 
you can bring it to me or the ADs directly. This list is not the 
appropriate place for that discussion.

I don't think it is productive to continue this discussion on the list. 
If you have further comments, please take them to private email.

pr