Subject: [Ltru] Editorial comments on draft-4646bis-12

From: "Doug Ewell" <[email-masked]>
Date: Sat, 15 Mar 2008 17:42:20 -0700

As far as I can tell, all of the following comments are editorial,
except perhaps the comment that an expected technical change was not
made.  If anyone spots a technical comment, please accept my apologies
and move it to a separate Subject line.  None of these should be
showstoppers; a few date back before the present draft, for which I
apologize.

    Section 2, introduction:

The space after the comma in the passage "primary for human
communication,such as" has been deleted.  Please add it back.

    Section 2.1, 2nd paragraph:

The last sentence mentions "the subtag registry" before it has actually
been defined or described anywhere.  I suggest adding a reference to
Section 3, or at the very least making the reference a bit more formal:

"Thus, a parser need not have an up-to-date copy (or any copy at all) of
the IANA Language Subtag Registry to perform most searching and matching
operations."

    Section 2.2.4:

This section should have all remaining references to "ISO 3166" changed
to "ISO 3166-1", in keeping with the recent change made in the ABNF.
For example, the passages in Section 2.2.4, item 3 about "ISO 3166
alpha-2 codes" could be misinterpreted or deliberately twisted to refer
to ISO 3166-2 alpha-2 code elements such as "OR".  I would actually
recommend searching the entire document for "ISO 3166" and changing each
instance to "ISO 3166-1" unless the result would be simply wrong.

    Section 3.1.6, 2nd paragraph from the end:

This section continues to state: "The field 'Preferred-Value' MUST NOT
be modified once created in the registry."  While AFAIK there was no
declared consensus to change this, I was quite sure there was popular
support for making this field mutable, so that users of the Registry
would not have to follow a sequence of P-V's, say from "i-hak" to
"zh-hakka" to "hak".  I had been delaying the release of
draft-4645bis-05 in anticipation of this change, but seeing none, I will
go ahead and release the draft with the same P-V's as before.  (Q: Why a
new draft-4645bis?  A: Because the ISO 639-3 code lists have changed.)

    Section 4.2, 3rd paragraph, just before bullet points:

Change spelling of "very" to "vary".

    Section 8, 29th bullet point (6th from the end):

The reference to RFC 4324 looks erroneous, and probably should be RFC
5234.

    Section 9 generally:

I was surprised to see RFC 4645, which pertained to RFC 4646, listed as
a normative reference while draft-4645bis, which pertains to the present
draft, is only listed as informative.  I think this is because Section
2.2.4, item 3 (E) makes reference to RFC 4645, and that item has not
been changed since it appeared in RFC 4646.

Regarding the list in question, only 830 "Channel Islands" remains, and
IMHO the 3166/MA would be crazy to assign this code now that GG and JE
have been individually assigned.  However, it is possible, and this
situation does justify a reference to RFC 4645.  But I still need a
normative/informative lawyer to explain to me why that reference is
normative while the reference to draft-4645bis is not.

    Section 9.1, 17th item (2nd from the end):

Change spelling of "Freitag" to "Freytag".  If Asmus were on the list, I
would leave it up to him to correct the spelling or form of his name,
but I don't think he is.

    Section 9.2, 9th item (5th from the end):

Change month "12" to "December".  Perhaps surprisingly, the rfc2xml tool
doesn't automatically substitute month names when a numeric month and no
day are given.  Perhaps one day...

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Addison Phillips <[email-masked]>
Date: Sat, 15 Mar 2008 18:52:27 -0700

Editor hat on; responses follow.
> communication,such as" has been deleted.  Please add it back.

Done.
> the IANA Language Subtag Registry to perform most searching and matching

Done, although I edited the sentence somewhat in order to address this 
issue. It now reads:

--
Thus, a parser need not have a list of valid tags or subtags (such as a 
copy of some version of the IANA Language Subtag Registry) in order to 
perform common searching and matching operations.
--

I admit to some pain removing the pithy "or any copy at all".

> instance to "ISO 3166-1" unless the result would be simply wrong.

Done. I did a search-and-replace. Note that a few instances of naked 
"ISO 3166" remain. These are all references to ISO 3166 as a standards 
body or in general. When in doubt, I made it ISO 3166-1.

> new draft-4645bis?  A: Because the ISO 639-3 code lists have changed.)

(editor hat off) I think a declaration from the chairs on this item is 
in order. I thought it was at or near consensus myself.

(retrieves editor's hat)

> Change spelling of "very" to "vary".

Done.

> The reference to RFC 4324 looks erroneous, and probably should be RFC

It's a typo. Fixed. Also: replaced with an <xref> so that it'll stay fixed.

> been changed since it appeared in RFC 4646.

Actually, this is deliberate and probably NOT an error. RFC 4645 did, in 
fact, define the list in question. There is a reference to the document 
in normative language in Section 2.2.4, item 3, subitem E.

All references to 4645bis (aka "registry-update") are informative in 
that the document plays no role in any of the normative language of 
4646bis. I suppose a case could be made that instructions to IANA on 
where to get the updated registry would be normative, though.

Incidentally, RFC 4645 was an *informative* reference to RFC 4646. It 
was promoted because normative language was added in this version 
requiring a look at the list in 4645 in that one very special case.

> would leave it up to him to correct the spelling or form of his name,

Done.

> day are given.  Perhaps one day...

Done.

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 16 Mar 2008 00:16:33 -0700

Addison Phillips <addison at yahoo dash inc dot com> wrote:

>> "Thus, a parser need not have an up-to-date copy (or any copy at all) 
>> of the IANA Language Subtag Registry to perform most searching and 
>> matching operations."
>
> Thus, a parser need not have a list of valid tags or subtags (such as 
> a copy of some version of the IANA Language Subtag Registry) in order 
> to perform common searching and matching operations.
> --
>

Actually I prefer my version, which is not only higher in pith but also 
avoids giving the impression that any old list of valid tags or subtags 
is an acceptable substitute for the LSR.

An alternative way to address the forward-reference issue would be "... 
the IANA Language Subtag Registry (described in Section 3) to 
perform..."  But this is not critical.

> body or in general. When in doubt, I made it ISO 3166-1.

Perfect.

> in order. I thought it was at or near consensus myself.

Should I wait on draft-4645bis-05 then, or go ahead with it?

>> I was surprised to see RFC 4645, which pertained to RFC 4646, listed 
>> as a normative reference while draft-4645bis, which pertains to the 
>> present draft, is only listed as informative...
>
> Actually, this is deliberate and probably NOT an error. RFC 4645 did, 
> in fact, define the list in question. There is a reference to the 
> document in normative language in Section 2.2.4, item 3, subitem E.

OK.

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 17 Mar 2008 17:27:42 -0700

I also took a pass through the document after Doug did so. Mostly editorial,
but I found a conflict that I think needs to be resolved. Comments in [...]

MD40
In this format, all non-initial two-letter subtags are uppercase, all
non-initial four-letter subtags are titlecase, and all other subtags are
lowercase.
=>
In this format, all two-letter region subtags are uppercase, all script
subtags are titlecase, and all other subtags are lowercase.

[reason: otherwise forces two/four letter subtags in extensions and
grandfathered, to have unintended case. Also consistent with other text in
the RFC about casing. Note: I think all the grandfathered tags should be
lowercased, for consistency, since their subtags are not formally
analyzable.]

MD41

"de-CH" represents German ('de') as used in Switzerland ('CH').
=>
"de-CH" represents German ('de') as used in Switzerland ('CH'), also known
as Swiss High German. It is distinguished from "gsw-CH", which represents
Swiss German (Schwyzertuetsch)

MD42
Users MUST NOT assign and use their own subtags, other than private-use
sequences (such as "en-x-personal") or by using subtags designated as
private-use in the registry (such as "no-QQ", where 'QQ' is one of a range
of private-use ISO 3166-1 codes). Otherwise they risk finding later that
their previously unassigned subtag was assigned a meaning that conflicts
with their chosen usage.
=>
Users MUST NOT assign and use their own subtags, other than private-use
sequences (such as "en-x-personal") or by using subtags designated as
private-use in the registry (such as "no-QQ", where 'QQ' is one of a range
of private-use ISO 3166-1 codes). Not only are they nonconformant, they also
risk finding later that their previously unassigned subtag was assigned a
meaning that conflicts with their chosen usage.

MD43

The IANA Language Subtag Registry ("the registry") is a machine-readable
file in the format described in this section, plus copies of the
registration forms approved in accordance with the process described in
Section 3.5 (Registration Procedure for
Subtags)<http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registrationProc>.
The existing registration forms for grandfathered and redundant tags taken
from RFC 3066 will be maintained as part of the obsolete RFC 3066 registry.
The remaining set of subtags created by either [RFC4645] (Ewell, D.,
"Initial Language Subtag Registry," September
2006.)<http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#RFC4645>or
[registry‑update] (Ewell, D., Ed., "Update to the Language Subtag Registry,"
September 2006.)<http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registry-update>will
not have registration forms created for them.

[Didn't understand the last sentence.]

MD44
4.1.  Choice of Language Tag ...
5. The same variant subtag MUST NOT be used more than once within a language
tag.

   - For example, the tag "de-DE-1901-1901" is not valid.

[Remove this. This section is on choice, and not validity. Such a tag is
invalid -- we should not be listing arbitrary conditions from validity
here.]

MD45

Requests to add a prefix to a variant subtag that imply a different semantic
meaning will probably be rejected.
=>
Requests to add a prefix to a variant subtag that imply a different semantic
meaning SHOULD be rejected.

MD46
The 'und' (Undetermined) primary language subtag identifies linguistic
content whose language is not known.
=>
The 'und' (Undetermined) primary language subtag identifies linguistic
content whose language is not determined.

MD47
Thus a language tag and its subtags can encompass a very wide range of
variation and yet remain valid in each particular instance.
=>
Thus a language tag and its subtags can encompass a very wide range of
variation and yet remain appropriate in each particular instance.
[validity of language tags is formal (2.9), and means something different
than whether the tag is appropriate for given content. We can speak of the
"application of a language tag" being valid, or an implementation being
valid, but need to avoid talking about a language tag being valid unless we
mean the formal status.]

MD48
Of course, existing content or implementations that use these tags remain
valid.
=>
Of course, application to existing content remains valid, and
implementations that use these tags remain conformant.

=============

*MD 48 Conflict*

We have two notions of "goodness" defined in Section 2.9. Well-formedness,
and validity.

Well-formed. It is well-formed when it follows the ABNF in Section 2.1. That
is clear.

Valid. We then say in 2.2.9 that it is valid when 3 conditions hold:

A tag is considered "valid" if it well-formed and it also satisfies these
conditions:

   - The tag is either a grandfathered tag, or all of its language,
   script, region, and variant subtags appear in the IANA language subtag
   registry as of the particular registry date.
   - There are no duplicate singleton (extension) subtags and no
   duplicate variant subtags.
   - For each subtag that has a 'Prefix' field in the registry, the
   Prefix matches the language tag using Extended Filtering (Phillips, A.
   and M. Davis, "Matching of Language Tags," September
2006.)<http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#RFC4647>[RFC4647].
That is, each subtag in the Prefix is present in the tag and in
   the same order. Furthermore, all of the Prefix's subtags MUST appear before
   the subtag. For example, the Prefix "zh-TW" matches the tag "zh-Hant-TW".

We also define "valid" for a given extension.

We explicitly do not list that a valid tag also satisfy all of the MUSTs in
sections 2.2.1 through 2.2.9. That is because:

   1. some of those MUSTs are actually requirements on the registration
   policy -- given that MUSTs have been obeyed, the contents of the registry
   can be assumed to be cool.
   2. some of those MUSTs are just reflecting the ABNF (eg that the
   script subtag MUST follow the language subtag
   3. all the other MUSTs are for extension tag stuff (and we define
   "valid for an extension" separately)

After scanning through all these MUSTs, there are conflicts between 2.9 plus
3.1.2, which are in terms of MUST, and 2.2.5 and 3.1.2 which are SHOULD and
"appropriate".
2.2.5.  Variant Subtags...The 'Prefix' indicates the language tag or tags
that would make a suitable prefix (with other subtags, as appropriate) in
forming a language tag with the variant. That is, each of the subtags in the
prefix SHOULD appear, in order, before the variant...

3.1.7.  Prefix Field

The 'Prefix' field contains an extended language range whose subtags are
appropriate to use with this subtag: each of the subtags in one of the
subtag's Prefix fields MUST appear before the variant in a valid tag. For
example, the variant subtag '1996' has a 'Prefix' field of "de". This means
that tags starting with the sequence "de-" are appropriate with this subtag,
so "de-Latg-1996" and "de-CH-1996" are both acceptable, while the tag
"fr-1996" is an inappropriate choice.
 3.1.2.  Record Definitions...For example, the 'Prefix' for the variant
'nedis' is 'sl', meaning that the tags "sl-nedis" and "sl-IT-nedis" are
appropriate while the tag "is-nedis" is not.

We can't say both that something is a SHOULD or simply "appropriate", and
also that it is a MUST. We need to change this uniformly to be either MUST
and use the term "valid", or change this uniformly to be SHOULD and use the
term "appropriate". That is, the document as it stands is contradictory, and
someone could take either position on how Prefixes need to be treated by
conformant implementations. So we have to resolve this one way or another.

My recommendation would be uniformly SHOULD/appropriate, mostly because
prefixes are edge cases that simply aren't worth testing for. I'm perfectly
happy saying that an implementation isn't conformant if it doesn't check the
registry for subtag validity; but prefixes are just not so important that
I'd be happy saying that an implementation isn't conformant if it doesn't
check for them.

The result would be changing 3.1.7 MUST to SHOULD and in 2.9 would be the
following (with minor wordsmithing):
A tag is considered "valid" when it satisfies these conditions:

   - The tag is well-formed.
   - The tag is either a grandfathered tag, or all of its language,
   script, region, and variant subtags appear in the IANA language subtag
   registry as of the particular registry date.
   - The tag has no duplicate singleton (extension) subtags and no
   duplicate variant subtags.
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Addison Phillips <[email-masked]>
Date: Mon, 17 Mar 2008 18:11:58 -0700

(editor hat on except where noted)

Mark Davis wrote:
> grandfathered, to have unintended case. Also consistent with other text 
> in the RFC about casing.

(and that said: hat off)

I tend to agree. However... I have code that depends on this. Actually, 
my code knows about extensions and private-use and doesn't apply the 
titlecase or uppercase rules after a singleton. I'd prefer to keep it 
that way.

> Note: I think all the grandfathered tags should 
> be lowercased, for consistency, since their subtags are not formally 
> analyzable.]

True, although they all work correctly.

(retrieves hat, dusts is slightly)

DONE. Therefore, I made an edit so that the casing rules read:

--
In this format, all subtags, including all those following singletons 
(that is, in extension or private-use sequences) are in lowercase. The 
exceptions to this are:  all other non-initial two-letter subtags are 
uppercase and all other non-initial four-letter subtags are titlecase.
--

> "de-CH" represents German ('de') as used in Switzerland ('CH'), also 
> known as Swiss High German. It is distinguished from "gsw-CH", which 
> represents Swiss German (Schwyzertuetsch)

DONE. This sort of thing doesn't belong in an example of 'region' 
subtags. I changed the example to use 'AT' (Austria) instead.

(hat off)
If you feel strongly that we need to incorporate the de/gsw thing into 
an example, let's put it into the section on tag choice.

(hat on)

> private-use in the registry (such as "no-QQ", where 'QQ' is one of a 
> range of private-use ISO 3166-1 codes). Otherwise they risk finding 
> later that their previously unassigned subtag was assigned a meaning 
> that conflicts with their chosen usage.
> =>
> private-use in the registry (such as "no-QQ", where 'QQ' is one of a 
> range of private-use ISO 3166-1 codes). Not only are they nonconformant, 
> they also risk finding later that their previously unassigned subtag was 
> assigned a meaning that conflicts with their chosen usage.

NOT DONE (need guidance). I think all of the 'theys' in your suggested 
version are confusing. Can you explain your edit? I don't understand 
what you're trying to achieve.

> Section 3.5 (Registration Procedure for Subtags) 
> <http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registrationProc>. 
> The existing registration forms for grandfathered and redundant tags 
> taken from RFC 3066 will be maintained as part of the obsolete RFC 3066 
> registry. The remaining set of subtags created by either [RFC4645] 
> (Ewell, D., "Initial Language Subtag Registry," September 2006.) 
> <http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#RFC4645> or 
> [registry‑update] (Ewell, D., Ed., "Update to the Language Subtag 
> Registry," September 2006.) 
> <http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registry-update> 
> will not have registration forms created for them.
> 

DONE. The are no registration forms for the items inserted specifically 
by RFC 4645 or RFC 4645bis. Since they weren't separately registered, 
there are no forms for IANA to archive. This looks awkward here, I note.

Thus, I split the latter part of the para off and edited it to read:

--
<t>Note: The existing registration forms for grandfathered and redundant 
tags taken from RFC 3066 have been maintained as part of the obsolete 
RFC 3066 registry. The subtags added to the registry by either <xref 
target="RFC4645"></xref> or <xref target="registry-update"></xref> do 
not have separate registration forms (so no forms are archived for these 
additions).</t>
--

> 
> MD44
> 
> 
>       4.1.  Choice of Language Tag
> 
> ...
> 5. The same variant subtag MUST NOT be used more than once within a 
> language tag.
> 
>     * For example, the tag "de-DE-1901-1901" is not valid. 
> 

DONE. I moved this to Section 2.2.5, where it fits with other usage 
requirements for variants.

> 
> MD45
> 
> Requests to add a prefix to a variant subtag that imply a different 
> semantic meaning will probably be rejected.
> =>
> Requests to add a prefix to a variant subtag that imply a different 
> semantic meaning SHOULD be rejected.

NOT DONE (need guidance). I think this normative language is 
unnecessary, but agree that we need to be fairly clear here. Perhaps we 
could change 'probably' to 'usually' or just remove it altogether?

> content whose language is not determined.

NOT DONE (need guidance). Could we say "cannot be determined"?

> [validity of language tags is formal (2.9), and means something 
> different than whether the tag is appropriate for given content. We can 
> speak of the "application of a language tag" being valid, or an 
> implementation being valid, but need to avoid talking about a language 
> tag being valid unless we mean the formal status.]

DONE.

> 
> MD48
> Of course, existing content or implementations that use these tags 
> remain valid.
> =>

(hat off)

Actually, this example could use either, since we're now talking about 
redundant tags---they are both valid and conformant.

(hat on)

DONE. To address this, I changed it thusly:

--
Of course, these tags, where applied to existing content or in existing 
implementations, remain valid (all of their subtags are in the registry, 
after all), while new tags or applications using these subtags become 
possible.
--

======

I have elided MD48 because I need to read it more deeply and suspect it 
needs some discussion?

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 17 Mar 2008 22:15:39 -0700

> DONE. Therefore, I made an edit so that the casing rules read:
I can live with that. But then  also change

"sgn-BE-fr" / "sgn-BE-nl" / "sgn-CH-de"
=>
"sgn-BE-FR" / "sgn-BE-NL" / "sgn-CH-DE"

> an example, let's put it into the section on tag choice.

ok, let's do that. It helps make the point that codes aren't necessarily
what you might think, prima facie.

> what you're trying to achieve.

Ok. The  wording right now suggests that the only reason for not doing the
MUST NOT is because of the possibility of collision. There are two reasons.

   1. It is not compliant.
   2. It is risky.

In my rewording, I just changed the Otherwise into the first clause, and the
"they" is the same as your original "they". Perhaps:

Not only is such assignment nonconformant, it also risks collision with a
future possible assignment. The private use subtags and sequences are
designed for this case.

> > <
> http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registrationProc
> >.
> > <
> http://inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#registry-update
> >

good

If we need to be clear about this, it should be a SHOULD. Otherwise we
should remove it.

> NOT DONE (need guidance). Could we say "cannot be determined"?

No.  The word "undetermined" is its definition. That is not the same as
"cannot be determined" (just like "unhappy" is not the same as "cannot be
happy"). Nor is it the same as "unknown". We should use exactly equivalent
phrasing to its definition, unless we explicitly decide to broaden it.

> DONE. To address this, I changed it thusly:

ok

> I have elided MD48 because I need to read it more deeply and suspect it

Absolutely. We need to get consensus on it -- it is not as simple as the
above cases, and requires discussion.

-- 
Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 18 Mar 2008 06:59:31 -0700

Mark Davis wrote:

>>> Note: I think all the grandfathered tags should be lowercased, for 
>>> consistency, since their subtags are not formally analyzable.]
>>
> =>"sgn-BE-FR" / "sgn-BE-NL" / "sgn-CH-DE"

I disagree with systematically normalizing the capitalization of 
grandfathered tags.  They are grandfathered and shouldn't be touched. 
If we want to say that capitalization of tags doesn't matter, then we 
should not create this unnecessary diff as though it did matter.

In particular, tags like "sgn-BE-fr" are grandfathered because they 
could not possibly fit the ABNF.  Changing the capitalization of such 
tags in the Registry might give the impression that their syntax has 
been thereby legitimized in some way, when in fact all it does is make 
the tags conform to a distillation of the capitalization rules for 
regular tags.

Also, though I haven't checked thoroughly, I'm sure there are some 
examples somewhere in the draft of private-use tags and subtags and 
extensions with "non-standard" capitalization, and this is a Good Thing, 
since the meaning of these subtags is determined outside this BCP, and 
might be clearer if capitalized in a "non-standard" way.

---

From: Addison Phillips <[email-masked]>
Date: Tue, 18 Mar 2008 08:05:30 -0700

Doug Ewell wrote:

> grandfathered tags.  They are grandfathered and shouldn't be touched. If 
> we want to say that capitalization of tags doesn't matter, then we 

Section 4.4 (canonicalization) provides for doing the case folding 
described here to all language tags. Prescribing a different form for 
the grandfathered tags isn't very appealing, as it would require 
pointless special cases for implementers.

> the tags conform to a distillation of the capitalization rules for 

I don't see why it would give that impression: they are still defined in 
a production called "irregular".

> might be clearer if capitalized in a "non-standard" way.

This may be true, but the canonical form is lowercase. CamelCase is nice 
for some things, but extensions should not rely on it--or be encouraged 
to do so.

Best Regards,

Addison