Subject: [Ltru] Two grandfathered tags for 'Hakka'

From: "Doug Ewell" <[email-masked]>
Date: Wed, 23 Jul 2008 00:47:03 -0600

In addition to the few pairs of language subtags that have the same 
Description, such that one or both will need to be modified to remove 
the ambiguity, we also have our old friends, the following two 
grandfathered tags:

Type: grandfathered
Tag: i-hak
Description: Hakka
Added: 1999-01-31
Deprecated: 2000-01-10
Preferred-Value: hak

Type: grandfathered
Tag: zh-hakka
Description: Hakka
Added: 1999-12-18
Deprecated: 2029-09-09
Preferred-Value: hak

It seems obvious to me that, just as we shouldn't need to change the 
Description field "Yiddish" for either 'he' or 'iw', because one is 
deprecated, we should not need to change "Hakka" for either of these 
tags, because *both* are deprecated.  Unfortunately, the wording in the 
current draft doesn't support that:

"... if a particular 'Description' field occurs in multiple records of a 
given type, then one of those records MUST be for a subtag listed in all 
the other records as Preferred-Value."

With the advent of the subtag 'hak', the tag "zh-hakka" will no longer 
be the Preferred-Value for "i-hak", so as it stands now, we would have 
to change the Description field for at least one of the deprecated 
grandfathered tags.

I'm sure there was a good reason why the rule was not simply stated as 
"deprecated subtags and tags don't count," but I can't think what it 
was.  I'd like to discuss revising this rule, so we don't have to do 
something silly.

--
Doug Ewell  *  Thornton, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Wed, 23 Jul 2008 08:17:32 -0700

We changed this draft from 4646 to update the Preferred-Values to point to the "last item" in the chain (one of these used to point to the other). I changed the rules to say:

<t>To help ensure that users do not become confused about which subtag to use, 'Description' fields assigned to a record of any specific type ('language', 'extlang', 'script', and so on)  MUST be unique within that given record type with the following exception: if a particular 'Description' field occurs in multiple records of a given type, then at most one of the records can omit the 'Deprecated' field and all records that share a 'Description' MUST have the same 'Preferred-Value'. This means that two records of the same type that share a 'Description' are also semantically equivalent and no more than one record with a given 'Description' is preferred for that meaning.</t>

Addison

Addison Phillips
Globalization Architect -- Lab126

Internationalization is not a feature.
It is an architecture.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Doug Ewell
> Sent: Tuesday, July 22, 2008 11:47 PM
> To: LTRU Working Group
> Subject: [Ltru] Two grandfathered tags for 'Hakka'
>

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 23 Jul 2008 23:06:30 -0700

Hi -

As a technical contributor...

> From: "Doug Ewell" <[email-masked]>
> To: "LTRU Working Group" <[email-masked]>
...
> the other records as Preferred-Value."
...
> was.  I'd like to discuss revising this rule, so we don't have to do 
...

I agree that what we have at the moment seems well-intended but
problematic.  The problem is that records of the grandfathered
and redundant types can refer to records of another type.

Randy

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 24 Jul 2008 21:45:38 -0600

"Phillips, Addison" <addison at amazon dot com> wrote:

> <t>To help ensure that users do not become confused about which subtag 
> to use, 'Description' fields assigned to a record of any specific type 
> ('language', 'extlang', 'script', and so on)  MUST be unique within 
> that given record type with the following exception: if a particular 
> 'Description' field occurs in multiple records of a given type, then 
> at most one of the records can omit the 'Deprecated' field and all 
> records that share a 'Description' MUST have the same 
> 'Preferred-Value'. This means that two records of the same type that 
> share a 'Description' are also semantically equivalent and no more 
> than one record with a given 'Description' is preferred for that 
> meaning.</t>

We're almost there.  This needs to be:  "... and all deprecated records 
that share a 'Description' MUST have the same 'Preferred-Value'."  Or 
maybe, "... and all records that share a 'Description' MUST have the 
same 'Preferred-Value', or none at all."  The way it reads now, 'he' and 
'iw' would have to have different descriptions because they do not have 
the same Preferred-Value (since 'he' has none).

We want different descriptions when the semantic is different.  I'm 
assuming here that unless there is a clear withdrawal/replacement chain 
from the ISO MA or RA, then the semantic is different.  When ISO 639-3 
gives us Borna (bwo, Ethiopia, also called Boro) and Borna (bxx, 
Democratic Republic of Congo), they are presumed different.  Hopefully 
we will be able to capture that intent in our rule.

Actually, we have another exception to the deprecated/preferred rule. 
All four of the region subtag records with Description field "PRIVATE 
USE" (AA, QM..QZ, XA..XZ, ZZ) need to be exempt from the 
unique-description rule.  Again, this is because the semantic is the 
same for all four.

---

From: John Cowan <[email-masked]>
Date: Fri, 25 Jul 2008 00:22:58 -0400

> unique-description rule.  Again, this is because the semantic is the 

+1.

Perhaps the groups should be unpacked now?  An extra 38 records is
nothing compared to the 6000+ we are adding.

-- 
Deshil Holles eamus.  Deshil Holles eamus.  Deshil Holles eamus.
Send us, bright one, light one, Horhorn, quickening, and wombfruit. (3x)
Hoopsa, boyaboy, hoopsa!  Hoopsa, boyaboy, hoopsa!  Hoopsa, boyaboy, hoopsa!
  --Joyce, Ulysses, "Oxen of the Sun"       [email-masked]
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 24 Jul 2008 22:33:25 -0600

> nothing compared to the 6000+ we are adding.

I wouldn't object, inasmuch as it would simplify LSR parsing, but that 
wasn't quite my point.  Whether we have four PRIVATE USE* records or 42, 
they should still have the same Description field since they have the 
same semantic.  (None of this artificial "PRIVATE USE-AA" junk.)

*I never understood why this particular Description field had to be in 
ALL CAPS, either.

---

From: John Cowan <[email-masked]>
Date: Fri, 25 Jul 2008 00:36:25 -0400

> *I never understood why this particular Description field had to be in 

You're the editor of 4645bis.  Fix 'em.

-- 
He played King Lear as though           John Cowan <[email-masked]>
someone had played the ace.             http://www.ccil.org/~cowan
        --Eugene Field
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 24 Jul 2008 22:49:52 -0600

John Cowan <cowan at ccil dot org> wrote:

>> *I never understood why this particular Description field had to be 
>> in ALL CAPS, either.
>

I'm a democratic editor; I do the bidding of the WG.

Fine: unless I hear otherwise in the next 72 hours (Monday, June 28 at 
5:00 UTC), I will change the four "PRIVATE USE" Descriptions to the 
calmer "Private Use".

Does anyone else support unpacking the subtag records that represent 
ranges?  Bear in mind that in addition to the regions, we would also get 
520 language subtags out of "qaa..qtz" and 50 script subtags out of 
"Qaaa..Qabx".

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 25 Jul 2008 10:29:31 +0200

> 5:00 UTC), I will change the four "PRIVATE USE" Descriptions to the 

I don't like the "titlecase" here, there is no name. Describing
these as "private use" (all lowercase) will be fine.

> 520 language subtags out of "qaa..qtz" and 50 script subtags out of 

No objection.

I would also not object (understatement...) if click letters were
spelled with the actual click letters in Unicode, rather than
keeping the ASCII fallbacks (in e.g. Description: =/Kx'au//'ein,
and a few others). (I've pointed this out a few times before.)

	/kent k

---

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 25 Jul 2008 08:13:30 -0700

I support unpacking and also support fixing case.

Addison

Addison Phillips

> Sent: Thursday, July 24, 2008 9:50 PM
> To: LTRU Working Group
> Subject: Re: [Ltru] Two grandfathered tags for 'Hakka'
>

---

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 25 Jul 2008 11:42:50 -0700

> the same Preferred-Value (since 'he' has none).

"none at all" doesn't work because that would allow any two non-deprecated (non-identical) items to share a description.

"all deprecated" doesn’t quite work either. The problem is that the non-deprecated item also has to share the description.

This gist of this is: all records that share a description must have the same preferred value. Not "preferred value *field*". The same *value*. So both 'iw' and 'he' prefer 'he' as the value. :-).

So the text ends up very similar to your suggestion:

<t>To help ensure that users do not become confused about which subtag to use, 'Description' fields assigned to a record of any specific type ('language', 'extlang', 'script', and so on)  MUST be unique within that given record type with the following exception: if a particular 'Description' field occurs in multiple records of a given type, then at most one of the records can omit the 'Deprecated' field; all deprecated records that share a 'Description' MUST have the same 'Preferred-Value'; and all non-deprecated records MUST be that 'Preferred-Value'. This means that two records of the same type that share a 'Description' are also semantically equivalent and no more than one record with a given 'Description' is preferred for that meaning.</t>

> Democratic Republic of Congo), they are presumed different.
> Hopefully we will be able to capture that intent in our rule.

Yikes. No, our rules don't appear to handle that. I'm not surprised that two (or more) languages use the same (English-like) string as their name. But it is difficult to disambiguate the two if that's all you have---which is the point of the "no duplicates" rule. I would expect that we'd edit these Description fields in the registry to clarify which is which, since we say:

--
Descriptions SHOULD contain all and only that information necessary
to distinguish one subtag from others that it might be confused with.
--

Addison

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 26 Jul 2008 12:21:30 -0600

Phillips, Addison <addison at amazon dot com> wrote:

>> We're almost there.  This needs to be:  "... and all deprecated 
>> 'Preferred-Value'." Or maybe, "... and all records that share a 
>> 'Description' MUST have the same 'Preferred-Value', or none at all." 
>> The way it reads now, 'he' and 'iw' would have to have different 
>> descriptions because they do not have the same Preferred-Value (since 
>> 'he' has none).
>
> "none at all" doesn't work because that would allow any two 
> non-deprecated (non-identical) items to share a description.

This passage would have immediately followed "at most one of the records 
can omit the 'Deprecated' field."  So there would not be two such items. 
For clarity, I probably should have said "and all *such* records that 
share..."

> "all deprecated" doesn’t quite work either. The problem is that the 
> non-deprecated item also has to share the description.

There might not be a non-deprecated item within that record type: 
witness "i-hak" and "zh-hakka".

Technically, these two grandfathered tags ("Hakka") are deprecated in 
favor of a language subtag ("Hakka Chinese") that has the same meaning 
but does not share the exact same description.  This is not a problem. 
After all, region subtag BU ("Burma") is deprecated in favor of MM 
("Myanmar") and I would hope nobody would suggest editing one of those 
two Description fields to restate what the P-V already states.

> This gist of this is: all records that share a description must have 
> the same preferred value. Not "preferred value *field*". The same 
> *value*. So both 'iw' and 'he' prefer 'he' as the value. :-).

That's tricky, and you might have to be careful to explain the 
difference between "preferred value" and "Preferred-Value," but it does 
work.

> at most one of the records can omit the 'Deprecated' field; all 
> deprecated records that share a 'Description' MUST have the same 
> 'Preferred-Value'; and all non-deprecated records MUST be that 

I think that's technically fine and plugs all the holes.  As an 
editorial nit, I would prefer to see:

"...and all non-deprecated records MUST have a 'Tag' or 'Subtag' value 
equal to that 'Preferred-Value'."

>> When ISO 639-3 gives us Borna (bwo, Ethiopia, also called Boro) and 
>> Borna (bxx, Democratic Republic of Congo), they are presumed 
>> different.  Hopefully we will be able to capture that intent in our 
>> rule.
>
> Yikes. No, our rules don't appear to handle that.

Well, that's the whole point of this exercise.  The duplication of 
"Borna" et al. is the problem we are trying to solve, and the 
duplications of "Yiddish" and "Hakka" et al. are the non-problems that 
prevent the solution from being simple and straightforward.

> I'm not surprised that two (or more) languages use the same 
> (English-like) string as their name. But it is difficult to 
> disambiguate the two if that's all you have---which is the point of 
> the "no duplicates" rule. I would expect that we'd edit these 
> Description fields in the registry to clarify which is which, since we 
> say:
>
> --

Correct, but what we're trying to do is state concrete rules for editing 
these fields.  As a general rule, we avoid modifying Description fields 
derived from ISO names; here we are justifying why we have to bend that 
rule.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 26 Jul 2008 12:40:31 +0900

I'm fine with "Private Use" or "private use", much better than
"PRIVATE USE". On the other hand, I don't think it's a good idea
to make our registry even longer by expanding these. It would expand
our registry by about 10%.

Regards,   Martin.

At 13:49 08/07/25, Doug Ewell wrote:
>Does anyone else support unpacking the subtag records that represent >ranges?  Bear in mind that in addition to the regions, we would also get >520 language subtags out of "qaa..qtz" and 50 script subtags out of >"Qaaa..Qabx".

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: John Cowan <[email-masked]>
Date: Sun, 27 Jul 2008 23:45:08 -0400

> our registry by about 10%.

I don't think a 10% expansion is a bad trade-off, considering how
big the registry is going to get, for simpler parsing.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
Heckler: "Go on, Al, tell 'em all you know.  It won't take long."
Al Smith: "I'll tell 'em all we *both* know.  It won't take any longer."
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 28 Jul 2008 00:20:07 -0700

I'm a bit split on this. I'm generally in favor of simpler parsing, but the
effort needed to parse the ranges used for Private Use is extremely small --
only a couple of (straightforward) lines of code.
So I could live with it either way.

Mark

---

From: Martin Duerst <[email-masked]>
Date: Wed, 30 Jul 2008 15:21:00 +0900

One additional argument is that people who hit the PU section
of the registry will shake their head and say "well, that's
why this page is so huge". They are of course wrong, but
I'd rather make people looking at the registry look at the
actual data (actual languages!) as much as possible.

Regards,    Martin.

At 16:20 08/07/28, Mark Davis wrote:
>I'm a bit split on this. I'm generally in favor of simpler parsing, but the effort needed to parse the ranges used for Private Use is extremely small -- only a couple of (straightforward) lines of code.
>
>On Sun, Jul 27, 2008 at 8:45 PM, John Cowan <<mailto:[email-masked]>[email-masked]> wrote:
>Martin Duerst scripsit:
>John Cowan    <mailto:[email-masked]>[email-masked]    http://ccil.org/~cowan
><mailto:[email-masked]>[email-masked]