Subject: [Ltru] Re: Registry change of the day: prefix with more than one subtag

From: "Doug Ewell" <[email-masked]>
Date: Mon, 9 Jul 2007 22:56:33 -0700

Addison Phillips <addison at yahoo dash inc dot com> replied to Stéphane 
Bortzmeyer:

>> First subtag(s) in the registry with a prefix which is more than one 
>> subtag. (And, yes, it triggered a bug in my programs.)
>
> You've got to think ahead with those unit tests :-). I must admit that 
> we've improved the text in draft-4646bis. In 4646 there is an example 
> (and a paragraph of text) identifying this case, but it is the section 
> on variant rather than in section 3.

Now It Can Be Told:  Stéphane's bug finally motivated me to, um, add 
complex-prefix support to my own tag-generating and -analyzing program, 
which I'd talked about quite a bit but had never released, due to (a) 
lack of said support and (b) lack of an installer package (which may be 
solved shortly).

While adding support for complex prefixes, I noticed that there is still 
some wiggle room in the rules about which variants should and should not 
be used with which prefixes.  Here is the relevant passage from 
draft-ietf-ltru-4646bis-06, Section 2.2.5, which is unchanged from RFC 
4646:

<<
Most variants that share a prefix are mutually exclusive. For example, 
the German orthographic variations '1996' and '1901' SHOULD NOT be used 
in the same tag, as they represent the dates of different spelling 
reforms. A variant that can meaningfully be used in combination with 
another variant SHOULD include a 'Prefix' field in its registry record 
that lists that other variant. For example, if another German variant 
'example' were created that made sense to use with '1996', then 
'example' should include two Prefix fields: "de" and "de-1996".

A human can easily see that "de-1901-1996" and "sl-rozaj-njiva" are 
meaningless, because the variants are clearly contradictory.  But a few 
variants with the same prefix can in fact be used together, such as 
"en-scouse-fonipa" or "en-boont-fonipa".  In fact, 'fonipa' and 'fonupa' 
have no prefix and can thus theoretically be used with any tag, but some 
combinations like "el-monoton-fonipa" are inappropriate nonetheless.

I can't think of a clean way to tell software which variants can be used 
together and which cannot, beyond the clear rule that all subtags in the 
prefix must be present -- for example, you can't use 'njiva' unless both 
'sl' and 'rozaj' are both present.

At http://users.adelphia.net/~dewell/prefix-anomaly.html you can find an 
illustration of this situation.  The drop-down list of variant subtags 
for "sl-rozaj" shows not only the reasonable choices, like 'fonipa' (no 
prefix) and 'njiva' (prefix = "sl-rozaj"), but also the inappropriate 
'nedis' (prefix = "sl").

What information is available in either RFC 4646(bis) or in the Registry 
that would help software to make this decision?  Does a prefix of 
"language with no variant" mean that the variant should not be used in 
the presence of any other variant?  How does one determine that 
"en-scouse-fonipa" is OK while "el-monoton-fonipa" is not?

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Tue, 10 Jul 2007 02:26:46 -0400

> "en-scouse-fonipa" is OK while "el-monoton-fonipa" is not?

I think there is no hope of getting this right automatically.  The
Prefix: field gives some guidance, but even it is only a SHOULD.
That said, people who use multiple prefixes probably wouldn't bother
unless they know what they are doing.  Indeed, the same can be
said of most kinds of generativity.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
   There was an old man                Said with a laugh, "I
     From Peru, whose lim'ricks all      Cut them in half, the pay is
       Look'd like haiku.  He              Much better for two."
                                             --Emmet O'Brien

---

From: Addison Phillips <[email-masked]>
Date: Tue, 10 Jul 2007 06:09:33 -0700

Doug Ewell wrote:
> "en-scouse-fonipa" or "en-boont-fonipa".  

Uh... fonipa has no prefix, so it doesn't share a prefix with either 
scouse or boont.

In fact, 'fonipa' and 'fonupa'
> combinations like "el-monoton-fonipa" are inappropriate nonetheless.

Yep. We have no mechanism for saying this. This would fall under the 
rubric of "Tag Content Wisely"
> prefix) and 'njiva' (prefix = "sl-rozaj"), but also the inappropriate 

Ur... nedis and rozaj share a prefix and thus fall under the general 
exclusion rule.

> the presence of any other variant?  

That is, in fact, what the RFC says. How else should we interpret:

 > Most variants that share a prefix are mutually exclusive.

How does one determine that
> "en-scouse-fonipa" is OK while "el-monoton-fonipa" is not?

Not via Prefixes. I'll note that "el-AQ-monoton" is also a dubious 
choice. There is an upper limit to the amount of information we can pack 
into the registry, I think. We could add the following text after the 
quoted paragraph though:

--
Some variants have no Prefix or have several different primary language 
prefixes. Such variants might be useful in combination with other, 
similar variants or with variants specific to a given language. This 
isn't always true and care is called for in selecting between these 
subtags. For example, a tag like "el-monoton-fonipa" is probably 
ill-considered, since both 'monoton' (Monotonic Greek) and 'fonipa' 
(International Phonetic Alphabet) can't both be used to transcribe the 
same document at the same time.
--

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 10 Jul 2007 07:35:48 -0700

Addison Phillips <addison at yahoo dash inc dot com> wrote:

>> meaningless, because the variants are clearly contradictory.  But a 
>> few variants with the same prefix can in fact be used together, such 
>> as "en-scouse-fonipa" or "en-boont-fonipa".
>

Sorry for being unclear.  'fonipa' does not have a prefix in the 
Registry, but it does have a prefix when used in actual tags such as 
"en-scouse-fonipa".

>> In fact, 'fonipa' and 'fonupa' have no prefix and can thus 
>> theoretically be used with any tag, but some combinations like 
>> "el-monoton-fonipa" are inappropriate nonetheless.
>

I agree that this is definitely a TCW situation.  Perhaps 
"en-scouse-boont" is also TCW and thus software can't be expected to 
make the "right" call.

>> ... The drop-down list of variant subtags for "sl-rozaj" shows not 
>> only the reasonable choices, like 'fonipa' (no prefix) and 'njiva' 
>> (prefix = "sl-rozaj"), but also the inappropriate 'nedis' (prefix = 
>> "sl").
>

Which is great for TCW.  But we may have variants in the future that 
have the same prefix -- that is, apply to the same language -- but deal 
with different aspects of "variation."  Right now we have variants for 
orthographies and for dialects.  Perhaps someday an alternative 
orthography will emerge for Armenian, which can be used with either 
Eastern or Western... you see where I'm headed.

>> Does a prefix of "language with no variant" mean that the variant 
>> should not be used in the presence of any other variant?
>

Then I hope an alternative orthography for Armenian never emerges.

> How does one determine that "en-scouse-fonipa" is OK while 
> "el-monoton-fonipa" is not?
>
> choice.

Not syntactically.  The RFC explicitly states that such combinations are 
doofy but allowable.  There is no mechanism and no wording that says 
"don't do this" as there is with Prefix.

> There is an upper limit to the amount of information we can pack into 
> the registry, I think.

No one agrees with that more than me.  I'm not looking to add more stuff 
to the Registry.  I'm looking for guidance from the RFC and the list.

> We could add the following text after the quoted paragraph though:
>
> --
> Some variants have no Prefix or have several different primary 
> language prefixes. Such variants might be useful in combination with 
> other, similar variants or with variants specific to a given language. 
> This isn't always true and care is called for in selecting between 
> these subtags. For example, a tag like "el-monoton-fonipa" is probably 

That's a good elaboration of TCW.

I guess the answer is that I have to allow "nedis" to be chosen even if 
"rozaj" has already been chosen, but I should mark it as questionable.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Thu, 12 Jul 2007 10:13:17 +0200

On Mon, Jul 09, 2007 at 10:56:33PM -0700,
 Doug Ewell <[email-masked]> wrote 
 a message of 72 lines which said:

> I can't think of a clean way to tell software which variants can be
> used together and which cannot,

I believe that we should handle this case under the general rule
"standards cannot prevent every stupid thing":

"Validity of a tag is not everything. A tag may be valid and
meaningless. This is unavoidable with a generative system like the
language subtag mechanism. So, ar-Cyrl-AQ - Arabic written with the
cyrillic script, as used in Antarctica - is perfectly valid but should
nevertheless be avoided because it has no relationship with the
reality."

There does not seem to be similar text in 4646bis. Do we think it
should be included or is it too paternalistic for readers?

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 12 Jul 2007 11:28:11 -0700

Hi -

As a technical contributor...

> From: "Stephane Bortzmeyer" <[email-masked]>
> To: "Doug Ewell" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Thursday, July 12, 2007 1:13 AM
> Subject: [Ltru] Re: Registry change of the day: prefix with more than onesubtag
>

Since the question seems to keep coming up, it's probably worth including.
For a heading for the section, I like your words: "standards cannot prevent
every stupid thing."

 I'd suggest putting it next to the section entitled "tag wisely."  :-)  * 0.5

Randy