Subject: [Ltru] Re: Identifying script (or global) variants

From: "Doug Ewell" <[email-masked]>
Date: Tue, 20 Feb 2007 23:14:57 -0800

Mark Davis <mark dot davis at icu dash project dot org> wrote:

> My main point was that we have a problem with multiple variants, 
> because the ordering doesn't make a difference in 4646, but we don't 
> canonicalize to any order. The simplest way to do that is 
> alphabetical, which I think is fine. Doug wanted something different, 
> so I sketched what something could look like. But I'm fine with 
> alphabetical.

I may have missed something here.  If you mean "canonicalize" in the 
sense of something a computer or other process does for purposes of 
comparison, that can be completely invisible to humans, then 
alphabetical is fine.  It is certainly better than adding a bunch of 
arbitrary-looking integers, that are bound to attract the same 
criticisms as the similarly conceived Unicode combining classes.

If you mean to specify a canonical order for subtags to *appear* within 
a tag, then neither alphabetical nor "general before specific" will 
work, because alphabetical breaks the prefix rule (as Frank and I said) 
and language-specific variants may be more important to protect from 
right-truncation than generic ones.

Example 1: two specific variants

    ca-valencia-coastal
    ca-coastal-valencia

As Frank pointed out, the second is not equivalent to the first if 
"coastal" has a Prefix of "ca-valencia".

Example 2: one generic, one specific

    sl-nedis-fonipa
    sl-fonipa-nedis

Since these two variants are unrelated, the order makes no real 
difference, so processes can swap them internally with no ill effect. 
Frank is right, though, that users may choose different orderings based 
on their idea of importance, and cannot in any case be expected to write 
variants in alphabetical order.

Example 3 (hypothetical): two generic variants

    en-scouse-pitman
    en-pitman-scouse

Like Example 1, the subtags can be swapped only if they are unrelated, 
never if one variant is part of the Prefix of the other.  In the case of 
generic variants, this seems likely.  Note the imaginary variant 
"pitman", intended to be generic since many languages could 
theoretically be written in Pitman shorthand.

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 21 Feb 2007 07:20:24 -0800

It appears that you didn't see my message of Feb 20, 2007 2:10 PM

-- 
Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 22 Feb 2007 06:25:23 -0800

in which he wrote:

> In that case, I'd make canonicalization put all general variants 
> before all specific variants, and within those two groups, put them in 
> alphabetical order.

That doesn't solve the problem of rearranging "ca-valencia-coastal" to 
"ca-coastal-valencia", where "coastal" is assumed to be a subvariant of 
"ca-valencia" and has that as its Prefix.  All of the variants involved 
here are specific, not general, and so belong to the same alphabetically 
sorted group.

I assume that "specific" and "general" variants are defined as those 
that do and do not have a Prefix, respectively.

What I don't understand is why all this machinery is preferable to a 
statement in RFC 4646bis that for matching purposes ONLY, the order of 
variants should be ignored.

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 22 Feb 2007 07:08:30 -0800

But that is what canonicalization is for; to put the tag into a form that
can be matched. If the items are out of order, then 4747 won't match
correctly. If there is a problem with the process I suggested, then we work
to fix the process. Here is another round (informal language at this point):

V3. canonicalization sorts the variants in the following way. Given two
variants A and B, apply each of the following steps to A and B, then B and
A, stopping when there is a hit.
1. if A has no prefix and B has a prefix, A comes before B
2. if A is part of a prefix (given the previous non-variant subtags) for B,
A comes before B
3. if A is alphabetically before B, A comes before B.

This also exposes a couple of issues with multiple variants that we haven't
yet accounted for:

   - We don't have a validity constraint that a variant can't occur twice
   (as we do for extensions); we should add it.
   - We also should add the constraint on registration that we can't have
   a cycle, eg that we can't have A as a valid prefix for B and B a valid
   prefix for A.

Mark

---

From: Martin Duerst <[email-masked]>
Date: Fri, 23 Feb 2007 12:49:06 +0900

At 23:25 07/02/22, Doug Ewell wrote:

>What I don't understand is why all this machinery is preferable to a statement in RFC 4646bis that for matching purposes ONLY, the order of variants should be ignored.

We should not change RFC 4647 by a little note in RFC 4646bis.

Regards,   Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]