Subject: [Ltru] Extended language subtags

From: "Doug Ewell" <[email-masked]>
Date: Sun, 27 Aug 2006 22:00:55 -0700

Randy and Martin both wrote on [email-masked] that we can begin 
discussing issues related to RFC 3066ter on this list, as long as we 
don't attempt to claim consensus on any of the issues.

That said, I'd like to start the broad discussion about extended 
language ("extlang") subtags.  I doubt we will reach consensus on all of 
this before the charter is approved.

My understanding, for well over a year, has been that languages that are 
included under a macrolanguage in ISO 639-3 would be mapped to extlang 
subtags, with that macrolanguage as their Prefix, while all other 
languages in 639-3 would be mapped to primary language subtags.  (Those 
languages that have an ISO 639-1 alpha-2 code element would continue to 
use it as the subtag instead of the corresponding alpha-3 code element, 
for compatibility with RFC 1766 and 3066.)

Using this model, the initial RFC 3066ter Registry would include 
approximately 7200 primary language subtags and approximately 350 
extlang subtags.

As recent discussions have shown, we will need to figure out what to do 
with the Registry if and when the ISO 639-3 mappings between individual 
languages and macrolanguages change.  There is plenty of lively debate 
about that and I don't plan to rehash it here.

One topic we have not addressed yet, which may involve extlang subtags, 
is sign languages.  ISO 639-3 lists 124 languages with "Sign Language" 
in their name.  It does not include the ISO 639-2 code element "sgn" 
("Sign Languages"), which denotes a collection of languages and is 
therefore out of scope for 639-3 (but which does exist in the RFC 
3066bis Registry).

There is a precedent in RFC 3066 for using the "sgn" code element 
together with an ISO 3166 country code element (plus optionally other 
symbols) to tag specific sign languages.  The classic example is 
"sgn-US" for American Sign Language (I will use it as my example for 
convenience only).  This system was devised by Michael Everson and the 
Deaf Action Committee for Sign Writing, and is described at 
http://www.evertype.com/standards/iso639/sgn.html.  Some of the 
sgn-combinations listed on that page are already registered under RFC 
3066, and consequently they are listed as either grandfathered or 
redundant in the RFC 3066bis Registry; others are not.

The ISO 639-3 code element for American Sign Language is "ase".  ("asl" 
is the code element for Asilulu, a language of Indonesia.)  If we 
convert such code elements directly to primary language subtags, as we 
will likely do for ordinary spoken/written languages that do not have a 
macrolanguage, then the RFC 3066ter tag for ASL would be "ase".

Alternatively, we could choose to add all 639-3 code elements for sign 
languages as *extlang* subtags, with a Prefix of "sgn".  Using this 
model, the RFC 3066ter tag for ASL would be "sgn-ase".  This approach 
would have the advantage (if such it is) that the sign-language nature 
would be indicated as part of the tag.  This does not necessarily imply 
any mutual compatibility between "sgn-xxx" languages, any more than the 
RFC 3066 scheme does.

In either case, the existing grandfathered tag "sgn-US" would presumably 
be deprecated and given a Preferred-Value of either "ase" or "sgn-ase".

I understand that Michael may not favor this approach, and may prefer to 
continue using the "sgn-US" system.  There is potentially some material 
already tagged using these RFC 3066 identifiers.  One significant 
drawback is that tags using this model that go beyond "sgn plus region" 
are not compatible with the RFC 3066bis syntax:

sgn-AU-NT -- uses an ISO 3166-2 country-subdivision code
sgn-IL-yi  -- uses an ISO 639-1 language code
sgn-SE-crp -- uses an ISO 639-2 language code

Another drawback, at least in my view, is that the generative model 
tells us that "sgn-XX" means "sign languages as used in region XX", 
which can be misleading in regions where multiple sign languages are in 
common use.  There is a strong identity between the concepts "American 
Sign Language" and "sign languages as used in the United States," but 
this may not hold for other sign languages.

It's important to remember that "deprecated" tags are not invalid; they 
are simply not recommended for future tagging.

Again, our goal right now needs to be to raise questions and consider 
scenarios, not to declare a Final Answer.

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
Editor, draft-ietf-ltru-initial

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Peter Constable" <[email-masked]>
Date: Sun, 27 Aug 2006 22:51:47 -0700

I think you need to decide: are we discussing extended language subtags, or the tagging of signed languages? Those are two different topics, and I think you're discussing the latter.

If we were to encode the 639-3 IDs for signed languages as extended language subtags with a prefix of "sgn", that would be an ad-hoc exception to the otherwise-applicable principle that every extended language subtag has a macrolanguage mapping to its designated prefix.

My inclination has always been to treat signed languages like any other language, meaning that we use their 639-3 IDs as primary language subtags. If Michael and the DAC want some way to indicate that content is in Sign Writing, the appropriate way to deal with that is to create a script ID in 15924 and use that -- i.e. tags like "ase-sgnw". 

(I find it odd that Michael saw that 15924 IDs were added for scripts that are long dead but for all his supposed interest in signed languages and Sign Writing there isn't an ISO 15924 ID for a single writing system used in relation to signed languages.)

Peter Constable

> -----Original Message-----
> From: Doug Ewell [mailto:[email-masked]]
> Sent: Sunday, August 27, 2006 10:01 PM
> To: LTRU Working Group
> Subject: [Ltru] Extended language subtags
>

---

From: John Cowan <[email-masked]>
Date: Mon, 28 Aug 2006 10:21:59 -0400

Peter Constable scripsit:

> My inclination has always been to treat signed languages like any
> other language, meaning that we use their 639-3 IDs as primary language
> subtags. 

This entirely contradicts your remarks of last February: see
http://www.alvestrand.no/pipermail/ietf-languages/2006-February/004061.html
I support your previous position.

> (I find it odd that Michael saw that 15924 IDs were added for scripts
> that are long dead but for all his supposed interest in signed
> languages and Sign Writing there isn't an ISO 15924 ID for a single
> writing system used in relation to signed languages.)

Michael isn't subscribed here, AFAIK.  I shall contact him
directly, not mentioning your name of course.

-- 
John Cowan                              [email-masked]
            http://www.ccil.org/~cowan
Humpty Dump Dublin squeaks through his norse
                Humpty Dump Dublin hath a horrible vorse
But for all his kinks English / And his irismanx brogues
                Humpty Dump Dublin's grandada of all rogues.  --Cousin James

---

From: "Peter Constable" <[email-masked]>
Date: Mon, 28 Aug 2006 08:21:08 -0700

> From: John Cowan [mailto:[email-masked]]

> http://www.alvestrand.no/pipermail/ietf-languages/2006-

Heh, heh... Caught in my error!

I completely forgot about that. OK, I think I was long inclined to prefer treating signed languages like other languages -- with primary subtags. But, in view of what I wrote last Feb, which was based on discussion with someone I believes understands the signed-language space, I'm willing to go along with using sgn as a prefix for signed languages as per what I wrote in that message. 

But, we should call that out in 3066ter as a special case in exception to the otherwise-applicable principle that extended language subtags must have a macrolanguage mapping in 639 to their prefix.

Peter Constable

---

From: John Cowan <[email-masked]>
Date: Mon, 28 Aug 2006 11:55:48 -0400

Peter Constable scripsit:

> I completely forgot about that. OK, I think I was long inclined to
> prefer treating signed languages like other languages -- with primary
> subtags. But, in view of what I wrote last Feb, which was based on
> discussion with someone I believes understands the signed-language
> space, I'm willing to go along with using sgn as a prefix for signed
> languages as per what I wrote in that message.

Similar things have happened to me often enough.  "A man convinced
against his will / Is of the same opinion still."  [*]

> But, we should call that out in 3066ter as a special case in exception
> to the otherwise-applicable principle that extended language subtags
> must have a macrolanguage mapping in 639 to their prefix.

Absolutely.

[*] Pedantic footnote:  This is a common misquotation of two lines from
Samuel Butler's _Hudibras_ that IMHO has more psychological depth than
the original, which says "He who complies ...".  External coercion is
one thing, coercion by reason another -- but it still feels like coercion.

-- 
Cash registers don't really add and subtract;           John Cowan
        they only grind their gears.                    [email-masked]
But then they don't really grind their gears, either;   
        they only obey the laws of physics.  --Unknown

---

From: John Cowan <[email-masked]>
Date: Mon, 28 Aug 2006 12:41:42 -0400

> any mutual compatibility between "sgn-xxx" languages, any more than the 

I support this model.

> be deprecated and given a Preferred-Value of either "ase" or "sgn-ase".

The tag "sgn-US" is actually redundant rather than grandfathered.

> this may not hold for other sign languages.

And indeed vice versa:  American Sign Language is used in Canada as well.

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
In the sciences, we are now uniquely privileged to sit side by side
with the giants on whose shoulders we stand.
        --Gerald Holton

---

From: John Cowan <[email-masked]>
Date: Mon, 28 Aug 2006 12:59:00 -0400

> about that and I don't plan to rehash it here.

Well, I will, then.  :-)

In ISO 639-3, code elements belong to one of three types:  macrolanguages,
languages encompassed by a macrolanguage, and languages not encompassed
by a macrolanguage.

Assuming we go with the model that macrolanguages and unencompassed
languages become language subtags, and encompassed languages become
extlang subtags, then there are a variety of changes in state that can
occur within 639-3 that we need to reflect in different ways.  ("We"
in all cases means ietf-languages.)

1) A new unencompassed language is added.  We add a new language subtag.

2) A new encompassed language is added.  We add it as an extlang tag,
and add the macrolanguage as a language tag if it is not already present.

3) A new macrolanguage is added, encompassing either already coded
languages or new languages or both.  We add a new language subtag for
the macrolanguage itself.  We treat the encompassed languages thus:

	a) If the language is already in the registry, we do nothing;
	we do not deprecate it as a language subtag or add it as an
	extlang subtag.  [This is a change from my previous position.]

	b) If the language is new, we add it as an extlang subtag.

4) An existing language is reclassified as a macrolanguage, encompassing
either already encoded languages or new languages or both.  We do not
change the existing language subtag.  The languages it encompasses are
treated as in case 3.  If the new macrolanguage is already encompassed,
then we have a nested macrolanguage; nothing special is done except that
any new languages it encompasses will have a two-part Prefix setting.

5) An existing unencompassed language is reclassified as encompassed by
a macrolanguage.  We do nothing.

6) An existing encompassed language is reclassified as unencompassed.
We do nothing.

7) An existing encompassed language is reclassified as encompassed by
a different macrolanguage.  We do nothing.

These proposals satisfy the axioms "Once a language tag, always a
language tag" and "Once an extlang tag, always an extlang tag."

--
"The serene chaos that is Courage, and the phenomenon   [email-masked]
of Unopened Consciousness have been known to the        John Cowan
Great World eons longer than Extaboulism."
"Why is that?" the woman inquired.
"Because I just made that word up", the Master said wisely.
        --Kehlog Albran, The Profit             http://www.ccil.org/~cowan

---

From: Addison Phillips <[email-masked]>
Date: Mon, 28 Aug 2006 10:57:20 -0700

> language tag" and "Once an extlang tag, always an extlang tag."

And go a bit further, in that they also adhere to a desire for stability 
extant in RFC 3066bis: "Once a preferred tag for a language, always the 
preferred tag for that language."

Note that deprecation of subtags in the registry tends to inform the 
user that "facts on the ground" have changed. Thus, 'NH' is deprecated 
because that "region" no longer exists. "en-NH" is still, on some level, 
the preferred tag for that language, but that language doesn't have a 
lot of meaning.

> Well, I will, then.  :-)

Good. In general, +1 to your summary.

> and add the macrolanguage as a language tag if it is not already present.

Not quite. The case is:

   "A new encompassed language is added. We add it as an extlang subtag 
with a prefix of its enclosing language subtag sequence."

If the macrolanguage is not present, the case is #1, as subtags can only 
be enclosed by other subtags already in the registry. This suggests 
that, when processing a list of new items, all macrolanguages MUST be 
processed first (at least mentally).

> change the existing language subtag. 

... change the existing language or extlang subtag.

> a different macrolanguage.  We do nothing.

Hm... an encompassed language is already an extlang. It could be given 
an additional Prefix. The existing Prefix would remain, however. I'm 
pretty sure this isn't a really hot idea, but it is a useful fudge, perhaps?

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: John Cowan <[email-masked]>
Date: Mon, 28 Aug 2006 15:04:23 -0400

> processed first (at least mentally).

Yes.  I was trying to cover the case where a macrolanguage and its encompassed
languages are added simultaneously.

> ... change the existing language or extlang subtag.

+1

> pretty sure this isn't a really hot idea, but it is a useful fudge, perhaps?

I think it's a very bad idea: it makes aaa-xxx and bbb-xxx synonymous, which
is something we avoid.  New axiom:  "No sets of language tags are synonymous,
unless all but one of them are deprecated."

-- 
John Cowan  [email-masked]   http://ccil.org/~cowan
Assent may be registered by a signature, a handshake, or a click of a computer
mouse transmitted across the invisible ether of the Internet. Formality
is not a requisite; any sign, symbol or action, or even willful inaction,
as long as it is unequivocally referable to the promise, may create a contract.
       --Specht v. Netscape

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 29 Aug 2006 06:57:17 +0200

John Cowan wrote:

>>> 7) An existing encompassed language is reclassified as
>>> encompassed by a different macrolanguage.  We do nothing.

>> Hm... an encompassed language is already an extlang. It
>> could be given an additional Prefix. The existing Prefix 
>> would remain, however. I'm pretty sure this isn't a really
>> hot idea, but it is a useful fudge, perhaps?

> I think it's a very bad idea: it makes aaa-xxx and bbb-xxx
> synonymous, which is something we avoid.  New axiom:  "No
> sets of language tags are synonymous, unless all but one of
> them are deprecated."

Here's what we have in 3066bis 2.2.2:

| Extended language subtag records, once they appear in the
| registry, MUST include exactly one 'Prefix' field indicating
| an appropriate language subtag or sequence of subtags that
| MUST always appear as a prefix to the extended language
| subtag.

We've guaranteed that 3066bis implementations will work with
3066ter.  Let's stick to that plan as long as possible.

Frank

---

From: "Doug Ewell" <[email-masked]>
Date: Mon, 28 Aug 2006 22:30:56 -0700

John Cowan <cowan at ccil dot org> wrote:

>> Alternatively, we could choose to add all 639-3 code elements for 
>> sign languages as *extlang* subtags, with a Prefix of "sgn"...
>
> I support this model.

So do I.  What I like about it is what Peter expressed in his February 
post:

"Having 'sgn' at the start immediately sets apart the modality, which 
will generally trump any other distinction in importance to the user."

That is, the fact that ASL and BSL are both "sign language" gives them 
much more in common with each other than with any spoken or written 
language.  This is similar to the way Cantonese and Mandarin are both 
identified as "Chinese," even by speakers of one who don't understand 
the other.

The only difference is that Sign Language per se isn't identified as a 
macrolanguage (or indeed at all) in ISO 639-3.  Peter is absolutely 
right that if we make all the sign languages extlangs under "sgn", then 
we need to call attention to this exceptional case in the RFC text. 
That's not a reason for not doing it, though.

>> In either case, the existing grandfathered tag "sgn-US" would 
>> presumably be deprecated and given a Preferred-Value of either "ase" 
>> or "sgn-ase".
>

Yes.  Well, yes and no.  The tag "sgn-US" itself is grandfathered, but 
in a very real sense the RFC 3066 registered meaning "American Sign 
Language" is very much grandfathered.  It is not the same thing as "sign 
languages used in the United States."  Plains Indian Sign Language is 
used in, and indigenous to, the U.S.