Subject: Re: [Ltru] Addition to ISO 639-3: [lyg]

From: [email-masked]
Date: Tue, 29 Apr 2008 08:36:15 -0500

Hello LTRU,

If this situation with 'kha' and 'lyg' is a big problem, maybe we should 
consider getting clarification from ISO 639-3 on what circumstances will 
cause them to narrow the scope of a subtag, as we feel they have done 
with Khasi.

I expect this situation will apply only with a very few instances of the 
larger category of Splits. To handle the introduction of a new code 
element for a language variety that had previously been considered to be 
within the denotation of an existing code element (a split), we (now) have 
4 possible cases:
Original code element is retired and two or more new code elements are 
created that account for the language varieties previously within the 
denotation of the original.
Original code element is changed in scope to a Macrolanguage, and two or 
more code elements are created that more specifically denote the language 
varieties within the denotation of the one original; all the new code 
elements are members of the macrolanguage.
Original code element is changed in scope to a Collection, and two or more 
code elements are created that more specifically denote the language 
varieties previously within the denotation of the one original; all the 
new code elements are under the hierarchy of the collective code element.
The new code element is defined and the original code element which 
implicitly included the language variety of the newly defined element is 
explicitly, but minutely, narrowed to exclude it. 

The criteria for this last case to possibly apply, and the motivations for 
defining such a case, were set forth in the comment on the creation of 
[lyg], as that instance is a new precedent: 
1. Recognition as a separate language is justifiable on linguistic 
sociolinguistic grounds, just as would
apply to any proposed new code element;
2. Ethno- and sociolinguistic evidence demonstrates a distinct identity 
and a linguistic awareness of
speakers such that the "dialect" or low variety is not generally a result 
of attempting to use the
standard form with insufficient ability (though there may be some speakers 
of the low form who do
not have a good command of the high form);
3. The proportion of known instances of use of the existing code element 
to actually be intended to mean
the low variety in question can be reasonably estimated and is very low 
(on the order of 4%, or no
more than 2 in 50).
4. Sources dependent on and informing the use of ISO 639-2 (e.g., the MARC 
Code List for Languages,
the IANA Language Subtag Registry) do not explicitly link the variety in 
question to an existing code
element.

There have been 35 simple splits thus far, 2 changes of scope (both to 
Macrolanguage), and now this one instance of explicit narrowing. I expect 
the various future split instances that ISO 639-3 will be handling to be 
in roughly the same proportions. In general (and in all instances thus 
far), cases 2, 3, and 4 apply when a new code element meeting the criteria 
of Part 3 is proposed, but it is implicitly or explicitly already within 
the denotation of an individual language code element within ISO 639-2, 
where there is a very strong resistance to deprecating a code element for 
the purpose of a split.

Best regards,

Joan Spanne
ISO 639-3/RA
SIL International
7500 W Camp Wisdom Rd
Dallas, TX 75236
[email-masked]

"Doug Ewell" <[email-masked]> 
2008-04-27 01:05 PM

To
"LTRU Working Group" <[email-masked]>
cc
"Frank Ellermann" <[email-masked]>, <[email-masked]>
Subject
Re: Addition to ISO 639-3: [lyg]

(moving this topic to LTRU)

Debbie Garside <debbie at ictmarketing dot co dot uk> wrote:

>>> I would like to see kha with the comment as mentioned previously and 
>>> also, if possible, a comment on lyg saying "see also kha". Taken 
>>> with the change/registration date, this then provides a complete and 
>>> tracked change history within the registry.
>>
>> You'd have to convince the 4645bis author to do this, in the RFC 4646 
>> subtag registry ISO 639-3 subtags such as "lyg" don't exist, and a 
>> kha-comment talking about an unregistered lyg-subtag makes no sense.
>
> Yes this is a problem. The sooner RFC464*bis are published and ISO 
> 639-3 made available the better.  Perhaps we should await the 
> publication of these RFC's before we try to do anything.

I'm not opposed to adding Debbie's comment to the 'kha' subtag in 
draft-4645bis, as long as this is not written into the process.  I'd 
hate to see people scream "process failure" if Michael and I don't catch 
every similar instance in the future.  These are just comments, after 
all, and not normative in any way.

I agree with CE that a comment under 'lyg' seems unnecessary.  People 
who have been using 'kha' to tag Lynghgam may need to be directed to the 
new 'lyg' subtag, but I don't see the purpose in pointing people from 
'lyg' back to 'kha' (except to provide a 639-6-like taxonomy, which I 
assume will be adequately provided by 639-6).  We already can't provide 
a complete and tracked change history, because of the possibility of 
un-deprecating subtags and changing or removing Preferred-Values.

If nobody objects -- and soon, because I'm working on draft-4645bis-05 
today -- I'll add the 'kha' comment to the new Registry contents, 
without comment in the prose.

If this situation with 'kha' and 'lyg' is a big problem, maybe we should 
consider getting clarification from ISO 639-3 on what circumstances will 
cause them to narrow the scope of a subtag, as we feel they have done 
with Khasi.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 30 Apr 2008 23:39:24 -0600

Debbie Garside <debbie at ictmarketing dot co dot uk> wrote:

> Although I think that this should form part of the formal process. I 
> think it should be written into the RFC as a "Should" for this type of 
> situation.

I'm concerned about using the word "should" (with or without RFC 2119 
uppercasing) for anything relating to Comments fields.  If the content 
absolutely has to be there, or else the Registry can't be used properly, 
then they shouldn't be comments; there should be a new field (i.e. "See 
Also") for this sort of must-have information.  And I say this knowing 
how Randy feels about finishing off this work, and not adding new things 
to it, and I say this 99.9% agreeing with him.

> With a comment field within 'lyg' directing Joe to 'kha' he can do a 
> secondary match.  With no comment he has no chance of finding the 
> documents unless he knows enough about the LSR to do a search for 
> comments regarding 'lyg' (which would bring up my proposed comment 
> within 'kha' but only in this particular instance).

Not only that, but this use case expands our -- somebody's -- task of 
digging beneath the surface of ISO 639-3 changes even more than I 
anticipated.  This is because of the four different paths that ISO 
639-3/RA can take when splitting a language, as outlined by Joan Spanne.

Suppose Joe, on his second week with Company A (luckily not having been 
sacked over the Lyngngam headache), receives a new request to find 
documents relating to Rangpuri, one of two languages split off from what 
used to be called Rajbanshi (the other "new" language is also called 
Rajbanshi).  This was another of the 2007 changes, one that follows the 
first of Joan's four cases.  Joe can find 'rkt' for Rangpuri, but is 
unlikely to find any correlation to either the old Rajbanshi ('rjb') or 
the new Rajbanshi ('rjs').  The subtag 'rjb' would have been deprecated, 
either with no Preferred-Value or with a P-V of 'rjs', but probably not 
with a P-V of 'rkt'.

Thus we would need to supply comments not only for the narrowing cases, 
which Joan expects to be comparatively rare, but also for what she 
called the "simple splits" like Rajbanshi, which constitute a 
significant percentage of the churn in ISO 639-3.  The "merge" cases 
would probably have to be commented as well, for similar reasons.  In 
essence this is a reverse Preferred-Value, telling the user how the 
language *used to be* tagged, whereas Preferred-Value leads the user 
from the old subtag to the new one.

And while I would love to think the rate of churn in ISO 639-3 is 
dwindling to a trickle, the Change Request Index page suggests 
otherwise: 45 proposed language changes already since the publication of 
the 2007 changes four months ago.

> At the moment these comments may or may not be there according to the 
> rules of the current RFC.  I don't think this is good enough.  We can 
> do better and I don't think it will cost a lot to do it.  A simple 
> rule that states "where the meaning of a subtag is narrowed and a new 
> subtag is issued as a result both subtags should have comment fields 
> denoting the change history".  A simple "watch this page" scenario on 
> the ISO 639-3 change notification page would assist in the process.

They don't post their changes on an incremental basis, with explanation, 
as ISO 639-2/RA does.  Rather, they make entire new files available for 
download, with the changes in place but not annotated, and only once a 
year do they publish a Summary of Outcomes document which explains the 
nature of the change.  It is then for us -- by which I assume is meant 
the Designated Experts, Michael and Doug -- to do the exegesis on the 
downloadable files, deciding which changes in ISO 639-3 require this 
type of "obligatory" comment.  The downloadable files are wonderful, 
don't get me wrong, but a list of changes posted on a Web page, in date 
order, would be necessary to make good use of WatchThatPage.

I do use WatchThatPage to find out about the new 639-3 files, and I had 
expected to do some exegesis on them already, such as identifying 
deprecated language subtags and their Preferred-Values.  These 
suggestions add an unknown amount of exegesis to what I was already 
going to do.  It's unlikely anyone else would volunteer to do it, to be 
honest.

We could extend this concept even further, by adding Comments fields to 
the region subtags for all of the former Soviet republics, pointing them 
back to 'SU', so that Joe could find documents related to "Kurdish as 
used in Armenia" that were tagged before Armenia was independent.  This 
is not meant to be sarcastic, but is a logical outgrowth of adding 
comments to language subtags pointing them back to "previously coded as" 
values.  I don't know where it would end.