Subject: [Ltru] "X" vs. 'X (macrolanguage)"

From: "Doug Ewell" <[email-masked]>
Date: Fri, 7 Dec 2007 22:09:35 -0800

As John pointed out, the first paragraph of Section 3.1.4 in 
draft-4646bis-10 disallows two or more "formatting variations of the 
same description," which basically means that for each subtag, either a 
non-inverted form like "Church Slavic" or an inverted form like "Slavic, 
Church" is allowed, but not both.

For 4645bis, I will be adding only the non-inverted forms of language 
subtags taken from the ISO 639-3 files, and will uninvert the existing 
descriptions ("Sotho, Southern") or remove inverted forms where both 
currently exist ("Ndebele, North" and "North Ndebele") for consistency.

There's only one source of major duplication in the Registry now, and 
that involves the language names that exist as both a macrolanguage and 
an individual language.  Swahili is an example:

Id Part2B Part2T Part1 Scope Language_Type Ref_Name Comment
swa swa swa sw M L Swahili (macrolanguage)
swh    I L Swahili (individual language)

When the individual languages are listed in the current Registry, they 
naturally do not have the word "(macrolanguage)" in their description:

Type: language
Subtag: sw
Description: Swahili
Added: 2005-10-16
Suppress-Script: Latn

I've been assuming that the only solution is to keep both names, putting 
the 639-3 name first, because we wanted to keep the 639-1 and/or -2 name 
as well:

Description: Swahili (macrolanguage)
Description: Swahili

But because of the no-inversion rule, it turns out we will be getting 
rid of some of the 639-1/-2 names anyway (see "Sotho, Southern" above). 
Is it still necessary to list both "X" and "X (macrolanguage)" as in the 
Swahili example above?

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://home.roadrunner.com/~dewell
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Sat, 8 Dec 2007 01:48:01 -0500

> Is it still necessary to list both "X" and "X (macrolanguage)" as in the 

I'd say remove the second line in favor of the first, which is definitive.

-- 
John Cowan
        [email-masked]
                I am a member of a civilization. --David Brin

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 8 Dec 2007 09:38:22 -0800

I'd favor:

Description: Swahili

A vanishingly small number of people will know what "macrolanguage" means or
how it should be translated, so having it be part of the name would be
clumsy. I also thought we were going to have macrolanguage as a field --
that would be clean way to do it, and if so, then including "
(macrolanguage)" would also be redundant.

Mark

-- 
Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 8 Dec 2007 09:56:38 -0800

Mark Davis wrote:

> I'd favor:
>
> Description: Swahili
>
> A vanishingly small number of people will know what "macrolanguage" 
> means or how it should be translated, so having it be part of the name 
> would be clumsy.

It's going to be confusing no matter what we do, because most people 
think 'sw' refers to Swahili, the individual language, and now we are 
telling them that language is really 'swh'.  This is worse than the 
Chinese case, where at least none of the encompassed languages is called 
simply "Chinese."

> I also thought we were going to have macrolanguage as a field -- that 
> would be clean way to do it, and if so, then including " 

But the macrolanguage 'sw' won't be marked in any special way; only the 
presence of other language subtags with "Macrolanguage: sw" will provide 
any clue.

I can see advantages and disadvantages to both sides here.  I also don't 
like exposing the term "macrolanguage" in user interfaces (c'mon, you 
know the Description fields will end up there) and forcing civilians to 
understand what a macrolanguage is.  But then what does it mean to have 
a choice between plain "Swahili" and "Swahili (individual language)"? 
And what about the people who said we MUST keep all the ISO 639 names 
intact, without changing so much as a hyphen or apostrophe, so the 
subtags could be related back to the standard?   I hope more people 
contribute their thoughts on this.

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 8 Dec 2007 10:44:58 -0800

Good points. My primary concern is with the presentation of language names
in UIs, and a listing of Swahili like the following is not going to be
comprehensible to anyone except for the 0.0000005% of people who are
familiar with these standards. It's a bit better with the names for Chinese,
as you point out.

...
Swahili (macrolanguage)
Swahili (individual language)
...

although even with Chinese, someone is going to be confused as to which they
should pick.

Now before someone says it, I realize that the names in a UI don't have to
be the same as the names in the registry. But the closer we get them to
being understandable by mortals, the more likely it is that we will have
non-confusing names show up in UIs. We might even leave the above in the
registry, but have a note in 4646bis about the names. So I think it's worth
our taking at least a little bit of time to discuss this.

The best I could think of off-hand was something like.
...
Swahili (general)
Swahili
...

and maybe some UI device like a link on general to get to a box with more
information.

As it turns out, there are just 4 ambiguous names after removing "
(macrolanguage)" and " (individual language)":

  dgo <http://www.sil.org/iso639-3/documentation.asp?id=dgo> Dogri
knn<http://www.sil.org/iso639-3/documentation.asp?id=knn>
Konkani  mly <http://www.sil.org/iso639-3/documentation.asp?id=mly> Malay
swh <http://www.sil.org/iso639-3/documentation.asp?id=swh> Swahili
  doi <http://www.sil.org/iso639-3/documentation.asp?id=doi> Dogri
kok<http://www.sil.org/iso639-3/documentation.asp?id=kok>
Konkani  msa <http://www.sil.org/iso639-3/documentation.asp?id=msa> Malay
swa <http://www.sil.org/iso639-3/documentation.asp?id=swa> Swahili
In all other cases, the names are different, often the result of some
adjective modifying the individual language. It appears that there are a
number of alternate names in SIL; perhaps we can use one of the alternate
names (or have a note that points to the possible use of alternate names?)

doi: Dhogaryali, Dogari, Dogri Jammu, Dogri-Kangri, Dogri Pahari, Dongari,
Hindi Dogri, Tokkaru
kok: Konkan Standard, Bankoti, Kunabi, North Konkan, Central Konkan,
Concorinum, Cugani, Konkanese
msa: Bahasa Malaysia, Bahasa Malayu, Malayu, Melaju, Melayu, Standard Malay
swa: Kiswahili, Kisuaheli

Mark

On Dec 8, 2007 9:56 AM, Doug Ewell <[email-masked]> wrote:

> http://home.roadrunner.com/~dewell <http://home.roadrunner.com/%7Edewell>

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 8 Dec 2007 20:11:46 +0100

Doug Ewell wrote:

> It's going to be confusing no matter what we do

Confusing is one thing, but two different language
codes with the same description would be *broken*.

3066 and 4646 went to great lengths that this is
not possible, including a promise from ISO in the
text.

If your beutifications result in an ambiguity for
"Swahili" pick a simpler algorithm where that won't
happen.  

> the macrolanguage 'sw' won't be marked in any 
> special way

The SIL pages claim that the description for "sw"
is "Swahili (macrolanguage)".  The 4646 registry
claims that it's "Swahili", at least one of them
is wrong.

> what about the people who said we MUST keep all
> the ISO 639 names intact, without changing so
> much as a hyphen or apostrophe, so the subtags
> could be related back to the standard?

Watching what happens with your beautification ;-)

Seriously, who lies about "sw", the registry or 
SIL ?  Did we miss a bulk update of 639-1 codes ?
I know that we talked about info in parentheses,
but not this "Swahili" ambiguity, or did we ?

 Frank

---

From: Peter Constable <[email-masked]>
Date: Sat, 8 Dec 2007 11:18:58 -0800

I don’t think at present anything identifies the entry for sw as being a macrolanguage entry. That’s the gap.

Peter

From: Mark Davis [mailto:[email-masked]]
Sent: Saturday, December 08, 2007 9:38 AM
To: John Cowan
Cc: Doug Ewell; LTRU Working Group
Subject: Re: [Ltru] "X" vs. 'X (macrolanguage)"

I'd favor:

Description: Swahili

A vanishingly small number of people will know what "macrolanguage" means or how it should be translated, so having it be part of the name would be clumsy. I also thought we were going to have macrolanguage as a field -- that would be clean way to do it, and if so, then including " (macrolanguage)" would also be redundant.

Mark
On Dec 7, 2007 10:48 PM, John Cowan <[email-masked]<mailto:[email-masked]>> wrote:

--
John Cowan
       [email-masked]<mailto:[email-masked]>
               I am a member of a civilization. --David Brin

_______________________________________________
Ltru mailing list
[email-masked]<mailto:[email-masked]>
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Peter Constable <[email-masked]>
Date: Sat, 8 Dec 2007 11:34:13 -0800

There is a deeper problem we should be considering: when does it make sense for people to use the macro-language or the individual-language IDs in cases like Swahili, Malay, etc.? In fact, when does it make sense to use which in cases for any macrolanguage ID in widespread use when there is a dominant encompassed language? This is an issue in these cases:

ar/ara vs. arb (Arabic)
kok vs. Knn (Konkani)
ms/msa vs. mly (Malay)
sw/swa vs. swh (Swahili)
uz/uzb vs. uzn (Uzbek)
zh/zho vs. cmn (Chinese)

These are all the cases in category 2 of ‘Macrolanguage analysis.txt’ which I sent as an attachment on 11/29. I didn’t include Dogri as I had no indication that doi is widely used, but we can certainly include it for consideration of this issue.

Peter

From: Mark Davis [mailto:[email-masked]]
Sent: Saturday, December 08, 2007 10:45 AM
To: Doug Ewell
Cc: LTRU Working Group
Subject: Re: [Ltru] "X" vs. 'X (macrolanguage)"

Good points. My primary concern is with the presentation of language names in UIs, and a listing of Swahili like the following is not going to be comprehensible to anyone except for the 0.0000005% of people who are familiar with these standards. It's a bit better with the names for Chinese, as you point out.

...
Swahili (macrolanguage)

although even with Chinese, someone is going to be confused as to which they should pick.

Now before someone says it, I realize that the names in a UI don't have to be the same as the names in the registry. But the closer we get them to being understandable by mortals, the more likely it is that we will have non-confusing names show up in UIs. We might even leave the above in the registry, but have a note in 4646bis about the names. So I think it's worth our taking at least a little bit of time to discuss this.

and maybe some UI device like a link on general to get to a box with more information.

As it turns out, there are just 4 ambiguous names after removing " (macrolanguage)" and " (individual language)":
dgo<http://www.sil.org/iso639-3/documentation.asp?id=dgo>

Dogri

Konkani

mly<http://www.sil.org/iso639-3/documentation.asp?id=mly>

Malay

swh<http://www.sil.org/iso639-3/documentation.asp?id=swh>

Swahili

doi<http://www.sil.org/iso639-3/documentation.asp?id=doi>

Dogri

Konkani

msa<http://www.sil.org/iso639-3/documentation.asp?id=msa>

Malay

swa<http://www.sil.org/iso639-3/documentation.asp?id=swa>

Swahili

In all other cases, the names are different, often the result of some adjective modifying the individual language. It appears that there are a number of alternate names in SIL; perhaps we can use one of the alternate names (or have a note that points to the possible use of alternate names?)

doi: Dhogaryali, Dogari, Dogri Jammu, Dogri-Kangri, Dogri Pahari, Dongari, Hindi Dogri, Tokkaru
kok: Konkan Standard, Bankoti, Kunabi, North Konkan, Central Konkan, Concorinum, Cugani, Konkanese
msa: Bahasa Malaysia, Bahasa Malayu, Malayu, Melaju, Melayu, Standard Malay
swa: Kiswahili, Kisuaheli

Mark
On Dec 8, 2007 9:56 AM, Doug Ewell <[email-masked]<mailto:[email-masked]>> wrote:

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://home.roadrunner.com/~dewell<http://home.roadrunner.com/%7Edewell>
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

---

From: Peter Constable <[email-masked]>
Date: Sat, 8 Dec 2007 11:39:32 -0800

> From: Frank Ellermann [mailto:[email-masked]]

> not possible, including a promise from ISO in the

Whate are you referring to? I am not aware that IETF requested any promises from ISO wrt descriptions.

> claims that it's "Swahili", at least one of them

SIL's pages in relation to this entry in ISO 639-3 are not wrong wrt ISO 639. I cannot comment generally on whether that info is considered appropriate in relation to some application of ISO 639; that is entirely beyond the scope of ISO 639.

Peter

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 8 Dec 2007 11:44:39 -0800

Mark Davis wrote:

> Now before someone says it, I realize that the names in a UI don't 
> have to be the same as the names in the registry. But the closer we 
> get them to being understandable by mortals, the more likely it is 
> that we will have non-confusing names show up in UIs. We might even 
> leave the above in the registry, but have a note in 4646bis about the 
> names. So I think it's worth our taking at least a little bit of time 
> to discuss this.
>

I assume you're talking about what you would do in a UI, not what the 
Registry should have.  We had a big squabble about keeping the ISO names 
intact.  In fact, at one point I was changing the parenthetical comments 
to "(general)" and "(specific)", or leaving out one or the other, but it 
was argued that people couldn't tie the Registry's Description fields 
back to the ISO standard.

So now the only thing I'm concerned about is whether to keep both the 
639-2 name, without the parenthetical comment, *and* the 639-3 name, 
which has it, or whether one of the two names can be jettisoned.

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 8 Dec 2007 21:50:25 +0100

> Whate are you referring to? I am not aware that
> IETF requested any promises from ISO wrt descriptions.

Bottom of <http://tools.ietf.org/html/rfc4646#page-9>:

The promise in RFC 3066 and 4646 is about alpha-2
and alpha-3 codes, for an existing alpha-3 code
for a language (identified by its description)
there won't be a new alpha-2 code (with the same
description because it's the same language).

My interpretation wrt the case at hand added in
parentheses.

> SIL's pages in relation to this entry in ISO 639-3
> are not wrong wrt ISO 639.

Then the 4646 registry must be wrong, it's supposed
to reflect whatever description ISO 639-1 offers for
an alpha-2 code.  Caveat, I looked up the ISO 639-1
"sw" on the SIL Web form, not some ISO 639-3 "sw".

 Frank

---

From: Peter Constable <[email-masked]>
Date: Sat, 8 Dec 2007 13:47:53 -0800

> description because it's the same language).

Right. That has nothing whatsoever to do with descriptions.

> "sw" on the SIL Web form, not some ISO 639-3 "sw".

There is no 639-3 "sw".

The site for 639-3 lists reference names as they exist in 639-3. The 639-2 site lists names based mainly on names as they existed in 639-1 and 639-2 prior to 639-3. (Some changes have been made since 639-3 was published) For 639-2, there isn't the same demand on the RA to add qualifiers to the names they record because there is no naming conflict in 639-2. (In cases where there is a naming conflict in 639-2, qualifiers have always been added.)

In principle, it's reasonable to expect the names for 639 would apply consistently across all parts. In practice, the 639-2 RA has been reluctant to make changes that would not be meaningful for their audience until they see some reasonable mitigation.

At this point (i.e. 4646bis era), I see no reason why the LSR could not rely solely on data provided by the 639-3 RA.

Peter