Subject: Re: [Ltru] A radical proposal (was: Re: extlang & deprecation)

From: "Doug Ewell" <[email-masked]>
Date: Wed, 2 Jul 2008 19:50:34 -0600

"Phillips, Addison" <addison at amazon dot com> wrote:

> I think it a mistake to water down "deprecated".

If you want "deprecated" to have its original strong meaning of "SHOULD 
NOT," that is fine, but we MUST be consistent about it.  We need to stop 
giving people the false impression, in drafts and mailing-list 
conversation, that it will be acceptable to use deprecated tags or 
subtags.  We need to stop trying to placate people in the X-Y versus Y 
debate by saying it doesn't matter much which form is deprecated, that 
the difference is only one of matching and fallback.  It will be a major 
difference.

> The problem we have with deprecation at present is the use of the 
> Preferred-Value field in 'extlang' records requires deprecation. If we 
> don't want to deprecate the extlang form, then we don't deprecate it. 
> The question is whether a tag of form "X-Y" is in its canonical form 
> or not. That is, is "zh-cmn-Hans-CN" canonical? Or is its canonical 
> form "cmn-Hans-CN"? It isn't necessarily true that non-canonical forms 
> are deprecated. For example, the tag "en-b-moo-a-foo" is not in 
> canonical form, but isn't necessarily deprecated.

I thought it was clear in my proposal that the no-extlang version would 
be preferred.  But you're right that nothing in the Registry would say 
so, which I concede is important.

> <AP> Canonicalization is not the same thing as permission to 
> normalize. Permission to normalize is much weaker. If we say that the 
> 'Y' form is canonical, it will save us having to modify aspects of 
> matching (we may still have to say something about matching). </AP>

I have been using the two terms interchangeably, and probably shouldn't 
have been.

> <AP> It doesn't remove the arbitrariness. It is just a different set 
> of arbitrariness. Personally, I prefer the smaller set of languages we 
> have now. But we could solve this by only initially registering a few 
> and allowing ietf-languages to register others. Or just register them 
> all. </AP>

How is it "just a different set of arbitrariness" if we handle all of 
the encompassed languages the same way, instead of hand-selecting a few 
macrolanguages like "zh" and treating their encompassed languages 
differently from, say, the languages encompassed by "fa"?

Everyone seems to prefer the smaller set of languages, and in fact many 
want it to be smaller than what we have now.  Something tells me that 
nobody has read the passage in draft-4645bis, Section 2.2, where I 
attempted to explain the arbitrariness we have now, so I'll repeat it 
here:

"If the language was encompassed by one of the [ISO639‑3] macrolanguages 
‘ar’ (Arabic), ‘kok’ (Konkani), ‘ms’ (Malay), ‘sw’ (Swahili), ‘uz’ 
(Uzbek), or ‘zh’ (Chinese), as determined by 
[iso‑639‑3‑macrolanguages_20080218], an extended language subtag was 
also added, with the primary language subtag of the macrolanguage as the 
value for the Prefix field. These macrolanguage subtags were already 
present in the Language Subtag Registry and were determined by the LTRU 
Working Group to have been used to represent a single dominant language 
as well as the macrolanguage as a whole, making the extended language 
mechanism suitable for languages encompassed by the macrolanguage."

Is everyone satisfied with this explanation of why these particular 
languages were chosen?  If we choose a different set, will we be able to 
explain that set in terms that are just as good or better?

I withdraw the radical proposal, but I hope it got us thinking a little 
about some of these issues.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Wed, 2 Jul 2008 21:53:57 -0700

> subtags.

No, we (at least most of us) have been consistent: it is always permitted that you can use a deprecated tag or subtag. From that perspective, it is "acceptable". However, it is not and has never been recommended. I have gone out of my way to open the door to people proposing that we deprecate in the other direction, noting why I don't prefer that position. But under no circumstances would I welcome a watered down deprecation. Either deprecate or don't.

> We need to stop trying to placate people in the X-Y
> versus Y
> the difference is only one of matching and fallback.  It will be a
> major difference.

+1. It matters a lot which form is deprecated in terms of how one implements matching. However, the difference in "deprecation direction" is much smaller than other differences.

> differently from, say, the languages encompassed by "fa"?

There is inherent arbitrariness in the existing set (no, sh: take a bow), plus a certain amount of instability in 639-3's handling so far. And we've had plenty of discussions of languages such as Moldovan. Ultimately, the languages that matter are the very short list John Cowan and Peter Constable bruited awhile ago. The others we have allowed in out of a (I think misguided) sense of "fair play". I support John Cowan's observation that this is strictly a backwards compatibility mechanism, for which we have a specific list of languages.

> languages were chosen?

Yes, except you missed 'sgn'.

>
> I withdraw the radical proposal, but I hope it got us thinking a
> little about some of these issues.

No reason to withdraw it: it is an option. As noted, I'm okay with certain aspects of it.

What we need is a sense of whether we have a consensus for something. All of these proposals ring hollow for me because they seem to be addressing inchoate fears of what someone *might* say about the proposed document, not what people have actually said.

Addison

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 3 Jul 2008 20:10:24 -0600

"Phillips, Addison" <addison at amazon dot com> wrote:

>> ... We need to stop giving people the false impression, in drafts and 
>> mailing-list conversation, that it will be acceptable to use 
>> deprecated tags or subtags.
>
> No, we (at least most of us) have been consistent: it is always 
> permitted that you can use a deprecated tag or subtag. From that 
> perspective, it is "acceptable". However, it is not and has never been 
> recommended.

If taggers are told in Section 3.1.5 that "validating processors SHOULD 
NOT generate these subtags" -- even though Section 2.1.9 does not 
mention the absence of deprecated subtags as a validity constraint --  
then it does not seem consistent to say that both X-Y and Y are "freely 
allowed."  Grudgingly, head-shakingly allowed, perhaps.

I will settle for whichever course the WG agrees upon (if it isn't a 
syntax error to say that), but I'm concerned about X-Y fans being misled 
about the social acceptability of using X-Y if it is deprecated.

>> the encompassed languages the same way, instead of hand-selecting a 
>> few macrolanguages like "zh" and treating their encompassed languages 
> There is inherent arbitrariness in the existing set (no, sh: take a 
> bow), plus a certain amount of instability in 639-3's handling so far.

That isn't what I meant.  We aren't responsible for the arbitrariness of 
the ISO standards.

> And we've had plenty of discussions of languages such as Moldovan. 
> Ultimately, the languages that matter are the very short list John 
> Cowan and Peter Constable bruited awhile ago. The others we have 
> allowed in out of a (I think misguided) sense of "fair play". I 
> support John Cowan's observation that this is strictly a backwards 
> compatibility mechanism, for which we have a specific list of 
> languages.

I've asked WG participants who would like to change the list of 
extlangable macrolanguages to provide a plausible explanation for their 
list.  I've seen some suggestions -- what we have minus 'sgn', just 
'zh', just 'zh' and 'ar' -- but no text explaining the choice.  I really 
don't want to write "because it felt right."

> Yes, except you missed 'sgn'.

I omitted the discussion of 'sgn' on purpose, as I thought it might 
distract from the rationale for choosing some ISO 639-3 macrolanguages 
but not others.  Also, I mistakenly thought the WG was in agreement 
about including 'sgn' on the list, and would be able to defend the 
decision.

> What we need is a sense of whether we have a consensus for something. 
> All of these proposals ring hollow for me because they seem to be 
> addressing inchoate fears of what someone *might* say about the 
> proposed document, not what people have actually said.

OK, I won't raise any more arguments about making the drafts defensible 
in IETF Last Call.  You're right, it is speculative; the people who 
raised the same objections last time that I'm focusing on now (e.g. 
cherry-picking) might not even be active in IETF any more.  Let's just 
do something, PLEASE, preferably some time before the London Olympics, 
and wait and see what happens.