Subject: Re: [Ltru] 3.1.4: description field uniqueness

From: "Doug Ewell" <[email-masked]>
Date: Tue, 8 Jul 2008 00:05:32 -0600

Peter Constable <petercon at microsoft dot com> wrote:

> Section 3.1.4 of the current draft has this wording:
>
> ----------------
> Each 'Description' field MUST be unique, both within the record in 
> which it appears and for the collection of records of the same type.
> ----------------
>
> I'm not positive I understand the intent for the second requirement, 
> unique "for the collection of records of the same type". It sounds 
> like that's saying that, if (say) a language record has a description 
> field with the value "foo", then no other language record can have a 
> description "foo".

We already have a problem with this, because "Dimli" is both the only 
Description field for subtag 'diq' and one of six Description fields for 
subtag 'zza'.  This happened because we committed to using all of the 
ISO 639-3 names.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 8 Jul 2008 22:45:49 -0600

Mark Davis <mark dot davis at icu dash project dot org> wrote:

> I disagree with this change. The descriptions need to be unique within 
> the type of field. We can always qualify the names, eg "Ainu (China)".

"Ainu (China)" came from ISO 639-3.  Up until now, we've obliged 
ourselves to keep the ISO names intact.  We've also allowed additional 
Description fields to be added, but even if we (read: ietf-languages) 
had taken advantage of that opportunity, that doesn't solve the 
uniqueness problem.

Then:

> It certainly *can* be made to be unique across all records, without 
> resorting to rocket science: change zza's Dimli by adding an 
> annotation, like "Dimli (Zaza)".

Iff we loosen the strict requirement to keep the exact ISO names.

Peter Constable <petercon at microsoft dot com> wrote:

> The problem is that, as things stand, we would be dependent on ISO 639 
> RAs / JAC to set names in such a way as to ensure that uniqueness 
> guarantee, and whereas we divide the alternate names for a given entry 
> into separate description fields they just have a single "name" 
> field - ie., they would have to ensure that no single name was ever 
> used in more than one entry, a uniqueness guarantee that goes far 
> beyond what they require. Clearly, that's not a reasonable 
> expectation. If we wanted to maintain this level of uniquess, then our 
> only option would be to add text in 3.4 allowing the Reviewer to edit 
> names coming from a source ISO standard by adding parenthetic 
> qualifiers to ensure uniqueness.

Exactly.

> But it's not clear to me why we would really need *that* level of 
> uniqueness.

Because a human or computer reading the Registry won't be able to tell 
which subtag to use for "Dimli," regardless of his, her, or its level of 
knowledge.  In cases like "Swahili (individual language)" versus 
"Swahili (macrolanguage)", at least the user has a hint.

Mark proposed:

> When creating or updating a record, prior to submitting the proposed 
> record to the ietf-languages list the Language Subtag Reviewer MUST 
> add annotations (in parentheses) if necessary to preserve the 
> uniqueness of descriptions, SHOULD remove duplicate or redundant 
> descriptions, and MAY edit descriptions to correct irregularities in 
> formatting (such as misspellings, inappropriate apostrophes or other 
> punctuation, or excessive or missing spaces).

I'm good with that, if the core-standards-traceability aficionados are.

I agree with Peter, though, that we should not cite UI as the reason for 
doing this, since we have said repeatedly that the Description fields 
aren't necessarily to be used verbatim in UI.  We did that for a very 
good reason, too: to avoid getting involved in endless, pointless, 
hopeless arguments over "preferred" or "culturally acceptable" names of 
languages.

Peter added:

> It means a bit more work for the Reviewer, and could lead to cases in 
> which a change in one entry in a source ISO standard leads to updates 
> to multiple records in the LSTR.

The Reviewer's clerical assistant thinks this extra work would be well 
worth the results.

"Phillips, Addison" <addison at amazon dot com> wrote:

> I can understand why Mark and others would want descriptions to be 
> unique too: how can users figure out which of the identically named 
> items to use for what? In practice it is probably fairly obvious, 
> provided one knows that, say, Dimli does appear twice and not just 
> once.

I don't think it would be obvious that a typical user would know this. 
And if the number of such cases is small (one at present), it would be 
better to fix the problem than to expect users to know an arcane detail 
like "watch out for Dimli."

> While it isn't much work to check it, I wouldn't be surprised to find 
> something slip through. I think we could accommodate this by 
> SHOULDifying uniqueness (we have already given the LSR broad editorial 
> powers regarding descriptions).

Only very recently, and the bulk of the WG probably hasn't had a chance 
to weigh in.  At least some of the aficionados are in Europe and may not 
have read this thread yet.

In the proposed text:

> They MUST be unique within a given type of subtag so that users do not 
> become confused about which subtag to use.

Maybe "unnecessarily confused."  With 8,000 language and extlang 
subtags, I don't think we can promise complete protection from 
confusion, but we can at least defend against gratuitous confusion.

> The Language Subtag Reviewer MUST edit any proposed first Descriptions 
> so as to ensure uniqueness among the first Descriptions (that is, 
> prevent collisions). The Language Subtag Reviewer SHOULD edit other 
> (non-first) Descriptions to preserve uniqueness, as well.
> ...
> Although not strictly required, it was edited to read "Dimli 
> (macrolanguage)" to prevent a collision. If 'zza' had had a first 
> description of "Dimli", then such editing to prevent a collision would 
> have been required.

We don't give any special normative status to the first Description 
field, although ISO 639-3 does.  So I don't think we can draw this 
distinction.  It would have to be required in all cases.