Subject: RE: [Ltru] Re: The most perverse tag

From: "Addison Phillips" <[email-masked]>
Date: Mon, 6 Jun 2005 10:02:11 -0700

I think our approach here may be flawed. We're trying to have normative and prescriptive values all in one requirement.

The minimum requirement should be either 26 characters (the longest practical 3066bis tag) or 33 if we add extlang support (for the longest practical 3066ter tag). The vast preponderance of tags will fall into this limit and neither is an impractical value (although 33 is maximally ugly).

The actual maximum for a meaningful tag (with three extlangs and two variants, please note) is actually 42 characters. The remaining 20 in the "62" value are private use (nearly a third!). We should document both of these numbers as the outer edge of a meaningful generative tag. The normative language for tags longer than 26 (or 33) should place the burden on the person choosing the tag, though, not applications, i.e.:

---
Language tags longer than this limit {ed note: 26 or 33} may be truncated by applications or protocols that limit tag sizes. When forming language tags, users and applications SHOULD avoid adding subtags that add no distinguishing value and SHOULD follow fields in the registry (such as Prefix and Suppress-Script) that limit subtag composition. See Section 4.1 (Choice of Language Tag).

Applications or protocols that must truncate a tag should first remove all extension subtags. If the tag is still too long, the tag should then be truncated by progressively removing subtags from the right of the tag. Note that private use subtags are removed first. When removing private use subtags, the singleton '-x-' should be removed along with the last remaining private use subtag. 
---

Then in section 4.4 (considerations for private use subtags) add:

---
Users of private use subtags should note that applications or protocols that truncate language tags (due to buffer length restrictions or other considerations) will generally remove private use subtags before removing other information from a tag. Extensive use or large numbers of private use subtags may be compromised by certain kinds of processing as a result.
---

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of John.Cowan
> Sent: 2005?6?6? 9:32
> To: Peter Constable
> Cc: [email-masked]
> Subject: Re: [Ltru] Re: The most perverse tag
> 
> Peter Constable scripsit:
> 
> > But we should consider what we think
> > should be required for 3066ter. In that case,
> >
> > lng-exa-exb-scrp-123-variant1
> >
> > is, I think, reasonably likely -- that's 29. I'm not sure how Addison
> > got 34, but if he can explain that, then I'd accept that number.
> 
> By adding an extra extlang.
> 
> --
> John Cowan  [email-masked]  www.ccil.org/~cowan
> www.reutershealth.com
> And now here I was, in a country where a right to say how the country
> should
> be governed was restricted to six persons in each thousand of its
> population.
> For the nine hundred and ninety-four to express dissatisfaction with the
> regnant system and propose to change it, would have made the whole six
> shudder as one man, it would have been so disloyal, so dishonorable, such
> putrid black treason.  --Mark Twain's Connecticut Yankee
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 6 Jun 2005 13:49:17 -0700

I think you are right about the items being mixed. But the language you have
is too strong. "We should document both of these numbers as the outer edge
of a meaningful generative tag." This is not the case, since private use
tags may certainly be meaningful, as may be extension subtags; otherwise
there would truly be no point to allowing them. So I suggest keeping the 63
recommendation. Details:

Suggested rephrasing:

>Language tags longer than this limit {ed note: 26 or 33} may be truncated
by applications or protocols that limit tag sizes.

And, unless we say otherwise, also language tags below this limit may be
truncated by applications... So change to:

=> Language tags may be truncated by applications or protocols that limit
tag sizes, although applications and protocols should not require truncation
to shorter than 63 bytes.

>When forming language tags, users and applications SHOULD avoid adding
subtags that add no distinguishing value and SHOULD follow fields in the
registry (such as Prefix and Suppress-Script) that limit subtag composition.
See Section 4.1 (Choice of Language Tag).

ok

> Applications or protocols that must truncate a tag should first remove all
extension subtags. If the tag is still too long, the tag should then be
truncated by progressively removing subtags from the right of the tag. Note
that private use subtags are removed first. When removing private use
subtags, the singleton '-x-' should be removed along with the last remaining
private use subtag.

Rephrase to get the right order, and to clarify that it is not "all
extension subtags" or nothing. Also, unless we want different results, they
should first canonicalize. Also, the singleton note also is true for the
other extensions, since otherwise the result is invalid (eg en-a-ABCDE can't
be truncated to en-a). So,

=>Applications or protocols that must truncate a tag should first
canonicalize the tag, then progressively remove extension and private use
subtags, starting from the right of the tag, until the result fits. When
removing private use or extension subtags, the singleton should be removed
along with the last remaining private use or extension subtag.  Note that
private use subtags are removed first. If the tag is still too long after
removing all private use and extension subtags, the tag should then be
further truncated by progressively removing subtags from the right of the
tag.

>Users of private use subtags should note that applications or protocols
that truncate language tags (due to buffer length restrictions or other
considerations) will generally remove private use subtags before removing
other information from a tag. Extensive use or large numbers of private use
subtags may be compromised by certain kinds of processing as a result.

"will generally" is too strong. And some other qualifications I find
unnecessary or misleading. We also don't alert them that *part* of the
content of an extension or private use sequence may be removed; e.g.
"en-a-sort-diction" may turn into "en-a-sort".

=>Users of private use or extension subtags should note that applications or
protocols that truncate language tags (due to buffer length restrictions or
other considerations) may remove private use or extension subtags. Large
numbers of private use or extension subtags in a tag may cause the rightmost
ones to be compromised in such cases. Specifications of private use or
extension mechanisms should be designed so as to degrade gracefully in the
face of possible truncation.

‎Mark

----- Original Message ----- 
From: "Addison Phillips" <[email-masked]>
To: "John.Cowan" <[email-masked]>; "Peter Constable"
<[email-masked]>
Cc: <[email-masked]>
Sent: Monday, June 06, 2005 10:02
Subject: RE: [Ltru] Re: The most perverse tag

I think our approach here may be flawed. We're trying to have normative and
prescriptive values all in one requirement.

The minimum requirement should be either 26 characters (the longest
practical 3066bis tag) or 33 if we add extlang support (for the longest
practical 3066ter tag). The vast preponderance of tags will fall into this
limit and neither is an impractical value (although 33 is maximally ugly).

The actual maximum for a meaningful tag (with three extlangs and two
variants, please note) is actually 42 characters. The remaining 20 in the
"62" value are private use (nearly a third!). We should document both of
these numbers as the outer edge of a meaningful generative tag. The
normative language for tags longer than 26 (or 33) should place the burden
on the person choosing the tag, though, not applications, i.e.:

---
Language tags longer than this limit {ed note: 26 or 33} may be truncated by
applications or protocols that limit tag sizes. When forming language tags,
users and applications SHOULD avoid adding subtags that add no
distinguishing value and SHOULD follow fields in the registry (such as
Prefix and Suppress-Script) that limit subtag composition. See Section 4.1
(Choice of Language Tag).

Applications or protocols that must truncate a tag should first remove all
extension subtags. If the tag is still too long, the tag should then be
truncated by progressively removing subtags from the right of the tag. Note
that private use subtags are removed first. When removing private use
subtags, the singleton '-x-' should be removed along with the last remaining
private use subtag.
---

---
Users of private use subtags should note that applications or protocols that
truncate language tags (due to buffer length restrictions or other
considerations) will generally remove private use subtags before removing
other information from a tag. Extensive use or large numbers of private use
subtags may be compromised by certain kinds of processing as a result.
---

Addison P. Phillips