Subject: [Ltru] Access to subtag registry by matching applications

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 4 Apr 2006 12:56:26 -0700

Hi -

As a technical contributor...

In the editor's copy at
http://www.inter-locale.com/ID/matching-diff-11-12.html
the proposed text reads:

| However, applications, protocols, or specifications are encouraged to  
| use information in the IANA Language Subtag Registry to canonicalize  
| language tags and ranges, mapping grandfathered and obsolete tags or  
| subtags into modern equivalents.  Such an implementation MAY also use  
| semantic information contained in the tags to augment matching. 

I am concerned that the first sentence may be misconstrued as encouraging
applications to access the subtag registry.  This would be contrary to
the WG's resolution of the concerns behind issue 968
https://rt.psg.com/Ticket/Display.html?id=968
(user "ietf", password "ietf", queue ltru-registry)

Consequently, I suggest this minor tweak to the proposed text's
first sentence:

  However, designers of applications, protocols, or specifications are
  encouraged to use information from the IANA Language Subtag Registry
  to support canonicalizing language tags and ranges, mapping
  grandfathered and obsolete tags or subtags into modern equivalents.

Regarding the second sentence, I believe it goes beyond the scope of
the algorithms defined by this BCP.  I suggest the following replacement
for the second sentence.
  The use of additional semantic information associated with subtags
  to augment matching is outside the scope of this memo.

Randy

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Addison Phillips" <[email-masked]>
Date: Tue, 4 Apr 2006 14:35:53 -0700

Comments follow

Addison

Addison Phillips
Internationalization Architect - Yahoo! Inc.

Internationalization is an architecture.
It is not a feature. 

> -----Original Message-----
> From: Randy Presuhn [mailto:[email-masked]]
> Sent: 2006年4月4日 12:56
> 
> applications to access the subtag registry.  

Agreed.
>   to augment matching is outside the scope of this memo.
I tend to disagree, from the perspective that we *are* documenting potential implementation details permitted by an application. However, strictly speaking such an implementation would deviate from the algorithms specified by draft-matching.

With an exception.

We give general license to implementations to specify how default content is located in the lookup algorithm (we require that they document what the mechanism is). Since this text is mostly an example anyway, recasting it for this particular purpose covers your concern and preserves the useful example. I suggest:

--
<t>Some forms of matching might benefit from the use of additional semantic information derived from subtags. For example, when defining the default content for a lookup process, an implementation might usefully include the primary language subtags 'nn' (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) as synonyms for the more general subtag 'no' (Norwegian). Or it might infer that content labeled "zh-Hans" (Chinese as written in the Simplified script) is more likely to match the range "zh-CN" (Chinese as used in China, where the Simplified script is predominant) than equivalent content labeled "zh-TW" (Chinese as used in Taiwan, where the Traditional script is predominant).</t>
--

---

From: "Addison Phillips" <[email-masked]>
Date: Tue, 4 Apr 2006 15:03:13 -0700

All:

I have posted a "draft-12a" on inter-locale. A diff from draft-11 is very difficult to interpret, so I have elected not to generate one. A diff from the editor's copy of draft-12 can be generated by following the URI:

http://tinyurl.com/nx8uf

I would suggest that we submit a draft-12 soon. I haven't done so because I've been waiting on one or another co-chair to tell me what to do with specific comments (i.e. whether we proceed with my editor's copy or roll back). I also notice that the issues list server appears to be dead, even using the IP address.

New copies are:

http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12a.txt
http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12a.html
http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12a.xml

Best Regards (for the editors),

Addison

Addison Phillips

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 4 Apr 2006 16:39:17 -0700

Hi -

> From: "Addison Phillips" <[email-masked]>
> To: "'Randy Presuhn'" <[email-masked]>; "'LTRU Working Group'" <[email-masked]>
> Sent: Tuesday, April 04, 2006 2:35 PM
> Subject: RE: [Ltru] Access to subtag registry by matching applications
...
>  Since this text is mostly an example anyway, recasting it for this
> particular purpose covers your concern and preserves the useful example. I suggest:
>
> --
> <t>Some forms of matching might benefit from the use of additional semantic
...

Change "forms of matching" to "forms of matching beyond the scope
of this memo" and I could live with your proposal.

Randy

---

From: Mark Davis <[email-masked]>
Date: Wed, 05 Apr 2006 10:52:19 -0700

We could make this more explicit, addressing Randy's concern, if we
added that

Each implementation MUST specify whether it is using semantic 
information to customize the matching, and if so, MUST define the 
mechanism used to do so.

Mark

---

From: Mark Davis <[email-masked]>
Date: Wed, 05 Apr 2006 11:33:18 -0700

Each implementation MUST specify whether it is using semantic information to customize the matching, and if so, MUST define the mechanism used to do so.

Mark

---

From: "Addison Phillips" <[email-masked]>
Date: Wed, 5 Apr 2006 15:35:23 -0700

> We could make this more explicit, addressing Randy's concern, if we

Yes, we could do this, but that also makes it clear that this is beyond the scope of the document anyway.

I think the real problem here is that we're thinking too specifically about examples such as 'nb'->'no'. The more general description is: users wish to tailor or customize the matching in a particular matching instance (so that, for example, 'zh-Hans' also matches 'zh-CN'). This isn't really "semantic" information (given that I might wish to associate the range "es-ES" with the tag "tlh-AQ", where no semantic linkage exists except in my own fevered mind). The problem here is that mappings of this sort can be described as (higher-level) features of the "application, protocol, or specification" and usually in terms of pre-processing the language priority list, the tags, or both. 

I think we would best limit ourselves (*if* we say anything at all) to something like:

--
Applications, protocols, or specifications, in addressing their particular requirements, can offer pre-processing or configuration options. For example, an implementation could allow a user to associate the language subtags 'nn' (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or perhaps the user could associate the range "zh-Hans" (Chinese as written in the Simplified script) with the tag "zh-CN" (Chinese as used in China, where the Simplified script is predominant). Such customization affects the operation of the underlying matching scheme. Conformant implementations SHOULD NOT enable such options by default and MUST offer a configuration or mode compatible with the unmodified matching scheme or schemes described in this document.
--

Thoughts?

Addison

Addison Phillips

> From: Mark Davis [mailto:[email-masked]]
> Sent: 2006年4月5日 11:33
> To: Addison Phillips
> Cc: 'Randy Presuhn'; 'LTRU Working Group'
> Subject: Re: [Ltru] Access to subtag registry by matching applications

---

From: John Cowan <[email-masked]>
Date: Wed, 5 Apr 2006 18:50:15 -0400

Addison Phillips scripsit:

> Applications, protocols, or specifications, in addressing their
> particular requirements, can offer pre-processing or configuration
> options. For example, an implementation could allow a user to
> associate the language subtags 'nn' (Nynorsk Norwegian) and 'nb'
> (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or
> perhaps the user could associate the range "zh-Hans" (Chinese as written
> in the Simplified script) with the tag "zh-CN" (Chinese as used in
> China, where the Simplified script is predominant). Such customization
> affects the operation of the underlying matching scheme. Conformant
> implementations SHOULD NOT enable such options by default and MUST
> offer a configuration or mode compatible with the unmodified matching
> scheme or schemes described in this document.

I don't like the last sentence.  Enabling such options by default, if they
are good ones, may be just the right thing to do: conformance isn't anything.
And the second half of the sentence says that a conformant application
MUST offer a conformant mode of operation, which is to say that conformance
entails conformance.

(BTW, graf 2 of Section 3 still talks about information items.)

> > Sent: 2006???4???5??? 11:33
> > To: Addison Phillips

-- 
But that, he realized, was a foolish            John Cowan
thought; as no one knew better than he          [email-masked]
that the Wall had no other side.                http://www.ccil.org/~cowan
        --Arthur C. Clarke, "The Wall of Darkness"

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 5 Apr 2006 15:54:28 -0700

Hi -

> To: "'Mark Davis'" <[email-masked]>
> Cc: "'Randy Presuhn'" <[email-masked]>; "'LTRU Working Group'" <[email-masked]>
> Sent: Wednesday, April 05, 2006 3:35 PM
...
> Applications, protocols, or specifications, in addressing their particular
> requirements, can offer pre-processing or configuration options. For example,
> an implementation could allow a user to associate the language subtags 'nn'
> (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general subtag
> 'no' (Norwegian). Or perhaps the user could associate the range "zh-Hans"
> (Chinese as written in the Simplified script) with the tag "zh-CN" (Chinese
> as used in China, where the Simplified script is predominant).

So far, so good.

> Such customization affects the operation of the underlying matching scheme.

Not necessarily; it could also be thought of as mapping a single tag into
a set of tags or ranges before offering them to the matching algorithm.

> Conformant implementations SHOULD NOT enable such options by default
> and MUST offer a configuration or mode compatible with the unmodified
> matching scheme or schemes described in this document.
...

If we're clear that we're only talking about such capabilities
being "built in" to the matching algorithm this would be ok, but
I'm concerned that this might be misconstrued as being too strong,
e.g., forbidding an application which used a conformant matching function
from providing these more elaborate / useful services.

Randy

---

From: "Addison Phillips" <[email-masked]>
Date: Wed, 5 Apr 2006 16:05:43 -0700

> (BTW, graf 2 of Section 3 still talks about information items.)

Good catch. Changed to:

<t>There are two types of matching scheme in this document. A matching
scheme that produces zero or more matching language tags is called
"filtering". A matching scheme that produces exactly one match for a given
request is called "lookup".</t>

> are good ones, may be just the right thing to do: conformance isn't

I don't like it either.

> a set of tags or ranges before offering them to the matching algorithm.

You're correct. I said as much in my preceding paragraphs too. It just isn't
expressed well here. What it probably ought to say is something like:

--
Modifying the language ranges or tags can affect the outcome of the matching
operation.
--

> from providing these more elaborate / useful services.

Let's forget the MUSTard. Question is: should we even mention the whole
customization thing?

Addison

Addison Phillips

> From: John Cowan [mailto:[email-masked]]
> Sent: 2006?4?5? 15:50
> To: Addison Phillips
> Cc: [email-masked]

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 5 Apr 2006 16:11:39 -0700

Hi -

> To: "'John Cowan'" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Wednesday, April 05, 2006 4:05 PM
...
...

I'd be equally happy without that last sentence or without the entire
paragraph.  I can understand the value of the tutorial material, but
don't feel strongly about retaining it.

Randy

---

From: Mark Davis <[email-masked]>
Date: Wed, 05 Apr 2006 16:43:51 -0700

I agree. Most of what Addison suggests is good, but the last sentence is 
too strong. Should be something like:

Conformant implementations MUST specify whether they allow such 
configuration options, and if so, SHOULD describe the effects they have 
on the modification process.

Mark

---

From: Mark Davis <[email-masked]>
Date: Wed, 05 Apr 2006 16:51:15 -0700

> Let's forget the MUSTard. Question is: should we even mention the 
whole customization thing?

Given that I think that any implementation that *didn't* do some 
customization would be unacceptable our customers, I think we have to 
allow for it. See my previous message for suggested rephrasing.

Mark

---

From: "Addison Phillips" <[email-masked]>
Date: Wed, 5 Apr 2006 17:45:15 -0700

Alright. I did a rewrite on your suggestion, though. The paragraph now reads:

--
<t>Applications, protocols, or specifications, in addressing their particular requirements, can offer pre-processing or configuration options. For example, an implementation could allow a user to associate the language subtags 'nn' (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or perhaps the user could associate the range "zh-Hans" (Chinese as written in the Simplified script) with the tag "zh-CN" (Chinese as used in China, where the Simplified script is predominant). These options will influence the results of a matching operation, so an application, protocol, or specification SHOULD document how ranges or tags are altered, prioritized, or compared in the subsequent match so that users can modify their tags or language priority lists appropriately.</t>
--

Addison Phillips

> Sent: 2006年4月5日 16:51
> Cc: 'John Cowan'; [email-masked]

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 5 Apr 2006 18:19:47 -0700

Hi -

Works for me, though others, finickier than I, might complain
about the SHOULD.

Randy

----- Original Message ----- 
From: "Addison Phillips" <[email-masked]>
To: "'Mark Davis'" <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, April 05, 2006 5:45 PM
Subject: RE: [Ltru] Access to subtag registry by matching applications

--
<t>Applications, protocols, or specifications, in addressing their particular requirements, can offer pre-processing or
configuration options. For example, an implementation could allow a user to associate the language subtags 'nn' (Nynorsk Norwegian)
and 'nb' (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or perhaps the user could associate the range "zh-Hans"
(Chinese as written in the Simplified script) with the tag "zh-CN" (Chinese as used in China, where the Simplified script is
predominant). These options will influence the results of a matching operation, so an application, protocol, or specification SHOULD
document how ranges or tags are altered, prioritized, or compared in the subsequent match so that users can modify their tags or
language priority lists appropriately.</t>
--

Addison Phillips

---

From: Mark Davis <[email-masked]>
Date: Wed, 05 Apr 2006 18:40:12 -0700

Works for me.

Randy Presuhn wrote:
>>> "filtering". A matching scheme that produces exactly one match for a
>>>       
>> given
>>     
>>>> I don't like the last sentence.  Enabling such options by default, if
>>>>         
>> they
>>     
>>>> Not necessarily; it could also be thought of as mapping a single tag
>>>>         
>> into
>>     
>>> You're correct. I said as much in my preceding paragraphs too. It just
>>>       
>> isn't
>>     
>>> Modifying the language ranges or tags can affect the outcome of the
>>>       
>> matching
>>     
>>> operation.
>>> --
>>>
>>>
>>>       
>>>> e.g., forbidding an application which used a conformant matching
>>>>         
>> function
>>     
>>>>> perhaps the user could associate the range "zh-Hans" (Chinese as
>>>>>           
>> written
>>     
>>>> I don't like the last sentence.  Enabling such options by default, if
>>>>         
>> they
>>     
>>>>>> Subject: Re: [Ltru] Access to subtag registry by matching
>>>>>>             
>> applications
>>     
>>>> that the Wall had no other side.
>>>>         
>> http://www.ccil.org/~cowan
>>

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 6 Apr 2006 09:07:17 -0700

To address the overzealous MUSTard (SHOULD), I did another minor rewrite of the para before submitting the text:

--
Applications, protocols, or specifications, in addressing their particular requirements, can offer pre-processing or configuration options. For example, an implementation could allow a user to associate or map a particular language range to a different value. Such a user might wish to associate the language range subtags 'nn' (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or perhaps the user could associate the range "zh-Hans" (Chinese as written in the Simplified script) with the language tag "zh-CN" (Chinese as used in China, where the Simplified script is predominant) because content is available with that tag. Documentation on how the ranges or tags are altered, prioritized, or compared in the subsequent match in such an implementation will assist users in making the best configuration choices.
--

Addison

Addison Phillips

> Sent: 2006年4月5日 18:40
> To: Randy Presuhn
> Cc: LTRU Working Group
> > <t>Applications, protocols, or specifications, in addressing their
> particular requirements, can offer pre-processing or
> > configuration options. For example, an implementation could allow a user
> to associate the language subtags 'nn' (Nynorsk Norwegian)
> > and 'nb' (Bokmal Norwegian) with the more general subtag 'no'
> (Norwegian). Or perhaps the user could associate the range "zh-Hans"
> > (Chinese as written in the Simplified script) with the tag "zh-CN"
> (Chinese as used in China, where the Simplified script is
> > predominant). These options will influence the results of a matching
> operation, so an application, protocol, or specification SHOULD
> > document how ranges or tags are altered, prioritized, or compared in the
> subsequent match so that users can modify their tags or

---

From: Marion Gunn <[email-masked]>
Date: Fri, 07 Apr 2006 11:59:38 +0100

Typographical comment: I'd like to see the full stop after '(Norwegan)'
replaced with either a comma or a a semi-colon. Reason: I think starting
a sentence with 'Or' is rather too dramatic in the context (good for
effect, bad for continuity). 

I'd also prefer 'the user may' to 'perhaps the user could', the former
being better EN-ISO (more authoritive, less casual language), IMHO.

Hope this helps,
mg

Scríobh Addison Phillips:
> Applications, protocols, or specifications, in addressing their particular requirements, can offer pre-processing or configuration options. For example, an implementation could allow a user to associate or map a particular language range to a different value. Such a user might wish to associate the language range subtags 'nn' (Nynorsk Norwegian) and 'nb' (Bokmal Norwegian) with the more general subtag 'no' (Norwegian). Or perhaps the user could...
> > >>>>> --
> > >>>>>
> > >>>>> Thoughts?
> > >>>>>
> > >>>>> Addison

-- 

Marion Gunn * EGTeo (Estab.1991)
27 Páirc an Fhéithlinn, Baile an 
Bhóthair, Co. Átha Cliath, Éire.
* [email-masked] * [email-masked] *

---

From: "Addison Phillips" <[email-masked]>
Date: Fri, 7 Apr 2006 07:59:29 -0700

Hi,

Integrating the two sentences into one would make a rather long run-on
sentence. Besides: a little drama in an RFC is probably a good thing. This
stuff tends to be rather dry.

Regarding the second comment, the word "may" has special meaning in an RFC.
It is an RFC 2119 keyword. In this particular set of documents we editors
have been trying to avoid these keywords except when they are intentionally
normative.

Addison

Addison Phillips

> From: Marion Gunn [mailto:[email-masked]]
> Sent: 2006?4?7? 4:00
> Cc: 'LTRU Working Group'
options.
> For example, an implementation could allow a user to associate or map a
> particular language range to a different value. Such a user might wish to
> associate the language range subtags 'nn' (Nynorsk Norwegian) and 'nb'
> perhaps the user could...
> > > >>>>> --
> > > >>>>>
> > > >>>>> Thoughts?
> > > >>>>>
> > > >>>>> Addison
> 
> --
>