Subject: [Ltru] summary of recommendations coming out of this week's teleconf

From: Peter Constable <[email-masked]>
Date: Sat, 1 Dec 2007 10:58:57 -0800

During the teleconf call this week, those of us that participated came to consensus on recommending these directions for 4646bis (at least, this is the understanding I came away with):

-          Do not pursue use of the extlang subtag; remove extlang from the ABNF.

-          Allow all 639-3 code elements as language subtags.

-          Deprecate all grandfathered tags like "zh-yue" in favour of just "yue".

-          Deprecate redundant "sgn-??" tags in favour of the 639-3 code element having the same denotation.

-          Come up with appropriate recommendations for matching or fallback between a given macrolanguage its encompassed languages.

We did not have discussion leading to any specific conclusions regarding matching/fallback or fields that might be added to entries to support that.

We did not discuss whether any macrolanguage IDs should be deprecated; e.g. whether zh should be deprecated in favour of cmn, yue, etc.

Peter

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Addison Phillips <[email-masked]>
Date: Sat, 01 Dec 2007 11:48:20 -0800

I believe Peter's summary captures the essence of what came out of the 
teleconference.

Addison

Peter Constable wrote:
> During the teleconf call this week, those of us that participated came 
> to consensus on recommending these directions for 4646bis (at least, 
> this is the understanding I came away with):
> 
>  
> 
> -          Deprecate all grandfathered tags like “zh-yue” in favour of 
> just “yue”.
> 
> -          Deprecate redundant “sgn-??” tags in favour of the 639-3 code 
> element having the same denotation.
> 
> -          Come up with appropriate recommendations for matching or 
> fallback between a given macrolanguage its encompassed languages.
> 
>  
> 
> We did not have discussion leading to any specific conclusions regarding 
> matching/fallback or fields that might be added to entries to support that.
> 
>  
> 
> We did not discuss whether any macrolanguage IDs should be deprecated; 
> e.g. whether zh should be deprecated in favour of cmn, yue, etc.
> 
>  
> 
>  
> 
> Peter
> 
>  
> 
> 
> ------------------------------------------------------------------------
> 

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Mon, 3 Dec 2007 09:29:27 +0100

On Sat, Dec 01, 2007 at 10:58:57AM -0800,
 Peter Constable <[email-masked]> wrote 
 a message of 262 lines which said:

> Do not pursue use of the extlang subtag; 

I understand

> remove extlang from the ABNF.

Warning, it means that tags which were well-formed in RFC 4646 (but
not valid since no extlang was registered) won't be well-formed
anymore. Isn't a violation of the stability principle? (Yes, the
"extlang" production in RFC 4646 was marked as "reserved for future
use" but it was legal nevertheless.)

---

From: Peter Constable <[email-masked]>
Date: Mon, 3 Dec 2007 13:59:40 -0800

> From: Stephane Bortzmeyer [mailto:[email-masked]]

> use" but it was legal nevertheless.)

Correct. Since there have never been registered extlang subtags, there might be parsers that allow for it but there should not be any tags that include them. If some such tags are out there in the wild, they would be well-formed but not valid in 4646, and neither well-formed nor valid in 4646bis (under this proposal).

Peter

---

From: "Felix Sasaki" <[email-masked]>
Date: Tue, 4 Dec 2007 10:35:06 +0900 (JST)

> Correct. Since there have never been registered extlang subtags, there
> might be parsers that allow for it but there should not be any tags that
> include them. If some such tags are out there in the wild, they would be
> well-formed but not valid in 4646, and neither well-formed nor valid in
> 4646bis (under this proposal).

Note that this change has consequences not only for implementations, but
also for specifications citing BCP 47 / RFC 4646. To give an example, see
e.g.
http://www.w3.org/Bugs/Public/show_bug.cgi?id=3079#c2
this is a comment on a W3C draft of "XML Schema 1.1 , part 2 - datatypes",
about the "language" data type. We were saying
"We would like to propose that you use the ABNF defined in RFC 4646. This
ABNF is stable."

So if extlang is not legal anymore, we will need to change that comment.

Felix

---

From: "Kent Karlsson" <[email-masked]>
Date: Tue, 4 Dec 2007 11:05:05 +0100

Felix Sasaki wrote:

> also for specifications citing BCP 47 / RFC 4646. To give an 
> example, see e.g.
> "We would like to propose that you use the ABNF defined in 
> RFC 4646. This ABNF is stable."

I, for one, don't mind keeping the extlang portions of the syntax,
which even allows for up to three levels of extlangs (which was
more than intented to be used 4646bis before the decision not
to use this mechanism at all). It does not seem particularly
harmful to keep it, even if never used for valid language tags.

	/kent k

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 4 Dec 2007 12:29:39 +0100

On Tue, Dec 04, 2007 at 11:05:05AM +0100,
 Kent Karlsson <[email-masked]> wrote 

> I, for one, don't mind keeping the extlang portions of the syntax,

I believe we must keep it, because we commited to stability. Once
well-formed, a tag should always be well-formed.

> It does not seem particularly harmful to keep it, even if never used
> for valid language tags.

+1 for me

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 4 Dec 2007 14:38:07 +0100

Stephane Bortzmeyer wrote:

> Once well-formed, a tag should always be well-formed.

There are no extlang tags yet, well-formed or otherwise.
Reducing "3" if possible in 4646bis was always the plan.

 Frank

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 4 Dec 2007 17:45:08 +0100

On Tue, Dec 04, 2007 at 02:38:07PM +0100,
 Frank Ellermann <[email-masked]> wrote 

> There are no extlang tags yet, well-formed or otherwise.

Are you sure? Did you read every XML document published to check the
language tag they indicate? It is *probably* true but I find
nevertheless annoying that we change the syntax in a way that will
make well-formed tags illegal. It will be difficult to claim that we
place stability over everything after that.

---

From: "Ira McDonald" <[email-masked]>
Date: Tue, 4 Dec 2007 13:22:47 -0500

Hi,

My two cents - Stephane is dead right - as an outsider who is a protocol
guy, I'm appalled that his argument isn't obvious to this list.

Cheers,
- Ira

-- 
Ira McDonald (Musician / Software Architect)
Chair - Linux Foundation Open Printing WG
Blue Roof Music / High North Inc
PO Box 221  Grand Marais, MI  49839
work: +1-906-494-2434
home: +1-906-494-2697
email: [email-masked]

---

From: John Cowan <[email-masked]>
Date: Tue, 4 Dec 2007 14:05:05 -0500

> guy, I'm appalled that his argument isn't obvious to this list.

(Disclaimer: I don't favor removing the extlang syntax from RFC 4646bis,
as I don't think removing it is necessary.)

The question is not so clear-cut as the terms "well-formed" and "valid"
(which are derived from XML, and ultimately from predicate logic) would
suggest.  Well-formed but not valid language tags are much more like
unassigned Unicode codepoints.  They are part of the space of possible
tags, but they have no meaning, and it is entirely inappropriate for
anyone to use them to tag anything.  Specifically, anyone who does is
subject to having his meanings misinterpreted if and when the tag ever
does become valid.

Removing the extlang syntax is like the ISO committee deciding to remove
the codepoints 110000 to 7FFFFFFF, as they did recently, in order to align
with Unicode's codespace.  No characters had ever been assigned there,
none were ever expected to be assigned there, many things would break
if any were ever assigned there.  So calling that a breach of backward
compatibility is true only in the most technical sense.

> > language tag they indicate? 

If any such documents exist, the language tags in them have no
interoperable meaning.

-- 
John Cowan       http://www.ccil.org/~cowan        <[email-masked]>
        You tollerday donsk?  N.  You tolkatiff scowegian?  Nn.
        You spigotty anglease?  Nnn.  You phonio saxo?  Nnnn.
                Clear all so!  `Tis a Jute.... (Finnegans Wake 16.5)

---

From: Shawn Steele <[email-masked]>
Date: Tue, 4 Dec 2007 11:23:50 -0800

> There are no extlang tags yet, well-formed or otherwise.

A quick web search shows several instances of extlang tags.  In many cases it seems that many of these "jumped the gun" with the description in RFC4646.  This includes several best practices docs and other sites.  The W3C's site even implies that zh-cmn should be used. (it does say that its "pending" but then pretty much describes extlang's as fait accompli.)

- Shawn

---

From: John Cowan <[email-masked]>
Date: Tue, 4 Dec 2007 14:30:40 -0500

Shawn Steele scripsit:

> A quick web search shows several instances of extlang tags.  In many
> cases it seems that many of these "jumped the gun" with the description
> in RFC4646.  This includes several best practices docs and other sites.
> The W3C's site even implies that zh-cmn should be used. (it does say
> that its "pending" but then pretty much describes extlang's as fait
> accompli.)

zh-cmn and a bunch of other zh-* are grandfathered.  What search did you use?

-- 
John Cowan                                [email-masked]
At times of peril or dubitation,          http://www.ccil.org/~cowan
Perform swift circular ambulation,
With loud and high-pitched ululation.

---

From: Addison Phillips <[email-masked]>
Date: Tue, 04 Dec 2007 11:59:00 -0800

> A quick web search shows several instances of extlang tags.  In many cases it seems that many of these "jumped the gun" with the description in RFC4646.  This includes several best practices docs a…

No, those are registered tags from RFC 3066 days.

Addison

---

From: Shawn Steele <[email-masked]>
Date: Tue, 4 Dec 2007 12:45:28 -0800

> zh-cmn and a bunch of other zh-* are grandfathered.  What search did you use?

http://www.w3.org/International/articles/language-tags for one example, I assume you're using search.live.com? ;-)  I assume someone'll update Wikipedia? "Extended Language Tags", "zh-cmn", "RFC 4646 extlang", etc.

I'm not trying to assign a "correctness" value, I'm just mentioning that lazy developers are likely to find sources such as this and work from there without reading 4646 or 4646bis.  It appears that some people took the reserved language part of RFC4646 and turned that into a best practice.

So if moving away from extlang, there should be an effort to reeducate these people :)

- Shawn

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 00:06:44 +0100

Ira McDonald wrote:

> I'm appalled that his argument isn't obvious to this list.

I'm appaled that simpler + shorter isn't obviously better
when longer never existed.  There's no practical problem
with never used extlang-s.  4646 implementations supporting
them in theory can't cause harm if they "misidentify" a tag
as "well formed" that never existed, and isn't well formed
any more in 4646bis.

On the other hand forcing 4646bis implementations to accept
something as well formed that doesn't exist *IS* harmful.  

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 00:16:58 +0100

Shawn Steele wrote:

> if moving away from extlang, there should be an effort to
> reeducate these people :)

The idea was that 4646 implementations can deal with whatever
happens in 4646bis wrt extlang-s, and "3" was identified as
worst case.  "1" was what I hoped for 4646bis, "0" is better.

Updating Wikipedia isn't our job - just a few days ago I've
updated the "frs" and "gsw" articles, IIRC "frs" still had
an old SIL code.  
 
 Frank

---

From: "Felix Sasaki" <[email-masked]>
Date: Wed, 5 Dec 2007 09:52:10 +0900 (JST)

> A quick web search shows several instances of extlang tags.  In many cases
> it seems that many of these "jumped the gun" with the description in
> RFC4646.  This includes several best practices docs and other sites.  The
> W3C's site even implies that zh-cmn should be used. (it does say that its
> "pending" but then pretty much describes extlang's as fait accompli.)

and this is the tip of an iceberg: We spend a fair amount of time to
evangelize BCP 47 with RFC 4646 syntax this year within W3C (not only XML
Schema WG), always saying "The ABNF is stable". I don't plan to go back to
these people to say now "stable, but ...", but if it can't be avoided ...

Felix

---

From: "Ira McDonald" <[email-masked]>
Date: Tue, 4 Dec 2007 20:19:49 -0500

Hi,

Ditto - I was just explaining extlang subtags in a localization overview two
weeks ago to an IEEE-ISTO PWG working group doing a Projector MIB.

I respectively suggest that the ABNF production be left intact, but an
ABNF comment (and preceding/following body text) be added that says
"DEPRECATED - present only for backward compatibility".

Cheers,
- Ira

PS - ABNF is supposed to be totally stable in PWG standards-track specs.

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 02:57:42 +0100

Felix Sasaki wrote:

> We spend a fair amount of time to evangelize BCP 47 with RFC 4646
> syntax this year within W3C (not only XML Schema WG)

That the WG claiming that IRI is just a modern name for URI, or do I
confuse this with XHTML modularization ?

> always saying "The ABNF is stable". I don't plan to go back to
> these people to say now "stable, but ..."

It's stable in the sense that it won't match any existing tag, and
it will continue to not match any existing tag.    It will only miss 
some non-existing tags, where a 4646bis implementation can be
smarter based on a simplified ABNF.

BTW, the discussion of the "most perverse tag" isn't affected 
by keeping or trimming the obsolete ABNF, zero extlang-s is
zero extlang-s no matter what the ABNF says.

 Frank

---

From: "Felix Sasaki" <[email-masked]>
Date: Wed, 5 Dec 2007 11:38:38 +0900 (JST)

> it will continue to not match any existing tag.

you are discussing in terms of data. I argued to working groups in terms
of an ABNF and asking them "please implement this ABNF in your spec, it is
stable *as an ABNF*".

This argument of stability of the ABNF (not data) was (and is) is a very
strong means for BCP 47 outreach to emerging technologies. Changing the
ABNF now means potentially loosing this argument.

Felix

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 4 Dec 2007 18:52:24 -0800

I agree completely with John. There are no possible current *valid* tags
that use extlang -- it was only a possibility that we left open for the
future. By removing extlang, we do no damage at all to existing data or
code.

Mark

On Dec 4, 2007 11:05 AM, John Cowan <[email-masked]> wrote:

> John Cowan       http://www.ccil.org/~cowan <http://www.ccil.org/%7Ecowan>       <
> [email-masked]>

-- 
Mark

---

From: John Cowan <[email-masked]>
Date: Tue, 4 Dec 2007 22:30:42 -0500

> future. By removing extlang, we do no damage at all to existing data or

As I said, I don't favor removing them -- there is no gain from doing so
except a theoretical nicety.

-- 
Well, I have news for our current leaders       John Cowan
and the leaders of tomorrow: the Bill of        [email-masked]
Rights is not a frivolous luxury, in force      http://www.ccil.org/~cowan
only during times of peace and prosperity.
We don't just push it to the side when the going gets tough.  --Molly Ivins

---

From: Felix Sasaki <[email-masked]>
Date: Wed, 05 Dec 2007 12:33:48 +0900

Mark Davis wrote:
> I agree completely with John. There are no possible current *valid* 
> tags that use extlang -- it was only a possibility that we left open 
> for the future. By removing extlang, we do no damage at all to 
> existing data or code.
how about existing specs that cite RFC 4646 and the "stable" ABNF?

Felix

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 4 Dec 2007 19:36:01 -0800

Sorry; didn't mean to imply that. I meant that I agree with your discussion
of why removing them is not an formal problem.

While I agree that we don't have to remove the syntax, removing them would
be more than a nicety: it makes the syntax checking significantly simpler,
as well as simplifying any APIs that are used with language tags.

Mark

On Dec 4, 2007 7:30 PM, John Cowan <[email-masked]> wrote:

> Rights is not a frivolous luxury, in force      http://www.ccil.org/~cowan<http://www.ccil.org/%7Ecowan>

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 4 Dec 2007 19:49:51 -0800

It doesn't cause a problem. Any existing tag that does not match the new
ABNF was, in fact, invalid under RFC 4646.

(The ABNF is the "convex hull" of the definition of valid language tags; all
valid language tags must follow the ABNF, but the vast majority of strings
that do fit the ABNF are not valid language tags.)

As far as I can see, we have the following alternatives:

   1. Remove extlang from the ABNF.
   2. Leave extlang in the ABNF, but document that it is "permanently
   reserved" (will never be used, a la ISO).
   3. Leave extlang in the ABNF, and just leave in the documentation from
   RFC 4646 that it is not used in the current version.

Here is my take.

   - #1 is the cleanest (for reasons given earlier).
   - #2 is functionally identical to #1, but may feel warmer and fuzzier
   to some people.
   - #3 we could live with, although it would be slightly suboptimal.

Mark

On Dec 4, 2007 7:36 PM, Mark Davis <[email-masked]> wrote:

> Sorry; didn't mean to imply that. I meant that I agree with your
> discussion of why removing them is not an formal problem.
>
> > tags
> > <http://www.ccil.org/%7Ecowan>

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 05:00:08 +0100

> how about existing specs that cite RFC 4646 and the "stable" ABNF?

How about W3C documents using raw IRIs in documents claiming to
be valid HTML 4 or XHTML 1 with a spec. talking about RFC 2396 ?
Or the odd system ID in the XML spec. working nowhere including
the W3C validator.  

IIRC you made sure that these spec.s say BCP 47, and where they
didn't follow your guidance its their fault, not yours.  It was
kind of obvious that the three extlang-s were reserved precisely
in the hope that 4646bis tags won't break 4646 parsers.  In fact
4646bis tags also won't break 3066 parsers, that's how they are
designed.  

OTOH new 4646bis implementations can be both simpler and smarter
than old 4646 or older 3066 implementations.  If folks need say
XML 1.0 2nd ed. as opposed to "XML 1.0 de jour" they'll have to
go to the trouble to spell it out, nothing's unusual about this.

BTW, the 4646bis drafts already modified the ABNF, the extlang
business is not the only update, there is already a new concept
of "irregular".

 Frank

---

From: John Cowan <[email-masked]>
Date: Tue, 4 Dec 2007 23:00:35 -0500

>    reserved" (will never be used, a la ISO).

I favor this.

-- 
Yes, chili in the eye is bad, but so is your    John Cowan
ear.  However, I would suggest you wash your    [email-masked]
hands thoroughly before going to the toilet.    http://www.ccil.org/~cowan
        --gadicath

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 05:11:37 +0100

>    - #2 is functionally identical to #1, but may feel warmer and
>         fuzzier to some people.

#3 is weird, and in #2 <extlang> needs to be replaced by <unused>
as name of the sytactical term.  It shouldn't feel warm and fuzzy.

 Frank (+1 for #1)

---

From: Felix Sasaki <[email-masked]>
Date: Wed, 05 Dec 2007 13:41:10 +0900

>>    reserved" (will never be used, a la ISO).
me too.

Felix

---

From: Felix Sasaki <[email-masked]>
Date: Wed, 05 Dec 2007 13:55:09 +0900

> go to the trouble to spell it out, nothing's unusual about this.

Maybe you are not aware of the situation that language tags are nearly 
the only IETF spec where W3C uses a BCP reference, or before that an 
"RFC XXX or its successor" reference - in difference to IRI, btw .

All I want is that BCP 47 will not loose this special status within W3C, 
and having as less changes to the ABNF as really necessary will make it 
much easier to achieve this. You might say you don't care about this 
status, but reading Ira's mail, it seems other people are in a similar 
situation.

Regarding the change of "irregular", this seems to me much easier to 
sell to people than deleting the extlang production.

Felix

---

From: Addison Phillips <[email-masked]>
Date: Tue, 04 Dec 2007 20:57:57 -0800

For compatibility, I'm for #2. Like Frank, I'd prefer #1, but would 
prefer to preserve the ABNF for compatibility purposes. And #2 appears 
to be a lot like what we have in draft-10, save that we could remove a 
lot of the normative gunk about extlang.

Addison

Mark Davis wrote:
> (The ABNF is the "convex hull" of the definition of valid language tags; 
> all valid language tags must follow the ABNF, but the vast majority of 
> strings that do fit the ABNF are not valid language tags.)
> 
>    3. Leave extlang in the ABNF, and just leave in the documentation
>       from RFC 4646 that it is not used in the current version.
> 
> Here is my take.
> 
>     * #1 is the cleanest (for reasons given earlier).
>     * #2 is functionally identical to #1, but may feel warmer and
>       fuzzier to some people.
>     * #3 we could live with, although it would be slightly suboptimal.
> 
> Mark
> 
> On Dec 4, 2007 7:36 PM, Mark Davis < [email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     While I agree that we don't have to remove the syntax, removing them
>     would be more than a nicety: it makes the syntax checking
>     significantly simpler, as well as simplifying any APIs that are used
>     with language tags.
> 
>     Mark
> 
> 
>     On Dec 4, 2007 7:30 PM, John Cowan <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>         Mark Davis scripsit:
> 
>         <mailto:[email-masked]>
>         Rights is not a frivolous luxury, in force      
>         http://www.ccil.org/~cowan <http://www.ccil.org/%7Ecowan>

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 4 Dec 2007 21:29:39 -0800

I'm good with that.

On Dec 4, 2007 8:57 PM, Addison Phillips <[email-masked]> wrote:

> >         http://www.ccil.org/~cowan <http://www.ccil.org/%7Ecowan> <
> http://www.ccil.org/%7Ecowan>

---

From: Peter Constable <[email-masked]>
Date: Wed, 5 Dec 2007 09:09:13 -0800

> From: John Cowan [mailto:[email-masked]]

> except a theoretical nicety.

The potential gain would be making it easier to grok and implement the syntax and explain it to others. It may (or may not) be that comments in the ABNF statement clearly indicating deprecation may allow that.

Peter

---

From: Peter Constable <[email-masked]>
Date: Wed, 5 Dec 2007 09:19:43 -0800

I think #3 would be more than just slightly sub-optimal. Many people would look just at the statement of the ABNF and go away and do things on that basis. The statement of the ABNF should somehow indicate clearly that extlang is DEAD, never to be resurrected.

We need to be clear about the decision we’re making: we are deciding to abandon extlang *permanently*. Removing extlang from the ABNF will leave something that is more-certainly stable in that particular regard. Of course, it doesn’t make the ABNF as a whole stable; but do keep in mind: we would be removing from the ABNF something that was reserved for possible future use. We’ve decided to abandon that future possibility. That is not a destabilizing change on the same order as destabilizing some aspect of the ABNF that was already in use. (Granted, though, some other parties may be concerned about any form of instability.)

Peter

From: Mark Davis [mailto:[email-masked]]
Sent: Tuesday, December 04, 2007 7:50 PM
To: John Cowan
Cc: Frank Ellermann; [email-masked]
Subject: Re: [Ltru] Re: Remove extlang from ABNF?

It doesn't cause a problem. Any existing tag that does not match the new ABNF was, in fact, invalid under RFC 4646.

(The ABNF is the "convex hull" of the definition of valid language tags; all valid language tags must follow the ABNF, but the vast majority of strings that do fit the ABNF are not valid language tags.)

 1.  Remove extlang from the ABNF.
 2.  Leave extlang in the ABNF, but document that it is "permanently reserved" (will never be used, a la ISO).
 3.  Leave extlang in the ABNF, and just leave in the documentation from RFC 4646 that it is not used in the current version.
Here is my take.

 *   #1 is the cleanest (for reasons given earlier).
 *   #2 is functionally identical to #1, but may feel warmer and fuzzier to some people.
 *   #3 we could live with, although it would be slightly suboptimal.
Mark
On Dec 4, 2007 7:36 PM, Mark Davis < [email-masked]<mailto:[email-masked]>> wrote:
Sorry; didn't mean to imply that. I meant that I agree with your discussion of why removing them is not an formal problem.

While I agree that we don't have to remove the syntax, removing them would be more than a nicety: it makes the syntax checking significantly simpler, as well as simplifying any APIs that are used with language tags.

Mark

On Dec 4, 2007 7:30 PM, John Cowan <[email-masked]<mailto:[email-masked]>> wrote:

--
Well, I have news for our current leaders       John Cowan
and the leaders of tomorrow: the Bill of        [email-masked]<mailto:[email-masked]>
Rights is not a frivolous luxury, in force       http://www.ccil.org/~cowan <http://www.ccil.org/%7Ecowan>
only during times of peace and prosperity.
We don't just push it to the side when the going gets tough.  --Molly Ivins

--
Mark

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 5 Dec 2007 17:06:08 +0100

> "RFC XXX or its successor" reference - in difference to IRI, btw .

Yes, some including me proposed standards track for 4646, but the
rough consensus was BCP :-(  You use 2119 (BCP 14) quite often in W3C
spec.s, but that's no moving target.  Unlike RFC 4646, were "moving
target" was predictable at least for extlang:

| reserved for future standardization, anticipating work that is
| currently under way on ISO 639.
[...]
| Their syntax is described here so that implementations can be
| compatible with any future revision of this document that does
| provide for their registration.
[...]
| "zh-gan" (registered under RFC 3066) might become a valid non-
| grandfathered (that is, redundant) tag in which the subtag 
| 'gan' might represent the Chinese dialect 'Gan'.

Note "might", it's not "shall".  We weren't sure, and the reasons
to drop "extlang" are rather complex (e.g. 639-3 is more volatile
than expected with its macrolanguages at the moment, not good for
a "stable forver" registry).

I still don't see the harm if an implementation based on RFC 4646
identifies a bogus (non-existing) tag as "well-formed", where a 
4646bis implementation would see "not well-formed" with the clean
ABNF proposal.  And both editors would apparently prefer "clean".

> You might say you don't care about this status

And I did, see above, 4646bis could be a nice "draft standard"
soon.  Dropping unused features in a DS is perfectly okay.  It's
also okay in a BCP, but apparently you tjink it's not.

> reading Ira's mail, it seems other people are in a similar 
> situation.

I'm not sure in what situation she would be with a simpler ABNF,
she used to be a "KISS" fan.  I've no idea what a printer could
do with a "well-formed" 4646 extlang tag when no extlangs exist.

> Regarding the change of "irregular", this seems to me much 
> easier to sell to people than deleting the extlang production.

In -10 it's still "extlang = [...] permanently reserved", in -11
it might be "unused = [...] permanently unused".  Only to keep
constructs well-formed that permanently won't exist.  Some folks
were less timid while introducing IPv6 literals in URLs, and W3C
documents use all kinds of mixtures, pure 2396, 2396 as amended
by ????, and the real thing, STD 66 (3986).  

After all "RFC" still stands for "request for comment", even BCPs
change (RFC 3066 was BCP 47 before, RFC 1766 was a proper PS).

 Frank

---

From: Peter Constable <[email-masked]>
Date: Wed, 5 Dec 2007 09:24:26 -0800

> From: Felix Sasaki [mailto:[email-masked]]

> >>    reserved" (will never be used, a la ISO).

"Permanently reserved" is not strong enough, IMO. It must be clear that extlang is *permanently dead*. I think #1 makes that clearer, but I'd accept a variation of #2 that left no doubt.

Peter

---

From: Felix Sasaki <[email-masked]>
Date: Thu, 06 Dec 2007 02:48:19 +0900

> also okay in a BCP, but apparently you tjink it's not.

exactly - or rather specs citing BCP 47. That's my whole point - the 
danger that specs writers might look at the dropped extlang and say 
"they are dropping features between versions of  BCP 47, so we better 
refer to an RFC *only* and even leave 'or its successor' out". If they 
would have done that already, we would have many "dead references" to 
RFC 4646 .

Felix

---

From: Felix Sasaki <[email-masked]>
Date: Thu, 06 Dec 2007 02:56:33 +0900

Peter Constable wrote:
>
> I think #3 would be more than just slightly sub-optimal. Many people 
> would look just at the statement of the ABNF and go away and do things 
> on that basis. The statement of the ABNF should somehow indicate 
> clearly that extlang is DEAD, never to be resurrected.
>
>  
>
> We need to be clear about the decision we’re making: we are deciding 
> to abandon extlang **permanently**. Removing extlang from the ABNF 
> will leave something that is more-certainly stable in that particular 
> regard. Of course, it doesn’t make the ABNF as a whole stable
>

I went through the draft looking for "stabl", and found it only in 
relation to validity (including the registry in general) and 
grandfathered sub tags. I found nothing about the ABNF. Secion 8 of RFC 
4646 says:

"... in no case is the syntax more permissive and processors based on 
the ABNF and other provisions of RFC 3066 (such as those described in 
[XMLSchema]) will be able to process the tags described by this document."

I think it would be very helpful (again esp. for outreach to other 
technologies) to have a statement in the main text about the stability 
and backwards compatibility of the ABNF. Of course one cannot predict 
the future of the ABNF, but expressing the intention might help already.

Felix

---

From: Peter Constable <[email-masked]>
Date: Wed, 5 Dec 2007 09:59:32 -0800

> refer to an RFC *only* and even leave 'or its successor' out".

But this is a key: we are *not* dropping any features. We are dropping the possibility of a future feature. The change to the ABNF (whether by removing the extlang subtag entirely or by renaming and/or comments) is to clean it up so that implementers do not implement for non-and-never-to-be-features.

Peter

---

From: Felix Sasaki <[email-masked]>
Date: Thu, 06 Dec 2007 03:08:42 +0900

> But this is a key: we are *not* dropping any features. We are dropping the possibility of a future feature. The change to the ABNF (whether by removing the extlang subtag entirely or by renaming an…

Looking closer to the spec than the ABNF, of course I see you point - 
but I'm still worried about the unclear status of the ABNF, see my other 
mail about the missing description of the ANBF stability.

Btw., I would be fine with your version of option #2 as well.

Felix

---

From: Shawn Steele <[email-masked]>
Date: Wed, 5 Dec 2007 11:32:55 -0800

> "Permanently reserved" is not strong enough, IMO. It must be clear that extlang is *permanently dead*. I think > #1 makes that clearer, but I'd accept a variation of #2 that left no doubt.

I agree with Peter that if its dead it should be made clear that it is completely dead.  "Permanently reserved" implies that software still needs to consider the possibility that such tags would exist, adding complexity that will never be used.

- Shawn

---

From: "Ira McDonald" <[email-masked]>
Date: Wed, 5 Dec 2007 16:36:49 -0500

Hi,

And, oddly, RFC 2119 does not define the two important conformance words used
widely in IETF specs "DEPRECATED" (dying) and "OBSOLETE" (dead).  But I urge
that an inline comment in the ABNF use the word OBSOLETE and the text make
clear that conforming implementations MUST NOT produce or consume "extlang"
subtags.

Cheers,
- Ira

---

From: Peter Constable <[email-masked]>
Date: Wed, 5 Dec 2007 13:44:52 -0800

> From: Ira McDonald [mailto:[email-masked]]

> clear that conforming implementations MUST NOT produce or consume

Of course, the only difference between that and removing extlang from the ABNF is the ability to claim, in *some* sense, the ABNF has not changed. But also of course, there is a very real change of syntax, so that claim is somewhat vacuous.

Peter

---

From: Addison Phillips <[email-masked]>
Date: Wed, 05 Dec 2007 15:14:14 -0800

Would making the ABNF read like the following be adequate:

...

language      = (2*3ALPHA [ extlang ]) ; shortest ISO 639 code
               / 4ALPHA                 ; reserved for future use
               / 5*8ALPHA               ; registered language subtag

extlang       = *3("-" 3ALPHA)         ; OBSOLETE (do not use)

...

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 5 Dec 2007 15:16:57 -0800

LGTM

On Dec 5, 2007 3:14 PM, Addison Phillips <[email-masked]> wrote:

> >>> "Permanently reserved" is not strong enough, IMO. It must be clear
> that extlang is *permanently dead*. I think > #1 makes that clearer, but I'd
> accept a variation of #2 that left no doubt.
> >> I agree with Peter that if its dead it should be made clear that it is
> completely dead.  "Permanently reserved" implies that software still needs
> to consider the possibility that such tags would exist, adding complexity
> that will never be used.
> >>
> >> - Shawn
> >>
> >>

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 6 Dec 2007 01:11:16 +0100

> extlang       = *3("-" 3ALPHA)         ; OBSOLETE (do not use)

s/extlang/obs-extlang/g

Corresponding prose somewhere:

| The <obs-extlang> construct was reserved in [RFC 4646] for future
| use, and later identified as unnecessary.  It will never be used,
| its sole purpose is a definition of formally "well-formed" tags
| (see Section X.Y) matching the definition given in [RFC 4646].

Of course I still don't like this rubbish, the folks wanting it did
not post a compelling example for what purpose they really need it.

What interoperability issue do they try to solve with <obs-extlang> ?
They get a parser ambiguity for <grandfathered> tags matching also
<obs-extlang>, in which way is that a good thing, less <irregular>
tags maybe ?

 Frank

---

From: Martin Duerst <[email-masked]>
Date: Thu, 06 Dec 2007 14:01:30 +0900

<chair hat on>
I think there is (at least) rough consensus that we do not want
to completely remove the extlang production from the ABNF in
RFC 4646bis. (I only see Frank cleary opposing this, everybody
else is either for it or is okay with it.)

>extlang       = *3("-" 3ALPHA)         ; OBSOLETE (do not use)

<chair hat off>
I would be okay with this, with some explaining text.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Thu, 06 Dec 2007 14:10:22 +0900

>4646bis (under this proposal).

My understanding is that if we leave the production in the ABNF,
such tags would still be well-formed. I don't think we can have
it both ways. The tags would be well-formed but obsolete.

I think we should be very clear about what the consequences of our
draft are. I think we are extremely clear on the fact that we don't
want anybody to generate tags with extlangs (except for a few pre-
registered ones (*)). But what about applications receiving these
tags? Are we okay with applications that accept tags with extlangs
as well-formed? Are we okay with applications that don't accept
tags with extlangs as well-formed? Are we okay with both? What
does that mean for existing implementations?

Regards,    Martin.

---

From: Shawn Steele <[email-masked]>
Date: Thu, 6 Dec 2007 00:40:44 -0800

> I would be okay with this, with some explaining text.

What do we expect OBSOLETE to do?  Ok, I get that I shouldn't use the obsolete tag to encode new data, but are applications expected to recognize such obsolete tags, even if they weren't ever valid?

If applications aren't expected to recognize them, then why declare it formally?  Why not just say in the prose that there use to be such an extlang thought, but its obsolete?  If someone reading RFC3066bis is really curious they could look at RFC3066 and see what the original idea was.

My concern is that if a developer/spec writer/tester is reading this they might feel that its best to go ahead and decide to recognize these in case any of these obsolete tags are encountered.  That could require a considerable expense depending on the depth of the effort.  If it were clear that none have ever been registered or legally allowed, and it was obsoleted before it was ever implemented, then the hypothetical development team could save that expense and effort.

- Shawn

---

From: Peter Constable <[email-masked]>
Date: Thu, 6 Dec 2007 08:00:31 -0800

> From: Martin Duerst [mailto:[email-masked]]

> else is either for it or is okay with it.)

I will go along if no consensus can be formed to remove it, though I still lean toward removing it.

Peter

---

From: Peter Constable <[email-masked]>
Date: Thu, 6 Dec 2007 10:46:57 -0800

I notice that Doug and Shawn also side with Frank in wanting extlang removed from the ABNF. So, perhaps the chair's conclusion is slightly premature.

Peter

> -----Original Message-----
> From: Peter Constable [mailto:[email-masked]]
> Sent: Thursday, December 06, 2007 8:01 AM
> To: [email-masked]
> Subject: RE: [Ltru] Re: Remove extlang from ABNF?
>

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 6 Dec 2007 21:37:17 +0100

> it both ways. The tags would be well-formed but obsolete.

The logic of <grandfathered> regular vs. irregular tags is affected
by <obs-extlang>.  If <obs-extlang> exists, then zh-min-nan is
regular, otherwise it's irregular.  I think.  I'm not going to
explain this to anybody who asks, I'd need a flow chart to grok it.

> I think we are extremely clear on the fact that we don't want 
> anybody to generate tags with extlangs (except for a few pre-
> registered ones (*)).

Did you forget a footnote (*) in your article ?  I'm not sure
that say zh-min-nan counts as using "pre-registered extlangs",
because min and nan were never registered as extlangs with
prefixes zh and min resp., RFC 4646 doesn't permit this.  It
only reserved the possibility.

OTOH zh-min-nan matches <language> + <obs-extlang>, messy... :-(

 Frank

---

From: Marion Gunn <[email-masked]>
Date: Fri, 7 Dec 2007 15:57:58 +0000

On 6 Dec 2007, at 18:46, scrIobh Peter Constable:

> I notice that Doug and Shawn also side with Frank in wanting  
> extlang removed from the ABNF. So, perhaps the chair's conclusion  
> is slightly premature.
>
>
> Peter

It would certainly seem to be premature, Peter, which I say right now  
because so do I (side with Frank and Doug and Shawn on this  
particular issue, most definitely).
mg

--
Marion Gunn
[email-masked]

- -
Marion Gunn * EGTeo (Estab.1991)
27 Páirc an Fhéithlinn, Baile an
Bhóthair, Co. Átha Cliath, Éire.
* [email-masked] * [email-masked] *