Subject: [Ltru] Proposals for changes to latest draft

From: "Mark Davis" <[email-masked]>
Date: Tue, 27 Feb 2007 07:58:53 -0800

I have a number of proposals for changes to the current draft. So that we
have specific wording to discuss, I posted a modified draft on the web at

http://docs.google.com/Doc?id=dfqr8rd5_4gh864p

The deletions are marked with strikeout, and the additions with yellow. (The
formatting is somewhat disturbed in using this mechanism, so ignore any
other formatting differences from the latest HTML draft on Addison's site).

Mark
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Tue, 27 Feb 2007 14:50:53 -0500

Mark Davis scripsit:

> The deletions are marked with strikeout, and the additions with yellow.
> (The formatting is somewhat disturbed in using this mechanism, so
> ignore any other formatting differences from the latest HTML draft on
> Addison's site).

Here's my take:

2.2.3 bullet 6 adds the notion that the hitherto private tag Qabx is to
be usurped to mean "no written content".  I agree that the "unwritten
languages" script tag does not do the job here, but I deny that it is
proper to steal a private-use tag, and I don't see any need to have such
an entity at all.

2.2.9:  I'm happy to add the notion of well-formed and valid tags
officially, since we use them informally.  I'm also happy with the idea
that repeated variant subtags are inherently invalid.  However, the change
to the interpretation of variant prefixes does not seem to deal correctly
with the case of a variant subtag with more than one Prefix: field.

3.1.2 has new language to prevent recursive variant subtags: thus if
avariant has a prefix of xx-bvariant, bvariant cannot have a prefix
of xx-avariant.  +1

3.1.4: specifies some types of irregularities in descriptions that may
be fixed.  Motherhood.  +1

3.2 is heavily recast with lots of "constitutional" baggage around the
LSR: two-year term, alternate appointed, MUST reflect the consensus
of informed opinion on the list (whose opinions are considered to
be informed?)  All of this is entirely unnecessary to say the least.
All we need is a statement affirming the right of the LSR to delegate
his duties while remaining responsible for them.

3.3 now has the LSR sending whole new registries to IANA on each change.
This is a very error-prone procedure, as entries could be added or dropped
in error.  The current procedure of instructing IANA which entries to
add or change (there are no drops) is now working well and should be
left alone.  Whole new registries are relevant when we add bulk changes
at the time of a new RFC.

3.5 has an editorial change.  +1.

4.1 adds another version of the stricture against -variant1-variant1
tags.  +1

4.4 adds a new rule for canonicalizing tags with multiple variant
subtags, thus:

Any sequence of variant tags MUST be reordered such that for each variant subtags A and B, A comes before B if

   1. A has no prefix and B has a prefix
   2. or A and B are not ordered by #1, and A is part of a prefix (given
      the previous non-variant subtags) for B
   3. or A and B are not ordered by #1 or #2, and A is alphabetically
      before B

I'm good with this.

5.1 has more language about sending whole registries.  -1 as noted above.

-- 
John Cowan    http://ccil.org/~cowan    [email-masked]
Economists were put on this planet to make astrologers look good.
        --Leo McGarry

---

From: Addison Phillips <[email-masked]>
Date: Tue, 27 Feb 2007 12:20:20 -0800

Comments follow on John's comments.

Addison

> proper to steal a private-use tag, and I don't see any need to have such

+1. Further, I'm not sure why one would tag content as "not written". 
Not including a script subtag does an admirable job of indicating the 
language in these cases.

> that repeated variant subtags are inherently invalid.  

I'm not sure I'm happy changing from measuring implementations to 
measuring tags. My problem is that it is more useful in practice to talk 
about the capabilities of a particular implementation.

> However, the change

The text on variant prefixes needs some minor massaging before 
inclusion, I think.

> avariant has a prefix of xx-bvariant, bvariant cannot have a prefix

+1, although it seems a bit pedantic.

> 3.1.4: specifies some types of irregularities in descriptions that may

+1

> his duties while remaining responsible for them.

I agree that we need a statement on LSR delegation.

I heavily agree with the change of SHOULD to MUST in Section 3.5 
requiring the LSR to actually announce decisions after two weeks 
(grrr....), but some of this section's proposed text seems duplicative.

I think a two-year term is prudent. I don't think we need to have an 
alternate. But the current, open-ended dictatorship is problematic to 
me. I think alternate wording is required to Mark's, though. Also, Mark 
renames the LSR incorrectly. I would suggest instead for the last paragraph:

---
The Language Subtag Reviewer chairs the [email-masked] 
discussion list. The reviewer MAY delegate any list or other 
administrative duties as necessary. As described in Section 3.5, the 
reviewer is solely responsible for determining consensus on a 
registration request and announcing the acceptance or rejection (or 
extension of discussion) of any particular request. The Language Subtag 
Reviewer is not permitted to delegate this responsibility. The decisions 
of the Language Subtag Reviewer MAY be appealed to the IESG under the 
same rules as other IETF decisions (see [RFC2026]). The IESG can reverse 
or overturn the decision of the Language Subtag Reviewer, provide 
guidance, or take other appropriate actions.
---

> at the time of a new RFC.

+1 for John's opposition to sending whole registries.

> 3.5 has an editorial change.  +1.

Actually, it's a substantive change to MUST, which I agree with.

> 4.1 adds another version of the stricture against -variant1-variant1

+1
>    3. or A and B are not ordered by #1 or #2, and A is alphabetically

I'm not. Canonicalization of this nature violates custom and practice in 
the formation of language tags. It has already been shown on the list 
that what one considers "wider" for a subtag another might consider 
"narrower". This requirement will break existing implementations for 
very little real-world improvement.

I'm okay with providing some guidance on tag choice and subtag ordering, 
but not with more algorithm here.

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.

---

From: John Cowan <[email-masked]>
Date: Tue, 27 Feb 2007 15:49:13 -0500

> Not including a script subtag does an admirable job of indicating the 

Omitting the script is technically ambiguous between "no script" and
"may or may not have a script, which may or may not be known".  What
I would like to see proved is that this difference actually makes a
difference in practice.  Typically other kinds of metadata will
differentiate textual from non-textual content.

> about the capabilities of a particular implementation.

We talk about valid or invalid or not well-formed tags all the time.
Consider your email http://www.alvestrand.no/pipermail/ietf-languages/2004-June/002034.html ,
which says "to know whether a tag is well-formed".

> requiring the LSR to actually announce decisions after two weeks 

+1.  I overlooked that.

> (grrr....), but some of this section's proposed text seems duplicative.

I could say more if I wished to.

> me. 

It is *not* a dictatorship.  Michael can be removed by petition to
the IESG.  Furthermore, anyone who wishes to overturn a particular
decision may institute an appeal.

> extension of discussion) of any particular request. 

The LSR is not a moderator, but a judge.  He does not announce,
but decides.  We are his advisors.  This is a revolutionary change.

> but not with more algorithm here.

"We'll argue that point," said Jack.
	--_Mr. Midshipman Easy_

-- 
A mosquito cried out in his pain,               John Cowan
"A chemist has poisoned my brain!"              http://www.ccil.org/~cowan
        The cause of his sorrow                 [email-masked]
        Was para-dichloro-
Diphenyltrichloroethane.                                (aka DDT)

---

From: Addison Phillips <[email-masked]>
Date: Tue, 27 Feb 2007 13:46:51 -0800

> differentiate textual from non-textual content.

Yes, it is technically ambiguous, but there doesn't seem to me to be a 
use case in which it matters.

> which says "to know whether a tag is well-formed".

Yes, yes, all very well. But this section of the document is about 
conformance requirements. Conformance of what? It really shouldn't be 
tags, since they are, ultimately, just strings that are being tested 
somehow. What matters is whether an implementation checks if a given 
string is well-formed, valid, or something else.

I agree that this section will read better if we use Mark's definitions 
based on tag conformance (allowing emails such as mine to pass muster). 
But the section should then proceed to talk about applying the terms to 
implementations. It makes no sense to talk about tag conformance when we 
mean implementations.

> decision may institute an appeal.

You are right that my characterization is not correct. But it is beside 
the point: I think that some review of performance is necessary. I note 
that the reviewer must frequently be prodded into action. There is, even 
now, a registration request that is many weeks old which has complete 
consensus which Michael has apparently approved but not forwarded 
(valencia). A regular renewal of the job would potentially improve 
performance without resorting to the unpleasant appeals process.

> but decides.  We are his advisors.  This is a revolutionary change.

No. It's not or at least not intended to be. Or, to put it another way, 
the reviewer has historically exercised broad (very broad) discretion in 
accepting and rejecting requests. My understanding of RFC 4646 is that a 
request that has met with broad consensus for inclusion and which 
otherwise meets the requirements of the RFC won't unreasonably be 
rejected. The reviewer is, as you say, a judge---s/he adjudicates 
whether sufficient consensus has been achieved, and, furthermore, that 
there are no fatal technical objections to the registration. This is why 
the person is considered by IANA to be an "expert reviewer".

If you don't like my proposed words to indicate this, do me the favor of 
proposing better ones.

I do note that Mark rather deeply changed the current draft's words, so 
my edit might be unnecessary. The current editor's copy says:

--
The Language Subtag Reviewer is appointed by the IESG for an indefinite 
term, subject to removal or replacement at the IESG's discretion. The 
Language Subtag Reviewer moderates the ietf-languages mailing list, 
responds to requests for registration, and performs the other registry 
maintenance duties described in Section 3.3 (Maintenance of the 
Registry). Only the Language Subtag Reviewer is permitted to request 
IANA to change, update, or add records to the Language Subtag Registry. 
The Language Subtag Reviewer MAY delegate list moderation and other 
clerical duties as needed.
--

To this, I think it important to indicate that the LSR has the duty of 
making the adjudication on a request.

> "We'll argue that point," said Jack.
"That I never will, sir,” replied Jack; “but the next time I argue it 
shall be, if possible, with power on my side, and, at all events, not 
quite so near a pond."

I look forward to your argument :-).

Addison

---

From: John Cowan <[email-masked]>
Date: Tue, 27 Feb 2007 18:07:24 -0500

> implementations. It makes no sense to talk about tag conformance when we 

Let's have both, then: definitions of well-formed and valid tags,
followed by definitions of validating and non-validating processors.

> [T]he reviewer has historically exercised broad (very broad) discretion in 
> rejected.

Very well, provided the reviewer is the judge of what is reasonable!
This reminds me of the lawyer's definition of "law", namely "the
law is what the judge says it is."  This works for lawyers, but not
for judges, who are sworn to do justice according to law.

> The reviewer is, as you say, a judge---s/he adjudicates 

Au contraire.  I believe that the reviewer has *always* acted on his
own judgement of whether the registration is good or bad, and *not*
on the basis of what he thinks the list consensus is.  I also believe
that this is The Right Thing, as shown by actual practice.

> my edit might be unnecessary. The current editor's copy says:

I'm good with that as far as it goes.

> making the adjudication on a request.

Sure.  But I do not want to constrain him to do what the list thinks.
He is not the list's servant, as an editor is; he is the sovereign expert
(though with a provision to overrule him).

> I look forward to your argument :-).

Another day.

-- 
May the hair on your toes never fall out!       John Cowan
        --Thorin Oakenshield (to Bilbo)         [email-masked]

---

From: [email-masked]
Date: Tue, 27 Feb 2007 15:32:43 -0800

> differentiate textual from non-textual content.

I think this is where I come out of hiding and repeat what I've previously 
stressed....

This matters to me. Distinguishing between a spoken variant used in 
dubbing and a written version used in subtitles is important. We maintain 
lists of valid "text languages" and "audio languages." These lists are 
unique with many similarities. This is how language is structured in most 
of the A/V standards I see.

Of course, there are languages that fall outside of this including tactile 
languages and visual (sign languages). But to me, zh-cmn (dubbed) is 
totally different from zh-cmn-Hant (subtitled) and I think the idea of a 
spoken language having a suppressed script is silly. That's really bad 
data, IMHO.

In short, I'd tag content as "not written" when it's spoken or signed. I 
never want to see a dubbing track identified as being Simplified Chinese. 
The suppress script tags in many ways makes RFC 4646 totally inappropriate 
for audio language tagging, but we also have no interest in using two 
different standards for the different uses.

This is why I support text in RFC 4646bis that specifically indicates that 
audio languages should not be tagged with scripts -- they are 
inappropriate. From this perspective, ISO 639-6 makes a lot of sense to my 
industry.

Outside of my immediate industry, there is a vast amount of video and 
animated content being added to the web every day. This needs tagging too.

Regards,

Karen Broome

---

From: Addison Phillips <[email-masked]>
Date: Tue, 27 Feb 2007 16:45:28 -0800

[email-masked] wrote:

> dubbing and a written version used in subtitles is important. We 
> maintain lists of valid "text languages" and "audio languages." These 
> lists are unique with many similarities. This is how language is 
> structured in most of the A/V standards I see.

I don't disagree with your purpose. I just note that you don't need a 
special language subtag to identify that the content is audio or 
otherwise non-texual.

> 
> Of course, there are languages that fall outside of this including 
> tactile languages and visual (sign languages). But to me, zh-cmn 
> (dubbed) is totally different from zh-cmn-Hant (subtitled) and I think 
> the idea of a spoken language having a suppressed script is silly. 
> That's really bad data, IMHO.

?? I'm not sure what you mean by a "suppressed script". Suppress-Script 
is an informational field in the registry that advises users about the 
best form of tag to use for the written forms of languages. It has 
nothing to do with spoken or otherwise signaled forms of language.

Various content is tagged with a language tag. The spoken/audio content 
is never tagged with tags that have a script subtag. The written content 
may include a script subtag (or not) depending on the usefulness of the 
script subtag in that context. It is entirely reasonable to think that a 
DVD might have an SAP track in "zh-cmn" and subtitling in "zh-cmn-Hant".

> never want to see a dubbing track identified as being Simplified 
> Chinese. The suppress script tags in many ways makes RFC 4646 totally 
> inappropriate for audio language tagging, but we also have no interest 
> in using two different standards for the different uses.

It would be silly to tag an audio track as "Simplified Chinese", but the 
Content-Type (or equivalent) makes clear that something tagged "zh-cmn" 
is probably audio, nu?

> 
> This is why I support text in RFC 4646bis that specifically indicates 
> that audio languages should not be tagged with scripts -- they are 
> inappropriate. 

Yes. This is just tag choice. The question here is whether we need to 
invent a fictional subtag to make clear that the content is "not 
written". There is an outer limit in the utility of coding everything in 
a language tag. Do you really want to see tags like "en-Qabx" for the 
English audio track? That's silly. "en" covers it just fine.

Addison

---

From: Addison Phillips <[email-masked]>
Date: Tue, 27 Feb 2007 17:04:49 -0800

> followed by definitions of validating and non-validating processors.

Agreed. I thought that's what I said :-).
> for judges, who are sworn to do justice according to law.

Yes, and hence the appeals process.

> on the basis of what he thinks the list consensus is. 

To be complete: the reviewer is sovereign in making the decision. 
However, the decision must be based on significant objections and these 
must be stated publicly on the list. List consensus or the lack thereof 
should certainly influence such a decision.

My bigger concerns with the LSR historically have been:

- there is a lack of timeliness in tracking the two-week rule
- that acceptance or rejection is not always clearly made on the list 
(is, for example, 'valencia' going to show up some time soon or not?)
- that objections are not always clear or clearly summarized
- that criteria for registration/rejection are not always clear

I believe that a registration in the face of overwhelming consensus 
against a subtag would be a truly bad thing. Similarly the reviewer 
should consider a countervailing consensus in support of a subtag 
closely before rejecting an item. The rejection cannot seem arbitrary: 
it must be based on something (RFCs 1766 and 3066 called this 
"significant objections", purposefully not defining "significant").

> I also believe

Yes, when done well. It is the Wrong Thing when applied haphazardly 
and/or inconsistently.

Mark's proposed changes to the term and nominating process would help 
ensure that this performance is occasionally monitored.

> (though with a provision to overrule him).

Good it is that I have not proposed we constrain him thus. I'm happy 
with the reviewer being the "sovereign expert", but the reviewer should 
also be aware that both "justice" and "the law" give a requester certain 
expectations of due process. I would not be concerned if there were a 
clear rejection of a subtag, even if it were entirely wrongheaded for 
the rejection to take place: the appeals process exists for precisely 
that purpose.

Addison

---

From: Martin Duerst <[email-masked]>
Date: Wed, 28 Feb 2007 17:27:27 +0900

Karen - A very simple (and maybe almost navie) question:
I assume that language isn't the only thing that you tag
for your material. I assume that there is other data, e.g.
how long (time-wise) some piece of data is, ..., and
most probably also whether something is audio, has subtitles,
and so on.

So what John is saying, and what I'm also assuming, is that
if you know, from some other data that you have anyway, that
something are subtitle, then you know that it must have some
kind of script, where as if you youw from other data that it's
audio, you know it can't have a script.

Regards,    Martin.

At 08:32 07/02/28, [email-masked] wrote:

>I think this is where I come out of hiding and repeat what I've previously stressed.... 
>
>This matters to me. Distinguishing between a spoken variant used in dubbing and a written version used in subtitles is important. We maintain lists of valid "text languages" and "audio languages." These lists are unique with many similarities. This is how language is structured in most of the A/V standards I see. 
>
>Of course, there are languages that fall outside of this including tactile languages and visual (sign languages). But to me, zh-cmn (dubbed) is totally different from zh-cmn-Hant (subtitled) and I think the idea of a spoken language having a suppressed script is silly. That's really bad data, IMHO. 
>
>In short, I'd tag content as "not written" when it's spoken or signed. I never want to see a dubbing track identified as being Simplified Chinese. The suppress script tags in many ways makes RFC 4646 totally inappropriate for audio language tagging, but we also have no interest in using two different standards for the different uses. 
>
>This is why I support text in RFC 4646bis that specifically indicates that audio languages should not be tagged with scripts -- they are inappropriate. From this perspective, ISO 639-6 makes a lot of sense to my industry. 
>
>Outside of my immediate industry, there is a vast amount of video and animated content being added to the web every day. This needs tagging too. 
>
>Regards, 
>
>Karen Broome 

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 28 Feb 2007 15:58:08 +0100

> http://docs.google.com/Doc?id=dfqr8rd5_4gh864p

I'll wait for an Internet Draft - with 'rfcdiff'
that gives me the usual style of diffs.

Frank

---

From: [email-masked]
Date: Wed, 28 Feb 2007 11:27:02 -0800

Responding to Martin and Addison:

To clarify: I'm not requesting a subtag to indicate audio language. It 
might be useful, but we can indicate that through context. However, when I 
previously suggested text to make it clear that script tags should NEVER 
be used with audio language, I felt like the text that was ultimately 
added was watered down. I probably need to recheck the recent draft and 
make the suggestions again. I'm not sure why this was opposed as it's 
incredibly useful to anyone dealing with audio language.

I don't think I want to tag audio language with an "unwritten" tag and 
agree that the language form can usually, but not always be determined 
through context. HTTP Content-Type doesn't help me much. I'm not writing 
web pages and I have both audio and dubbed languages identified in the 
same  file. (I'm writing XML and using these in databases.) 

All I am really pushing for is a stronger "thou shalt not use script tags 
with audio content" as this isn't as obvious as you might think to the 
people who assign these tags (this problem extends beyond my employer).

The Suppress-Script still bothers me, though I know my chances of getting 
this changed are nil. But for the purposes of argument:

Addison writes:

How does the registry know the "best" form of tag for any individual's 
usage? To me the "suppress" in suppress-script indicates "you needn't list 
this tag because it is the default for this language."  The "best" form 
for my audio identification needs is most definitely not Latn and I'm not 
"suppressing" anything if I don't use the tag. The field isn't named 
"suggested-script." 

>From my viewpoint, indicating whether the language is spoken or written 
does fall under the concept of language identification and is not 
equivalent to proposals to include truly ancillary information that does 
not specifically serve to identify a language. This is where ISO 639-6 may 
offer a better alternative for many of my identification needs. It 
resolves the ambiguity of the "en" tag in RFC 4646.

I'm going to Japan on Friday for a week and a half and probably won't be 
able to get to just about anything until I return, unfortunately. You 
could take another look at my previous suggestions with respect to not 
using script tags with audio content.

Regards,

Karen Broome
Metadata Systems Designer
Sony Pictures Entertainment
310.244.4384

Martin Duerst <[email-masked]> 
02/28/2007 12:27 AM

To
[email-masked], John Cowan <[email-masked]>
cc
LTRU Working Group <[email-masked]>
Subject
Re: [Ltru] Proposals for changes to latest draft

Regards,    Martin.

>Of course, there are languages that fall outside of this including 
tactile languages and visual (sign languages). But to me, zh-cmn (dubbed) 
is totally different from zh-cmn-Hant (subtitled) and I think the idea of 
a spoken language having a suppressed script is silly. That's really bad 
data, IMHO.