Subject: [Ltru] Re: Upcoming changes to BCP47 (language tag) syntax

From: [email-masked]
Date: Tue, 22 Jan 2008 11:54:34 -0800

Addison wrote:
> 
> - Use 'cmn' when you need to indicate "Mandarin Chinese" as distinct 
> from Chinese in general. This case is rare and should be avoided 
> whenever possible. A better way to write that would be: "Do NOT use 
> 'cmn' unless you have a Very Good Reason."

I'm a little worried by this text. These tags are intended to identify 
audio and text content. It is common for those classifying audiovisual 
content to maintain separate lists of "dubbing" (or audio language) and 
"subtitle" or (text languages). This strategy is built into many of the 
standards we use today.

I need to distinguish dubbed Taiwanese Mandarin from Mainland Mandarin 
from Cantonese. With RFC 4646bis as it is currently written, I planned to 
consider "zh" to be a text or subtitle language, with extensions for 
script variants, and use "cmn" and "yue" for the audio languages with 
their own variants. (In subtitle form, I need to distinguish Mandarin, 
Taiwanese, and Cantonese.) Previous ISO language standards mostly ignored 
audio forms of Chinese creating a lot of dirty data in my industry because 
we need these classifications. I hope we're not making this mistake again.

The problem with the strategy above is that I will then have two unique 
tags for the same language audience (dubbed or subtitled mainland Mandarin 
speakers, for example). I thought this was a good argument for extlang, 
which has been removed from RFC 4646bis.

I don't think that the use of cmn will be "very rare" so I would not like 
to Addison's advice to avoid "cmn" written into any guidelines.

This sounds more and more like there's a narrowing the semantic of the 
original "zh" tag. We've got a lot of Cantonese and Taiwanese content 
categorized against the zh tag today and I know this is true of the other 
massive content stores at other studios. 

Regards,

Karen Broome
Sony Pictures Entertainment

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Tue, 22 Jan 2008 15:53:00 -0500

[email-masked] scripsit:

> Taiwanese, and Cantonese.) 

That's the most sensible approach for audio.  The warnings are basically
about situations where there is a lot of pre-existing tagged content,
which doesn't much apply to audio.

> massive content stores at other studios. 

No, it's just warning you that de facto it's been narrowed already,
particularly in the text domain, since most written Chinese is in
fact Mandarin.  (And the same for Standard Arabic, and Swahili,
and a bunch of other cases.)

-- 
Andrew Watt on Microsoft:                       John Cowan
Never in the field of human computing           [email-masked]
has so much been paid by so many                http://www.ccil.org/~cowan
to so few! (pace Winston Churchill)

---

From: [email-masked]
Date: Tue, 22 Jan 2008 17:20:15 -0800

> which doesn't much apply to audio.

We have A LOT of historical audio tagged as zh and zho. Audio content 
wasn't invented yesterday or even the day before, so I think I disagree 
with your premise here. I'm concerned that if these warnings are too 
strongly worded, you might scare off those with a legitimate use for the 
tag. As I have pointed out before, web video content is increasing and 
clear guidelines should be in place.

> and a bunch of other cases.)

I think this is an assumption that should be called into question. There 
are many audio archives, studios, web sites, and other audiovisual content 
producers with systems that now use either ISO 639-1 or ISO 639-2 to 
categorize "Chinese" audio content. There are fewer content collections 
categorized against RFC 3066 or later. I believe this could cause a lot of 
confusion amongst the owners of those collections as they sort out what 
they have on the more granular levels now allowed by RFC 4646 and 4646bis. 

I think the idea that "zh" is a synonym for Mandarin is rooted in the text 
mentality and the de facto narrowing you see there does not hold true for 
audio uses. Most archival standards today use "zho" or "chi" and there are 
a lot of audio and moving image archives out there. An increasing number 
of these moving image collections are being digitized for online formats. 
If zh=mandarin in both spoken and written contexts, any Cantonese or 
Taiwanese content in these collections today will be irretrievable until 
the collection owners can afford to hire language experts to separate the 
Cantonese apples from the Mandarin oranges. 

Regards,

Karen Broome

---

From: John Cowan <[email-masked]>
Date: Tue, 22 Jan 2008 23:47:43 -0500

[email-masked] scripsit:

> with your premise here. 

Sure.  But it could be any Chinese language; there's no presumption of
Mandarin in the audio domain.

> audio uses. 

We are in violent agreement.

-- 
De plichten van een docent zijn divers,         John Cowan
die van het gehoor ook.                         [email-masked]
      --Edsger Dijkstra                         http://www.ccil.org/~cowan

---

From: Martin Duerst <[email-masked]>
Date: Wed, 23 Jan 2008 14:52:06 +0900

Karen - Can you propose some text changes that would remove your concerns
for this text?

Regards,   Martin.

At 10:20 08/01/23, [email-masked] wrote:

>We have A LOT of historical audio tagged as zh and zho. Audio content wasn't invented yesterday or even the day before, so I think I disagree with your premise here. I'm concerned that if these warnings are too strongly worded, you might scare off those with a legitimate use for the tag. As I have pointed out before, web video content is increasing and clear guidelines should be in place. 
>
>> 
>I think this is an assumption that should be called into question. There are many audio archives, studios, web sites, and other audiovisual content producers with systems that now use either ISO 639-1 or ISO 639-2 to categorize "Chinese" audio content. There are fewer content collections categorized against RFC 3066 or later. I believe this could cause a lot of confusion amongst the owners of those collections as they sort out what they have on the more granular levels now allowed by RFC 4646 and 4646bis. 
>
>I think the idea that "zh" is a synonym for Mandarin is rooted in the text mentality and the de facto narrowing you see there does not hold true for audio uses. Most archival standards today use "zho" or "chi" and there are a lot of audio and moving image archives out there. An increasing number of these moving image collections are being digitized for online formats. If zh=mandarin in both spoken and written contexts, any Cantonese or Taiwanese content in these collections today will be irretrievable until the collection owners can afford to hire language experts to separate the Cantonese apples from the Mandarin oranges. 
>
>Regards, 
>
>Karen Broome 

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: John Cowan <[email-masked]>
Date: Wed, 23 Jan 2008 01:10:07 -0500

Martin Duerst scripsit:

> Karen - Can you propose some text changes that would remove your
> concerns for this text?

Please note that I have already provided text that generalizes the
warning from Chinese to several additional languages, but Addison has
not yet incorporated this.

-- 
You escaped them by the will-death              John Cowan
and the Way of the Black Wheel.                 [email-masked]
I could not.  --Great-Souled Sam                http://www.ccil.org/~cowan

---

From: [email-masked]
Date: Wed, 23 Jan 2008 09:27:47 -0800

John, 

If you send me your text, I'll take a look. I'm a bit buried in other 
duties right now and haven't read the latest drafts. I could probably take 
a look this weekend.

Regards,

Karen Broome
Metadata Systems Designer
Sony Pictures Entertainment
310.244.4384

---

From: John Cowan <[email-masked]>
Date: Wed, 23 Jan 2008 14:24:58 -0500

> duties right now and haven't read the latest drafts. I could probably take 

See http://www1.ietf.org/mail-archive/web/ltru/current/msg09238.html

-- 
It was dreary and wearisome.  Cold clammy winter still held way in this
forsaken country.  The only green was the scum of livid weed on the dark
greasy surfaces of the sullen waters.  Dead grasses and rotting reeds loomed
up in the mists like ragged shadows of long-forgotten summers.
        --"The Passage of the Marshes"          http://www.ccil.org/~cowan