Subject: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

From: Daniel LaVon Billings <[email-masked]>
Date: Wed, 12 Aug 2020 17:56:04 +0000

Urdu is listed as Arab script reference in the subtag registry when it should have the newer approved Aran designation:

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7Cdaniel%40churchofjesuschrist.org%7C37060a5c0a8a4fb04c8c08d83ee334fe%7C61e6eeb35fd74aaaae3c61e8deb09b79%7C0%7C1%7C637328492850938946&sdata=6uXT5tBTUTGojI%2BkP2HtSYHJ%2BRzqKYTdnvvCZG79jhk%3D&reserved=0>

Urdu should be using the Aran script, not the Arab script:

%%
Type: language
Subtag: ur
Description: Urdu
Added: 2005-10-16
Suppress-Script: Arab
%%

%%

Type: script

Subtag: Aran

Description: Arabic (Nastaliq variant)

Added: 2014-12-11

%%

How can we get the subtag registry to be updated?

Daniel Billings |  Internationalization and Translation Systems Manager
Language Services and Area Support
Publishing Services Department
[email-masked]<mailto:[email-masked]>

"We shall not fight our battles alone. There is a just God who presides over the destinies of nations, and who will raise up friends to fight our battles for us. The battle, sir, is not to the strong alone; it is for the vigilant, the active, the brave." - Patrick Henry

---

From: Peter Constable <[email-masked]>
Date: Wed, 12 Aug 2020 19:56:58 +0000

If the proposal is to the Suppress-Script value for the ur language entry, then I definitely disagree.

In general, I'm not sure why script-variant IDs like Aran or Latg exist in ISO 15924 at all, and I'm not aware of any application contexts in which they are actually used. For any languages written using Arabic script, including languages in Pakistan, Afghanistan and Iran, I think it's generally likely that applications would use tags with a script subtag Arab or else without any script subtag. And changing the Suppress-Script field for ur would likely break some applications that would expect tags of the form "ur-Arab" were it not for the current Suppress-Script value.

If it were allowed, I wouldn't have objected to adding a second Suppress-Script field with Aran. But RFC5646 only permits one Suppress-Script field.

Peter

From: Ietf-languages <[email-masked]> On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 10:56 AM
To: [email-masked]
Subject: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7C%7C474b1783d597405620a608d83ef45726%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637328566431979306&sdata=Aqs%2FaRTTuH5TxwkwPvdsE8nT60U2YBnpq6D1wBz%2FC6E%3D&reserved=0>

%%
Type: language
Subtag: ur
Description: Urdu
Added: 2005-10-16
Suppress-Script: Arab
%%

%%

Type: script

Subtag: Aran

Added: 2014-12-11

%%

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 12 Aug 2020 14:40:31 -0600

Hi Daniel,

 �

The purpose of Suppress-Script in BCP 47 is to improve compatibility between BCP 47 applications and those written to older specifications, which did not support script subtags.

 �

There was a great deal of concern, in the mid-’00s when RFC 4646 was being developed, that the new script subtags would be overused, so that, for example, users who previously tagged English content as “en” or “en-US” would start tagging it as “en-Latn” or “en-Latn-US” instead. This would add virtually no information to the tag, because English is normally written in the Latin script; but it could cause compatibility problems with processes that did not understand the script subtag. Suppress-Script was devised as a way to discourage users from adding unnecessary script subtags like this. It is optional, pragmatic, and suggestive in nature; it does not attempt to provide a scholarly reference about the language.

 �

By changing the Suppress-Script for Urdu from ‘Arab’ to ‘Aran’, we would be essentially saying that the tag “ur-Arab” does add significant information beyond the tag “ur”, which is not true (most Urdu content is indeed written in the Arabic script) and in my opinion would be a step backward. Note that there is no corresponding script subtag for “Arabic script (Naskh variant).”

 �

I suspect, somewhat echoing Peter, that most users do not even know ‘Aran’ exists or why it is separately encoded. While I understand some of the thought process behind this proposal, I agree that the change should not be made.

 �

--

Doug Ewell | Thornton, CO, US | ewellic.org

 �

 �

From: Ietf-languages <[email-masked]> On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 11:56
To: [email-masked]
Subject: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

 �

 �

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7Cdaniel%40churchofjesuschrist.org%7C37060a5c0a8a4fb04c8c08d83ee334fe%7C61e6eeb35fd74aaaae3c61e8deb09b79%7C0%7C1%7C637328492850938946&sdata=6uXT5tBTUTGojI%2BkP2HtSYHJ%2BRzqKYTdnvvCZG79jhk%3D&reserved=0> 

 �

 �

%%

Type: language

Subtag: ur

Description: Urdu

Added: 2005-10-16

Suppress-Script: Arab

%%

 �

---

From: Daniel LaVon Billings <[email-masked]>
Date: Wed, 12 Aug 2020 20:39:58 +0000

Culturally, they only use the Aran variant script for Urdu, so logically applications should look for a Nastaliq font as opposed to a Naskh font whenever "ur" is being called. Otherwise, we will be locked in a situation where our technical limitations are holding back the appropriate culturally significant script from being recognized as the standard in the language. The registry should reflect that cultural significance or else there is no reason for Aran to exist at all in ISO 15924.

From: Peter Constable <[email-masked]>
Sent: Wednesday, August 12, 2020 1:57 PM
To: Daniel LaVon Billings <[email-masked]>; [email-masked]
Subject: RE: Suggestion to update Urdu Script Designation in the subtag registry

Peter

From: Ietf-languages <[email-masked]<mailto:[email-masked]>> On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 10:56 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry<https://na01.safelinks.protection.outlook..com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7C%7C474b1783d597405620a608d83ef45726%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637328566431979306&sdata=Aqs%2FaRTTuH5TxwkwPvdsE8nT60U2YBnpq6D1wBz%2FC6E%3D&reserved=0>

%%
Type: language
Subtag: ur
Description: Urdu
Added: 2005-10-16
Suppress-Script: Arab
%%

%%

Type: script

Subtag: Aran

Added: 2014-12-11

%%

---

From: Daniel LaVon Billings <[email-masked]>
Date: Wed, 12 Aug 2020 20:49:25 +0000

It seems like it could be generally assumed that Arab was created to signify the Naskh variant because otherwise, there isn’t a reason for creating the Aran script code. We need our applications to use a Nastaliq font whenever Urdu is called since that is the standard for Urdu, but this subtag registry currently is in competition with that ideology. We have plenty of use cases to use cmn-Latn (for Romanized Chinese text) or other variants of the standard tagging that we know why they are different from the standard, but in Urdu’s case, we would never want Urdu to use a non-Nastaliq font.

“We shall not fight our battles alone. There is a just God who presides over the destinies of nations, and who will raise up friends to fight our battles for us. The battle, sir, is not to the strong alone; it is for the vigilant, the active, the brave.” – Patrick Henry

From: Doug Ewell <[email-masked]>
Sent: Wednesday, August 12, 2020 2:41 PM
To: Daniel LaVon Billings <[email-masked]>; [email-masked]
Subject: RE: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

Hi Daniel,

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7Cdaniel%40ChurchofJesusChrist.org%7C545e43ff302c400ec52a08d83efffb94%7C61e6eeb35fd74aaaae3c61e8deb09b79%7C0%7C1%7C637328616450395840&sdata=ZtjuTkXdvC29fzHO9Rjti0Ae1Tnqvt0C4cMUSBQhlxI%3D&reserved=0>

%%
Type: language
Subtag: ur
Description: Urdu
Added: 2005-10-16
Suppress-Script: Arab
%%

%%

Type: script

Subtag: Aran

Added: 2014-12-11

%%

Daniel Billings |  Internationalization and Translation Systems Manager
Language Services and Area Support
Publishing Services Department
daniel@churchofjesuschrist..org<mailto:[email-masked]>

“We shall not fight our battles alone. There is a just God who presides over the destinies of nations, and who will raise up friends to fight our battles for us. The battle, sir, is not to the strong alone; it is for the vigilant, the active, the brave.” – Patrick Henry

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 12 Aug 2020 15:14:48 -0600

You are, of course, perfectly at liberty to tag content as “ur-Aran” to specify the Nastaliq variant, just as you can use “cmn-Latn” to specify Mandarin written in Latin. Neither BCP 47 nor the contents of the Registry is locking or holding back anyone in that regard.

 �

Suppress-Script isn’t really meant as a font selection device for any language. There are hundreds or thousands of languages known to be written predominantly in a particular script, for which there is no Suppress-Script value.

 �

We can certainly check with ISO 15924/RA-JAC to see if there is any unstated expectation that ‘Arab’ implies the Naskh variant.

 �

--

 �

 �

From: Ietf-languages <[email-masked]> On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 14:49
To: Doug Ewell <[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

 �

 �

Daniel �Billings �| � Internationalization and Translation Systems Manager

[email-masked] <mailto:[email-masked]> 

 �

 �

From: Doug Ewell <[email-masked] <mailto:[email-masked]> > 
Sent: Wednesday, August 12, 2020 2:41 PM
To: Daniel LaVon Billings <[email-masked] <mailto:[email-masked]> >; [email-masked] <mailto:[email-masked]> 
Subject: RE: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

 �

Hi Daniel,

 �

 �

 �

 �

 �

--

 �

 �

From: Ietf-languages <[email-masked] <mailto:[email-masked]> > On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 11:56
To: [email-masked] <mailto:[email-masked]> 
Subject: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

 �

 �

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7Cdaniel%40ChurchofJesusChrist.org%7C545e43ff302c400ec52a08d83efffb94%7C61e6eeb35fd74aaaae3c61e8deb09b79%7C0%7C1%7C637328616450395840&sdata=ZtjuTkXdvC29fzHO9Rjti0Ae1Tnqvt0C4cMUSBQhlxI%3D&reserved=0> 

 �

 �

%%

Type: language

Subtag: ur

Description: Urdu

Added: 2005-10-16

Suppress-Script: Arab

%%

 �

---

From: Mark Davis ☕️ <[email-masked]>
Date: Wed, 12 Aug 2020 14:26:25 -0700

Doug is right (as usual)

And moreover, most software assumes that variant subtags won't be used
in language subtags. You won't get a fallback from Latg to Latn, or from
Aran to Arab.If they do occur, then they will at best be ignored — and at
worst disrupt normal processing, such as: matching a user's preferred
languages to a system's supported languages, or picking the allowed
characters from the script (for script variants, Unicode Unicode the empty
set of characters).

Mark

On Wed, Aug 12, 2020 at 2:14 PM Doug Ewell <[email-masked]> wrote:

> You are, of course, perfectly at liberty to tag content as “ur-Aran” to
> specify the Nastaliq variant, just as you can use “cmn-Latn” to specify
> Mandarin written in Latin. Neither BCP 47 nor the contents of the Registry
> is locking or holding back anyone in that regard.
>
>
>
> Suppress-Script isn’t really meant as a font selection device for any
> language. There are hundreds or thousands of languages known to be written
> predominantly in a particular script, for which there is no Suppress-Script
> value.
>
>
>
> We can certainly check with ISO 15924/RA-JAC to see if there is any
> unstated expectation that ‘Arab’ implies the Naskh variant.
>
>
>
> --
>
> *From:* Ietf-languages <[email-masked]> *On Behalf Of *Daniel
> LaVon Billings
> *Sent:* Wednesday, August 12, 2020 14:49
> *To:* Doug Ewell <[email-masked]>; [email-masked]
> *Subject:* Re: [Ietf-languages] Suggestion to update Urdu Script
> Designation in the subtag registry
>
>
>
> It seems like it could be generally assumed that Arab was created to
> signify the Naskh variant because otherwise, there isn’t a reason for
> creating the Aran script code. We need our applications to use a Nastaliq
> font whenever Urdu is called since that is the standard for Urdu, but this
> subtag registry currently is in competition with that ideology. We have
> plenty of use cases to use cmn-Latn (for Romanized Chinese text) or other
> variants of the standard tagging that we know why they are different from
> the standard, but in Urdu’s case, we would never want Urdu to use a
> non-Nastaliq font.
>
>
>
> *Language Services and Area Support*
>
> *Publishing Services Department*
>
> daniel@churchofjesuschrist..org <[email-masked]>
>
>
>
> “We shall not fight our battles alone. There is a just God who presides
> over the destinies of nations, and who will raise up friends to fight our
> battles for us. The battle, sir, is not to the strong alone; it is for the
> vigilant, the active, the brave.” – Patrick Henry
>
>
>
> *From:* Doug Ewell <[email-masked]>
> *Sent:* Wednesday, August 12, 2020 2:41 PM
> *To:* Daniel LaVon Billings <daniel@ChurchofJesusChrist..org
> <[email-masked]>>; [email-masked]
> *Subject:* RE: [Ietf-languages] Suggestion to update Urdu Script
> Designation in the subtag registry
>
>
>
> Hi Daniel,
>
>
>
> The purpose of Suppress-Script in BCP 47 is to improve compatibility
> between BCP 47 applications and those written to older specifications,
> which did not support script subtags.
>
>
>
> There was a great deal of concern, in the mid-’00s when RFC 4646 was being
> developed, that the new script subtags would be overused, so that, for
> example, users who previously tagged English content as “en” or “en-US”
> would start tagging it as “en-Latn” or “en-Latn-US” instead. This would add
> virtually no information to the tag, because English is normally written in
> the Latin script; but it could cause compatibility problems with processes
> that did not understand the script subtag. Suppress-Script was devised as a
> way to discourage users from adding unnecessary script subtags like this.
> It is optional, pragmatic, and suggestive in nature; it does not attempt to
> provide a scholarly reference about the language.
>
>
>
> By changing the Suppress-Script for Urdu from ‘Arab’ to ‘Aran’, we would
> be essentially saying that the tag “ur-Arab” does add significant
> information beyond the tag “ur”, which is not true (most Urdu content is
> indeed written in the Arabic script) and in my opinion would be a step
> backward. Note that there is no corresponding script subtag for “Arabic
> script (Naskh variant).”
>
>
>
> I suspect, somewhat echoing Peter, that most users do not even know ‘Aran’
> exists or why it is separately encoded. While I understand some of the
> thought process behind this proposal, I agree that the change should not be
> made.
>
>
>
> --
>
> *From:* Ietf-languages <[email-masked]> *On Behalf Of *Daniel
> LaVon Billings
> *Sent:* Wednesday, August 12, 2020 11:56
> *To:* [email-masked]
> *Subject:* [Ietf-languages] Suggestion to update Urdu Script Designation
> in the subtag registry
>
>
>
> Urdu is listed as Arab script reference in the subtag registry when it
> should have the newer approved Aran designation:
>
>
>
>
> https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7Cdaniel%40ChurchofJesusChrist.org%7C545e43ff302c400ec52a08d83efffb94%7C61e6eeb35fd74aaaae3c61e8deb09b79%7C0%7C1%7C637328616450395840&sdata=ZtjuTkXdvC29fzHO9Rjti0Ae1Tnqvt0C4cMUSBQhlxI%3D&reserved=0>
>
>
>
> *Language Services and Area Support*
>
> *Publishing Services Department*
>
> daniel@churchofjesuschrist...org <[email-masked]>
>
>
>
> “We shall not fight our battles alone. There is a just God who presides
> over the destinies of nations, and who will raise up friends to fight our
> battles for us. The battle, sir, is not to the strong alone; it is for the
> vigilant, the active, the brave.” – Patrick Henry
>
>
> _______________________________________________
> Ietf-languages mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-languages
>

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 12 Aug 2020 15:27:06 -0600

Think of it this way: By changing the Suppress-Script for Urdu from 'Arab' to 'Aran', we would merely be telling processes that create tags:

"Don't use 'ur-Aran' because the script subtag 'Aran' is redundant. The simpler tag 'ur' means the same thing and is more widely recognized, and you should use that instead."

That might not be what you are trying to achieve.

---

From: Peter Constable <[email-masked]>
Date: Wed, 12 Aug 2020 21:37:08 +0000

> We can certainly check with ISO 15924/RA-JAC to see if there is any unstated expectation that ‘Arab’ implies the Naskh variant.

I am certain that ‘Arab’ does not imply _any_ style variant of Arabic script. One has to look no further than Unicode character properties to see that.

Peter

From: Ietf-languages <[email-masked]> On Behalf Of Doug Ewell
Sent: Wednesday, August 12, 2020 2:15 PM
To: 'Daniel LaVon Billings' <daniel=[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

From: Ietf-languages <[email-masked]<mailto:[email-masked]>> On Behalf Of Daniel LaVon Billings
Sent: Wednesday, August 12, 2020 14:49
To: Doug Ewell <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

From: Doug Ewell <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, August 12, 2020 2:41 PM
To: Daniel LaVon Billings <daniel@ChurchofJesusChrist..org<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

Hi Daniel,

https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.iana.org%2Fassignments%2Flanguage-subtag-registry%2Flanguage-subtag-registry&data=02%7C01%7C%7Cdbdb28a6412b4e4afe8908d83f04c432%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637328636993289640&sdata=n%2Fcaqly1MrrPN%2FIWcU6MPW%2BjagOcsems%2FMRyd10KnaA%3D&reserved=0>

%%
Type: language
Subtag: ur
Description: Urdu
Added: 2005-10-16
Suppress-Script: Arab
%%

%%

Type: script

Subtag: Aran

Added: 2014-12-11

%%

Daniel Billings |  Internationalization and Translation Systems Manager
Language Services and Area Support
Publishing Services Department
daniel@churchofjesuschrist...org<mailto:[email-masked]>

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 13 Aug 2020 01:42:57 +0100

On Wed, 12 Aug 2020 21:37:08 +0000
Peter Constable <[email-masked]> wrote:

> I am certain that ‘Arab’ does not imply _any_ style variant of Arabic
> script. One has to look no further than Unicode character properties
> to see that.

What do Unicode character properties have to do with it?  They may
choose their value symbols from ISO 15924, but Unicode properties are
concerned with characters and their interactions, at which level there
is no difference between Arab and Aran. Unicode uses the symbol Arab.

Richard.

---

From: Peter Constable <[email-masked]>
Date: Thu, 13 Aug 2020 01:05:06 +0000

> at which level there is no difference between Arab and Aran. Unicode uses the symbol Arab 

Exactly. It's an ostensive demonstration that the range of style variants encompassed by Arab is unlimited: it denotes Arabic script in any style. 

Also keep in mind what agency functions as the ISO 15924 RA: The Unicode Consortium.

Peter

-----Original Message-----
From: Ietf-languages <[email-masked]> On Behalf Of Richard Wordingham
Sent: Wednesday, August 12, 2020 5:43 PM
To: [email-masked]
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

> script. One has to look no further than Unicode character properties 

What do Unicode character properties have to do with it?  They may choose their value symbols from ISO 15924, but Unicode properties are concerned with characters and their interactions, at which level there is no difference between Arab and Aran. Unicode uses the symbol Arab.

Richard.

_______________________________________________
Ietf-languages mailing list
[email-masked]
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fietf-languages&amp;data=02%7C01%7C%7Ca3bc12db55ad456d246f08d83f21dd32%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637328761972858429&amp;sdata=066%2FNYe6jORkxI7%2B%2Fg25WSi4rqOA2PP35C5FYzBq1ZY%3D&amp;reserved=0

---

From: r12a <[email-masked]>
Date: Thu, 13 Aug 2020 11:54:24 +0100

> unstated expectation that ‘Arab’ implies the Naskh variant.

I would hope not, since Naskh is only one of several writing styles used 
for Arabic.  These include Naskh, Nastaliq (Aran), Ruq'a, Kano, Kufi, 
and so on.  If Arab was equated with naskh only, we'd be stuck for what 
to use to represent text written in the other styles.

I would have thought that, generally speaking, the presence of ur would 
already indicate that an application should by default use a nastaliq 
font for Urdu (and ks for Kashmiri), without the need to further 
qualify.  Additional subtags are mostly useful for modifying the default 
assumptions that come with the language, rather than completing the intent.

It seems to me that Aran might appeal for languages such as Persian, 
which are commonly written in naskh style, but can be written in a kind 
of nastaliq, so the -Aran marker could help to indicate that 
distinction. In a similar way, then, -Arab could be used after ur to 
indicate that a non-nastaliq font should be used.  But the problem here 
seems to be that -Arab and -Aran only work for a tiny subset of the 
actual list of writing-style identifiers that are actually needed,. 
There are also other places where it would be useful to distinguish 
between particular styles. For example, Hausa in Arabic script can be 
written with the hafs or warsh orthographies (typically requiring 
different fonts because they include different character repertoires), 
but in Nigeria Hausa also uses the Kano writing-style.  One might also 
want to label text that uses a magrebi style font in North Africa. Etc.

I think the -Arab subtag is mostly useful for languages (such as the 
many in Central Asia and nearby) that can be written in more than one 
script, and where you need to explicitly distinguish whether, say, 
Latin, Arabic, or Cyrillic, should be used for a given bit of text. But 
even there my personal preference is only to use the script tag when i 
need to make a meaningful distinction, not all the time.

I find myself wondering whether Aran ought really to have been a variant 
subtag, to which we could add others for different writing styles.  In 
particular, because some of the usage distinctions just mentioned can't 
be expressed by combining language and region tags.

ri

---

From: Hugh Paterson III <[email-masked]>
Date: Thu, 13 Aug 2020 14:49:27 +0200

ri,

In Nigeria Hausa can also be written with the Latin script. Where can I go
to find what the basic default assumptions are for a language tag? Is the
default always Latin?

In an interesting case where I have done some research. A language of the
Ivory Coast and Liberia share a language called Dan. I know of 4
orthographies used in print in Dan all of them Latin or Latin with
borrowings from Cyrillic . One in Liberia, three in CI. So dnj_ci is not
sufficient to distinguish the three in the Ivory Coast. My work focuses on
the production of optimized keyboard layouts which are orthography
specific, so I use the -x- component of bcp47 to distinguish the texts and
the tools. This seems to be a valid way but is it the best way? It seems to
me that the script and the orthography layers are independent (and perhaps
also the writing style), and orthography is only explicitly addressed by
the sub-tag registry. I.e the German language tags including the 1996
related tag.

All the best,
- Hugh

On Thu, Aug 13, 2020 at 12:54 PM r12a <[email-masked]> wrote:

> for Arabic.  These include Naskh, Nastaliq (Aran), Ruq'a, Kano, Kufi, and
> so on.  If Arab was equated with naskh only, we'd be stuck for what to use
> to represent text written in the other styles.
>
> already indicate that an application should by default use a nastaliq font
> for Urdu (and ks for Kashmiri), without the need to further qualify.
> Additional subtags are mostly useful for modifying the default assumptions
> that come with the language, rather than completing the intent.
>
> It seems to me that Aran might appeal for languages such as Persian, which
> are commonly written in naskh style, but can be written in a kind of
> nastaliq, so the -Aran marker could help to indicate that distinction. In a
> similar way, then, -Arab could be used after ur to indicate that a
> non-nastaliq font should be used.  But the problem here seems to be that
> -Arab and -Aran only work for a tiny subset of the actual list of
> writing-style identifiers that are actually needed,. There are also other
> places where it would be useful to distinguish between particular styles.
> For example, Hausa in Arabic script can be written with the hafs or warsh
> orthographies (typically requiring different fonts because they include
> different character repertoires), but in Nigeria Hausa also uses the Kano
> writing-style.  One might also want to label text that uses a magrebi style
> font in North Africa. Etc.
>
> I think the -Arab subtag is mostly useful for languages (such as the many
> in Central Asia and nearby) that can be written in more than one script,
> and where you need to explicitly distinguish whether, say, Latin, Arabic,
> or Cyrillic, should be used for a given bit of text. But even there my
> personal preference is only to use the script tag when i need to make a
> meaningful distinction, not all the time.
>
> particular, because some of the usage distinctions just mentioned can't be
> expressed by combining language and region tags.
>
>
> ri
>
>
>
-- 
All the best,
-Hugh

Sent from my iPhone
Paris, France

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 13 Aug 2020 15:08:59 +0100

On Thu, 13 Aug 2020 14:49:27 +0200
Hugh Paterson III <[email-masked]> wrote:

> ri,
> 
> In Nigeria Hausa can also be written with the Latin script. Where can
> I go to find what the basic default assumptions are for a language
> tag? Is the default always Latin?

Unless there is a suppress-script tag for the language, the default is
not normatively defined.  The ICU database, CLDR, seems to have
empirical fallback rules, but they're not trustworthy.  They have
(have had?) such howlers as the Saurashtra script being the default for
the Saurashtra language.  (The proposal for the script's encoding
noted that speakers usually used a different script for their
language.  I have reported the howler.)  It's a lot of work to dig this
information out, and sane fallback might even depend on the context, as
I've noticed in the choice of digits for page numbering in Thai -
magazines use (Western) Arabic digits, while the government sponsored
dictionary uses Thai digits. The Northern Thai nameplates in a Chiang
Mai university will use the Tai Tham script, but if you're trying to
spread the word of God, you'll use the Thai script with Bangkok spelling
rules.  The recent New Testament translation into Northern Thai used
both scripts because of the competing requirements. (The two NT versions
correspond very exactly.) Most Northern Thai speakers can function in
Thai.

Richard.

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 13 Aug 2020 10:20:17 -0600

> tag? Is the default always Latin?

I realize you know most of what follows, because SIL. Much of this explanation is for others.

When I want to find out details like this about a language, I first go to the ISO 639-3/RA site:
https://iso639-3.sil.org/code/hau

That points me to four sites for more information:
Ethnologue:	https://www.ethnologue.com/language/hau
Glottolog:	https://glottolog.org/glottolog?iso=hau#12/11.1513/8.7804
Multitree:	http://www.multitree.org/codes/hau.html
Wikipedia:	https://en.wikipedia.org/wiki/Hausa_language

Unfortunately I can no longer recommend Ethnologue to anyone except professionals in the field, because it now requires a minimal subscription at USD 480 per year (more capable subscriptions cost much, much more) to see basic information like speaker population, location, and writing system.

Glottolog is focused on language family assignments, on which experts often disagree, but it does include location and alternate names, as well as a ton of references.

Multitree appears to be down at present, so I can't say anything about it except that. My understanding is that it is also focused on relationships between languages.

Wikipedia is what it always is: There's a ton of information; the question is whether you feel you can trust the expertise and accuracy of the contributors.

The IANA Language Subtag Registry was never intended to be an encyclopedic reference to languages, their names, or their usage. It lists the subtags corresponding to languages, scripts, regions, and language varieties that are necessary to construct good and meaningful tags for identifying and locating content.

There is no "default" writing system expressed or implied in either the Registry or BCP 47. Suppress-Script, as stated earlier, is a mechanism to help people avoid using script subtags that are unlikely to add useful information. But to find out what script is most common for any given language, or other important information about the language, encyclopedic references are often needed. And the absence of a Suppress-Script field for a language subtag is no indication of anything.

Richard Wordingham replied:

> not normatively defined.

There is nothing normative about Suppress-Script.

---

From: Peter Constable <[email-masked]>
Date: Thu, 13 Aug 2020 16:29:33 +0000

If typographic styles are to be indicated in a tag that is primarily intended to declare the _language_ of content, then what are the limits to that?

For some languages written with Latin script, certain styles of fonts are preferred. For example, the Puyallup nature just south of where I live use fonts with a distinctive style. Should Puyallup content be marked up with a tag that includes something indicating that distinct style? If another language doesn’t normally use the Puyallup style fonts but some documents do, should those documents be tagged to indicate that style?

Or within a single language, there may be different typographic preferences. I’m sure you’re familiar with issues regarding accented capitals in French: one convention allows for them, while another convention would never have accented capitals. Do we need variant subtags to indicate such distinctions? Or take this a step further: there are lots of Latin ligatures that _could_ be used in a document but aren’t required. Should language tags reflect levels of ligation that are used?

Certainly there are stylistic preferences associated with some languages, such as Nastaliq for Urdu. (Mind, I suspect Urdu speakers wouldn’t expect or want a Nastaliq font to be used for UI strings on a phone.) I could perhaps see some benefit in having a way to qualify fonts for such distinctions, and to capture that language/style association as a default in something like CLDR. But aside from characterizing font resources, it’s not clear to me where is it would be beneficial to use ‘Aran’, either in a language tag or some other context.

Peter

From: Ietf-languages <[email-masked]> On Behalf Of r12a
Sent: Thursday, August 13, 2020 3:54 AM
To: Doug Ewell <[email-masked]>; 'Daniel LaVon Billings' <daniel=[email-masked]>
Cc: [email-masked]
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

I would hope not, since Naskh is only one of several writing styles used for Arabic.  These include Naskh, Nastaliq (Aran), Ruq'a, Kano, Kufi, and so on.  If Arab was equated with naskh only, we'd be stuck for what to use to represent text written in the other styles.

I would have thought that, generally speaking, the presence of ur would already indicate that an application should by default use a nastaliq font for Urdu (and ks for Kashmiri), without the need to further qualify.  Additional subtags are mostly useful for modifying the default assumptions that come with the language, rather than completing the intent.

It seems to me that Aran might appeal for languages such as Persian, which are commonly written in naskh style, but can be written in a kind of nastaliq, so the -Aran marker could help to indicate that distinction. In a similar way, then, -Arab could be used after ur to indicate that a non-nastaliq font should be used.  But the problem here seems to be that -Arab and -Aran only work for a tiny subset of the actual list of writing-style identifiers that are actually needed,. There are also other places where it would be useful to distinguish between particular styles. For example, Hausa in Arabic script can be written with the hafs or warsh orthographies (typically requiring different fonts because they include different character repertoires), but in Nigeria Hausa also uses the Kano writing-style.  One might also want to label text that uses a magrebi style font in North Africa. Etc.

I think the -Arab subtag is mostly useful for languages (such as the many in Central Asia and nearby) that can be written in more than one script, and where you need to explicitly distinguish whether, say, Latin, Arabic, or Cyrillic, should be used for a given bit of text. But even there my personal preference is only to use the script tag when i need to make a meaningful distinction, not all the time.

I find myself wondering whether Aran ought really to have been a variant subtag, to which we could add others for different writing styles.  In particular, because some of the usage distinctions just mentioned can't be expressed by combining language and region tags.

ri

---

From: Mark Davis ☕️ <[email-masked]>
Date: Thu, 13 Aug 2020 09:52:58 -0700

CLDR does have likely subtags, which provide default interpretations for
incomplete language tags.

https://unicode-org.github.io/cldr-staging/charts/38/supplemental/likely_subtags.html

For Hausa, the default script varies by region.
https://unicode-org.github.io/cldr-staging/charts/38/supplemental/likely_subtags.html#ha

Richard, these are all 'best available data', and are continually being
refined. It is especially a problem for longer-tail languages. Where there
are howlers, we'd like to know about them. Saurashtra is still an issue
(see the above page), but I see no bug report with the term "Saurashtra",
except for the following 2, which appear unrelated.

CLDR-11994 <https://unicode-org.atlassian.net/browse/CLDR-11994>

Review localeId display names
<https://unicode-org.atlassian.net/browse/CLDR-11994>

CLDR-9408 <https://unicode-org.atlassian.net/browse/CLDR-9408>

Review and fix primarily spoken languages
<https://unicode-org.atlassian.net/browse/CLDR-9408>

https://unicode-org.atlassian.net/issues/?filter=-2&jql=text~Saurashtra%20and%20project%3Dcldr%20and%20status%20!%3Ddone

Could you file a bug report for Saurashtra and any other cases you know of?

Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 13 Aug 2020 11:53:10 -0600

For some of the styling scenarios Peter describes, I could imagine creating a BCP 47 extension, so taggers could write something like "fr-s-plaincap" or "fr-s-accntcap".

For the OP’s original scenario of Urdu in Nastaliq, I don’t have any issue with specifying “ur-Aran” when the choice of a Nastaliq font is considered important, especially since 'Aran' already exists. It's just not what Suppress-Script is for.

From: Ietf-languages <[email-masked]> On Behalf Of Peter Constable
Sent: Thursday, August 13, 2020 10:30
To: r12a <[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Suggestion to update Urdu Script Designation in the subtag registry

Peter

---

From: Mark Davis ☕️ <[email-masked]>
Date: Thu, 13 Aug 2020 11:10:59 -0700

Mark

On Thu, Aug 13, 2020 at 10:53 AM Doug Ewell <[email-masked]> wrote:

> For some of the styling scenarios Peter describes, I could imagine
> creating a BCP 47 extension, so taggers could write something like
> "fr-s-plaincap" or "fr-s-accntcap".
>
> For the OP’s original scenario of Urdu in Nastaliq, I don’t have any issue
> with specifying “ur-Aran” when the choice of a Nastaliq font is considered
> important, especially since 'Aran' already exists.

It is certainly legal to specify ur-Aran. However, as I remarked before, I
don't think it will accomplish what Daniel would intend with it — and in
the worst case could cause very significant degradation in quality (such a
request for a localized data for ur-Aran actually returning English
instead, even though ur is available).

If people don't understand the ramifications of using variant script tags,
they definitely shouldn't be using them.

> It's just not what Suppress-Script is for.
>

Definitely agreed.

> If typographic styles are to be indicated in a tag that is primarily
> intended to declare the _language_ of content, then what are the limits to
> that?
>
> For some languages written with Latin script, certain styles of fonts are
> preferred. For example, the Puyallup nature just south of where I live use
> fonts with a distinctive style. Should Puyallup content be marked up with a
> tag that includes something indicating that distinct style? If another
> language doesn’t normally use the Puyallup style fonts but some documents
> do, should those documents be tagged to indicate that style?
>
> Or within a single language, there may be different typographic
> preferences. I’m sure you’re familiar with issues regarding accented
> capitals in French: one convention allows for them, while another
> convention would never have accented capitals. Do we need variant subtags
> to indicate such distinctions? Or take this a step further: there are lots
> of Latin ligatures that _could_ be used in a document but aren’t required.
> Should language tags reflect levels of ligation that are used?
>
> Certainly there are stylistic preferences associated with some languages,
> such as Nastaliq for Urdu. (Mind, I suspect Urdu speakers wouldn’t expect
> or want a Nastaliq font to be used for UI strings on a phone.) I could
> perhaps see some benefit in having a way to qualify fonts for such
> distinctions, and to capture that language/style association as a default
> in something like CLDR. But aside from characterizing font resources, it’s
> not clear to me where is it would be beneficial to use ‘Aran’, either in a
> language tag or some other context.
>
>
> Peter
>

---

From: John Cowan <[email-masked]>
Date: Thu, 13 Aug 2020 15:06:33 -0400

On Thu, Aug 13, 2020 at 12:29 PM Peter Constable <[email-masked]> wrote:

> fonts with a distinctive style.
>

As I understand the theory of script variants (and Michael probably
understands it better), they are about semi-legibility.  I can read Latn
text in pretty much any font (always excepting Stop <
http://www.identifont.com/show?374>, which combines the virtues of ugliness
with those of unreadability).  But I would find it tiring and painful to
read an English-language book in Latg, and impossible to read one in Latf,
because these variants are only semi-legible to someone who knows Latn
well.  (Legibility should not be confused with intelligibility, of course:
I can't understand Irish text whether it is Latn or Latg.)

Exactly when to say something is a mere font variant (and so unrecognized
by standards), a script variant, or a closely related but different script,
is always going to be a question, the same question as "Language or
dialect" but transposed to the key of scripts.

For instance, is the script of <
https://upload.wikimedia.org/wikipedia/commons/9/9a/Vesti-1631.jpg>, a page
of the first Russian-language newspaper ever, a mere handwriting font of
Cyrs (pre-Petrine Cyrilic), or another Cyrillic script variant, or
effectively a different script altogether?  I don't know.

Should Puyallup content be marked up with a tag that includes something
> indicating that distinct style? If another language doesn’t normally use
> the Puyallup style fonts but some documents do, should those documents be
> tagged to indicate that style?
>

If it made them illegible to speakers of that language, then I'd say yes.

John Cowan          http://vrici.lojban.org/~cowan        [email-masked]
A rabbi whose congregation doesn't want to drive him out of town isn't
a rabbi, and a rabbi who lets them do it isn't a man.    --Jewish saying

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 13 Aug 2020 20:29:22 +0100

On Thu, 13 Aug 2020 16:29:33 +0000
Peter Constable <[email-masked]> wrote:

> But aside from
> characterizing font resources, it’s not clear to me where is it would
> be beneficial to use ‘Aran’, either in a language tag or some other
> context.

That rather raises the question of why Aran is registered in BCP 47!

However, I think Aran makes sense in catalogues, though my reasoning is
based on the apparently analogous situation of Latn, Latf and Latg, but
where Latn is not used if Latf or Latg were appropriate.  If various
versions of rendered text, e.g. scanned in text, were available, for
reading I would choose Latn because I read that faster than Latf or
Latg.  (This breaks down if the 'Latn' object uses Carolingian
minuscule!)

Of course, this is not a text processing use.

Richard.

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 13 Aug 2020 20:29:34 +0100

On Thu, 13 Aug 2020 11:53:10 -0600
"Doug Ewell" <[email-masked]> wrote:

> For the OP’s original scenario of Urdu in Nastaliq, I don’t have any
> issue with specifying “ur-Aran” when the choice of a Nastaliq font is
> considered important, especially since 'Aran' already exists. It's
> just not what Suppress-Script is for.

I am now very confused.  BCP 47 says one SHOULD NOT tag text as
en-Latn.  So, if I were cataloguing the various texts cluttering my
house and recording their language and script, I would have some that
were English in the Latin script and some that were English in the Thai
script.*  So, if I chose to use BCP 47 to record the language, the
former should be recorded as "en" rather than as "en-Latn" and the
latter could be recorded as "en-Thai".  Then, when I came to use the
catalogue, I would know that those labelled as "en-Thai" were in the
Thai script, but for those labelled "en", I should be unsure of the
script; it would be improper to assume that the script was the Latin
script, though that would be the best **guess**. 

*These arise for teaching and, at risk, as semi-encrypted messages.

In the implausible case that I knew that something was in English but
hadn't looked at it to determine the script, I presume I should record
the language and script as "en-Zyyy".

Have I got all this right?

> From: Ietf-languages <[email-masked]> On Behalf Of
> Peter Constable Sent: Thursday, August 13, 2020 10:30
> intended to declare the _language_ of content, then what are the
> limits to that? 
> 
> For some languages written with Latin script, certain styles of fonts
> are preferred. For example, the Puyallup nature just south of where I
> live use fonts with a distinctive style. Should Puyallup content be
> marked up with a tag that includes something indicating that distinct
> style? If another language doesn’t normally use the Puyallup style
> fonts but some documents do, should those documents be tagged to
> indicate that style?
> 
> convention would never have accented capitals. Do we need variant
> subtags to indicate such distinctions? Or take this a step further:
> there are lots of Latin ligatures that _could_ be used in a document
> but aren’t required. Should language tags reflect levels of ligation
> that are used?
> 
> Certainly there are stylistic preferences associated with some
> languages, such as Nastaliq for Urdu. (Mind, I suspect Urdu speakers
> wouldn’t expect or want a Nastaliq font to be used for UI strings on
> a phone.) I could perhaps see some benefit in having a way to qualify
> fonts for such distinctions, and to capture that language/style
> association as a default in something like CLDR. But aside from

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 13 Aug 2020 13:57:45 -0600

Richard Wordingham wrote:

> However, I think Aran makes sense in catalogues, though my reasoning
> is based on the apparently analogous situation of Latn, Latf and Latg,
> but where Latn is not used if Latf or Latg were appropriate. [...]

The subtag 'Aran' is included in the Registry because the code element [Aran] is in ISO 15924, and we have to be extremely careful with the notion of cherry-picking code elements out of a core standard and leaving others behind. We don't know all the reasons why someone would want to use BCP 47, except that we all know (I hope) that web pages aren't the only use case. A tag or subtag that might not make any sense for one context might be very appropriate for another.

Initially, Daniel Billings simply asked for the Suppress-Script for 'ur' to be changed from 'Arab' to 'Aran', without stating the presumed benefit. Subsequent posts showed that he has document-processing applications which select a font based on the script subtag of the content, or based on the Suppress-Script of the language if no explicit script subtag is provided (I hope I have that right).

Well, specifying 'Aran' instead of assuming 'Arab' might make a lot of sense in that case, for printing documents in Urdu where the app doesn't already know what font to pick for Urdu. It might also be appropriate for cataloguing library holdings, as Richard said. It would make no sense at all for tagging plain text, because in plain text there is no difference between calligraphic styles. All of these are potential applications for BCP 47.

---

From: Shawn Steele <[email-masked]>
Date: Thu, 13 Aug 2020 20:53:33 +0000

Wouldn't the better approach be to let the app know that Urdu, particularly Urdu in Arabic, should use this style?

There are numerous cases where a particular "script" has different font preferences depending on the language being displayed.

-Shawn

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 13 Aug 2020 22:04:45 +0100

On Thu, 13 Aug 2020 09:52:58 -0700
Mark Davis ☕️ <[email-masked]> wrote:

> Saurashtra is
> still an issue (see the above page), but I see no bug report with the
> term "Saurashtra", except for the following 2, which appear unrelated.

I've now raised CLDR-14087 on the expansions of the languages
"Saurashtra" and "Northern Thai", and on the expansion of the script
"Ogham" on its own.  For the latter, I rely on Michael Everson's
statement that the language most often written in Ogham nowadays is
English.

Richard.

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 13 Aug 2020 15:08:47 -0600

Richard Wordingham wrote:

> en-Latn.

Not unless you have a particular reason for calling out the script. For example, you might have two parallel English texts, one in normal Latin and one in the Gaelic variety. It might be appropriate to tag them as "en-Latn" and "en-Latg" because you want to call special attention to the distinction. RFC 5646, Section 4.1, item 2, first bullet point shows another example.

That is not the same as reflexively tagging every piece of "en" text as "en-Latn", which was the concern 15 years ago.

> So, if I were cataloguing the various texts cluttering my house and
> recording their language and script, I would have some that were
> English in the Latin script and some that were English in the Thai
> script.*
>

And presumably the Latin-script and Thai-script texts were not parallel texts presented in contrast, as posited above.

> So, if I chose to use BCP 47 to record the language, the former should
> be recorded as "en" rather than as "en-Latn" and the latter could be
> recorded as "en-Thai".

Yes, because English is so overwhelmingly written in Latin script and not in Thai or other script (your personal collection notwithstanding) that this could be reasonably assumed for the former.

> Then, when I came to use the catalogue, I would know that those
> labelled as "en-Thai" were in the Thai script, but for those labelled
> "en", I should be unsure of the script; it would be improper to assume
> that the script was the Latin script, though that would be the best
> **guess**.

Maybe I'm unclear about the difference between "assume" and "guess" in this context.

Your, or some institutional, knowledge of English should indicate that English is written predominantly in Latin. The Suppress-Script for 'en' is there because English is written so predominantly in Latin script that to say so in a language tag is normally redundant.

To project this onto the Urdu-in-Nastaliq scenario, you would need to conclude that tag consumers encountering plain "ur" would be so likely to assume "ur-Aran" (specifically Nastaliq), and so unlikely to assume "ur-Arab" (Arabic with no indication as to calligraphic style), that "ur-Arab" is a special case to be called out in the tag, sort of like "en-Thai".

> the language and script as "en-Zyyy".

I would presume that if you just didn't look at it, it should be "en". If you looked at it and literally cannot determine what the script is, either because you don't have the knowledge or resources to identify it or because it's a new Voynich or whatever, then it would be "en-Zyyy".

---

From: Mark Davis ☕️ <[email-masked]>
Date: Thu, 13 Aug 2020 16:14:02 -0700

Thanks!

Mark

---

From: Richard Wordingham <[email-masked]>
Date: Fri, 14 Aug 2020 01:26:21 +0100

On Thu, 13 Aug 2020 15:08:47 -0600
"Doug Ewell" <[email-masked]> wrote:

> >> For the OP’s original scenario of Urdu in Nastaliq, I don’t have
> >> any issue with specifying “ur-Aran” when the choice of a Nastaliq
> >> font is considered important, especially since 'Aran' already
> >> exists. It's just not what Suppress-Script is for.  
> >

> Not unless you have a particular reason for calling out the script.
> For example, you might have two parallel English texts, one in normal
> Latin and one in the Gaelic variety. It might be appropriate to tag
> them as "en-Latn" and "en-Latg" because you want to call special
> attention to the distinction. RFC 5646, Section 4.1, item 2, first
> bullet point shows another example.

> That is not the same as reflexively tagging every piece of "en" text
> as "en-Latn", which was the concern 15 years ago.

> And presumably the Latin-script and Thai-script texts were not
> parallel texts presented in contrast, as posited above.

Parallel texts would indeed be a rare case.

> > So, if I chose to use BCP 47 to record the language, the former
> > should be recorded as "en" rather than as "en-Latn" and the latter
> > could be recorded as "en-Thai".  

> Yes, because English is so overwhelmingly written in Latin script and
> not in Thai or other script (your personal collection
> notwithstanding) that this could be reasonably assumed for the former.

> > labelled as "en-Thai" were in the Thai script, but for those
> > labelled "en", I should be unsure of the script; it would be
> > improper to assume that the script was the Latin script, though
> > that would be the best **guess**.  

> Maybe I'm unclear about the difference between "assume" and "guess"

If what I did with one of them depended on the script, then if I "guess"
I need to check the script: if I "assume", I don't check.

Now, I can't rely on the suppress-script field to assume that "en"
means "en-Latn"; that would be an improper use of the field.  Is that
correct?

> > In the implausible case that I knew that something was in English
> > but hadn't looked at it to determine the script, I presume I should
> > record the language and script as "en-Zyyy".  

> I would presume that if you just didn't look at it, it should be
> "en". If you looked at it and literally cannot determine what the
> script is, either because you don't have the knowledge or resources
> to identify it or because it's a new Voynich or whatever, then it
> would be "en-Zyyy".

Surely an English text in your conscript Ewellic should be
labelled "en-Zzzz", for 'uncoded script'.  So are you saying that the
script is only 'undetermined' if an attempt to determine its script
code has failed?

You seem to be saying that my index SHOULD NOT use a BCP 47 tag to
record whether a text in English is in the Latin script.  On the
other hand, it could be used to record the script of Northern Thai
texts.

Would a rule that the script must be indicated somehow make a
difference, e.g. by making plain "en" or "ur" imply that the script
subtag had been suppressed?

Richard.

---

From: Richard Wordingham <[email-masked]>
Date: Fri, 14 Aug 2020 01:40:35 +0100

On Thu, 13 Aug 2020 13:57:45 -0600
"Doug Ewell" <[email-masked]> wrote:

> It
> would make no sense at all for tagging plain text, because in plain
> text there is no difference between calligraphic styles.

What about "de" (or "de-Latn") v. "de-Latf"?  Fraktur has
meaning-sensitive ligaturing rules.  Still, that's rather exceptional.

Richard.

---

From: Martin Hosken <[email-masked]>
Date: Fri, 14 Aug 2020 10:57:24 +0700

Dear Hugh,

> to find what the basic default assumptions are for a language tag? Is the

This is an area that SIL has been working on recently. You can find the results of our work at https://github.com/silnrsi/langtags/tree/master/pub in json and summary txt. The json has much more information.

The work is based on a theory of tag sets. A set of tags that are considered semantically equivalent with a suggested default and full tag for each set. The reason for producing such a database is that when it comes to 'normal' tagging, people generally skip tag elements that add no distinction from the notional default. Therefore this is a need for a list of those defaults for systems that need that information.

Notice that we have not included aran, latv or any of the other variant scripts and have just kept to the core scripts. The tag sets would get rather big if we included those as well. If people are actually using those variants, I would be open to including them. But I do agree that I consider them unhelpful noise and that information about such things as font choice would be better held in the CLDR. In fact we have done this in our more humble embracing and extending of the CLDR to support minority languages for which there is much less information here at https://github.com/silnrsi/sldr/

We hope that that work to collect information on minority languages will help enable better support for them in wider technologies.

One difficulty we have is that with so many dialects and the informal requests for orthography variants (like the one you list below), we are not sure how to go about registering all these variants. We do not want to become a registrar of dialect information. It's bad enough running ISO 639-3. But neither does this list probably want to wade through all the various orthography variants and dialects that arise at the local level. So, currently, they are all -x-. Is anyone willing to help with that?

> the sub-tag registry. I.e the German language tags including the 1996

GB,
Martin

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 14 Aug 2020 14:00:39 +0900

Hello Richard, others,

> meaning-sensitive ligaturing rules.  Still, that's rather exceptional.

For ligatures like fi, the meaning-sensitive ligaturing rules of German 
apply to plain Latn, too. You might not always see them e.g. on Web 
pages, but in good print, they definitely should be used. It's true that 
Fraktur has more of these, but that's only a quantative difference.

Regards,   Martin.

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 13 Aug 2020 23:17:57 -0600

Richard Wordingham wrote:

>>> improper to assume that the script was the Latin script, though that
>>> would be the best **guess**.
>

Um, OK.

> means "en-Latn"; that would be an improper use of the field.  Is that

I have the feeling I'm being set up for something, but here goes:

The Suppress-Script field on 'en' means that IF content tagged as "en" is written at all — it could be spoken, or even some other modality — it is PROBABLY written in the Latin script. You cannot be 100% certain that it is written, or that it is written in the Latin script. If you need 100% certainty, the text should be tagged as "en-Latn".

But it does not follow from this that all English text written in the Latin script should be tagged "en-Latn". Most of the time, the assumption is correct and adequate. This is why the word SHOULD is used, instead of MUST.

> Surely an English text in your conscript Ewellic should be labelled
> "en-Zzzz", for 'uncoded script'.

I could do that, or I could use a script subtag in the private-use area (e.g. "en-Qabe") or a private-use subtag (e.g. "en-x-ewellic"). Each of these approaches has its pros and cons.

> So are you saying that the script is only 'undetermined' if an attempt
> to determine its script code has failed?

That is my understanding. I welcome the opinions of others on this question. I don't think it is covered in RFC 5646, and I don't believe the subject has come up on this list before.

Do you have a concrete use case surrounding this?

> record whether a text in English is in the Latin script.  On the other
> hand, it could be used to record the script of Northern Thai texts.

In your everyday life, it is not necessary under normal circumstances to point out that your car has four wheels, because the overwhelming majority of cars do.

That is how I view Suppress-Script. The distinction is not between English per se and Northern Thai per se, nor between the Latin script and other scripts.

> subtag had been suppressed?

A rule that the script must be indicated would be the exact opposite of what we were trying to accomplish in 2005, which is backward compatibility with the huge volume of existing language-tagged data.

Suppression of a script subtag isn't usually that kind of active thought process, like "I really want to indicate that this English text is in Latin script, but BCP 47 said I mustn't." It's more a matter of not stating that your car has four wheels, which might be obvious enough without saying so, or which might not even be relevant.

---

From: Michael Everson <[email-masked]>
Date: Fri, 14 Aug 2020 10:39:21 +0100

On 12 Aug 2020, at 20:56, Peter Constable <[email-masked]> wrote:
> 
> In general, I’m not sure why script-variant IDs like Aran or Latg exist in ISO 15924 at all, and I’m not aware of any application contexts in which they are actually used. For any languages written using Arabic script, including languages in Pakistan, Afghanistan and Iran, I think it’s generally likely that applications would use tags with a script subtag Arab or else without any script subtag. And changing the Suppress-Script field for ur would likely break some applications that would expect tags of the form “ur-Arab” were it not for the current Suppress-Script value.

They exist for bibliographical purposes for identification of books which might be in Latg (Gaelic script), Latf (blackletter), etc., where such identification would be useful to the end user. 

Michael Everson

---

From: Michael Everson <[email-masked]>
Date: Fri, 14 Aug 2020 10:39:40 +0100

On 12 Aug 2020, at 21:39, Daniel LaVon Billings <daniel=[email-masked]> wrote:
> 
> Culturally, they only use the Aran variant script for Urdu, so logically applications should look for a Nastaliq font as opposed to a Naskh font whenever “ur” is being called. Otherwise, we will be locked in a situation where our technical limitations are holding back the appropriate culturally significant script from being recognized as the standard in the language. The registry should reflect that cultural significance or else there is no reason for Aran to exist at all in ISO 15924.

ISO 15924 isn’t CLDR.

Michael Everson

---

From: Richard Wordingham <[email-masked]>
Date: Fri, 14 Aug 2020 13:32:05 +0100

On Thu, 13 Aug 2020 23:17:57 -0600
"Doug Ewell" <[email-masked]> wrote:

> > If what I did with one of them depended on the script, then if I
> > "guess" I need to check the script: if I "assume", I don't check.  
> 
> Um, OK.
> 
> > means "en-Latn"; that would be an improper use of the field.  Is
> > that correct?  
> 

Well, I had the strong feeling that what you have been saying would
*mostly* work, so I'm constructing a plausible situation where it seems
not to work.  Beyond that, I'm just asking you to check my logic. 

> The Suppress-Script field on 'en' means that IF content tagged as
> "en" is written at all — it could be spoken, or even some other
> modality — it is PROBABLY written in the Latin script. You cannot be
> 100% certain that it is written, or that it is written in the Latin
> script. If you need 100% certainty, the text should be tagged as
> "en-Latn".

> But it does not follow from this that all English text written in the
> Latin script should be tagged "en-Latn". Most of the time, the
> assumption is correct and adequate. This is why the word SHOULD is
> used, instead of MUST.

That seems to answer the issue.  If the language tag is describing
something that already exists, then if one may need to know the script
from just the tag, one should not heed suppress-script - which is why
the word 'SHOULD' is used.

> I could do that, or I could use a script subtag in the private-use
> area (e.g. "en-Qabe") or a private-use subtag (e.g. "en-x-ewellic").
> Each of these approaches has its pros and cons.
> 
> > So are you saying that the script is only 'undetermined' if an
> > attempt to determine its script code has failed?  
> 
> That is my understanding. I welcome the opinions of others on this
> question. I don't think it is covered in RFC 5646, and I don't
> believe the subject has come up on this list before.

> Do you have a concrete use case surrounding this?

I was coming to the conclusion that the ambiguity between "en" for
"en-Latn" and "en" for "English, but I won't tell you the script", was
resolved by the tags Zxxx, Zyyy and Zzzz.  Then the latter meaning for
text gets translated to en-Zyyy, and then the abbreviation of en-Latn
to en is resolved by the suppress-script.

> In your everyday life, it is not necessary under normal circumstances
> to point out that your car has four wheels, because the overwhelming
> majority of cars do.

But this information could be vital for positioning it over an
inspection pit.

> That is how I view Suppress-Script. The distinction is not between
> English per se and Northern Thai per se, nor between the Latin script
> and other scripts.

> A rule that the script must be indicated would be the exact opposite
> of what we were trying to accomplish in 2005, which is backward
> compatibility with the huge volume of existing language-tagged data.

I was thinking of a rule applying to that index of mine, not one for
universal use.

If we need to know the script of an item, then the pre-existing
language tagging simply lacks the information - we need to treat is as
undetermined.  I thought the more critical issue was old tag processors
that would choke if the script were provided.  The downside of the
solution was that for existing items, the script would need to be
determined if it was needed.  What can be silly is that absence of
information is intended to be the usual case.  Often, however, the
script needs to determined on a character by character basis, so
tagging by script is unnecessary and can be ignored.  There can be a
big practical difference between saying, 'this paragraph is in the
Devanagari script' and saying, 'the interface language is to use the
Devanagari script'. (There are some odd effects around though - I've
seen Thai line-breaking applied to stuff in a word-spacing 'complex'
script because I hadn't overridden the default language for complex
scripts, and Thai has its own paragraph justification options.)

> Suppression of a script subtag isn't usually that kind of active
> thought process, like "I really want to indicate that this English
> text is in Latin script, but BCP 47 said I mustn't." It's more a
> matter of not stating that your car has four wheels, which might be
> obvious enough without saying so, or which might not even be relevant.

But matters when choosing an inspection pit!  But a garage would
probably check how many wheels the car had, rather than rely on the
owner's statement.

Richard.