Subject: [Ietf-languages] Between language and script in Burmese

From: Simon Cozens <[email-masked]>
Date: Wed, 17 Nov 2021 09:20:05 +0000

Hello!

I've been working on a system font which covers a number of minority 
languages and scripts of Burma, some of which are not currently 
addressable because they lack IETF (and OpenType) script/language tags, 
or where the correct tag combination is not obvious.

The Burmese script has many language-specific and context-specific 
variant forms (see UTN11 - 
https://www.unicode.org/notes/tn11/UTN11_4.pdf - for examples), and the 
boundary between script and language is not always obvious. Some of 
these differences in letterforms are encoded separately in Unicode and 
some of them as allographs. It's all a bit of mess.

The easy problem we have is the Thai Mon language. This is a variant of 
the Mon language used by Mon people in Thailand. It has its own distinct 
script tradition. 
(https://www.unicode.org/L2/L2020/20163-arakanese-mon.pdf) There's no 
distinct language subtag but I believe mnw-TH is enough to distinguish 
this language - although we may have to pull some OpenType strings to 
enable that distinction to select Thai Mon specific orthographic forms.

The hard problem we have is that some of these language-specific variant 
orthographies are used to write text in *other* languages. In that 
sense, they are essentially functioning as *different scripts* to 
standard Myanmar.

For example: a document written in the Shan language using the Shan 
variant orthography of Burmese is clearly shn-Mymr, and setting the Shan 
language in a document should be enough to activate the Shan variant 
forms. No problems here. And a document written in Pali using the 
standard Burmese orthography is obviously pl-Mymr, and because it's 
standard Burmese, a Burmese font doesn't need to do any magic to get the 
right glyphs.

But what is a document written in the Pali language using the Shan (or 
Khamti, or Mon) orthography? Do we need variant tags to distinguish the 
flavour of Burmese script used in these cases? Shouldn't Shan, Khamti 
and Mon actually be separate scripts? And if not, how on earth are we 
going to get browsers to choose the Shan forms for this document, 
without pretending that it's actually written in the Shan language?

Any advice would be helpful!

Thanks,
Simon

---

From: Peter Constable <[email-masked]>
Date: Wed, 17 Nov 2021 20:03:45 +0000

Hi, Simon

You've touched on the easier cases that can be supported now in BCP 47. For Mon, you suggested mnw-TH could be used, though I would also consider mnw-Thai versus mnw-Mymr: if the only or main distinction to be made in content is the script used, then I'd use the script ID, not the region subtags. But if there are dialect differences between what is used in eastern Myanmar versus western Thailand, then that would be reason to use the region subtags.

For Pali written in Myanmar script, just that much information should be tagged using pl-Mymr, and that tag alone would convey _nothing_ about orthographic or typographic variants. (Though for some language tags a variant that is much more frequently used than others gets tagged as an unmarked case, without any additional subtag to reflect the distinction. Cf. suppress-script, for example.) If there are finer distinctions to be made-certainly for orthographic differences-then additional subtags would be needed. If orthographic distinctions correlate closely with region differences, then region subtags could be used to capture that. But for the cases you mention, region is not a good correlate. For those cases, variant subtags would be needed.

Of course, mapping from language tags to OpenType Layout language system tags to implement typographic distinctions is a related but separate matter.

Peter

-----Original Message-----
From: Ietf-languages <[email-masked]> On Behalf Of Simon Cozens
Sent: Wednesday, November 17, 2021 1:20 AM
To: [email-masked]
Subject: [Ietf-languages] Between language and script in Burmese

Hello!

I've been working on a system font which covers a number of minority languages and scripts of Burma, some of which are not currently addressable because they lack IETF (and OpenType) script/language tags, or where the correct tag combination is not obvious.

The Burmese script has many language-specific and context-specific variant forms (see UTN11 -
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.unicode.org%2Fnotes%2Ftn11%2FUTN11_4.pdf&amp;data=04%7C01%7C%7C3be526f409fd4088a03208d9a9ab9c8f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727376793324395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=k%2BeqHiUinoNECnQao0SiMRDlRhdZ%2B8KMAwRvtEPscoM%3D&amp;reserved=0 - for examples), and the boundary between script and language is not always obvious. Some of these differences in letterforms are encoded separately in Unicode and some of them as allographs. It's all a bit of mess.

The easy problem we have is the Thai Mon language. This is a variant of the Mon language used by Mon people in Thailand. It has its own distinct script tradition. 
(https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.unicode.org%2FL2%2FL2020%2F20163-arakanese-mon.pdf&amp;data=04%7C01%7C%7C3be526f409fd4088a03208d9a9ab9c8f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727376793324395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=PQYVW4W9Zwu6Vk2EfJSpR4snPN4P8bJQYwN8PclcHyc%3D&amp;reserved=0) There's no distinct language subtag but I believe mnw-TH is enough to distinguish this language - although we may have to pull some OpenType strings to enable that distinction to select Thai Mon specific orthographic forms.

The hard problem we have is that some of these language-specific variant orthographies are used to write text in *other* languages. In that sense, they are essentially functioning as *different scripts* to standard Myanmar.

For example: a document written in the Shan language using the Shan variant orthography of Burmese is clearly shn-Mymr, and setting the Shan language in a document should be enough to activate the Shan variant forms. No problems here. And a document written in Pali using the standard Burmese orthography is obviously pl-Mymr, and because it's standard Burmese, a Burmese font doesn't need to do any magic to get the right glyphs.

But what is a document written in the Pali language using the Shan (or Khamti, or Mon) orthography? Do we need variant tags to distinguish the flavour of Burmese script used in these cases? Shouldn't Shan, Khamti and Mon actually be separate scripts? And if not, how on earth are we going to get browsers to choose the Shan forms for this document, without pretending that it's actually written in the Shan language?

Thanks,
Simon

_______________________________________________
Ietf-languages mailing list
[email-masked]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fietf-languages&amp;data=04%7C01%7C%7C3be526f409fd4088a03208d9a9ab9c8f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727376793324395%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=TUhhHlRbKHXmwTcwJLCzdNap1S%2F0G4VbUhxV2%2BQinXw%3D&amp;reserved=0

---

From: Simon Cozens <[email-masked]>
Date: Wed, 17 Nov 2021 21:39:54 +0000

On 17/11/2021 20:03, Peter Constable wrote:
> You've touched on the easier cases that can be supported now in BCP 47. For Mon, you suggested mnw-TH could be used, though I would also consider mnw-Thai versus mnw-Mymr

I'm not sure this is right. mnw-Thai would be Mon written in the Thai 
script (not what we want), and mnw-Mymr would be Mon (not Thai Mon) 
written in Myanmar script; neither of which would be referring to the 
Thai Mon language written in the Thai Mon orthographic variant of 
Myanmar script.

> For Pali written in Myanmar script, just that much information should be tagged using pl-Mymr, and that tag alone would convey _nothing_ about orthographic or typographic variants.

Right. So that's obviously not enough.

> If there are finer distinctions to be made-certainly for orthographic differences-then additional subtags would be needed. If orthographic distinctions correlate closely with region differences, then region subtags could be used to capture that. But for the cases you mention, region is not a good correlate. For those cases, variant subtags would be needed.

I think we are going to need variant subtags to denote orthographic 
forms of the Myanmar script preferred by the Shan, Khamti and Mon 
communities. Advice on how to get these variants registered would be 
appreciated!

> Of course, mapping from language tags to OpenType Layout language system tags to implement typographic distinctions is a related but separate matter.

Well, yes. I'll be pestering you about registering some new OTL language 
system tags once we've got something for them to map from!

Simon

---

From: Martin Hosken <[email-masked]>
Date: Thu, 18 Nov 2021 09:05:11 +0700

Dear All,

Sorry, I was away for a few days.

> Thai Mon language written in the Thai Mon orthographic variant of 

I agree with Simon here. mnw-Thai is Mon in Thai script for which there is no evidence (which seems odd to me).

> communities. Advice on how to get these variants registered would be 

Yes. I'm not sure why there is a need to register all these Pali orthographies (since a change in underlying orthography is a different orthography for the language concerned). What's wrong with simply tagging it shn? Is there a need to distinguish pi in this way.

Please do also bear in mind that there is a small plethora of Shan orthographies, which need their variants and into which Pali my be represented. For example: Cushing, Common, Modern, Old, etc. Perhaps we need to cut the Shan pie and register variants for those, before we address Pali.

Khamti is recommended to be encoded using variation selectors which would alleviate the need for a contrastive tag, although there is still the font variation regarding using filled dots over unfilled dots, which re-raises the need. Mon has no stylistic variation over Burmese that I know of, although there may be spelling/encoding differences between pi-Mymr-x-mnw vs pi-Mymr.

> system tags once we've got something for them to map from!

It all depends how much of a hurry you are in and how important it is to distinguish pi as a separate language based on these orthographies. There's no free lunch here and with the free and easy code switching that happens with Pali and these other languages, I merely raise the question of how important marking pi is for them.

Yours,
Martin

> https://www.ietf.org/mailman/listinfo/ietf-languages

---

From: Richard Wordingham <[email-masked]>
Date: Thu, 18 Nov 2021 19:47:27 +0000

On Thu, 18 Nov 2021 09:05:11 +0700
Martin Hosken <[email-masked]> wrote:

> > communities. Advice on how to get these variants registered would
> > be appreciated!  
> 
> Yes. I'm not sure why there is a need to register all these Pali
> orthographies (since a change in underlying orthography is a
> different orthography for the language concerned). What's wrong with
> simply tagging it shn? Is there a need to distinguish pi in this way.

I suppose you'll point to the low demand for Latin spell-checkers, but
while plain text seems to be handled well, there are some stereotypical
character-selection differences between Mon and Burmese Pali, and Shan
Palis have a very striking spelling difference, and differ amongst
themselves.  I have strong doubts abut the stabilities of the Shan Pali
orthographies.

On the other hand, I don't think we need any special typographical
support for Mon pi-Mymr.  Text-to-speech would be massively
different.  (There are a few recordings on English Wiktionary.)

Richard.

---

From: Simon Cozens <[email-masked]>
Date: Fri, 19 Nov 2021 08:10:59 +0000

On 18 Nov 2021, at 02:05, Martin Hosken <[email-masked]> wrote:
> 
> What's wrong with simply tagging it shn? Is there a need to distinguish pi in this way.

Because the implication is that if you want a document to appear in a variant *script* you have to change the *language* tag - and change it to something which is not actually the language of the document! That can’t be right, surely?

> Khamti is recommended to be encoded using variation selectors

Poor Khamti: language tagging for *my* languages but variant selectors for yours…

> It all depends how much of a hurry you are in and how important it is to distinguish pi as a separate language based on these orthographies. 

I don’t think we do anything specific in the font for Shan Pali; we just wanted to know the answer to the question “how should I correctly tag a document to get this appearing with the right letter forms?”

Thai Mon is different - we really do want support for that, so will need an OTL tag and to have shaping engines apply that tag to mnw-TH documents.

But Shan Pali is a useful case because it does ask the question of whether something like Shan - and all these variant forms of Burmese orthography - is a language or a script (or a script variant). Some of them diverge quite strongly from the standard Burmese letterforms but IETF considers them all Mymr. I’d question that.

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 19 Nov 2021 19:21:58 +0900

Hello Simon, others,

> But Shan Pali is a useful case because it does ask the question of whether something like Shan - and all these variant forms of Burmese orthography - is a language or a script (or a script variant)…

It's not the IETF that decides on script codes. The IETF just pulls 
together various ISO standards into a single language tag, and has some 
escape hatches such as variant subtags when the ISO standards are not 
enough.

Regards,   Martin.

---

From: r12a <[email-masked]>
Date: Fri, 19 Nov 2021 12:49:59 +0000

fwiw, here are some thoughts for my 2p:

It seems to me that the less you have to remember that a particular 
combination of tags means x or y, the better.  If we can use tags that 
are easy to understand and guess without having to remember special 
rules or conventions, then we're much better off.

I think the language tag should always reflect the actual language of 
the text, regardless of script or orthography. Labelling something as 
shn when it's not will mess up things like voice browsers, spell 
checkers, and possibly layout, opentype features, etc.

We actually appear to have a few script tags that are in fact 
orthography tags, such as aran, syrn, and syrj (see 
https://r12a.github.io/app-subtags/?lookup=aran,syrn,syrj) but it's 
never been clear to me why that is a good idea (except for hant, and 
hans, but the situation is somewhat  different there).  I'm inclined to 
think that script tags should just be distinctive at the script level.

I'm also wary of the idea of using region tags to specify a particular 
orthography (like we used to with zh-TW, and zh-CN), unless you really 
want to identify the orthography only on the basis of a particular 
regional standard (like en-GB), because (a) the usage may not be cleanly 
defined by a region tag (eg. things like Azeri, Kurdish, etc. or use of 
zh-CN for Singapore) and (b) it may be difficult to know/remember which 
region tag should be used. These relationships with region may also 
change in the future, as people migrate or countries mutate.

On the other hand, we have *loads* of variant tags that are related to 
specific orthographies. See 
https://r12a.github.io/app-subtags/?find=ortho which list around 50.

So i'm inclined to think that perhaps the solution is to submit a 
request for new orthographic variant subtags.

ri

Martin J. Dürst wrote on 19/11/2021 10:21:
>> But Shan Pali is a useful case because it does ask the question of 
>> whether something like Shan - and all these variant forms of Burmese 
>> orthography - is a language or a script (or a script variant). Some 
>> of them diverge quite strongly from the standard Burmese letterforms 
>> but IETF considers them all Mymr. I’d question that.
>
> together various ISO standards into a single language tag, and has 
> some escape hatches such as variant subtags when the ISO standards are 
> not enough.
>
> Regards,   Martin.
>

---

From: John Cowan <[email-masked]>
Date: Fri, 19 Nov 2021 22:03:19 -0500

On Fri, Nov 19, 2021 at 7:50 AM r12a <[email-masked]> wrote:

> We actually appear to have a few script tags that are in fact orthography
> tags, such as aran, syrn, and syrj (see
> https://r12a.github.io/app-subtags/?lookup=aran,syrn,syrj) but it's never
> been clear to me why that is a good idea (except for hant, and hans, but
> the situation is somewhat  different there).  I'm inclined to think that
> script tags should just be distinctive at the script level.
>

They aren't orthography tags in the sense you mean: they are script
*variant* tags.  Consider German.  A German text has the same orthographic
conventions whether it is written in Antiqua (ordinary Latin script) or
Fraktur.  In a sense, the distinction is merely a matter of the fonts
used.  However, people who know German but are not familiar with Fraktur
will find it almost as unintelligible as if it were in some different
script altogether, and so one may tag a German book in Fraktur as de-Latf
rather than de-Latn to mark this distinction.

There also exist Fraktur books in Danish, Norwegian, Swedish, Estonian,
Latvian, and perhaps Finnish (unless it was only used for Finland Swedish,
I don't know), so it is not just a matter of variations in writing German.
Indeed, 𝔗𝔥𝔢 𝔑𝔢𝔴 𝔜𝔬𝔯𝔨 𝔗𝔦𝔪𝔢𝔰 uses it in the banner at the top
of the front page, which could be tagged en-Latf.  The story for Nastaliq
(and in principle for Kufic, Tulut, Diwani, Sini, etc.) and for the three
variants of Syrian is exactly the same.

---

From: Richard Wordingham <[email-masked]>
Date: Sat, 20 Nov 2021 08:14:16 +0000

On Wed, 17 Nov 2021 21:39:54 +0000
Simon Cozens <[email-masked]> wrote:

> > Of course, mapping from language tags to OpenType Layout language
> > system tags to implement typographic distinctions is a related but
> > separate matter.  

> Well, yes. I'll be pestering you about registering some new OTL
> language system tags once we've got something for them to map from!

Is there precedent for allocating pi-Tham-MM and pi-Tham-LA different
OTL language tags?  Closer to home, is there a tag that can get one
French Classical Greek lettering?

Richard.

---

From: Simon Cozens <[email-masked]>
Date: Sat, 20 Nov 2021 08:26:20 +0000

On 20/11/2021 08:14, Richard Wordingham wrote:
> OTL language tags? 

As mentioned elsewhere, we don't do anything special for Shan Pali but 
we are going to need something to enable Thai Mon.

> Closer to home, is there a tag that can get one

Greek script with French document conventions would be grek/FRA.
(https://typedrawers.com/discussion/comment/54645/#Comment_54645

---

From: Richard Wordingham <[email-masked]>
Date: Sat, 20 Nov 2021 10:21:51 +0000

On Sat, 20 Nov 2021 08:26:20 +0000
Simon Cozens <[email-masked]> wrote:

> > Is there precedent for allocating pi-Tham-MM and pi-Tham-LA
> > different OTL language tags?   

> As mentioned elsewhere, we don't do anything special for Shan Pali
> but we are going to need something to enable Thai Mon.

I was asking about a similar case where the OTL language tags already
arguably exist - SHN (if you accept the Tai Khuen as Shan) and LAO.
And there are documents where the vernacular is indisputably Shan
(Myanmar script, though I don't rule out obsolete unencoded characters)
and the Pali is in the Tai Tham ('Yuan') script. 

> (https://typedrawers.com/discussion/comment/54645/#Comment_54645

That's not a BCP 47 tag.

Richard.

---

From: Peter Constable <[email-masked]>
Date: Mon, 22 Nov 2021 23:34:40 +0000

Sorry, I misunderstood

> This is a variant of the Mon language used by Mon people in Thailand. It has its own distinct script tradition.

to mean Mon written in a different script from Mymr script rather than using a distinctive style of Mymr script.

Peter

-----Original Message-----
From: Simon Cozens <[email-masked]> 
Sent: Wednesday, November 17, 2021 1:40 PM
To: Peter Constable <[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Between language and script in Burmese

On 17/11/2021 20:03, Peter Constable wrote:
> You've touched on the easier cases that can be supported now in BCP 
> 47. For Mon, you suggested mnw-TH could be used, though I would also 
> consider mnw-Thai versus mnw-Mymr

I'm not sure this is right. mnw-Thai would be Mon written in the Thai script (not what we want), and mnw-Mymr would be Mon (not Thai Mon) written in Myanmar script; neither of which would be referring to the Thai Mon language written in the Thai Mon orthographic variant of Myanmar script.

> If there are finer distinctions to be made-certainly for orthographic differences-then additional subtags would be needed. If orthographic distinctions correlate closely with region differences, th…

I think we are going to need variant subtags to denote orthographic forms of the Myanmar script preferred by the Shan, Khamti and Mon communities. Advice on how to get these variants registered would be appreciated!

> Of course, mapping from language tags to OpenType Layout language system tags to implement typographic distinctions is a related but separate matter.

Well, yes. I'll be pestering you about registering some new OTL language system tags once we've got something for them to map from!

Simon

---

From: Peter Constable <[email-masked]>
Date: Tue, 23 Nov 2021 00:01:30 +0000

I should have elaborated: 

For Mon (Mymr script) as used in Thailand, potentially mnw-Mymr-TH could be used if the style correlates particularly with the region TH (vs. MM). But as Richard Ishida pointed out, using region subtags might not be the best choice, and there are several precedents for using variant subtags to distinguish orthographic variants. 

One thing that should be clarified for "Thai Mon" (i.e., Mon as written in Thailand) is the extent to which the differences are orthographic versus typographic in nature. 

I'd characterize "orthographic" to mean spelling-i.e., what Unicode character sequences are used to spell words. By "typographic", I mean how the text is presented in terms of glyphs and their shaping/layout. For example, since U+103F GREAT SA is encoded as an atomic character, distinct from the sequence <101E SA, 1039 VIRAMA, 101E SA>, that's an orthographic difference, not a typographic difference. On the other hand, differences such as where loop terminals appear would be a typographic difference.

Btw, looking at https://www.unicode.org/L2/L2020/20163-arakanese-mon.pdf, there may be some open issues not yet resolved as to whether some Thai Mon forms should be treated as different presentations (typographic) or rather should be encoded as separate characters (orthographic).

However much these might be considered orthographic versus typographic distinctions, though, they are potential candidates for distinguishing between them using BCP 47 tags.

Peter

-----Original Message-----
From: Ietf-languages <[email-masked]> On Behalf Of Peter Constable
Sent: Monday, November 22, 2021 3:35 PM
To: Simon Cozens <[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Between language and script in Burmese

Sorry, I misunderstood

Peter

-----Original Message-----
From: Simon Cozens <[email-masked]>
Sent: Wednesday, November 17, 2021 1:40 PM
To: Peter Constable <[email-masked]>; [email-masked]
Subject: Re: [Ietf-languages] Between language and script in Burmese

Simon
_______________________________________________
Ietf-languages mailing list
[email-masked]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fietf-languages&amp;data=04%7C01%7C%7C87e8c8f845574ae974de08d9ae10b052%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637732208978652256%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=gNnXd198qY955SsyrI0Y2JCo9w90sLqd94fR7pVXeuE%3D&amp;reserved=0

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 27 Nov 2021 15:55:41 -0700

I wonder if posting a list of the combinations of language, script, and typographic or orthographic variety that are under consideration, along with the appropriate BCP 47 tag for each or a note that there isn't one, would help others as much as it would help me.

Such a list might make it easier to compare similar tagging possibilities and identify where existing combinations of subtags are just right, where they are possible but ill-advised or non-intuitive, and where new variant subtags are needed.

I know exceedingly little about typography in this part of the world, so perhaps the confusion is just mine.

--
Doug Ewell, CC, ALB | Lakewood, CO, US | ewellic.org