Subject: Re: [Ietf-languages] Forms for subtag kmpre20c

From: Michael Everson <[email-masked]>
Date: Sun, 1 Dec 2019 14:14:01 +0000

> I am still not happy with it. How many centuries of orthography is this tag supposed to support?

Whatever is in Khmer script that's written in an old spelling. How
does it matter?

> What process will be able to do anything with it?

The database I'm needing subtags for is mainly bibliographical, it has
the title for each text in two flavors: the original (old) spelling as
appearing on the manuscript, and the equivalent in modern spelling
(Chuon Nath style).

> And again, what are the reforms? What are their dates?

The reform process is long and complex. I have put the reference to
this article describing it several times:

https://englishkyoto-seas.org/2015/04/vol-4-no-1-sasagawa/

I'm not sure what else I can do... Should I copy paste the article
content into an email? Here's a short summary:

1915: establishment of the committee for editing a Khmer dictionary,
start of the debates between phonetic vs. etymological spellings
1926: establishment of a second committee led by Chuon Nath, using
mostly etymological spellings
1920s: printeries using mostly reformed orthography, as it was largely
under the control of those who favored reform (including the French
and reformist monks at the Institut Bouddhique), whereas manuscripts
were generally produced by traditional scribes and scholars and used
non-reformed orthography
1938: first edition of the Dictionnaire Cambodgien by Chuon Nath
1967: 5th and final edition of the Dictionnaire Cambodgien
1967-1974: Khmer becomes main language in education
1972: reform by Loch Phlaeng and the Khmerization movement (more based
on phonetic, less letters, less diphtongues), used officially from
1985 to 2009
2009: official use reverts to Chuon Nath's Dictionnaire Cambodgien

So I suppose you could arbitrarily pick 1967 and 1972 as dates for the
two reforms, but it's not clearcut at all.

> Our tags generally point TO a reference, and don’t specify themselves by relation to what they are NOT.

Well, I guess I'll keep this in a private subtag then. There's no
homogeneity in the pre-reform Khmer spelling. There is no tag that
could be defined to point TO it, because it doesn't exist as a
homogeneous concept that can be agreed upon.

> What sort of Khmer? Modern Khmer (whenever that dates from and to)? Or is this tag supposed to include Old and Middle Khmer?

Why does it matter? My data has whatever Khmer there is in the
Buddhist texts from the 16th to the 20th century (most of them copied
from previous sources, there is no way of knowing when a manuscript
started to circulate so some probably predate the 16th c.). For what
it's worth, I also have a lot of Pāli written using old Khmer
spelling.

> As to the reform or reforms, should there be a subtag that points to an authoritative source for one or more of these?

1967 edition of the "Dictionnaire Cambodgien" by Chuon Nath could be a
good reference for the first one

For the second reform, I have no source... It it talked about in the
article from 2015 and in
https://www.persee.fr/doc/befeo_0336-1519_1999_num_86_1_3414 but I
don't have an exact reference. I'll try to get one.

> This is underspecified and it’s not satisfactory so far.

And it will not be specified further because it cannot.

I suppose one way to unlock the situation would be to propose a subtag
for Chuon Nath's spelling style, I'll do that let's see how it goes.

Best,
-- 
Elie

_______________________________________________
Ietf-languages mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf-languages

---

From: Élie Roux <[email-masked]>
Date: Sun, 1 Dec 2019 15:26:48 +0100

> We have en (English, unspecified (apart from being modern English, not Old or Middle English), and we have en_US and en_GB for main variants, typically involving and we have en_GB-oxendict for a specific spelling variant.
>
> It seems to me that you need a subtag for modern spelling; assuming that Chuon Nath is the name of the prevalent reform, for instance, you could have km-chuonnat or km-chn1938 or the like.
>
> In fact if there were a need you could have subtags referrng to various editions of the reformed orthographies.
>
> But really, we use subtags for what things ARE, not for what the are not.

Ok yes, makes sense. I will do that once I manage to gather more
documents and info, which should happen in Spring.

So, to continue the analogy with English: the -oxendict subtype refers
to the general style right? What I mean is: in the same way that the
Oxford Dictionary
- has different editions with (I suppose marginally) different spelling
- does not encompass all possible words of all possible strings tagged
with -oxendict

I could propose a -dictcamb (for Dictionnaire Cambodgien), which could refer to:
- the general work, not a specific edition
- a general style of spelling, defined by a dictionary that doesn't
encompass all possible words (especially loan ones, even though it
refers to Pali a lot)

right?

I think the main difference is that there will likely not be any new
edition of the Dictionnaire Cambodgien (we could assume that if
someone was to take over the project of a new edition, they would not
keep the French name), and that people usually refer to the 1967
(last) edition. In that case, what would be the customary type of
langtag? "-dica1967" or "-dictcamb"?

Best,
-- 
Elie

---

From: Michael Everson <[email-masked]>
Date: Sun, 1 Dec 2019 14:54:53 +0000

Doug, I’ve rejected the current version of the proposal. 

On 1 Dec 2019, at 14:26, Élie Roux <[email-masked]> wrote:
> Ok yes, makes sense. I will do that once I manage to gather more documents and info, which should happen in Spring.

Whenever you can. It doesn’t have to be a thesis.

> So, to continue the analogy with English: the -oxendict subtype refers to the general style right? What I mean is: in the same way that the Oxford Dictionary
> - does not encompass all possible words of all possible strings tagged with -oxendict

In this case, en_GB has two main traditions of spelling. Oxford uses one which spells centre, colour, civilize, and the other tradition spells centre, colour, civilise. The use of this tag is chiefly to select a spell-check dictionary, I think. The one orthography closest to en_GB-oxendict is en_CA.

> - the general work, not a specific edition

If the orthography differs from edition to edition and you need to make that distinction, you may need more than one subtag. 

> - a general style of spelling, defined by a dictionary that doesn’t encompass all possible words (especially loan ones, even though it refers to Pali a lot)
> 
> right?

I tend to think that subtags can be used to mark a variety of different contexts but selection of a spell-check dictionary is a very practical one.

> langtag? "-dica1967" or "-dictcamb”?

dc1967 would be adequate I should think. It depends on the number of things you think you need to differentiate.

Michael

---

From: Richard Wordingham <[email-masked]>
Date: Sun, 1 Dec 2019 18:50:39 +0000

On Sun, 1 Dec 2019 14:14:01 +0000

> Whatever is in Khmer script that's written in an old spelling. How

You are not the only consumer of subtagging.  For example, etymologies
on the English Wiktionary often tag words as 'Old Khmer'. 

> appearing on the manuscript, and the equivalent in modern spelling

So why does that need a tag?  What of a spelling that used the glottal
stop latter ('QA') when Chuon Nath prescribes a vowel-specific vowel
letter?  Even though the original spelling is in a way more modern
('before its time'), would not that have two spellings recorded for it?

Is this database restricted to manuscripts?

OT: How do you handle glyph differences that are close to being
character differences?  One example, if I understand Antelme, is that
from a lumper's perspective U+17BE used to look like *<U+17C1 E,
U+17B7 I>, but this proscribed combination contrasts with U+17BE in
Thai (perhaps in a wider sense).

> > And again, what are the reforms? What are their dates?  

I hope this is for identification purposes.  Several systems seem to
have been concurrent.

> two reforms, but it's not clearcut at all.

Your account hasn't made clear when the use of CVC orthographic
syllables was largely abandoned.  That's not a phonetic v. etymological
difference.  Are we looking at at least three distinguishable writing
systems?

> > Our tags generally point TO a reference, and don’t specify
> > themselves by relation to what they are NOT.  
> 

That doesn't stop manuscripts being classified as 'Middle English',
despite its extreme lack of homogeneity.

> For what

OT: 'Pāli written using old Khmer spelling' is an interesting concept.
Would you care to educate me by elaborating?  I've just been surprised
by the form of some 150 year old printed Thai-script Pali.  It's
different to the two living orthographies supported on Wiktionary.

Richard.

---

From: Élie Roux <[email-masked]>
Date: Sun, 1 Dec 2019 20:57:58 +0100

> on the English Wiktionary often tag words as 'Old Khmer'.

Hmm sure, but in my mind this is a different issue: if the -kmpre20c
were to exist, I can't really see why it couldn't be applied to
anything written in Khmer script using non-reformed spelling: Old,
middle, modern Khmer, all sorts of dialects, Pali, Sanskrit, Chinese,
etc. That's what I meant, I'm not sure how you interpreted my point
though?

> So why does that need a tag?

To change the Lucene analyzer and for the UI (for each string in our
database, we have a tag, and there's a little popup saying what
language is the string). Note that we're dealing mostly with
low-resource languages, and sometimes non-standardized scripts and
transliterations; so we had to come up with a very lengthy list of
private lang tags:
https://github.com/buda-base/owl-schema/blob/master/lang-tags.md and
some conventions to refer to the scripts (and script flavors) that are
not in Unicode:
https://github.com/buda-base/owl-schema/blob/master/types/scripts.ttl
(we didn't include the SEA scripts yet, but there are quite a few).

>  What of a spelling that used the glottal

Well, you lost me here, my knowledge of Khmer is close to
non-existant, but I'm copying Trent Walker who has a very vast
knowledge of these matters!

> Is this database restricted to manuscripts?

Currently yes, in the future we could imagine cataloging modern publications.

> Thai (perhaps in a wider sense).

Another question I won't pretend to be able to get the answer of :)

> I hope this is for identification purposes.  Several systems seem to

Several in the XXth century, and before that there doesn't seem to be
any study; so I'm not even sure we can confidently speak of concurrent
systems, it might have been just very wild.

> difference.  Are we looking at at least three distinguishable writing

Another question that's not for me...

> despite its extreme lack of homogeneity.

Well, I really have no clue about the evolution of the Khmer language,
in my mind the tag was only about the evolution of spelling
conventions: the database we have has titles in old spelling + new
spelling, it's just a mapping of characters and not a change of
language (if I understand correctly). So in my mind this is a
different situation from the Middle English / English situation where
the language itself is different, not just the spelling.

> Would you care to educate me by elaborating?

Hmm, I would love to do so but I'm not sure I'm the best person... I
can refer you to http://www.trentwalker.org/unfoldingbuddhism which
has some most excellent scholarship and scans of Leporellos, some of
which contain Pali using the old spelling.

>  I've just been surprised

Hmm, that could be an entry point to an interesting discussions
(perhaps to be had outside the list), can you tell me what you're
referring to?

Best,
-- 
Elie

---

From: Richard Wordingham <[email-masked]>
Date: Mon, 2 Dec 2019 14:12:37 +0000

On Sun, 1 Dec 2019 20:57:58 +0100
Élie Roux <[email-masked]> wrote:

> > You are not the only consumer of subtagging.  For example,
> > etymologies on the English Wiktionary often tag words as 'Old
> > Khmer'.  
> 

If that's truly the case, the proper tag is und-Khmr.  You then hit the
problem that language tagging doesn't handle exclusions.  At least,
Michael Everson said it doesn't and I have no reason to disbelieve
him.  It also makes sense to me as a policy.

> (we didn't include the SEA scripts yet, but there are quite a few).

And this immediately undermines the previous generality, as it includes
things like pi-Khmr.

> Currently yes, in the future we could imagine cataloging modern

What about printed Khmer-script missionary texts from 1893 printed in
Hong Kong?

> the language itself is different, not just the spelling.

Most living languages are continually changing, though I can't comment
on how hard it is to apply a knowledge of Modern khmer to understand
Old Khmer and Middle Khmer.  Some sources talk of a change of script,
though what that entails I don't know.  Certainly vowel symbols have
been added (apparently based on Thai additions), and the distinction
between Indic b/v has been reinstated.  Wikipedia starts Modern Khmer
in the later 18th century.

The fact that Middle English as a whole certainly lacked a standard
orthography is no bar to its being classified as a language.  The lack
of standards is therefore not of itself a bar to adding variants for
Old Khmer (if you truly have such materials), Middle Khmer and I
suggest is not necessarily a bar to 'pre 20th century' Modern Khmer.
It might be necessary to provide evidence of a time depth to the
variations - in which case we should try to get the experts involved. A
lack of agreement on classifying substantial documents could be, though
one should expect borderline documents.  How far back the BCP 47
language 'km' should go is another matter, probably not within IETF
jurisdiction.

> > OT: 'Pāli written using old Khmer spelling' is an interesting
> > concept. Would you care to educate me by elaborating?  
> 

I was chasing up the references in
https://en.wikipedia.org/wiki/Talk:Thai_script#Sanskrit_and_Pali_Orthographies ,
in particular to the book "Pali Phonetic Edition" "พระไตรปิฎกสัชฌายะ
ฉบับเสียงอ่านปาฬิ", available at
http://plcthinktank.com/e-Book/Resource/0000005319.pdf.  The Wikipedia
article currently only documents the academic, abugida writing of Pali
in the Thai script, with a very misleading reference to the
'alphabetical' writing found in most books of chants. The books shows a
third scheme in use in a 19th century publication.  The third scheme is
primarily also an abugida, but has (encoded) vowel killers (two!)
different to the one used as a vowel killer nowadays, and marks
short /a/ in closed syllables with mai hanakat in much the same way as
the corresponding symbol Khmer samyok sannya is used. Michell's
"Siamese-English dictionary" (1892) noted that Siamese gramarians
regarded this as a vowel symbol.  The third scheme seems to have a
name, 'karayut', so it seems almost ready for registration.  However, I
haven't found names for the modern abugidic and alphabetic systems of
writing Pali.  One practical application would be to distinguish the
variants in spelling dictionaries.  It's the modern, unnamed varieties
for which spelling dictionaries would have most use.

Richard.

---

From: Élie Roux <[email-masked]>
Date: Mon, 2 Dec 2019 15:36:46 +0100

> If that's truly the case, the proper tag is und-Khmr.

Why not. Generally speaking I think all lang tags should all have a
macrolanguage though. Most our database is composed of Tibetan; most
of it is Classical but we also have old and modern, but we don't care
about the distinction in the lang tags. I certainly find und-Tibt (or
actually und-x-ewts as we're using transliteation) quite ugly and I
would much prefer bo to be a macrolanguage (like zh) instead of an
individual one. But that's not a problem I can solve... In the
meantime I'll use km as it's much more user friendly.

> You then hit the

Sure

> And this immediately undermines the previous generality, as it includes

What generality? I'm not following

> What about printed Khmer-script missionary texts from 1893 printed in

We don't have any yet and we have no acquisition plan for such texts.
What about them?

> suggest is not necessarily a bar to 'pre 20th century' Modern Khmer.

Sure.

> variations - in which case we should try to get the experts involved.

That's what we're trying to do, but we can't expect to stop all
operations before we get the answers, and they probably won't be there
in the next few decades... and I need to tag my data in the meantime.

Best,
-- 
Elie

---

From: Richard Wordingham <[email-masked]>
Date: Mon, 2 Dec 2019 16:45:00 +0000

On Mon, 2 Dec 2019 15:36:46 +0100
Élie Roux <[email-masked]> wrote:

> > And this immediately undermines the previous generality, as it
> > includes things like pi-Khmr.  
> 

You said that you would like to use "-kmpre20c" for Khmer script
materials that were "written in Khmer script using non-reformed
spelling: Old, middle, modern Khmer, all sorts of dialects, Pali,
Sanskrit, Chinese, etc".  Then you referenced your convention on
github, which sensibly gives "pi-Khmr" for Pali in Khmer script.

However, I may have misunderstood you.  I though you meant you would
use "km-kmpre20c", but perhaps you just meant you would suffix it to
other languages.  On closer examination I saw pi-Khmr-x-kmpre20c.  The
general view here is that the latter should not happen without explicit
sanction.

> > What about printed Khmer-script missionary texts from 1893 printed
> > in Hong Kong?  
> 

You wrote, "The database I'm needing subtags for is mainly
bibliographical, it has the title for each text in two flavors: the
original (old) spelling as appearing on the manuscript, and the
equivalent in modern spelling (Chuon Nath style)".  I therefore
wondered if your database were restricted to manuscripts.  

> in the next few decades... and I need to tag my data in the meantime.

You probably need to make provision for automatically converting the
tags later.

Richard.