Subject: Re: [Ietf-languages] First cut at a BCP 47 extension structure for ISO TR 21636 (was: Language subtag registration form)

From: "Doug Ewell" <[email-masked]>
Date: Fri, 27 Nov 2020 23:29:32 -0700

Sebastian Drude wrote:

> I believe we should not worry about the personal varieties at this
> point.

I think this would be a prudent exclusion. As with the 'u' and 't'
extensions, keys can be added after the initial rollout, but I can't
imagine a scenario in which encoding this dimension would ever be
practical.

> Probably we should form a small committee to continue this discussion,
> instead of involving (and spamming, at this point) the whole list.  I
> am certainly willing to be instrumental in (contributing to) forming
> and driving such a group, in whichever role I can contribute best.

Maybe a custom mailing list could be formed. Everyone here would need to
be eligible to join the discussion if they choose to.

> Points that I can see now that would need to be discussed, besides
> flashing out what you have begun:
>
> -- interaction with existing subtags, in particular dialects --
> probably we want to avoid synonym language tags composed according to
> different frameworks

I can envision this becoming a real time and effort sink. As just one
example, script subtags imply "written" (one implies "not written"), and
this extension would provide a "medium" dimension. What if they disagree
within the same tag? Trying to cherry-pick the 21636 values to exclude
those that might conflict with other subtags would probably be very
messy.

> -- can more than one value in the SAME dimension be indicated? (I would
> argue yes, if that is syntactically okay)

Syntactically, sure:

Sometimes this will make sense, such as for "communicative functioning."
In many cases it clearly won't, as in combining "formal" and "informal"
situations. Some semantic restrictions might be appropriate in this
extension to guard against the latter. Our approach in BCP 47 has simply
been to advise tag creators to "tag wisely," discouraging nonsensical
combinations but not making them invalid.

> -- the 'certainty' and similar "adjectives" (yes, that is how I would
> see them; -- e.g. primary vs. secondary modality, genuine vs.
> imitated, ...)

I knew I had missed some in reading quickly through the NP document. If
there are many, this would require some thought.

> -- default values

I know CLDR has this concept for the 'u' extension. It may just amount
to defaulting true/false values to true. If this is needed on a
per-dimension basis, there could be a "default" attribute on one value
in the code list. But interpreting this would require tag consumers to
have access to the code list, which is not usually desirable for
syntactic analysis.

> -- requirements for a registry, and its feasability, and finally
> implementation

Oh, we'll go there.

> One question I have: need the values in the key-value-pairs be unique
> over different languages?  As an example, can two dialects of
> different languages share the same string xyz as a designation used as
> value in ...v-...-sp-xyz?

The existing, analogous rule for variants is a point of controversy
here. BCP 47 says that variants should not be defined to have different
meanings depending on the language. So we have variant subtags like
'pinyin' that can apply to both Chinese and Tibetan, because it's nearly
the same romanization scheme for both languages; and we have variants
that can apply to almost any language, such as the one that means
"written in IPA." But we can't have something like 'western' because the
concept of "language X as used in the western part of country Y" has
different meanings for different values of X and Y. Not everyone here
agrees where to draw the line.

I assume that many of the 21636 concepts have the same meaning
regardless of the language, but this needs more study.

> Alternatively/additionally, we could use the glottocodes for the major
> dialects already included in the Glottolog (they are a string of 4
> letters and 4 digits), and come to an agreement with the Glottolog
> folks to extend their list of dialects.
> (It would be excellent to be in touch with them anyways, also for
> cases where ISO is less accurate than the Glottologue.)

When I saw this originally, I wasn't in a position to respond, but it
alarmed me. Establishing Glottolog as a competing standard to ISO 639
for encoding language information in BCP 47, even in different subtag
types, can only lead to confusion and duplicate representations, exactly
what you expressed a desire to avoid earlier.

Then I saw the following, and became even more alarmed:

> However, differently from when BCP 47 was created, Glottolog now is an
> impressive and very complete and accurate work.  Many scholars combine
> it, or even prefer it over ISO 639, it is used in WIkipedia, etc., so
> perhaps it is the time for rethinking the relationship between BCP and
> Glottolog, especially for the cases of languages that are missing in
> the Ethnologue / ISO 639.

So, just to be 100% clear: the core standards that are used as the
source for subtag structure in BCP 47, and for values of certain subtag
types in the Language Subtag Registry, are fixed. They cannot be swapped
in and out. Any proposal for "rethinking the relationship between BCP
[47] and Glottolog," to the extent that means switching from ISO 639
code elements to Glottolog code elements, is completely off the table.

(Incidentally, Wikipedia uses an extension of ISO 639 for language
coding, and only displays Glottolog code elements in language boxes. The
Portuguese Wikipedia is at pt.wikipedia.org, not
port1283.wikipedia.org.)

The core standards were chosen deliberately, for a variety of reasons
such as existing practice, suitability, and stability promises. We know
there are other encoding systems for languages, some of which may even
have isolated or perceived advantages over ISO 639. But this is what we
have chosen. There is no significant likelihood this will be overturned.

--
Doug Ewell | Thornton, CO, US | ewellic.org

---

From: Sebastian Drude <[email-masked]>
Date: Sat, 28 Nov 2020 16:43:48 -0300

Thanks again, Dear Doug.

My comments below.

Sebastian

-- 

Museu P.E. Goeldi, CCH, Linguistica ▪ Av. Perimetral, 1901
Terra Firme, CEP: 66077-530 ▪ Belém do Pará – PA ▪ Brazil
[email-masked] ▪ +55 (91) 3217 6024 ▪ +55 (91) 983733319
Priv: Tv. Juvenal Cordeiro, 184, Apt 104 ▪ 66070-300 Belém

> imagine a scenario in which encoding this dimension would ever be
Indeed.

> be eligible to join the discussion if they choose to.

Yes, that would be great.  I am not connected any more to any service 
where I could do so, whom could we ask?  The LinguistList?

> those that might conflict with other subtags would probably be very

I understand.  We would then, as you write below, just trust the wit and 
common sense of thouse creating tags.

> combinations but not making them invalid.

That is fine.  One could need it for at least two cases:

- more general and more specific indications (dialects, sociolects...)

- idiolects belonging to intersections of varieties (e.g., showing all 
the defining criteria for more than one dialect, in border areas)

For the first case, several ever more specific indications would only be 
admissible if each is relevant for some appliaction, and one cannot 
presume that the language-tag-consumer knows of logical implications 
(South Tirol German implies Bavarian which implies High German, for 
instance).

> there are many, this would require some thought.
At this point, I do not foresee this to be heavily used, but what do I 
know about possible needs in 20 years?  I would need to compile a list 
of such "adjectives", perhaps there is one more case besides the three 
we have identified here.  Again (see below, next comment), there are 
default values, and only exceptions would need coding.

> have access to the code list, which is not usually desirable for

I understand.  A common objection to the 8-dimension-approach is that it 
is very cumbersome to indicate values for each dimension for each 
resource, and I could not agree more.

One obvious solution is to imply default values and code only 'deviant' 
varieties.  The default values could be: (1) the respective "standard 
varieties" in the case of the space dimension, (2) "current period" for 
time, (3) "middle-class" or "socially neutral" for social group, (5) 
"neutral" for register, (7) "full" proficiency, and (8) "regular 
functioning" (no 'anomaly') for the commun. funct. dimension.  (We leave 
(6), the person dimension, out, as discussed.)
For (3) the medium dimension, the default value would perhaps depend on 
(a) the media carrier ("oral" for an audio recording, "written" for a 
text document or PDF, although one may want to indicate the specific 
writing system etc. -- that is currently also the case), and (b) the 
language (sign languages would have the signed modality as default for 
videos, other languages the multimodal modality; Latin would have the 
written modality as default independent of the media carrier, and so forth).

> regardless of the language, but this needs more study.
Indeed, except for dialects, I would assume that most values are 
universal, but some languages may have additional values 
(south-east-asian languages are famous for their many registers based on 
the social status of speaker and addressee, for instance, and certain 
sociolects will only exist for, say, indian casts but not elsewhere).  I 
do not see any problem with that, each value is conceived as universal 
and each language picks the values it needs.

Similarly, there can be different time extensions for periods and epochs 
(time dimension) -- middle English covers very different centuries from 
middle Persian.  But it would still be fine to use these labels for each 
language; the concrete time span is less crucial than the respective 
variety being named by, e.g. "middle".

> what you expressed a desire to avoid earlier.
I agree.  I was not at all proposing to complement, let alone replace, 
the ISO 639 identifiers in the main language subtags or any other 
crucial area of BCP 47 by glottocodes.

> in and out.

Absolutely, I am aware of that and agree with it.  I am sorry if I 
expressed myself in a way that was easy to misunderstand.  I would not 
be so heavily involved in ISO TC37/SC2 if I wanted to replace ISO 639 by 
Glottocodes.

> Any proposal for "rethinking the relationship between BCP
Sure.
> Portuguese Wikipedia is at pt.wikipedia.org, not

Exactly.

Still, that is what I mean: while not changing the current setting for 
the URLs, Wikipedia at some time recognized that the Glottocodes exist, 
and are important enough to be now a standard information given for each 
language.

Similarly, when I said "perhaps it is the time for rethinking the 
relationship between BCP and Glottolog, especially for the cases of 
languages that are missing in the Ethnologue / ISO 639.", I meant that 
allowing for people to use Glottocodes in the "x" private use area 
whithout any official status is too little/weak, it does not do them 
justice, so to say.
Instead, one could think of introducing an extension, say, with subtag 
"g" (if that is not already assigned -- where can I find the list of 
existing extensions?), to be followed by a glottocode, in order to allow 
for a user to indicate some variety that is well defined by its 
Glottolog entry.  Especially in cases where ISO does not have an 
appropriate language code (yet), a user could use, for instance, 
*mis-g-adha1238*, where the g-extension subtag *adha1238* is the 
Glottocode for the extinct Adhari language, which is not (yet) in ISO 
639.  Perhaps even a better solution can be imagined than using the 
special purpose ISO 639 identifier "mis" -- /miscellaneous language (no 
ISO 639 code element is assigned for this language or included in the 
part of ISO 639 used by a given application)/.

> have chosen. There is no significant likelihood this will be overturned.

Absolutely, and I would not have it otherwise, unless ISO becomes 
unfeasible for some reason, or in practice everybody is abandoning it 
for some good reason -- but I do not see that coming.

> _______________________________________________
> Ietf-languages mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-languages

---

From: Mark Davis ☕️ <[email-masked]>
Date: Sat, 28 Nov 2020 18:32:20 -0800

> BCP 47 says that variants should not be defined to have different
meanings depending on the language.

1. I think the actual phrasing implies something slightly different:

   Requests to add a 'Prefix' field to a variant subtag that imply a
   different semantic meaning SHOULD be rejected.  For example, a
   request to add the prefix "de" to the subtag '1994' so that the tag
   "de-1994" represented some German dialect or orthographic form would
   be rejected.  The '1994' subtag represents a particular Slovenian
   orthography, and the additional registration would change or blur the
   semantic meaning assigned to the subtag.
The main purpose of that clause is that if variant1 modifies prefix1 in a
particular way, then variant1 shouldn't modify prefix2 in a very different
way. It doesn't apply when there isn't a prefix, such as with fonipa, or
anything else that represents a well-defined modification of previous
subtags. It also doesn't apply to defined extensions; it is up to the spec
for that extension to determine the right mechanisms.

Certainly it doesn't not require gratuitously different spellings for a
general-purpose subtag, based on the locales it is applied to. That is,
there is no necessity to have subtags for a 'register' dimension of
'formal' that add extra letters or numbers just to be different.

fr-v-re-formalfr
de-v-fr-formde
en-v-fr-formen
en-v-fr-formengb
...

However, it would be very ill-advised to define a subtag 'formal' for a
*register* dimension, and use that same subtag for a different dimension
like *space*.

Moreover, for the highest degree of utility, the functional application of
each value should be applicable across languages. For example, if 'formal'
for one language is reserved for the emperor, while 'formal' in another
language is for any people that you don't know well, then it becomes very
difficult for developers to apply the value consistently.

2. There was an issue around multiple subtag values. Eg
fr-v-xx-abcdef-ghijk. If you allow them, you need to specify precisely
which combinations are allowed, and what the semantics of any combinations
means.

In CLDR, for example, we allow a few different combinations, currently. The
examples in the following are from the machine-readable specification for
CLDR extension key-value pairs.

*incremental*: combinations where success elements 'narrow' the scope of
the previous element, eg:

            <type name=*"islamic"* description=*"Islamic calendar"*/>

            <type name=*"islamic-umalqura"* description=*"Islamic calendar,
Umm al-Qura"* since=*"24"*/>

            <type name=*"islamic-tbla"* description=*"Islamic calendar,
tabular (intercalary years [2,5,7,10,13,16,18,21,24,26,29] - astronomical
epoch)"* since=*"24"*/>

            <type name=*"islamic-civil"* description=*"Islamic calendar,
tabular (intercalary years [2,5,7,10,13,16,18,21,24,26,29] - civil epoch)"*
since=*"24"*/>

            <type name=*"islamic-rgsa"* description=*"Islamic calendar,
Saudi Arabia sighting"* since=*"24"*/>

*multiple*: any ordered list, such as collation reordering codes, eg:

            <type name=*"space"* description=*"Whitespace reordering code,
see LDML Part 5: Collation"* since=*"21"*/>

            <type name=*"punct"* description=*"Punctuation reordering code,
see LDML Part 5: Collation"* since=*"21"*/>

            <type name=*"symbol"* description=*"Symbol reordering code
(other than currency), see LDML Part 5: Collation"* since=*"21"*/>

            <type name=*"currency"* description=*"Currency reordering code,
see LDML Part 5: Collation"* since=*"21"*/>

            <type name=*"digit"* description=*"Digit (number) reordering
code, see LDML Part 5: Collation"* since=*"21"*/>

            <type name=*"REORDER_CODE"* description=*"Other collation
reorder code — for script, see LDML Part 5: Collation"* since=*"21"*/>

*any*:

            <type name=*"PRIVATE_USE"* description=*"Private use transform
identifier. All subfields consistent with rfc6497 (that is, subtags of 3-8
alphanum characters) are valid, and do not require registration."* since=
*"21.0.2"*/>

3. I would advise proceeding slowly and carefully; as remarked earlier you
need to guarantee that there will not be backwards compatibility problems
or people won't touch the extension with a 10 foot pole. So it would be
best to make sure that each dimension's core values are well defined, with
lots of examples from multiple languages, before adding each of them.

Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 28 Nov 2020 20:53:44 -0700

Sebastian Drude wrote:

>> Maybe a custom mailing list could be formed. Everyone here would need
>> to be eligible to join the discussion if they choose to.
>

That shouldn’t be necessary. Many folks have access to Mailman and can spin up an ad-hoc mailing list. I know Michael Everson has done so on several occasions, including one for a Unicode encoding proposal I’ve been involved with that really couldn’t have succeeded without it. But Michael doesn’t run a mailing-list business, and I hesitate to pester him for this.

>>> -- can more than one value in the SAME dimension be indicated? (I
>>> would argue yes, if that is syntactically okay)
>

I need to read Mark’s response about this, and the issues surrounding it, a bit more thoroughly. I was under the impression that a given extension could quite happily allow you to string together “fr-v-xx-abcdef-ghijk”, or not, depending on how the spec is written. Mark seems to imply some inherent limitations on this in BCP 47. Fortunately we have plenty of time to figure out what can and cannot, or should not, be done here.

> For the first case, several ever more specific indications would only
> be admissible if each is relevant for some appliaction, and one cannot

Nothing about BCP 47 tagging is ever intended to presume that the user (at either end) knows anything about language family hierarchies, about which there is much disagreement anyway.

I suspect I am not reading this comment carefully enough, and taking it out of context.

>>> -- the 'certainty' and similar "adjectives" (yes, that is how I
>>> would see them; -- e.g. primary vs. secondary modality, genuine vs.
>>> imitated, ...)
>>
>> I knew I had missed some in reading quickly through the NP document.
>> If there are many, this would require some thought.
>

No, I literally meant I did not even know there were three at present. I thought only "certainty" fell into this category. That's why I need to read through that part of the NP document again.

But yes, the mechanism does need to allow for future modifiers of this type, just like the 'u' extension allows for additional keys, as well as additional values within each key.

>>> -- default values
>
> I understand.  A common objection to the 8-dimension-approach is that
> it is very cumbersome to indicate values for each dimension for each
> One obvious solution is to imply default values and code only
> 'deviant' varieties.  The default values could be: (1) the respective
> "standard varieties" in the case of the space dimension, (2) "current
> period" for time, (3) "middle-class" or "socially neutral" for social
> group, (5) "neutral" for register, (7) "full" proficiency, and (8)
> "regular functioning" (no 'anomaly') for the commun. funct. dimension.
> (We leave (6), the person dimension, out, as discussed.)
>
> For (3) the medium dimension, the default value would perhaps depend
> on (a) the media carrier ("oral" for an audio recording, "written" for
> a text document or PDF, although one may want to indicate the specific

I have to confess that it never occurred to me that users of this extension, or indeed users of the TR in any form, would always be expected to provide values for all eight (or seven) dimensions, and that a defaulting mechanism would be necessary to permit eliding some of them.

> crucial area of BCP 47 by glottocodes.

Thank you very much. I am glad in this case that I simply misread this and panicked for no reason.

> the URLs, Wikipedia at some time recognized that the Glottocodes
> exist, and are important enough to be now a standard information given
> for each language.

Well, I mean, we also know that Glottocodes exist. For that matter, we also know the Linguasphere coding system exists. Whether we consider these to be “standards,” whether on a par with ISO 639 or not, can be debated.

> existing extensions?),

https://www.iana.org/assignments/language-tag-extensions-registry

> to be followed by a glottocode, in order to allow for a user to
> indicate some variety that is well defined by its Glottolog entry.
> Especially in cases where ISO does not have an appropriate language
> code (yet), a user could use, for instance, mis-g-adha1238, where the
> g-extension subtag adha1238 is the Glottocode for the extinct Adhari
> language, which is not (yet) in ISO 639.  Perhaps even a better
> solution can be imagined than using the special purpose ISO 639
> identifier "mis" -- miscellaneous language (no ISO 639 code element is
> assigned for this language or included in the part of ISO 639 used by
> a given application).

ISO 639-3/RA does have a well-established process to add (and modify, and delete) code elements to reflect linguistic realities. The reviewers may simply have not seen the documentation for Adhari (Old Azeri) that they were looking for. Submitting a proposal to add a language like this to 639-3, and thus to the Registry, might be more productive than building a mechanism to swap in another coding system, ISO 2022-like. This is especially true if the Adhari example isn’t intended to represent hundreds or thousands of language missing from 639-3.

--
Doug Ewell, CC, ALB | Thornton, CO, US | ewellic.org

---

From: Mark Davis ☕️ <[email-masked]>
Date: Sun, 29 Nov 2020 07:59:14 -0800

> I was under the impression that a given extension could quite happily
allow you to string together “fr-v-xx-abcdef-ghijk”, or not, depending on
how the spec is written. Mark seems to imply some inherent limitations on
this in BCP 47. Fortunately we have plenty of time to figure out what can
and cannot, or should not, be done here.

Your impression is right, sorry that I was unclear. Bcp47 does not impose
structure; they were just examples of how we ended up adding structure so
as to provide for clear semantics when parsing multiple values.

BTW, for a new email group there are far easier and more functional
mechanisms available in this millennium ...

On Sat, Nov 28, 2020, 19:53 Doug Ewell <[email-masked]> wrote:

> That shouldn’t be necessary. Many folks have access to Mailman and can
> spin up an ad-hoc mailing list. I know Michael Everson has done so on
> several occasions, including one for a Unicode encoding proposal I’ve been
> involved with that really couldn’t have succeeded without it. But Michael
> doesn’t run a mailing-list business, and I hesitate to pester him for this.
>
> I need to read Mark’s response about this, and the issues surrounding it,
> a bit more thoroughly. I was under the impression that a given extension
> could quite happily allow you to string together “fr-v-xx-abcdef-ghijk”, or
> not, depending on how the spec is written. Mark seems to imply some
> inherent limitations on this in BCP 47. Fortunately we have plenty of time
> to figure out what can and cannot, or should not, be done here.
>
> Nothing about BCP 47 tagging is ever intended to presume that the user (at
> either end) knows anything about language family hierarchies, about which
> there is much disagreement anyway.
>
> No, I literally meant I did not even know there were three at present. I
> thought only "certainty" fell into this category. That's why I need to read
> through that part of the NP document again.
>
> But yes, the mechanism does need to allow for future modifiers of this
> type, just like the 'u' extension allows for additional keys, as well as
> additional values within each key.
>
> >>> -- default values
> >
> I have to confess that it never occurred to me that users of this
> extension, or indeed users of the TR in any form, would always be expected
> to provide values for all eight (or seven) dimensions, and that a
> defaulting mechanism would be necessary to permit eliding some of them.
>
> Well, I mean, we also know that Glottocodes exist. For that matter, we
> also know the Linguasphere coding system exists. Whether we consider these
> to be “standards,” whether on a par with ISO 639 or not, can be debated.
>
> ISO 639-3/RA does have a well-established process to add (and modify, and
> delete) code elements to reflect linguistic realities. The reviewers may
> simply have not seen the documentation for Adhari (Old Azeri) that they
> were looking for. Submitting a proposal to add a language like this to
> 639-3, and thus to the Registry, might be more productive than building a
> mechanism to swap in another coding system, ISO 2022-like. This is
> especially true if the Adhari example isn’t intended to represent hundreds
> or thousands of language missing from 639-3.
>
> --

---

From: Sebastian Drude <[email-masked]>
Date: Sun, 29 Nov 2020 18:16:58 -0300

Thanks again, dear Doug,

and again my comments below.

Hava a nice (rest of) 1st advent sunday,

Sebastian

-- 

On 29/11/2020 00:53, Doug Ewell wrote:
> several occasions, including one for a Unicode encoding proposal I’ve 
> been involved with that really couldn’t have succeeded without it. But 
> Michael doesn’t run a mailing-list business, and I hesitate to pester 
> him for this. 
Right. If things where normal, I could ask my department, but people are 
working from home etc....

> I need to read Mark’s response about this, and the issues surrounding it, a bit more thoroughly. I was under the impression that a given extension could quite happily allow you to string together “…
Yes, let's go on this with care.  I understood that there are 
possibilities of several values for the same key, which would mostly 
depend on how we set that up. I am confident we figure that out.

> Nothing about BCP 47 tagging is ever intended to presume that the user (at either end) knows anything about language family hierarchies, about which there is much disagreement anyway.

This is what I would expect.

> I suspect I am not reading this comment carefully enough, and taking it out of context.
No need to rush.

> But yes, the mechanism does need to allow for future modifiers of this type, just like the 'u' extension allows for additional keys, as well as additional values within each key.

Looking at the latest version of the document, there are indeed a number 
of such 'modifying' statements.

Certainty (values: documented, confirmed, certain, inferred, assumed, 
possible, unclear, ...).

Dialects being influenced by another language (Example: Urdu as spoken 
in London, influenced by: English).  Dialectal varieties that emerge if 
the father speaks one and the mother another dialect, or the parents 
speak one dialect, but the person in question grows up speaking another 
dialect with peers -- sometimes not dialectal bi-'lingualism' emerges, 
but mixed forms.

Primary modality and secondary modality.  This may actually be relevant 
in many situations.

Proficiency: framework of rating, or theory of language acquisicion applied

Communicative functioning: degree of (informally said:) anomaly

I guess that this can all be easily solved by adding further two-letter 
keys to provide these indications in their values as if they were 
additional dimensions.  The rules then would have to state that they are 
only allowed if the major dimension is present.

> I have to confess that it never occurred to me that users of this extension, or indeed users of the TR in any form, would always be expected to provide values for all eight (or seven) dimensions, a…

Indeed, that was never the intention, but as soon as people understand 
that in principle each and every speech event or resource inevitably has 
some value on each of the eight dimensions, people could feel obliged, 
for the sake of coherence or completeness or whatever, to indicate all 8 
of them every time.  Absolutely not feasible, and not what anybody would 
want, let alone require.  Still, it came up as an argument for rejecting 
the whole framework.

Of course, if one indicates only one value for one dimension (which is 
perfectly fine), or no variation value at all, as will be the case of 
the vast majority of all language tags, the status of the (other) 
dimensions of linguistic variation can usually be inferred by assuming 
something like default values, and that could be made explicit 
somewhere.  That is what I meant.  That does not necessarily imply that 
default values are a formal part of the extension.

> Thank you very much. I am glad in this case that I simply misread this and panicked for no reason.
No, I expressed myself badly here I guess.  Sorry for that.

> Well, I mean, we also know that Glottocodes exist. For that matter, we also know the Linguasphere coding system exists. Whether we consider these to be “standards,” whether on a par with ISO 639 or…

Sure.  This depends a lot of the community, of course.  In diversity 
linguistics, people accept Glottocodes as a valid alternative, if not 
superior, to ISO 639-3.  And I have to agree that the scholar soundness 
is impressive, certainly very different from the idiosyncratic 
Linguasphere (despite "a resourceful conception, many valid insights and 
much knowledge", as I wrote elsewhere <http://hdl.handle.net/10125/24814>).

> https://www.iana.org/assignments/language-tag-extensions-registry
I see, only two, "t" and "u", are registered.  Plenty of room to 
register "g" and "v".

> The reviewers may simply have not seen the documentation for Adhari (Old Azeri) that they were looking for.

This was just a fictious case, taking one of the first languages with a 
Glottocode that apparently does not have any ISO 639 entry.  There are 
no real "reviewers" (reviewing what?).

> Submitting a proposal to add a language like this to 639-3, and thus to the Registry, might be more productive than building a mechanism to swap in another coding system, ISO 2022-like. This is especially true if the Adhari example isn’t intended to represent hundreds or thousands of language missing from 639-3.

There are not too many languages in the Glottolog which are not in the 
Ethnologue, certainly not thousands, perhaps a couple of hundreds.  
Still, certain communities may be content to just take the existing 
Glottocode instead of taking up the burden of starting a process to add 
the language to ISO 639.  (There may be little overlap with the 
communities using BCP 47 language tags, but that may change.)

In addition to languages not (yet) in ISO, there are certainly several 
thousands varieties (mainly, but not only dialects) of languages which 
already do have a Glottocode, which *may* be interesting to allow as 
subtags or at least an extension.  I don't understand this deeply enough 
at this point to advocate for or against anything, I mainly want to draw 
your attention to this.

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 30 Nov 2020 08:46:41 +0900

On 30/11/2020 06:16, Sebastian Drude wrote:
(sorry, not sure that was actually Sebastian, or Doug, or somebody else)

> Certainty (values: documented, confirmed, certain, inferred, assumed, 

If these values indicate some kind of difference in grammar (e.g. 
subjunctive) or intonation (for audio) that is indicative of these 
distinctions, this might be fine as part of language tagging.

If however the intent here is to indicate something about the truth 
value of the data (text/audio/...) in question, similar to fact checks 
(e.g. Washington Post Pinocchios,...), then I think that should very 
clearly be kept out of language tagging.

Regards,   Martin.