Subject: [Ltru] Re: Language transformation

From: "Doug Ewell" <[email-masked]>
Date: Fri, 1 Apr 2005 21:58:30 -0800

John Cowan <jcowan at reutershealth dot com> wrote:

>> On the other hand, if these are implemented as variant subtags,
>> there is no need for a separate registry to provide further
>> explanation or description of the variants.  They are listed in the
>> registry, with a recommended prefix, and that is that.
>
> This overstates the case.  In order to make practical use of such
> orthographies, they must be defined somewhere, and if the users of
> trans*tions find it useful to have a registry of these things, then
> LTRU has nothing to say about it.
>
> After all, if you want to know what "nn" codes for, the registry will
> tell you "Nynorsk", but if you want to know what Nynorsk is, or how to
> recognize it, or how to spell-check it, you must look elsewhere.  At
> the moment there is no single "elsewhere" to look: this may be
> accounted good or bad.

I don't disagree that it would be helpful to have a reference to what
'wadegile' or 'mccnrsch' means.  I would think the registration forms
might be a good place to start.

What concerns me is the idea of a Transliteration Board that maintains
its own registry of transliterations, and tells the Language Subtag
Reviewer and ietf-languages which variant subtags to register, based on
what they have already approved for their registry.

If Tomas wants a separate organization to be responsible for vetting
these, he should pursue the extension-RFC path, which does exactly that,
but requires the use of a singleton such as -t-.  If he wants to be able
to write "zh-wadegile" without the -t-, then he is talking about
variants, and those will be proposed and debated and approved (or
rejected) using the same process that will apply to all other variants.

-Doug Ewell
 Fullerton, California
 http://users.adelphia.net/~dewell/

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 1 Apr 2005 23:13:07 -0800

Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:

>> Last Call for the registry draft is scheduled for May
>
> I really don't care about timetables decreed by somebody
> else, they are free to fix it when necessary.

We're trying to hammer out the details of a stable mechanism.  May vs.
June might not make that much difference, but we can't drag this out
indefinitely.

>> Can you think of a use case for 200?
>
> As an optional replacement for old cs-CS / sk-CS / rom-CS
> tags.  We want backwards compatibility with BU / DD / YD,
> therefore we should also offer it for the old CS.

That's not the type of backward compatibility that BU and DD and YD
provide.  They are there precisely so that existing content with those
tags *does not* have to be retagged with MM or DE or YE.

> So what is the correct replacement for rom-CS if you're
> very sure that the CS is the old CS ?  Just rom ?  I've
> no idea which other languages might be relevant in the
> old CS.  Maybe hu-CS, what is this, hu-SK, or is hu-CZ
> also possible ?
>
> If both are possible somebody trying to fix it could need
> a 200 created by a better "split" rule.  Or he needs an
> info to use hu-SK if hu-CS was the old CS.  It's messy,
> minority languages in the old CS could be also minority
> languages in the new CS (rom, hu, yi, de, what else ?).

I won't fight hard on this 200 issue, but we need to come to some sort
of agreement that not every problem related to the splitting and merging
and shifting of countries is going to be solved within language tags.
Originally you didn't even want pre-1995 region subtags to be allowed;
now the correct "replacement" for old CS (withdrawn in 1993) is
important?

-Doug Ewell
 Fullerton, California

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 1 Apr 2005 23:23:01 -0800

Addison Phillips <addison dot phillips at quest dot com> wrote:

>> Depending on our editors this could already specify the
>> record-jar or similar format, so that you have a chance
>> to create a proper first I-D in this format.
>
> It doesn't because that debate seems to have fallen off without a
> genuine consensus being reached. I *think* the consensus is record-
> jar. And maybe also XML.

I remember some pretty firm objections to XML.

> At present I intend that when I next sit down to update the editor's
> copy (that usually being my job in our editing duopoly) I'll
> incorporate the changes necessary to make record-jar the format with
> no mention of XML. Any objections?

No objection at all.  I'm ready with record-jar any time you folks are.

I would appreciate some guidance on the exact flavor of record-jar that
people want.  There are a few choices to be made:

- max line length (80, 72, etc.)
- character(s) used for comments ("%%" versus "#")
- line breaking convention (backslash-newline, newline-space, etc.)

> The additional rule you suggest would be, btw:
>
> 3. Create an M49-based registration for any 3166 code that was
> withdrawn after Date A and subsequently assigned a new value before
> Date B (in Cowan's terms).

I'm glad to see this (and the other two rules) expressed as general
rules, not special exceptions.

-Doug Ewell
 Fullerton, California

---

From: John Cowan <[email-masked]>
Date: Sat, 2 Apr 2005 03:35:13 -0500

> - max line length (80, 72, etc.)

No opinion on this one.

> - character(s) used for comments ("%%" versus "#")

In record-jar the %% is a separator, and anything following that on the
same line is not part of the data structure.  So IMHO comments should
be handled as a separate "comment:" field.

> - line breaking convention (backslash-newline, newline-space, etc.)

Record-jar is based on RFC (2)822, so newline-space(s) is the convention.

-- 
John Cowan                              [email-masked]
http://www.reutershealth.com            http://www.ccil.org/~cowan
Humpty Dump Dublin squeaks through his norse
                Humpty Dump Dublin hath a horrible vorse
But for all his kinks English / And his irismanx brogues
                Humpty Dump Dublin's grandada of all rogues.  --Cousin James

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 2 Apr 2005 01:58:59 -0800

John Cowan <jcowan at reutershealth dot com> wrote:

> In record-jar the %% is a separator, and anything following that on
> the same line is not part of the data structure.  So IMHO comments
> should be handled as a separate "comment:" field.

There are actually two types of comments.  There are the headers, like
this:

# IANA Language Subtag Registry
# This registry lists all valid subtags for language tags
# created under RFC 3066bis.

and then there are the comments that belong to a single record, and go
at the end like this:

# deprecated 1989

For the latter, I agree that these should be a true field.  For the
former, I've kept them more or less as they were in the bar-delimited
file, introduced by a "#" in column 1.  But the documentation for the
"Open-RJ" library says comments should begin with "%%", so I couldn't be
sure.

> Record-jar is based on RFC (2)822, so newline-space(s) is the

Again, I trusted the Open-RJ page and went with backslash-newline.  I'll
change this.

OK, so right now I have something like this:

# IANA Language Subtag Registry
# etc. etc.

Type:        date
Date:        2005-03-30

# language codes: ISO 639 and registered codes

# ISO 639-1 (alpha-2) codes

Type:        language
Subtag:      aa
Description: Afar
Date:        2004-07-06
%%
Type:        language
Subtag:      ab
Description: Abkhazian
Date:        2004-07-06
%%
etc. etc.

If the group agrees on this format, or have concrete suggestions for
improvement, then that's what I'll post Wednesday or thereabouts.  I'm
also waiting to see what the consensus is on 200.  So far, one person
would like to see it back.

-Doug Ewell
 Fullerton, California

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sat, 02 Apr 2005 16:23:12 +0200

>people want.  There are a few choices to be made:

I need to translate this to our system management language through a simple 
C program.

>- max line length (80, 72, etc.)

No special requirement.

>- character(s) used for comments ("%%" versus "#")

"%%" could help to transliterate it to "//" while "#" may happen to be used 
elsewhere.

>- line breaking convention (backslash-newline, newline-space, etc.)

No special need.

>rules, not special exceptions.

Our intent is to add the expiration date of a code to the concerned code 
element. If you have the information very good. We currently miss the 
3166-2 revisions. But it is likely that 3166-2 will not be muched used in 
APMD draft context.
jfc

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 02 Apr 2005 22:43:23 +0200

> I remember some pretty firm objections to XML.

And one reason pro XML (clear support for Unicode character
references in an otherwise US-ASCII text).  JFTR, I prefer
record-jar in the body of the I-D registry.

> - max line length (80, 72, etc.)

It's not "unlimited", therefore you can pick any reasonable
limit.  For an I-D 72 is perfect.

> - character(s) used for comments ("%%" versus "#")

%% is the terminator for an entry.  As far as I'm concerned an
empty line is also okay.  The %% idea is for formats, where
the begin of the next entry isn't obvious. and where empty
lines could be lost, that's not the case for the I-D registry.

Comments introduced by # are fine.  If you stick to %%, then
maybe % is better than # (?)

> - line breaking convention (backslash-newline, newline-space,

CrLfSp / CrLfSpSp / ... as in folded mail / news header fields.
CrLf plus one or more spaces (maybe more are better readable ?)

                      Bye, Frank

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 2 Apr 2005 13:05:46 -0800

JFC (Jefsey) Morfin <jefsey at jefsey dot com> wrote:

> I need to translate this to our system management language through a
> simple C program.
> ...
> "%%" could help to transliterate it to "//" while "#" may happen to be
> used elsewhere.

If you're converting this to another format, you may have to do special
processing on "%%" anyway.

Question for the group:  Do I need "%%" at the beginning and end of
*all* records?  Right now I omit it if the otherwise-adjacent records
are separated by at least one comment, as shown in my previous examples.
(Or perhaps that's a reason to use "%%" for comments instead of "#".)

> Our intent is to add the expiration date of a code to the concerned
> code element. If you have the information very good.

Code elements don't have an expiration date.  They're good forever, or
until the MA withdraws or deprecates them.  Subtags are good forever, or
until the language tagging mechanism is totally replaced.

> We currently miss the 3166-2 revisions. But it is likely that 3166-2
> will not be muched used in APMD draft context.

ISO 3166-2 code elements (for country subdivisions) are one to three
characters long, and wouldn't fit within the proposed language tag
syntax.

I don't know what you mean by "APMD."  A quick Web search led me to
information on power management for laptops, which I'm sure is not what
you meant.

-Doug Ewell
 Fullerton, California

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 02 Apr 2005 23:17:13 +0200

Doug Ewell wrote:

 [200]
> That's not the type of backward compatibility that BU and DD
> and YD provide.  They are there precisely so that existing
> content with those tags *does not* have to be retagged with
> MM or DE or YE.

Point.  Let's forget 200, it's a hopeless case, and the rules
in 3066bis enforce that it can never happen again.

> we need to come to some sort of agreement that not every
> problem related to the splitting and merging and shifting of
> countries is going to be solved within language tags.

Mark claims that this is only an "academic exercise".  Maybe he
thinks that CS, DE, VN, YE, and YU were all academic exercises.
So far we discussed three solutions:

1 - Fix it for past problems (before 1988), ignore 200, DE, YE,
    and YU.  That's now ready, and 200 can't happen again.

2 - Add UN numbers for regions when they are otherwise "lost".
    You don't like it, because it only works by updating old
    tags.  I also don't like it, but it's the only emergency
    exit in this "academic exercise" (about twice per decade)
    with something like the draft -00 rules.

3 - Use UN numbers for stable entries, otherwise allow and
    accept that all alpha-2 codes are instable by design.

I don't see a fourth solution,  Sooner or later "1" loses some
regions, that's an observation for the simple rules in draft
-00 and before, demonstrated by YE and YU.

Now if you and Mark would say that you understand the problem
and ignore it intentionally, then we can go with solution "1".

Is the problem clear ?  Or should I elaborate the BY + RU => RU
and US => US + XX "academic exercises" ?

                            Bye, Frank

---

From: John Cowan <[email-masked]>
Date: Sat, 2 Apr 2005 19:12:48 -0500

> (Or perhaps that's a reason to use "%%" for comments instead of "#".)

You need it between each record.  You do not need it at the beginning
or the end, but it doesn't hurt.  You should not omit it in any
circumstance.

To recap the history, record-jar format descends from cookie-jar format
(not Web cookies, but fortune cookies), which is just chunks of unformatted
text separated by "%%" lines.  Record-jar just adds to that the idea that
each chunk is made up of RFC (2)822 headers.

-- 
John Cowan  www.ccil.org/~cowan  [email-masked]  www.reutershealth.com
Monday we watch-a Firefly's house, but he no come out.  He wasn't home.
Tuesday we go to the ball game, but he fool us.  He no show up.  Wednesday he
go to the ball game, and we fool him.  We no show up.  Thursday was a
double-header.  Nobody show up.  Friday it rained all day.  There was no ball
game, so we stayed home and we listened to it on-a the radio.  --Chicolini

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 03 Apr 2005 01:39:52 +0200

I do not understand all this endless academic debate. ICANN (IANA) is a 
member of the ISO 3166 MA group. I suggest you try to locate who is the 
IANA person in charge and ask him what is the fix the IANA supports. Then 
you publish an RFC on the issue to help him/her go and impose to ISO. As 
Randy says standards reflect personal references: make sure your personal 
reference becomes the rule.

Let be realistic: this WG will probably not rule the world, and will not 
prevent changes (and their resulting many possible mistakes). Any rigid 
system will not scale. And will not adapt in continuity with past 
situations (prior 1988 or 1974). The only rule you can apply is "parameter 
in context" what means you must know the date of the system a document 
refers to.

In real life, ISO 3166-2 will be used. You cannot imagine that en-US is 
descriptive enough when en-PT can offer an accuracy down to a group of 56 
people.

jfc

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 03 Apr 2005 05:23:06 +0200

At 03:35 03/04/2005, Doug Ewell wrote:
> >> Code elements don't have an expiration date.  They're good forever,
> >> or until the MA withdraws or deprecates them.  Subtags are good
> >> forever, or until the language tagging mechanism is totally replaced.
> >
> > What makes their expiration date.
>
>Can you predict when those events will happen?  If not, does it make
>sense to reserve a field for it?

I do not plan to reserve a field but to add the year in the code element. 
But the approach we _consider_ is quite different and much more Internet like.

>Never mind; this is the structure for your draft, and you will design it
>in the way you think is best.

Yes. But the idea is not to design it differently from APMD for the 
XML/HTML/CLDR applications (t)hey know better what they want and did not 
document the CLDR specific needs. But to maintain the underlaying 
information available to every possible application format. Including the 
currently discussed format. Them being free to use it or not, the way they 
think the best.

> >> information on power management for laptops, which I'm sure is not
> >> what you meant.
> >
> > Addison Philips' and Mark Davis' Draft.
>
>Aha.
>
>Please write to me publicly, on the list, on issues related to the
>Working Group.

Sorry, privacy was not intended. I copy the list.
jfc

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 5 Apr 2005 20:00:40 -0700

Hi -

> From: "Frank Ellermann" <[email-masked]>
> To: <[email-masked]>
> Sent: Saturday, April 02, 2005 1:43 PM
> Subject: [Ltru] Re: Proposed changes to region subtags
...
...

Pardon the naive question:
Couldn't we simply use Unicode character references (as needed)
in a record-jar format?

Randy

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 07 Apr 2005 00:19:11 +0200

Randy Presuhn wrote:

> Couldn't we simply use Unicode character references (as
> needed) in a record-jar format?

That would be an ad-hoc format, an ABNF for the complete
record-jar format with this idea is of course possible.

Essentially we'd need &#xabcd; or &#xabcde; - let's ignore
&#x10bcdef;  Anything else not starting with &#x is ASCII.

But there are no Unicode points above 126 in the draft
registry at the moment, so why do we discuss this now ?

                        Bye, Frank