Subject: [Ltru] Solving the UTF-8 problem; was Language Tag Modification 1694acad;

From: "CE Whitehead" <[email-masked]>
Date: Mon, 02 Jul 2007 11:03:27 -0400

Hi, I'm confused as to whether or not persons with Thai Windows only (do 
they exist?? I thought so) can see all Latin-1 characters properly.

Also, I think adding a comment to transliterations into ascii that overlap 
would not be too much trouble.

(But this is needed only if there are persons who have operating systems 
anywhere on earth who cannot see these characters).

Finally, I do not think it is too much work to provide a ferw 
transliterations of non-ascii characters, given all the other stuff that 
goes into language subtag entries.

And I do not mind whether it is the officialregistry in ascii and the 
unofficial in utf-8 or the other way around.

Thanks.

(I've put all this again below for people who like the context.)

--C. E. Whitehead

Doug Ewell dewell at roadrunner.com
Mon Jul 2 00:58:48 CEST 2007

>What are we going to do when the ISO 639-3 code list is finalized and we 
>have to deal with adding the following pairs of languages, whose names 
>differ only by diacritical marks?

>aru  Arua
>arx  Aruá

>bfa  Bari
>mot  Barí

>kgm  Karipúna
>kuq  Karipuná

>sbe  Saliba
>slc  Sáliba

>wbf  Wara
>tci  Wára

>Are we going to include an ASCII version of every name that contains an 
>accented letter?  There are several hundred in ISO 639-3.

I have not seen ISO 639-3; but as it so much work  just to put in the 
comments, description, etc., that it should be trivial to add in the ascii 
name!
As for the languages that differ only by the diacritical mark you might need 
a comment about this
somewhere; I think it can be handled!  (for non-latin charcters and for 
Latin if needed, eg. if there are persons in the world who cannot see these)

>Section 3.1 mentions transcription of non-Latin Description fields into the 
>Latin script.  It does not talk about providing a pure-ASCII equivalent for 
>every non-ASCII French- or Spanish-language string, and I don't believe 
>that was the WG's intention.  Transcriptions are useful when the content is 
>in Arabic or Cyrillic or Han, to make the material available to 
>Latin-script-only readers.  Providing "transcriptions" like "4#xE8;eme 
>(4eme)" merely announces to the world that we can't solve our own technical 
>character-encoding problems without resorting to unwieldy kludges.

Are there people who only have say Thai windows, who would appreciate the 
transcriptions?

That's what I was thinking; sorry if I'm wrong.

Stephane Bortzmeyer bortzmeyer at nic.fr
Mon Jul 2 15:55:33 CEST 2007 wrote:

>On Sun, Jul 01, 2007 at 03:58:48PM -0700,
>Doug Ewell <dewell at roadrunner.com> wrote a message of 161 lines which 
>said:

> > Another possibility is to have IANA post an official version of the
> > Registry in one encoding, such as UTF-8, and additional, unofficial
> > versions in other encodings, such as Latin-1 or hex NCRs.
>Why not? Currently, we do exactly the opposite: IANA publishes the
>official registry in hex NCR
>(http://www.iana.org/assignments/language-subtag-registry) and
>langtag.net publishes an unofficial version in UTF-8
>(http://www.langtag.net/registries/language-subtag-registry.utf8).

Fine with me!

--C. E. Whitehead
[email-masked]

_________________________________________________________________
Picture this – share your photos and you could win big!  
http://www.GETREALPhotoContest.com?ocid=TXT_TAGHM&loc=us

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Mon, 2 Jul 2007 22:43:39 -0700

> they exist?? I thought so) can see all Latin-1 characters properly.

Not if they are unable to run either IE 4.0 or SC UniPad, either of which is 
supposed to work on any Windows 95 Rev B or later system.  If you tell me 
what OS and browser you have at your disposal, I will tell you whether 
should be able to view 
http://langtag.net/registries/language-subtag-registry.utf8 correctly, 
either online or offline.  I could also provide you with a Windows 
95-compatible program that would provide a best-fit display of non-Latin-1 
characters, but I suppose the library won't let you run it.

Remember also that the Registry exists to provide names and descriptions 
that are sufficient for tag producers and consumers to make sensible tagging 
decisions.  Does anyone disagree with that?  It is not just for casual 
browsing.

> would not be too much trouble.

Nobody has ever said it would be difficult to add layers of comments.  It 
would be troublesome, at least for me, to introduce unnecessary confusion by 
folding names like Arua and Aruá together and then adding comments to try to 
explain what we meant, instead of just spelling the names right.

> anywhere on earth who cannot see these characters).

Anywhere on earth?  This is your operational criterion, not mine.

> goes into language subtag entries.

Nobody has ever said it would be "too much work" to add transliterations. 
See above.

> I have not seen ISO 639-3;

Then trust the examples I gave.

> but as it so much work  just to put in the comments, description, etc., 
> that it should be trivial to add in the ascii name!

See above.  Quick question: How many people feel my efforts on RFC 4646 have 
been guided by what is easiest, rather than what is right?

>> (http://www.langtag.net/registries/language-subtag-registry.utf8).

Can we pursue this official version/unofficial version strategy, as an 
alternative to loading up the Registry with excise?

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 3 Jul 2007 09:11:46 +0200

On Mon, Jul 02, 2007 at 10:43:39PM -0700,
 Doug Ewell <[email-masked]> wrote 
 a message of 74 lines which said:

> Can we pursue this official version/unofficial version strategy, as
> an alternative to loading up the Registry with excise?

It seems a good strategy, very "better to light a candle than to
complain about obscurity".

I would say -1 to any decision of having *several* official versions
(because it would be more work for IANA and more opportunity to have a
discrepancy, something which is not serious with unofficial versions,
but could be dramatic between official versions).

But allow me a little troll: if we choose UTF-8, what about
normalization?

1) Do not mention it (this would mean that IANA would be free to
suddenly canonicalize the registry, thus making it different in a
byte-to-byte comparison)

2) Mandate NFC or NFD (which means an automatic registry checker would
have to check it)

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 3 Jul 2007 07:00:15 -0700

> 2) Mandate NFC or NFD (which means an automatic registry checker would 

There's actually nothing new here, since the Registry is already using 
Unicode with hex NCRs as the encoding scheme, and we would just be 
changing it to Unicode with UTF-8 as the encoding scheme.

However, it wouldn't hurt to specify NFC somewhere in the draft.  This 
is what we are already using and what the IETF and W3C seem to prefer. 
Descriptions and comments are supposed to be non-normative, so I'm not 
sure any user's tools would *have* to do any checking or correcting, 
though of course ours should.

---

From: Peter Constable <[email-masked]>
Date: Tue, 3 Jul 2007 08:07:18 -0700

+1 to specifying NFC (whether we use UTF-8 or NCRs).

Peter

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Doug Ewell
Sent: Tuesday, July 03, 2007 7:00 AM
To: [email-masked]; LTRU Working Group
Subject: Re: Solving the UTF-8 problem; was Language Tag Modification 1694acad;

_______________________________________________
Ietf-languages mailing list
[email-masked]
http://www.alvestrand.no/mailman/listinfo/ietf-languages

---

From: Peter Constable <[email-masked]>
Date: Tue, 3 Jul 2007 08:13:56 -0700

From: [email-masked] [mailto:[email-masked]] On Behalf Of CE Whitehead

Users of Windows 3.1 Thai would not be able to see Latin-1 characters. But we should not be designing this registry to accommodate the limitations of Windows 3.1, or any other obsolete software.

Peter