Subject: [I18ndir] Review of new characters for Unicode 12.0.0

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 00:49:43 +0000

There were some talks about doing a review of the new characters for 
Unicode 12.0.0, for 'due diligence'.

Here are the results such a review of new characters for Unicode 12.0.0, 
starting off http://www.unicode.org/charts/PDF/Unicode-12.0/.
I also used a small Ruby program that I wrote, attached. In order for it 
to work, you have to make sure you use the latest (and greatest :-) 
version of Ruby that supports Unicode 12.0.0.

Telugu
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-0C00.pdf
new U+0C77 TELUGU SIGN SIDDHAM -> disallowed, okay for sign used at the 
beginning of texts as an invocation

Lao
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-0E80.pdf
new Lao letters and a sign (virama) for Pali and Sanskrit
-> pvalid, which is okay for historical letters and marks

Vedic Extensions
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1CD0.pdf
new U+1CFA VEDIC SIGN DOUBLE ANUSVARA ANTARGOMUKHA -> pvalid, okay 
because this is graphically a base letter (used as a base for a 
combining nasal sign)

Miscellaneous Symbols and Arrows
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-2B00.pdf
two new symbols -> disallowed, okay

Supplemental Punctuation
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-2E00.pdf
new U+2E4F CORNISH VERSE DIVIDER -> disallowed, okay

Latin Extended-D
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-A720.pdf
4 casing pairs, 3 upper-case letters completing case pairs with existing 
lower-case letters. Upper-case are disallowed (-> okay), lower-case are 
pvalid. Among these 4, the following three may merit a closer look:
U+A7BB LATIN SMALL LETTER GLOTTAL A
U+A7BD LATIN SMALL LETTER GLOTTAL I
U+A7BF LATIN SMALL LETTER GLOTTAL U
They are graphically very similar (if not identical) to the following 
three from the Latin extensions for Vietnamese:
(see https://www.unicode.org/charts/PDF/U1E00.pdf)
U+1EA3 ả LATIN SMALL LETTER A WITH HOOK ABOVE
U+1EC9 ỉ LATIN SMALL LETTER I WITH HOOK ABOVE
U+1EE7 ủ LATIN SMALL LETTER U WITH HOOK ABOVE
I have sent a mail to Unicode experts to get more information on these.

Latin Extended-E
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-AB30.pdf
two new letters for Sinological transcription, digraphs with hooks
-> pvalid, okay

Elymaic
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-10FE0.pdf
new script Elymaic, 23 letters, thereof one ligature
-> pvalid, okay

Newa
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11400.pdf
U+1145F NEWA LETTER VEDIC ANUSVARA -> pvalid, okay

Takri
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11680.pdf
U+116B8 TAKRI LETTER ARCHAIC KHA -> pvalid, okay

Nandinagari
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-119A0.pdf
new script Nandinagari, all letters pvalid, one sign (U+119E3) 
disallowed, okay

Soyombo
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11A50.pdf
two alternate visarga signs in Soyombo
pvalid, okay [maybe these are targets for a variant set]

Tamil Supplement
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11FC0.pdf
Tamil supplement: signs for fractions, traditional units, etc.
disallowed, okay

Egyptian Hieroglyph Format Controls
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-13430.pdf
Egyptian Hieroglyph Format Controls for quadrats -> all disallowed, okay

Miao
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-16F00.pdf
Miao letters and marks for writing additional languages -> all pvalid, okay

Ideographic Symbols and Punctuation
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-16FE0.pdf
two additional symbols -> disallowed, okay

Tangut
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-17000.pdf
6 additional Tangut characters -> pvalid, okay

Small Kana Extension
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1B130.pdf
historic small Hiragana/Katakana extensions -> pvalid, okay
[Should be excluded from general use because they may be confused with 
the larger versions by everyday users, but this is only really a problem 
for the last character (U+1B167 KATAKANA LETTER SMALL N) because the 
others won't usually appear in names anyway.]

Nyiakeng Puachue Hmong
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E100.pdf
new script, all characters except one are are pvalid
I have done some additional checks on the following two:
U+1E14E NYIAKENG PUACHUE HMONG LOGOGRAM NYAJ
   o used to represent the word for money or currency
U+1E14F NYIAKENG PUACHUE HMONG CIRCLED CA
   o used to indicate ownership
   -> U+1E108  nyiakeng puachue hmong letter ca
Currently the former is pvalid but the later is disallowed.
It may well be that it would be more appropriate to have the former be 
disallowed (as currently the '$' sign is disallowed in LDH) but allow 
the latter, if it is e.g. appearing in composite words which may appear 
in domain names. But according to the descriptions in
http://unicode.org/L2/L2017/17002r3-n4780r3-nyiakeng-puachue-hmong.pdf,
we are fine as is.

Wancho
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E2C0.pdf
new script, all letters/digits/tone marks pvalid, one currency sign 
disallowed, okay

Adlam
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E900.pdf
U+1E94B ADLAM NASALIZATION MARK -> pvalid, okay
[Looks somewhat like an apostrophe, but that can be said of a Hebrew 
Yod,... too.]

Ottoman Siyaq Numbers
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1ED00.pdf
These are numbers, but they are not positional, and there's no zero
-> disallowed, okay
[If these were in present-day use, we might want to check whether these 
might appear in e.g. company names and such, but these are only used 
historically, so we're okay.]

Enclosed Alphanumeric Supplement
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F100.pdf
U+1F16C RAISED MR SIGN, has a NFKC composition -> disallowed, okay

Transport and Map Symbols
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F680.pdf
two additional transport-related emoji -> disallowed, okay

Geometric Shapes Extended
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F780.pdf
colored circles and squares -> disallowed, okay

Supplemental Symbols and Pictographs
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F900.pdf
additional emoji of various kinds -> disallowed, okay

Chess Symbols
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1FA00.pdf
turned and other variants of chess symbols for variants of chess
-> disallowed, okay

Symbols and Pictographs Extended-A
http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1FA70.pdf
additional emoji -> disallowed, okay

Regards,    Martin.

---

From: Asmus Freytag <[email-masked]>
Date: Sun, 17 Mar 2019 21:12:50 -0700

> version of Ruby that supports Unicode 12.0.0.
...

> I have sent a mail to Unicode experts to get more information on these.

For A7BD (and it's uppercase, which is not relevant for IDNs) the 
documents say:

"The code positions could be U+A7BA and U+A7BB (next available code 
positions), properties as follow:
A7BA;LATIN CAPITAL LETTER EGYPTOLOGICAL YOD;Lu;0;L;;;;;N;;;;A7BB;
A7BB;LATIN SMALL LETTER EGYPTOLOGICAL YOD;Ll;0;L;;;;;N;;;A7BA;;A7BA
There are not confusable with any existing code points, but the new 
characters could be confused with the sequence of the Latin letter I, 
Latin letter IOTA, Greek letter IOTA, followed by a diacritical mark 
resembling the top of the glyph. Given the existing confused situation, 
this does not exacerbate the risk in a meaningful way. There is a 
limited need to use a YOD in general purpose identifiers; its main 
purpose is to represent transliteration of historic Egyptian text 
originally written using Egyptian hieroglyphs." (WG2 N4792R2)

On this basis, they could be exceptionally DISALLOWED for IDNA.

The other two characters are for transcriptions of Ugaritic - also 
something not desperately needed and exceptional "disallowed" could be 
defended.

See WG2 document N3487.

A./

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 04:58:35 +0000

Hello Asmus,

Many thanks for the followup, also on the Unicode side.

On 2019/03/18 13:12, Asmus Freytag wrote:
> ....
> There are not confusable with any existing code points,

After close examination, I indeed realized that the above Vietnamese 
characters have hooks (half-circles with a small downwards stroke 
attached at the lower end) whereas the 'glottal' characters have 
something more like U+2019 ’ RIGHT SINGLE QUOTATION MARK. So at least on 
close inspection, they are distinguishable. Making sure that diacritic 
marks are easily distinguishable in a requirement of any font that's 
claiming to be suited for IDNA anyway, so this isn't new territory.
[I remember the famous font designer Chuck Bigelow talking about this in 
a keynote at an Unicode conference many many years ago, see also
http://cajun.cs.nott.ac.uk/compsci/epo/papers/volume6/issue3/bigelow.pdf, 
section entitled "The design of diacritics".]

> but the new 

For others, this is at 
https://unicode.org/wg2/docs/n4792r2-egyptologicalYod.pdf.

> something not desperately needed and exceptional "disallowed" could be 

Yes. The risk is then that some as-yet unalphabetized language community 
picks up some of these letters, and later finds out they cannot use them 
in IDNs.

Anyway, I'd be open to any kind of conclusion if it can be made quickly 
(weeks rather than years).

Given they are indeed distinct from the 'Vietnamese' characters, I'd be 
okay with just going with the result of the algorithm (i.e. PVALID), but 
they might be candidates for your 'problematic characters' document.

> See WG2 document N3487.

That's http://www.unicode.org/L2/L2008/08272-n3487-egyptological-yod.pdf.

Regards,    Martin.

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 18 Mar 2019 00:21:44 -0700

> they might be candidates for your 'problematic characters' document.

Effectively you are saying that these could be fine for some (many?) 
zones, yet
we'd agree that for others it would be unwise to allow them.

The discussion of how "unwise" is documented is something that we need to
have separately. There are separate strands of that discussion going on in
parallel.

A./

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 08:35:04 +0000

A kind reviewer found some errors in the attached program. Fortunately, 
it turned up that these errors didn't affect the output of the program 
at all. But it's better to have the fixed version for future reference, 
so I'm attaching it. I'm apologizing for the confusion.

Regards,   Martin.

---

From: "Patrik Fältström" <[email-masked]>
Date: Mon, 18 Mar 2019 11:41:50 +0100

Note that beta version of 12.1.0 already exists.

   Patrik

On 18 Mar 2019, at 1:49, Martin J. Dürst wrote:

> There were some talks about doing a review of the new characters for Unicode 12.0.0, for 'due diligence'.
>
> Here are the results such a review of new characters for Unicode 12.0.0, starting off http://www.unicode.org/charts/PDF/Unicode-12.0/.
> I also used a small Ruby program that I wrote, attached. In order for it to work, you have to make sure you use the latest (and greatest :-)
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-0C00.pdf new U+0C77 TELUGU SIGN SIDDHAM -> disallowed, okay for sign used at the beginning of texts as an invocation
>
> Lao
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-0E80.pdf new Lao letters and a sign (virama) for Pali and Sanskrit
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1CD0.pdf new U+1CFA VEDIC SIGN DOUBLE ANUSVARA ANTARGOMUKHA -> pvalid, okay because this is graphically a base letter (used as a base for a combining nasal sign)
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-2B00.pdf two new symbols -> disallowed, okay
>
> Supplemental Punctuation
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-2E00.pdf new U+2E4F CORNISH VERSE DIVIDER -> disallowed, okay
>
> Latin Extended-D
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-A720.pdf 4 casing pairs, 3 upper-case letters completing case pairs with existing lower-case letters. Upper-case are disallowed (-> okay), lower-case are pvalid. Among these 4, the following three may merit a closer look:
> They are graphically very similar (if not identical) to the following three from the Latin extensions for Vietnamese:
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-AB30.pdf two new letters for Sinological transcription, digraphs with hooks -> pvalid, okay
>
> Elymaic
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-10FE0.pdf new script Elymaic, 23 letters, thereof one ligature
> -> pvalid, okay
>
> Newa
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11400.pdf U+1145F NEWA LETTER VEDIC ANUSVARA -> pvalid, okay
>
> Takri
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11680.pdf U+116B8 TAKRI LETTER ARCHAIC KHA -> pvalid, okay
>
> Nandinagari
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-119A0.pdf new script Nandinagari, all letters pvalid, one sign (U+119E3)
> disallowed, okay
>
> Soyombo
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11A50.pdf two alternate visarga signs in Soyombo
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-11FC0.pdf Tamil supplement: signs for fractions, traditional units, etc.
> disallowed, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-13430.pdf Egyptian Hieroglyph Format Controls for quadrats -> all disallowed, okay
>
> Miao
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-16F00.pdf Miao letters and marks for writing additional languages -> all pvalid, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-16FE0.pdf two additional symbols -> disallowed, okay
>
> Tangut
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-17000.pdf 6 additional Tangut characters -> pvalid, okay
>
> Small Kana Extension
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1B130.pdf historic small Hiragana/Katakana extensions -> pvalid, okay
> [Should be excluded from general use because they may be confused with the larger versions by everyday users, but this is only really a problem for the last character (U+1B167 KATAKANA LETTER SMALL N) because the others won't usually appear in names anyway.]
>
> Nyiakeng Puachue Hmong
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E100.pdf new script, all characters except one are are pvalid
> It may well be that it would be more appropriate to have the former be disallowed (as currently the '$' sign is disallowed in LDH) but allow the latter, if it is e.g. appearing in composite words which may appear in domain names. But according to the descriptions in
> http://unicode.org/L2/L2017/17002r3-n4780r3-nyiakeng-puachue-hmong.pdf, we are fine as is.
>
> Wancho
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E2C0.pdf new script, all letters/digits/tone marks pvalid, one currency sign disallowed, okay
>
> Adlam
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1E900.pdf U+1E94B ADLAM NASALIZATION MARK -> pvalid, okay
> [Looks somewhat like an apostrophe, but that can be said of a Hebrew Yod,... too.]
>
> Ottoman Siyaq Numbers
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1ED00.pdf These are numbers, but they are not positional, and there's no zero -> disallowed, okay
> [If these were in present-day use, we might want to check whether these might appear in e.g. company names and such, but these are only used historically, so we're okay.]
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F100.pdf U+1F16C RAISED MR SIGN, has a NFKC composition -> disallowed, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F680.pdf two additional transport-related emoji -> disallowed, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F780.pdf colored circles and squares -> disallowed, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1F900.pdf additional emoji of various kinds -> disallowed, okay
>
> Chess Symbols
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1FA00.pdf turned and other variants of chess symbols for variants of chess -> disallowed, okay
>
> http://www.unicode.org/charts/PDF/Unicode-12.0/U120-1FA70.pdf additional emoji -> disallowed, okay
>
>
> Regards,    Martin.
>
> -- 
> I18ndir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i18ndir

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 11:09:23 +0000

Hello Patrik,

> Note that beta version of 12.1.0 already exists.

Thanks, I didn't know.

However, please note that it's currently bogus.

The core file, UnicodeData-12.1.0d1.txt, contains exactly one additional 
line when compared with the 12.0.0 version:

32FF;SQUARE ERA NAME SAYUU;So;0;L;<square> 5DE6 53F3;;;;N;;;;;

What will change once we know the actual era name is 'SAYUU' and '5DE6 
53F3'. It currently says 左右 (left-right), and we can be pretty sure that 
that won't be the new area name. First, because it would be a rather 
silly era name, and second because it starts with an 'S', and one of the 
conditions for the new area name is that its romanized name doesn't 
start with the same letter as the most recent ones, which excludes 'H', 
'S', 'T', and most probably also 'M'.

What we know is that there will be a new character at U+32FF and that 
this character will be DISALLOWED because it will have a kompatibility 
decomposition. So we already have everything we need for the IDNA data :-).

Regards,    Martin.

---

From: "Patrik Fältström" <[email-masked]>
Date: Mon, 18 Mar 2019 12:23:06 +0100

Martin, as I have implemented IDNA2008 from scratch, i.e. not using any libraries at all, let me suggest we sync on what output format we use so that we can do "diff" between your and my lists. My list for 12.0.0 is btw attached.

May I suggest what my program is doing (of course), which is as follows:

<Codepoint>;<Derived property value>;<Rule(s) that matched>;<Name>

:
:
0058;DISALLOWED;AB;LATIN CAPITAL LETTER X
0059;DISALLOWED;AB;LATIN CAPITAL LETTER Y
005A;DISALLOWED;AB;LATIN CAPITAL LETTER Z
005B;DISALLOWED;;LEFT SQUARE BRACKET
005C;DISALLOWED;;REVERSE SOLIDUS
005D;DISALLOWED;;RIGHT SQUARE BRACKET
005E;DISALLOWED;;CIRCUMFLEX ACCENT
005F;DISALLOWED;;LOW LINE
0060;DISALLOWED;;GRAVE ACCENT
0061;PVALID;AE;LATIN SMALL LETTER A
0062;PVALID;AE;LATIN SMALL LETTER B
0063;PVALID;AE;LATIN SMALL LETTER C
0064;PVALID;AE;LATIN SMALL LETTER D
0065;PVALID;AE;LATIN SMALL LETTER E
0066;PVALID;AE;LATIN SMALL LETTER F
0067;PVALID;AE;LATIN SMALL LETTER G
0068;PVALID;AE;LATIN SMALL LETTER H
0069;PVALID;AE;LATIN SMALL LETTER I
006A;PVALID;AE;LATIN SMALL LETTER J
:
:

   Patrik

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 11:39:23 +0000

Hello Patrik,

> Note that beta version of 12.1.0 already exists.

About 42 minutes after this message you sent another in the same thread 
that shows up saying it's 35.7 MB heavy. My mailer (Thunderbird) doesn't 
seem able to handle it. Can you tell us what it is about? Thanks.

Regards,   Martin.

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 11:44:44 +0000

Hello Patrik,

Please ignore the mail below. I have been able to get the gist of your 
mail at 
https://mailarchive.ietf.org/arch/msg/i18ndir/1P6_sK59EnAEYV89FHeQ43791Ec.

I suggest we use something else than email for transferring really large 
files. I'll follow up tomorrow.

Regards,    Martin.

---

From: "Patrik Fältström" <[email-masked]>
Date: Mon, 18 Mar 2019 13:27:58 +0100

On 18 Mar 2019, at 12:39, Martin J. Dürst wrote:

> About 42 minutes after this message you sent another in the same thread that shows up saying it's 35.7 MB heavy. My mailer (Thunderbird) doesn't seem able to handle it. Can you tell us what it is about? Thanks.

Ouch...sorry...

   paf

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 18 Mar 2019 23:47:22 +0000

Hello Patrik,

On 2019/03/18 21:29, Patrik Fältström wrote:
> [Now without my list of code points]

In the meantime, with a bit more patience, I have been able to extract 
the file and have been able to look at it in an editor.

> Martin, as I have implemented IDNA2008 from scratch, i.e. not using any 
> libraries at all,

I'm also not using any libraries at all, just relying on the Unicode 
version in Ruby to be the same as the target version. But I understand 
what you're saying.

> let me suggest we sync on what output format we use so 
> that we can do "diff" between your and my lists. My list for 12.0.0 is 
> btw attached.

I think it's a good idea to have this for some cross-checks. But one of 
the features of my program was that it listed only the new characters, 
because these were the ones I wanted to review. So I'll definitely keep 
this ability.

> <Codepoint>;<Derived property value>;<Rule(s) that matched>;<Name>

I'm fine with that. I'll have to do more work on supporting ranges in 
UnicodeData.txt and supporting the rules that I ignored in my first 
quick attempt. So please don't rely on my work for moving forward.

Regards,   Martin.

> .

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 18 Mar 2019 18:35:52 -0700

> However, please note that it's currently bogus.

Unicode 12.1.0 is in the special situation that a new character is known 
to be needed by an entire country from a fixed date; with known 
properties other than its shape (and therefore name and decomposition).

Therefore, unlike other betas, there are three known items of 
"last-minute correction" that are known to definitely apply: name, glyph 
and decomposition mapping (but not decomposition type).

As Martin points out, the IDNA2008 status calculation is unaffected and 
this makes thus a good test case for overlapping the IDNA update work 
with the beta.

A./

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 19 Mar 2019 07:11:34 +0100

On 19 Mar 2019, at 0:47, Martin J. Dürst wrote:

> I'm also not using any libraries at all, just relying on the Unicode version in Ruby to be the same as the target version. But I understand what you're saying.

Well, Ruby do use ultimately libidn2 for IDNA2008, so yes, you do use libraries.

> I think it's a good idea to have this for some cross-checks. But one of the features of my program was that it listed only the new characters, because these were the ones I wanted to review. So I'll definitely keep this ability.

Ok, I of course have that as well :-)

That operates on the file format I did show you.

> I'm fine with that. I'll have to do more work on supporting ranges in UnicodeData.txt and supporting the rules that I ignored in my first quick attempt. So please don't rely on my work for moving forward.

No problem, I already do have two implementations myself. One that do use libidn2 in python3, one in ruby that do not, and then I compare with IANA as I know they do have a completely code base as well.

   Patrik

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 19 Mar 2019 11:40:19 +0000

Hello Patrik,

> Well, Ruby do use ultimately libidn2 for IDNA2008, so yes, you do use libraries.

No, Ruby itself does not provide an implementation of IDNA2008. There 
are gems for that, e.g. https://rubygems.org/gems/idn-ruby, which uses 
libidn. But I'm not using any gems in my code.

Regards,   Martin.

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 19 Mar 2019 12:57:37 +0100

On 19 Mar 2019, at 12:40, Martin J. Dürst wrote:

> No, Ruby itself does not provide an implementation of IDNA2008. There are gems for that, e.g. https://rubygems.org/gems/idn-ruby, which uses libidn. But I'm not using any gems in my code.

Ok, sorry, I misread your code!!!

My apologies!

   paf