Subject: [Lucid] Communication.

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Mar 2015 22:37:31 +0000

I feel like I'm speaking a different language, or at least not worrying about the same problems.

Andrew keeps asking, I've read https://datatracker.ietf.org/doc/draft-sullivan-lucid-prob-stmt/?include_text=1 several times now...

It looks like the PRECIS effort and Unicode aren't speaking the same language.  Which is unfortunate, since PRECIS is attempting to suggest ways to use Unicode.  

I think Asmus said that well " Character encoding is not the process of numbering entities by shape. It's the process of numbering entities by their underlying identity in writing."  

The PRECIS effort's language is all about characters are encoded by shape.  Unicodes language defined them as being encoded by their underlying identity.

-Shawn

PS: I love the sentence that starts off RFC 6885: "If a protocol expects to compare two strings and is prepared only for those strings to be ASCII, then using Unicode code points in those strings requires they be prepared somehow."  That makes me want to go hide in a cave or something.

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 19 Mar 2015 16:59:22 -0600

On 3/19/15 4:37 PM, Shawn Steele wrote:
> Andrew keeps asking, I've read
> https://datatracker.ietf.org/doc/draft-sullivan-lucid-prob-stmt/?include_text=1
> several times now...
>
> It looks like the PRECIS effort and Unicode aren't speaking the same
> language.  Which is unfortunate, since PRECIS is attempting to
> suggest ways to use Unicode.
>
> I think Asmus said that well " Character encoding is not the process
> of numbering entities by shape. It's the process of numbering
> entities by their underlying identity in writing."
>
> The PRECIS effort's language is all about characters are encoded by
> shape.  Unicodes language defined them as being encoded by their
> underlying identity.

Really? Now I need to ask if you have read draft-ietf-precis-framework:

http://datatracker.ietf.org/doc/draft-ietf-precis-framework/

Just as with IDNA2008, we structured PRECIS so that decisions are made 
algorithmically based on the properties of code points (which together 
constitute the code point's identity), not the shapes of glyphs.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Mar 2015 23:14:06 +0000

> Just as with IDNA2008, we structured PRECIS so that decisions are made algorithmically based on the properties of code points (which together constitute the code point's identity), not the shapes of glyphs.

Yet this discussion is about whether that algorithm correctly handles the characters based on the shapes of the glyphs.

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 19 Mar 2015 17:15:52 -0600

> Yet this discussion is about whether that algorithm correctly handles the characters based on the shapes of the glyphs.

Humans aren't computers and they make decisions and get confused based 
on appearance. I don't see the two perspectives as mutually exclusive.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Mar 2015 23:49:15 +0000

> Humans aren't computers and they make decisions and get confused based on appearance. I don't see the two perspectives as mutually exclusive.

They're orthogonal though.  Clearly some characters are confusable yet necessary (Cyrillic А and Latin A).  The example in the appendix point out other cases where multiple code points are confusable.  The only real difference between the examples in PRECIS/lucid's Appendix A is questioning of the validity of the same-script similarities in Appendix A, bases primarily in the appearance.

I find l̦ and ļ fairly difficult to distinguish, yet the distinction between ț and ţ (easier to tell, but the same combining marks) is important to some.   Should l̦ and ļ be forbidden or unified because I can't see a distinction and I have no clue how people would use them?  l̦ and ļ aren't in Appendix A (are they supposed to be?)  

For that matter, I have a hard time seeing a difference between i and i̇, however IDNA makes particular use of the odd combination in the casing/normalization rules.  Those aren't in Appendix A.

Should i and i̇ be in Appendix A?  What about l̦ and ļ?  If not, can someone explain why they're special?

-Shawn

---

From: Asmus Freytag <[email-masked]>
Date: Thu, 19 Mar 2015 16:52:48 -0700

On 3/19/2015 3:59 PM, Peter Saint-Andre - &yet wrote:
> Just as with IDNA2008, we structured PRECIS so that decisions are
> made algorithmically based on the properties of code points (which
> together constitute the code point's identity), not the shapes of glyphs.
>
> Peter
>
Peter,

nobody argues that IDNA or PRECIS are not based on
character properties.  They most clearly are.

What I was trying to highlight is the fact that there may be
a misunderstanding of how Unicode (and pretty much all
character encoding, except for, the most trivial cases)
is conceived.

By basing your specification on properties, you can
ensure uniqueness of underling encoded identity,
but, unless the particular properties happen to take into
account information on shape, you won't get uniqueness
in shape.

I'm all for helping IDNA and PRECIS by having Unicode
provide some additional properties that can help address
the case of "apparent identity" where the shapes agree,
but the identity actually is distinct.

A./

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Mar 2015 23:52:35 +0000

Sorry, I was a little vague there.  IDNA went to special effort to make i and i happen by mapping İ to i̇.  So it seems peculiar to me to question it now.

-Shawn

-----Original Message-----
From: Shawn Steele 
Sent: Thursday, March 19, 2015 4:49 PM
To: 'Peter Saint-Andre - &yet'; [email-masked]
Subject: RE: [Lucid] Communication.

-Shawn