Subject: RE: [Geopriv] Open discussion: Revised Civic LO draft

From: "Martin Thomson" <[email-masked]>
Date: Fri, 2 Sep 2005 10:17:09 +1000

Hi Andy,

This is one that I had to do a little reading on to be sure that I was doing
the right thing.  Unicode and scripts are not something that I have dealt
with before, so my interpretation may not be completely accurate.

The way I understand it is that unicode (UTF-8, UTF-16, etc...) specifies a
means of encoding characters.  Every character also has an associated script
(the characters that I am using are in the script called "latin" for
instance).  This is where the DHCP civic draft confused me, and perhaps
Henning can clear this up, but I think that specifying script is redundant.
That is, by specifying a particular codepoint in one of the unicode
standards, you refer to a specific character, and that character has a
script.

The harm in including the information would be a comflict between what one
scheme is telling you, and what the other says.  That sort of conflict is
dangerous - which one has precedent.  If I specify character U+203B
(japanese kome) and the cyrillic script, how does that help?  The language
attribute should disambiguate sufficiently (japanese kome vs. urdu paragraph
separator).

Cheers,
Martin

-----Original Message-----
From: Andrew Newton [mailto:[email-masked]] 
Sent: Thursday, 1 September 2005 10:06 PM
To: Martin Thomson
Cc: GEOPRIV
Subject: Re: [Geopriv] Open discussion: Revised Civic LO draft

Martin,

I haven't had my morning coffee, so I'm likely not to get this  
right.  But in Section 2.3, you mention dropping 'script' in favor of  
Unicode (which XML does natively).  Isn't it still helpful to have  
the script information?  Even if UTF-8 or UTF-16 can do everything,  
what harm is done by continuing to include that information?

-andy

On Sep 1, 2005, at 12:04 AM, Martin Thomson wrote:

> Hi All,
>
> I haven't received any comments on the civic draft.  Just in case
> this got lost in the noise last week, I'd like to remind you of the  
> existence of this draft.
>
> Ta,
> Martin
>
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Thomson, Martin [WOLL:5500:EXCH]
> Sent: Monday, 22 August 2005 1:16 PM
> To: GEOPRIV
> Subject: [Geopriv] Open discussion: Revised Civic LO draft
>
>
> Hi Everyone,
>
> I have submitted a draft that updates the civic address format for  
> PIDF-LO.  This includes Henning's DHCP types as well as a few more.
>
>     http://www.ietf.org/internet-drafts/draft-thomson-revised-civic- 
> lo-00.txt
>
> If you have any comments, let fly.  I'd be happy to discuss any of  
> the decisions that have been made.
>
> Cheers,
> Martin
>
> _______________________________________________
> Geopriv mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/geopriv
>

_______________________________________________
Geopriv mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/geopriv

---

From: Andrew Newton <[email-masked]>
Date: Thu, 1 Sep 2005 20:51:36 -0400

On Sep 1, 2005, at 8:17 PM, Martin Thomson wrote:

> Hi Andy,
>
> This is one that I had to do a little reading on to be sure that I  
> was doing the right thing.  Unicode and scripts are not something  
> that I have dealt with before, so my interpretation may not be  
> completely accurate.
>
> The way I understand it is that unicode (UTF-8, UTF-16, etc...)  
> specifies a means of encoding characters.  Every character also has  
> an associated script (the characters that I am using are in the  
> script called "latin" for instance).  This is where the DHCP civic  
> draft confused me, and perhaps Henning can clear this up, but I  
> think that specifying script is redundant.  That is, by specifying  
> a particular codepoint in one of the unicode standards, you refer  
> to a specific character, and that character has a script.
>
> The harm in including the information would be a comflict between  
> what one scheme is telling you, and what the other says.  That sort  
> of conflict is dangerous - which one has precedent.  If I specify  
> character U+203B (japanese kome) and the cyrillic script, how does  
> that help?  The language attribute should disambiguate sufficiently  
> (japanese kome vs. urdu paragraph separator).

Perhaps my original question should have been, "So why does dhcp- 
civic have scipts?"

Like XML, it is also using UTF-8.  But that draft does have this  
explanation:

    The same civic and postal address information can often be rendered
    in multiple languages and scripts.  For example, Korean addresses  
are
    often shown in Hangul, Latin and Kanji, while some older cities have
    multiple language variants (Munich, Muenchen and Monaco, for
    example).  Since DHCPv4 and DHCPv6 do not currently support a
    mechanism to query for a specific script or language, the DHCP  
server
    SHOULD provide all common renderings to the client and MUST provide
    at least the rendering in the language and script appropriate to the
    location indicated.

So it isn't necessarily about the UTF-8.  It is about the protocol  
interaction between the sender and receiver.  And I think that  
perhaps we have the same interaction with PIDF-LO, and therefore the  
'scripts' out to be put back in (though in XML it is usually called  
'lang' or 'language').  Do we envision a receiver of PIDF-LO asking  
for the Italian version of the same document after receiving the  
English version?  I doubt it.

-andy

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 01 Sep 2005 21:02:27 -0400

Andrew Newton wrote:

> out to be put back in (though in XML it is usually called  'lang' or 
> 'language').  Do we envision a receiver of PIDF-LO asking  for the 
> Italian version of the same document after receiving the  English 
> version?  I doubt it.

This is fortunately the same thing: both use RFC 3066 language tags (see 
http://www.w3.org/TR/2004/REC-xml-20040204/#sec-lang-tag).

[Pretty soon it will be 3066bis, judging from the discussion on the ietf 
mailing list...]

Henning

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 01 Sep 2005 21:07:53 -0400

UTF-8 mostly specifies the script, but due to "Han unification", this 
isn't always true.

In most cases, there would indeed be no need to include the script.

Martin Thomson wrote:
> specifies a means of encoding characters.  Every character also has an 
> associated script (the characters that I am using are in the script 
> called "latin" for instance).  This is where the DHCP civic draft 
> confused me, and perhaps Henning can clear this up, but I think that 
> specifying script is redundant.  That is, by specifying a particular 
> codepoint in one of the unicode standards, you refer to a specific 
> character, and that character has a script.
> 
> The harm in including the information would be a comflict between what 
> one scheme is telling you, and what the other says.  That sort of 
> conflict is dangerous - which one has precedent.  If I specify character 
> U+203B (japanese kome) and the cyrillic script, how does that help?  The 
> language attribute should disambiguate sufficiently (japanese kome vs. 
> urdu paragraph separator).

---

From: Ted Hardie <[email-masked]>
Date: Thu, 1 Sep 2005 18:24:29 -0700

At 9:07 PM -0400 9/1/05, Henning Schulzrinne wrote:
>UTF-8 mostly specifies the script, but due to "Han unification", this isn't always true.

Also called the "CJK" unification, because the ideographs that are common to
Chinese, Japanese, and Korean aren't repeated 3 times.  This is to save space
in the tables.

The larger point, though, is what the real danger is in PIDF-LO document
giving a code point and a surprising script association.  I've been trying
to think of cases where a code point being reported in a surprising script
would actually influence the use made of the PIDF-LO in protocol processing.
I haven't come up with any, largely because I don't see the "script, code point"
pair getting used as a tuple.  Am I missing some obvious case?

The one concern I can see is attempting to use the script as a indicator of
how to display something so that a surprising script is used to mask aspect
of the PIDF-LO object in the rendering.  If a user agent sees "Latin" and chooses
a pretty but limited font as a result, then hits CJK characters, things might go
missing that would be displayed.  But this seems to be something to include
in the Security Considerations rather than a strong risk.  Is the risk greater
here than it appears?

			regards,
				Ted Hardie

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 01 Sep 2005 21:43:10 -0400

Clearly, this mis-association can happen in XML with the lang (which is 
really a language-script) tag, and it doesn't seem to have raised any 
particular concerns that I have heard, although that's probably due to 
my fairly limited horizon in this area.

Since both the DHCP draft and XML use (or will use) 
http://www.ietf.org/internet-drafts/draft-ietf-ltru-registry-12.txt in 
whole or in part, the same security considerations there would appear to 
apply.

There is a somewhat-related paragraph:

    The language tag associated with a particular information item is of
    no consequence whatsoever in determining whether that content might
    contain possible homographs.  The fact that a text is tagged as being
    in one language or using a particular script subtag provides no
    assurance whatsoever that it does not contain characters from scripts
    other than the one(s) associated with or specified by that language
    tag.

Henning