Subject: [idn] stringprep comment 2

From: "Soobok Lee" <[email-masked]>
Date: Wed, 30 Jan 2002 02:42:34 +0900

In stringprep  section
6,

" 6. Unassigned
Code Points in Stringprep Profiles

This section describes two different
types of strings in typical
protocols where internationalized strings are
used: "stored strings" and
"queries". Of course, different Internet protocols
use strings very
differently, so these terms cannot be used exactly in every
protocol
that needs to use stringprep. In general, "stored strings" are
strings
that are used in protocol identifiers and named entities, such as
names
in digital certificates, DNS domain name parts, and names of SNMP
objects.
"Queries" are strings that are used to match against strings that
are
stored identifiers, such as user-entered names for digital
certificate
authorities, DNS lookups, and SNMP requests.
"

Is
this classification from  over-simplifications ?
only two
types?

Is
there any 3rd type possible?

To be complete, 'typical' ,'cannot .. exactly'
and 'in general' had better be removed and

the classification should  first define
"strored strings" precisely and  then

define the complement set
as the "queries"  if it
is proved that any 3rd type does not exist.

I have doubt that there
are many *gray areas* that  do not fall into either
categories.

Soobok Lee

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Tue, 29 Jan 2002 12:24:25 -0800

At 2:42 AM +0900 1/30/02, Soobok Lee wrote:
>
>In stringprep  section 6,
>
>" 6. Unassigned Code Points in Stringprep Profiles
>
>This section describes two different types of strings in typical
>protocols where internationalized strings are used: "stored strings" and
>"queries". Of course, different Internet protocols use strings very
>differently, so these terms cannot be used exactly in every protocol
>that needs to use stringprep. In general, "stored strings" are strings
>that are used in protocol identifiers and named entities, such as names
>in digital certificates, DNS domain name parts, and names of SNMP objects.
>"Queries" are strings that are used to match against strings that are
>stored identifiers, such as user-entered names for digital certificate
>authorities, DNS lookups, and SNMP requests. "
>
>
>Is this classification from  over-simplifications ? only two types?
>Is there any 3rd type possible?

Maybe, but we could not think of any, and no one has suggested any 
yet. Do you have an example that we should deal with?

>To be complete, 'typical' ,'cannot .. exactly' and 'in general' had 
>better be removed and
>the classification should  first define "strored strings" precisely and  then
>define the complement set  as the "queries"  if it is proved that 
>any 3rd type does not exist.

That would indeed be complete. It is also impossible to do for the 
hundreds (thousands?) of Internet protocols.

>I have doubt that there are many *gray areas* that  do not fall into 
>either categories.

Your doubt may be correct, but we cannot act on your doubt without 
some concrete examples.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Adam M. Costello" <[email-masked]>
Date: Wed, 30 Jan 2002 04:09:10 +0000

Soobok Lee <[email-masked]> wrote:

> 6. Unassigned Code Points in Stringprep Profiles
>
> protocols where internationalized strings are used: "stored strings"
> and "queries".
>
> Is this classification from over-simplifications ? only two types?  Is

I think what this section really depends on is that whenever two strings
are compared, at least one of the strings must have been prepared using
strict stringprep (no unassigned code points allowed), while the other
may have been prepared using loose stringprep (unassigned code points
allowed).  This allows false negatives (where the strings don't match
even though you wish they did), but prevents false positives (where the
strings match even though they shouldn't).

Applications and protocols need to do whatever is necessary to guarantee
that whenever two strings are compared, at least one was prepared using
strict stringprep.  Typically they would satisfy that requirement by
classifying strings into "stored strings" and "query strings", using
strict stringprep on the stored strings, and never comparing two query
strings.  But it's up to each application/protocol to define that
classification for its own strings.

Perhaps the document should say something like that.

AMC

---

From: "Soobok Lee" <[email-masked]>
Date: Wed, 30 Jan 2002 14:33:24 +0900

Hi, Adam
----- Original Message -----
From: "Adam M. Costello"
To:
Sent: Wednesday, January 30, 2002 1:09 PM
Subject: Re: [idn] stringprep comment 2

> Soobok Lee wrote:

In many cases, label strings are imported or transmitted from other applicationss,
not generated by the application itself. that makes it difficult to classify them.

For example, let's think about IRC servers using email addresses as nicknames.
One IRC participator announces him as [email-masked] using old iDNA-aware IRC client,
and his partner invites him as [email-masked]' using new client,
while A == B modulo new stringprep ,but A != B modulo old stringprep.

The similar situations come with other applications using email addresses as
such session/user identifiers, which are not necessariliy permanent identifiers.
In such cases, comparing two "query strings" are inevitable and make troubles.

Another situtations.
In composing email messages, senders' email addresses are "stored strings" or "queries"??
the recipients addresses can be classified as the same or not ?
the email addresses in the addressbook?
It seems very subtle question.

Soobok Lee

---

From: "Adam M. Costello" <[email-masked]>
Date: Wed, 30 Jan 2002 07:33:28 +0000

Soobok Lee <[email-masked]> wrote:

> In many cases, label strings are imported or transmitted from other
> applicationss, not generated by the application itself. that makes it
> difficult to classify them.
>
> ...comparing two "query strings" are inevitable and make troubles.
>

I'm starting to agree.

Hmmm, is it even necessary to avoid comparing two query strings?  Even
if you compare two strings prepared with loose stringprep (unassigned
code points allowed), the worst that can happen is still a false
negative (where they don't match when they should), you still can't get
a false positive (where they match when they shouldn't).

So maybe we could simply remove all this talk about stored strings
and query strings.  The Stringprep spec could make this yet another
parameter: the profile would have to specify, along with the mapping
tables etc, a single bit indicating whether unassigned code points are
prohibited or not.  The Stringprep spec would retain the discussion
about what can happen when unassigned code points are not prohibited.

The Nameprep document would define two profiles: strict Nameprep and
loose Nameprep, which differ only in that bit.  The IDNA spec would
require registrars to use strict Nameprep, and would let applications
decide for themselves when/if to allow loose Nameprep.

How's that?

AMC

---

From: "Soobok Lee" <[email-masked]>
Date: Thu, 31 Jan 2002 11:30:21 +0900

----- Original Message -----
From: "Adam M. Costello"
To: "IETF idn working group"
Sent: Wednesday, January 30, 2002 4:33 PM
Subject: Re: [idn] stringprep comment 2

> a false positive (where they match when they shouldn't).

Right. but, The latter can be as catastrophic as the former.

> decide for themselves when/if to allow loose Nameprep.

Using iDN doesn't necessarily mean the application should *generate* the
ACE label. As examplified in my previous posting, the IRC server may
get chatters' email addresses only from chatting clients,not making on its own,
therefore, the IRC servers can't know whether the iDN email addresses
are made using strong nameprep or weak one. the IRC server are left with
no clue to decide whether to compare them or not.

if each application vendor adopts its own different nameprep profile,
applications behaviors may be unpredictable across applications
for end users. then, IDN can't fulfill the essesntial conditions of successful
universal identifier system.

To shot down moving targets,we need moving guns and moving hands and moving eyes.

Soobok Lee

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 31 Jan 2002 22:03:02 +0000

This message argues both sides of the issue.  :)

Soobok Lee <[email-masked]> wrote:

> The latter can be as catastrophic as the former.

I assume you meant that false negatives (where names don't match when
they should) can be as catastrophic as false positives (where names
match when they shouldn't).  Can you back up that claim?

> applications behaviors may be unpredictable across applications for
> end users.

Do you have a suggestion?  What should happen when an application
encounters a name that uses code points newer than the application's
version of nameprep?  If the application prohibits unassigned code
points, then the name will never match anything, because ToASCII will
fail.  If the application allows unassigned code points, then the name
will never match the wrong thing, and might sometimes match the right
thing (in practice, I think it usually will work).  Which is preferable?
The conservative approach (never match) is more predictable, but the
other approach (match if you're lucky) might make users happier.

Wait, I just realized why we needed to avoid comparing two strings
that have both been prepared using loose stringprep.  If they both use
unassigned code points that turn out to be prohibited in future versions
of nameprep, then they might match even though they are both invalid
names.  That's a false positive, which is bad.  So we do indeed need to
avoid such comparisons.

Disregard my suggestion from my last message.

Perhaps the stringprep spec should say that applications may use loose
stringprep only if they know for sure that the name will never be
compared against a name that was also prepared using loose stringprep.
If there's no way to know, then you must use strict stringprep.

In the case of DNS, if the IDNA spec requires authoritative servers
to use strict nameprep, then clients are free to prepare queries
using loose nameprep.  Other protocols could in principle use similar
methods--requiring strict nameprep at "one end" (whatever that means for
that protocol) so that the "other end" can use loose nameprep.

But how practical is that?  Take email headers for example.  Who has any
idea what will be done with domain names that appear in email headers?

Maybe it would be a lot simpler and safer just to prohibit unassigned
code points always.  If you want to use new characters, you'll just have
to upgrade your software to the new nameprep, sorry.

Can we get some more people involved in this thread?  I think Soobok is
right that the existing wording in stringprep about "stored strings" and
"query strings" is going to be very difficult to interpret in practice,
and something needs to be done about it, but I don't know what.

AMC

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 31 Jan 2002 14:17:43 -0800

The comparison of loose vs loose should never be done, for the reasons
you outline. That leaves

a) strict vs strict
b) strict vs loose

(a) is always safe.

(b) works for situations like DNS, where there is a server on one side
storing strict keys, and queries are made on the other and may be
loose or strict. It would work for similar situations, such as email,
if at one end there is a server that does validation (of, in this
case, email names).

Mark
—————

Πόλλ’ ἠπίστατο ἔργα, κακῶς δ’ ἠπίστατο πάντα — Ὁμήρου Μαργίτῃ
[For transliteration, see http://oss.software.ibm.com/cgi-bin/icu/tr]

http://www.macchiato.com

----- Original Message -----
From: "Adam M. Costello" <[email-masked]>
To: "IETF idn working group" <[email-masked]>
Sent: Thursday, January 31, 2002 14:03
Subject: Re: [idn] stringprep comment 2

> > applications behaviors may be unpredictable across applications
for

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 31 Jan 2002 22:03:02 +0000

AMC

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 31 Jan 2002 14:17:43 -0800

a) strict vs strict
b) strict vs loose

(a) is always safe.

Mark
—————

http://www.macchiato.com

----- Original Message -----
From: "Adam M. Costello" <[email-masked]>
To: "IETF idn working group" <[email-masked]>
Sent: Thursday, January 31, 2002 14:03
Subject: Re: [idn] stringprep comment 2

for