Subject: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 15 Aug 2006 07:31:47 -0500

>I was selected as General Area Review Team reviewer for this specification
> (for background on Gen-ART, please see
> http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).

This is a re-review, my previous review was for 06, with Scott as 
shepherding AD, before IETF 65. I'm  reading the deltas from 06 (in the 
spirit of not finding new problems with previously-reviewed text).

Summary: Again, nearly ready for publication as Proposed Standard, with some 
(new) items that do need to be addressed before publication.

Thanks,

Spencer

Review Comments:

2.2.  Purpose

   Collations abstraction layer for comparison functions so that these
   comparison functions can be used in multiple protocols.

I am just barely able to parse this sentence so that it's not a sentence 
fragment. I think the problem is that "functions" is being used as a verb 
and as a noun in the same sentence. I saw later in the document that you had 
changed "function"-the-noun to "operation", so should be easy to fix. But 
this isn't an editorial comment, because I'm not sure what the sentence is 
saying.

4.2.2.  Equality
    ...
    In this specification, the return values of the equality test are
    called "match", "no-match" and "undefined".  This is not a
    specification, merely a choice of phrasing.

What does the last sentence mean? (Brian Carpenter asked me, so he doesn't 
know, either).

5.2.  Operations

...

   Although the collation's substring function provides a list of
   matches, a protocol need not provide all that to the client.  It may
   provide only the first matching substring, or even just the
   information that the substring search matched.

Hmmm. I am trying to remember that you're not defining a protocol, only 
describing what protocols do and don't do, but I'm trying to read this from 
the application's perspective, and having a hard time understanding how (for 
example) an application that is trying to display what is matching responds 
when the protocol only provides an indication that something matched. You 
may say this is what the protocol developers are supposed to worry about 
("if you think applications will want to display what matches, you'd better 
define the protocol so that this information is returned"), and that's OK. 
I'm just struggling a bit here.

6.  Use by Existing Protocols

...

   IMAP [16] also collates, although that is explicit only when the
   COMPARATOR [18] extension is used.  The built-in IMAP substring
   operation and the ordering provided by the SORT [17] extension may
   not meet the requirements made in this document.

   Other protocols may be in a similar position.

   In IMAP, the default collation is i;ascii-casemap, because its
   operations most closely resembles IMAP's built-in operations.

EDITORIAL: I'm guessing that the previous paragraph should be moved up one? 
At the very least, I'm confused because I'm not sure if the top paragraph in 
this extract describes the differences between i;ascii-casemap and IMAP's 
built-in operations or is talking about something else.

9.1.1.  ASCII Numeric Collation Description

   The "i;ascii-numeric" collation is a simple collation intended for
   use with arbitrary sized unsigned decimal integer numbers stored as
   octet strings.  US-ASCII digits (0x30 to 0x39) represent digits of
   the numbers.  Before converting from string to integer, the input
   string is truncated at the first non-digit character.  All input is
   valid; strings which do not start with a digit represent positive
   infinity.

Is it obvious to everyone except me that leading zeros are ignored? The 
examples giving a little further down say so - is making this point in 
examples normative enough?

9.2.1.  ASCII Casemap Collation Description

...

   The i;ascii-casemap collation is well suited to to use with many
   internet protocols and computer languages.  Use with natural language
   is often inappropriate: even though the collation apparently supports
   languages such as Italian and English, in real-world use it tends to
   stumble over words such as "naive", names such as "Llwyd", people and
   place names containing non-ASCII, euro and pound sterling symbols,
   quotation marks, dashes/hyphens, etc.

OK, this may be inadvertantly funny - are "naive" and "Llwyd" supposed to 
include a non-ascii character, or is that sentence saying something else? 
(Welcome to the world of the RFC Editor)

13.  Open Issues

    ... adding a
    note to the RFC editor to possibly replace the 3066 reference

>From Brian: Surely this needs to be done?

>From Spencer: I'm thinking that the "checking the SP SP "1" SP SP string for 
correctness" also needs to be done pretty soon :-0

_______________________________________________
Gen-art mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/gen-art

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 16 Aug 2006 00:00:55 +0200

Spencer Dawkins writes:
>> I was selected as General Area Review Team reviewer for this 
>> specification (for background on Gen-ART, please see 
> shepherding AD, before IETF 65. I'm  reading the deltas from 06 (in 
> the spirit of not finding new problems with previously-reviewed 
> text).
>
> Summary: Again, nearly ready for publication as Proposed Standard, 
> with some (new) items that do need to be addressed before 
> publication.
>
> Thanks,
>
> Spencer
>
> Review Comments:
>
> 2.2.  Purpose
>
> I am just barely able to parse this sentence so that it's not a 
> sentence fragment. I think the problem is that "functions" is being 
> used as a verb and as a noun in the same sentence. I saw later in the 
> document that you had changed "function"-the-noun to "operation", so 
> should be easy to fix. But this isn't an editorial comment, because 
> I'm not sure what the sentence is saying.

It is saying "Arnt cannot search and replace".

    Collations provide a multi-protocol abstraction layer for comparison 
functions so that these
    comparison functions can be used in multiple protocols.

(Maybe strike "layer". Not sure  yet. Must look at it when I'm 100% awake.)

> What does the last sentence mean? (Brian Carpenter asked me, so he 
> doesn't know, either).

It means: I'm not defining what these three return values are called, 
only naming them so I can talk about them. If you implement this, 
you're free to call them anything you want. You can use a C++ enum 
type, or -1/0/1, or whatever.

This rather awkward phrasing is a result of conflicting reviewer requests.

I could say «The return values of the equality test are called "match", 
"no-match" and "undefined" in this document.» Would that be clear 
enough?

> Hmmm. I am trying to remember that you're not defining a protocol, 
> only describing what protocols do and don't do, but I'm trying to 
> read this from the application's perspective, and having a hard time 
> understanding how (for example) an application that is trying to 
> display what is matching responds when the protocol only provides an 
> indication that something matched. You may say this is what the 
> protocol developers are supposed to worry about ("if you think 
> applications will want to display what matches, you'd better define 
> the protocol so that this information is returned"), and that's OK.

Exactly.

Some current-day protocols are defined such that 'x contains y' returns 
true/false. Servers implementing such protocols should still be able to 
use collations.

> I'm just struggling a bit here.

Understandably.

> EDITORIAL: I'm guessing that the previous paragraph should be moved up one?

No... the «Other protocols» bit applies to the text above it, not to the 
text below it. But I'll change it if one other person agrees with you.

> At the very least, I'm confused because I'm not sure if the top 
> paragraph in this extract describes the differences between 
> i;ascii-casemap and IMAP's built-in operations or is talking about 
> something else.

I'll explain it, but I'm not sure you want to know ;)

In IMAP, the server advertises which extensions it supports. The client 
does not advertise anything. Because of this asymmetry, the server 
sometimes has to act in a way which satisfies both an unextended and an 
extended client. This is such as case: The server must do substring and 
sorting operations, and it often cannot tell whether the client knows 
about collations.

What's needed is a default collation which is close enough to 
current-day IMAP behaviour that unextended clients are not surprised by 
what the server does. i;ascii-casemap is that. It may or may not be a 
perfect match for IMAP/SORT as specified (Alexey Melnikov and I haven't 
found a difference), and it's at least close to what servers typically 
implement.

> Is it obvious to everyone except me that leading zeros are ignored? 
> The examples giving a little further down say so - is making this 
> point in examples normative enough?

It's specified in 2244, so I don't think it's very important. This 
document merely registers a collation which has been specified for a 
decade and implemented in many products.

> OK, this may be inadvertantly funny - are "naive" and "Llwyd" supposed 
> to include a non-ascii character, or is that sentence saying 
> something else? (Welcome to the world of the RFC Editor)

I would write naïve if I could. I assume people know that naive and 
naïve are both common spellings.

Llwyd is thus spelt. The Welsh consider ll a separate letter and sort it 
between l and m.

> From Spencer: I'm thinking that the "checking the SP SP "1" SP SP 
> string for correctness" also needs to be done pretty soon :-0

Each and every time I run xml2rfc, actually. I grew bored :(

Arnt

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 15 Aug 2006 16:12:33 -0700

On Aug 15, 2006, at 3:29 PM, Spencer Dawkins wrote:
>>
>> OK, this may be inadvertantly funny - are "naive" and "Llwyd"  
>> supposed to include a non-ascii character, or is that sentence  
>> saying something else? (Welcome to the world of the RFC Editor)
>
> Spencer-reply: I guess my point was that this was extremely subtle  
> for those of us who don't work with i18n comparison all day long.  
> Perhaps 'Welsh names such as "L1wyd", when the Welsh consider "ll"  
> a separate letter and sort it between "1" and "m"'? But you're  
> going to have to figure out how to get "naïve" into an RFC...  
> Perhaps your AD can step in front of this speeding bullet?

Not sure there's a speeding bullet here.  I happened to know what  
Arnt meant by this stuff -- my own favourite example is sorting  
"Canada" followed by "canal" then "cantor", *then* followed by   
"caña" and finally "cañada" (last two with tildes above the n's if  
you can't see them), if you're using Spanish.

Without the ability to put ñ or ï into Internet Drafts, we could  
probably still make this more readable for people unfamiliar with  
i18n and sorting issues.  Here's a stab:

Use with natural language  is often inappropriate: even though the  
collation apparently supports  languages such as Italian and English,  
in real-world use it tends to mis-sort a number of types of string:
     * words such as "naive" (if spelled with an accent, the accented  
character could push the word to the wrong spot in a sorted list),
    * names such as "Llwyd" (which in Wales/Welsh or in Spanish  
should sort after single-L names like Lyza),
    * people and  place names containing non-ASCII,
    * strings containing euro and pound sterling symbols, quotation  
marks, dashes/hyphens, etc.

Lisa

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 15 Aug 2006 17:29:28 -0500

Hi, Arnt,

For a sleepy guy, you explained things pretty well. Please check below to 
see if we still have stuff to think about... this round is tagged 
"Spencer-reply:". But I think we're very close to "good to go".

Thanks,

Spencer

----- Original Message ----- 
From: "Arnt Gulbrandsen" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; "General Area Review 
Team" <[email-masked]>; "Lisa Dusseault" <[email-masked]>
Sent: Tuesday, August 15, 2006 5:00 PM
Subject: Re: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt

Spencer Dawkins writes:
> Summary: Again, nearly ready for publication as Proposed Standard, with 
> some (new) items that do need to be addressed before publication.
>
> Thanks,
>
> Spencer
>
> Review Comments:
>
> 2.2.  Purpose
>
> and as a noun in the same sentence. I saw later in the document that you 
> had changed "function"-the-noun to "operation", so should be easy to fix. 
> But this isn't an editorial comment, because I'm not sure what the 
> sentence is saying.

Spencer-reply: This works for me, with a slight preference for striking 
"layer", but either way would work for me.

Spencer-Reply: That would work for me - I know Brian needed help, too, and 
he's the one that needs to be happy. But we're within an RFC Editor note of 
finished on this.

> describing what protocols do and don't do, but I'm trying to read this 
> from the application's perspective, and having a hard time understanding 
> how (for example) an application that is trying to display what is 
> matching responds when the protocol only provides an indication that 
> something matched. You may say this is what the protocol developers are 
> supposed to worry about ("if you think applications will want to display 
> what matches, you'd better define the protocol so that this information is 
> returned"), and that's OK.

Exactly.

Spencer-reply: If you stuck "In this way, servers can use collations to 
support protocols that are defined such that 'x contains y' returns 
true-false." at the end of the paragraph, I'd "get it" without help.

Understandably.

Spencer-reply: Better not change it, because I was confused when I suggested 
it (this does not augur well for suggested changes). But please see next 
item.

> At the very least, I'm confused because I'm not sure if the top paragraph 
> in this extract describes the differences between i;ascii-casemap and 
> IMAP's built-in operations or is talking about something else.

Spencer-reply: Again, your explanation helped a lot and I'd like to hijack 
it into the document. Something like "In IMAP, the default collation is 
i;ascii-casemap, since its operations are understood to match IMAP's 
built-in operations."?

Spencer-reply: ack.

Spencer-reply: I guess my point was that this was extremely subtle for those 
of us who don't work with i18n comparison all day long. Perhaps 'Welsh names 
such as "L1wyd", when the Welsh consider "ll" a separate letter and sort it 
between "1" and "m"'? But you're going to have to figure out how to get 
"naïve" into an RFC... Perhaps your AD can step in front of this speeding 
bullet?

> From Spencer: I'm thinking that the "checking the SP SP "1" SP SP string 
> for correctness" also needs to be done pretty soon :-0

Arnt

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 Aug 2006 09:32:37 +0200

You have another choice with ï which is to write it in Unicode
notation, U+00EF (LATIN SMALL LETTER I WITH DIAERESIS) if I am not
mistaken.

Neither I nor Google knows how Unicode represents Welsh double L.
Maybe they don't? It's sorted between LY and M in the dictionary
at http://www.aber.ac.uk/~gpcwww/gpc_pdfs.htm#DANGOSEIRIAU

(BTW llywd and lloyd are different words, but as proper names
they are confused in English spelling.)

    Brian

Lisa Dusseault wrote:
>> Perhaps 'Welsh names such as "L1wyd", when the Welsh consider "ll"  a 
>> separate letter and sort it between "1" and "m"'? But you're  going to 
>> have to figure out how to get "naïve" into an RFC...  Perhaps your AD 
>> can step in front of this speeding bullet?
> 
> 
> Not sure there's a speeding bullet here.  I happened to know what  Arnt 
> meant by this stuff -- my own favourite example is sorting  "Canada" 
> followed by "canal" then "cantor", *then* followed by   "caña" and 
> finally "cañada" (last two with tildes above the n's if  you can't see 
> them), if you're using Spanish.
> 
> probably still make this more readable for people unfamiliar with  i18n 
> and sorting issues.  Here's a stab:
> 
>    * names such as "Llwyd" (which in Wales/Welsh or in Spanish  should 
> sort after single-L names like Lyza),
> ------------------------------------------------------------------------
>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 16 Aug 2006 10:53:26 +0200

Brian E Carpenter writes:
> Maybe they don't?

Right. That's really the only possibility given the Unicode design 
principles, which include round-trip compatibility with existing 
character sets.

> It's sorted between LY and M in the dictionary at 
> http://www.aber.ac.uk/~gpcwww/gpc_pdfs.htm#DANGOSEIRIAU
>
> (BTW llywd and lloyd are different words, but as proper names they are 
> confused in English spelling.)

Different as in not identical or not equal? A collation can easily be 
defined where they are equal ;)

Arnt

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 16 Aug 2006 01:56:26 -0700

> at http://www.aber.ac.uk/~gpcwww/gpc_pdfs.htm#DANGOSEIRIAU 
Unicode doesn't have a special character for "Ll" (and boy am I glad it 
doesn't....)
Unicode doesn't do language-specific collations all that well - CLDR does.

See these URLs:

http://en.wikipedia.org/wiki/Latin_alphabet#Collating_sequence_with_extensions
http://en.wikipedia.org/wiki/Common_Locale_Data_Repository

But at the moment http://unicode.org/cldr/data/common/collation/ doesn't 
seem to have a "cy" entry....

and that's probably far beyond what any of you (except for Martin, who 
probably knows it all already) ever wanted to know :-)

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 16 Aug 2006 11:30:29 +0200

> defined where they are equal ;)

They actually mean different things.

lloyd seems to have something to do with death, and
llwyd means grey-haired among many other things.

llowydd is also a word (and dd is another letter in Welsh, not just two d's).

Which is also more than you needed to know ;-)

     Brian

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 19:08:06 +0900

At 16:32 06/08/16, Brian E Carpenter wrote:
>You have another choice with $B}(Bwhich is to write it in Unicode

The point is that it's just represented as two "ASCII" Ls,
LL in upper case, Ll in title case, and ll in lower case.
Still sorting algorithms will do the right thing (if they
are told to sort for Welsh or traditional Spanish, and
know how to do that, that is).

Regards,     Martin.

>It's sorted between LY and M in the dictionary
>>>> saying sometFrom [email-masked] Wed Aug 16 06:44:38 2006
Date: Wed, 16 Aug 2006 19:08:06 +0900
To: Brian E Carpenter <[email-masked]>,
	Lisa Dusseault <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt
Cc: Arnt Gulbrandsen <[email-masked]>,
	General Area Review Team <[email-masked]>, [email-masked]

At 16:32 06/08/16, Brian E Carpenter wrote:
>You have another choice with $B}(Bwhich is to write it in Unicode

The point is that it's just represented as two "ASCII" Ls,
LL in upper case, Ll in title case, and ll in lower case.
Still sorting algorithms will do the right thing (if they
are told to sort for Welsh or traditional Spanish, and
know how to do that, that is).

Regards,     Martin.

>It's sorted between LY and M in the dictionary
>>>> saying sometFrom [email-masked] Wed Aug 16 06:44:38 2006
Date: Wed, 16 Aug 2006 19:08:06 +0900
To: Brian E Carpenter <[email-masked]>,
	Lisa Dusseault <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt
Cc: Arnt Gulbrandsen <[email-masked]>,
	General Area Review Team <[email-masked]>, [email-masked]

At 16:32 06/08/16, Brian E Carpenter wrote:
>You have another choice with $B}(Bwhich is to write it in Unicode

The point is that it's just represented as two "ASCII" Ls,
LL in upper case, Ll in title case, and ll in lower case.
Still sorting algorithms will do the right thing (if they
are told to sort for Welsh or traditional Spanish, and
know how to do that, that is).

Regards,     Martin.

>It's sorted between LY and M in the dictionary
>>> I would write na$B}W(Be if I could. I assume people know that naive and
>>> na$B}W(Be are both common spellings.
>>>
>>> Perhaps 'Welsh names such as "L1wyd", when the Welsh consider "ll"  a separate letter and sort it between "1" and "m"'? But you're  going to have to figure out how to get "na$B}W(Be" into an RFC...  Perhaps your AD can step in front of this speeding bullet?
>> 
>> Not sure there's a speeding bullet here.  I happened to know what  Arnt meant by this stuff -- my own favourite example is sorting  "Canada" followed by "canal" then "cantor", *then* followed by   "ca$B�B(B" and finally "ca$B�B(Bda" (last two with tildes above the n's if  you can't see them), if you're using Spanish.
>> Without the ability to put $B�(Bor $B}(Binto Internet Drafts, we could  
>> probably still make this more readable for people unfamiliar with  i18n and sorting issues.  Here's a stab:
>>    * names such as "Llwyd" (which in Wales/Welsh or in Spanish  should sort after single-L names like Lyza),

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:40:07 +0900

>>>
>>>
>>> I would write na=1B$B}W=1B(Be if I could. I assume people know that n=
aive and
>>> na=1B$B}W=1B(Be are both common spellings.
>>>
>>> Llwyd is thus spelt. The Welsh consider ll a separate letter and  sor=
t it
>>> between l and m.
>>>
>>> Spencer-reply: I guess my point was that this was extremely subtle =20
>>> for those of us who don't work with i18n comparison all day long. =20
>>> Perhaps 'Welsh names such as "L1wyd", when the Welsh consider "ll"  a=
 separate letter and sort it between "1" and "m"'? But you're  going to h=
ave to figure out how to get "na=1B$B}W=1B(Be" into an RFC...  Perhaps yo=
ur AD can step in front of this speeding bullet?
>>=20
>> Not sure there's a speeding bullet here.  I happened to know what  Arn=
t meant by this stuff -- my own favourite example is sorting  "Canada" fo=
llowed by "canal" then "cantor", *then* followed by   "ca=1B$B=81B=1B(B" =
and finally "ca=1B$B=81B=1B(Bda" (last two with tildes above the n's if  =
you can't see them), if you're using Spanish.
>> Without the ability to put =1B$B=81=01=1B(Bor =1B$B}=01=1B(Binto Inter=
net Drafts, we could =20
>> probably still make this more readable for people unfamiliar with  i18=
n and sorting issues.  Here's a stab:
>> Use with natural language  is often inappropriate: even though the =20
>> collation apparently supports  languages such as Italian and English, =
=20
>>     * words such as "naive" (if spelled with an accent, the accented =20
>>    * names such as "Llwyd" (which in Wales/Welsh or in Spanish  should=
>>    * strings containing euro and pound sterling symbols, quotation =20
>>=20
>> ----------------------------------------------------------------------=
--

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked] =
   =20

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:40:07 +0900

aive and
t it
net Drafts, we could =20
=20
--

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:40:07 +0900

Some small comments below:

At 07:00 06/08/16, Arnt Gulbrandsen wrote:
>Spencer Dawkins writes:
>>> I was selected as General Area Review Team reviewer for this specification (for background on Gen-ART, please see http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).
>>
>> This is a re-review, my previous review was for 06, with Scott as shepherding AD, before IETF 65. I'm  reading the deltas from 06 (in the spirit of not finding new problems with previously-reviewed text).
>>
>> Summary: Again, nearly ready for publication as Proposed Standard, with some (new) items that do need to be addressed before publication.
>>
>> Thanks,
>>
>> Spencer
>>
>> Review Comments:
>>
>> 2.2.  Purpose
>>
>> I am just barely able to parse this sentence so that it's not a sentence fragment. I think the problem is that "functions" is being used as a verb and as a noun in the same sentence. I saw later in the document that you had changed "function"-the-noun to "operation", so should be easy to fix. But this isn't an editorial comment, because I'm not sure what the sentence is saying.
>
>    Collations provide a multi-protocol abstraction layer for comparison functions so that these

I'd leave in layer, but take out the first "multi-protocol", to give
"Collations provide an abstraction layer for comparison functions so
that these comparison functions can be used in multiple protocols."

>> What does the last sentence mean? (Brian Carpenter asked me, so he doesn't know, either).
>
>It means: I'm not defining what these three return values are called, only naming them so I can talk about them. If you implement this, you're free to call them anything you want. You can use a C++ enum type, or -1/0/1, or whatever.
>
>I could say ォThe return values of the equality test are called "match", "no-match" and "undefined" in this document.サ Would that be clear enough?

That would also be fine by me.

>> Is it obvious to everyone except me that leading zeros are ignored? The examples giving a little further down say so [email-masked]>
Cc: General Area Review Team <[email-masked]>,
	Lisa Dusseault <[email-masked]>, [email-masked]

Some small comments below:

At 07:00 06/08/16, Arnt Gulbrandsen wrote:
>Spencer Dawkins writes:
>>> I was selected as General Area Review Team reviewer for this specification (for background on Gen-ART, please see http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).
>>
>> This is a re-review, my previous review was for 06, with Scott as shepherding AD, before IETF 65. I'm  reading the deltas from 06 (in the spirit of not finding new problems with previously-reviewed text).
>>
>> Summary: Again, nearly ready for publication as Proposed Standard, with some (new) items that do need to be addressed before publication.
>>
>> Thanks,
>>
>> Spencer
>>
>> Review Comments:
>>
>> 2.2.  Purpose
>>
>> I am just barely able to parse this sentence so that it's not a sentence fragment. I think the problem is that "functions" is being used as a verb and as a noun in the same sentence. I saw later in the document that you had changed "function"-the-noun to "operation", so should be easy to fix. But this isn't an editorial comment, because I'm not sure what the sentence is saying.
>
>    Collations provide a multi-protocol abstraction layer for comparison functions so that these

I'd leave in layer, but take out the first "multi-protocol", to give
"Collations provide an abstraction layer for comparison functions so
that these comparison functions can be used in multiple protocols."

>> What does the last sentence mean? (Brian Carpenter asked me, so he doesn't know, either).
>
>It means: I'm not defining what these three return values are called, only naming them so I can talk about them. If you implement this, you're free to call them anything you want. You can use a C++ enum type, or -1/0/1, or whatever.
>
>I could say ォThe return values of the equality test are called "match", "no-match" and "undefined" in this document.サ Would that be clear enough?

That would also be fine by me.

>> Is it obvious to everyone except me that leading zeros are ignored? The examples giving a little further down say so [email-masked]>
Cc: General Area Review Team <[email-masked]>,
	Lisa Dusseault <[email-masked]>, [email-masked]

Some small comments below:

At 07:00 06/08/16, Arnt Gulbrandsen wrote:
>Spencer Dawkins writes:
>>> I was selected as General Area Review Team reviewer for this specification (for background on Gen-ART, please see http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).
>>
>> This is a re-review, my previous review was for 06, with Scott as shepherding AD, before IETF 65. I'm  reading the deltas from 06 (in the spirit of not finding new problems with previously-reviewed text).
>>
>> Summary: Again, nearly ready for publication as Proposed Standard, with some (new) items that do need to be addressed before publication.
>>
>> Thanks,
>>
>> Spencer
>>
>> Review Comments:
>>
>> 2.2.  Purpose
>>
>> I am just barely able to parse this sentence so that it's not a sentence fragment. I think the problem is that "functions" is being used as a verb and as a noun in the same sentence. I saw later in the document that you had changed "function"-the-noun to "operation", so should be easy to fix. But this isn't an editorial comment, because I'm not sure what the sentence is saying.
>
>    Collations provide a multi-protocol abstraction layer for comparison functions so that these

I'd leave in layer, but take out the first "multi-protocol", to give
"Collations provide an abstraction layer for comparison functions so
that these comparison functions can be used in multiple protocols."

>> What does the last sentence mean? (Brian Carpenter asked me, so he doesn't know, either).
>
>It means: I'm not defining what these three return values are called, only naming them so I can talk about them. If you implement this, you're free to call them anything you want. You can use a C++ enum type, or -1/0/1, or whatever.
>
>I could say ォThe return values of the equality test are called "match", "no-match" and "undefined" in this document.サ Would that be clear enough?

That would also be fine by me.

>> Is it obvious to everyone except me that leading zeros are ignored? The examples giving a little further down say so - is making this point in examples normative enough?
>
>It's specified in 2244, so I don't think it's very important. This document merely registers a collation which has been specified for a decade and implemented in many products.

I didn't know that. If that's the case, we should provide a reference.
(we have RFC 2244 in the list of references, but no pointer to it
from 9.1, and we might want to make RFC 2244 normative).

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:58:27 +0900

At 08:12 06/08/16, Lisa Dusseault wrote:

>>>OK, this may be inadvertantly funny - are "naive" and "Llwyd" supposed to include a non-ascii character, or is that sentence saying something else? (Welcome to the world of the RFC Editor)

>Without the ability to put $B�(Bor $B}(Binto Internet Drafts, we could probably still make this more readable for people unfamiliar with i18n and sorting issues.  Here's a stab:
>
>Use with natural language  is often inappropriate: ev is making this point in examples normative enough?
>

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:58:27 +0900

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:58:27 +0900

>Use with natural language  is often inappropriate: even though the collation apparently supports  languages such as Italian and English,

It apparently supports English, but Italian comes with accents
(a bit less than French, but e.g. both "is" and "and" are written
as "e", the former with an accent grave), so that's a bad
example. I think the examples here are usually languages like
Hawai'ian or Swahili.

Regards,    Martin.

>    * words such as "naive" (if spelled with an accent, the accented character could push the word to the wrong spot in a sorted list), 
>   * strings containing euro and pound sterling symbols, quotation marks, dashes/hyphens, etc.
>
>Lisa

en though the collation apparently supports  languages such as Italian and English,

It apparently supports English, but Italian comes with accents
(a bit less than French, but e.g. both "is" and "and" are written
as "e", the former with an accent grave), so that's a bad
example. I think the examples here are usually languages like
Hawai'ian or Swahili.

Regards,    Martin.

>    * words such as "naive" (if spelled with an accent, the accented character could push the word to the wrong spot in a sorted list), 
>   * strings containing euro and pound sterling symbols, quotation marks, dashes/hyphens, etc.
>
>Lisa

en though the collation apparently supports  languages such as Italian and English,

It apparently supports English, but Italian comes with accents
(a bit less than French, but e.g. both "is" and "and" are written
as "e", the former with an accent grave), so that's a bad
example. I think the examples here are usually languages like
Hawai'ian or Swahili.

Regards,    Martin.

>    * words such as "naive" (if spelled with an accent, the accented character could push the word to the wrong spot in a sorted list), 
>   * strings containing euro and pound sterling symbols, quotation marks, dashes/hyphens, etc.
>
>Lisa

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Aug 2006 17:25:42 +0900

>I am just barely able to parse this sentence so that it's not a sentence fragment. I think the problem is that "functions" is being used as a verb and as a noun in the same sentence. I saw later in…

"comparison functions" is nominal in both cases. What's missing is the verb after
the first word, e.g.:

Collations *provide an* abstraction layer...     or
Collations *function as* an abstraction layer

or so.

>What does the last sentence mean? (Brian Carpenter asked me, so he doesn't know, either).

I suggest changing the sentence:
    This is not a specification, merely a choice of phrasing.
To something like:
    Each protocol has to specify how to represent these values.
or something in that direction. Also, maybe adding a sentence
to 5.2 mentioning this would help overall understanding.

>Hmmm. I am trying to remember that you're not defining a protocol, only describing what protocols do and don't do, but I'm trying to read this from the application's perspective, and having a hard time understanding how (for example) an application that is trying to display what is matching responds when the protocol only provides an indication that something matched. You may say this is what the protocol developers are supposed to worry about ("if you think applications will want to display what matches, you'd better define the protocol so that this information is returned"), and that's OK. I'm just struggling a bit here.

To take a specific example, assume that IMAP provides a function to select
all the emails that contain a specific string (IMAP probably does provide
such a function, but I don't know for sure). IMAP will only tell you which
mails matched, and then your MUA will have to download these to show the
details.

>EDITORIAL: I'm guessing that the previous paragraph should be moved up one? At the very least, I'm confused because I'm not sure if the top paragraph in this extract describes the differences between i;ascii-casemap and IMAP's built-in operations or is talking about something else.

I'm reading it that i;ascii-casemap is a rather separate thing. The
second (one line) para applies to the first, and so the order of the
paras shouldn't be changed, I guess. But I'm not an IMAP expert.

Apart from that, there is a small typo: change
"operations most closely resembles" to "operations most closely resemble".

>Is it obvious to everyone except me that leading zeros are ignored? The examples giving a little further down say so - is making this point in examples normative enough?

It's not only the examples that say so, but also the details of the
equality operation. But I agree that adding "Leading zeroes are allowed
but ignored." or so to the first
paragraph would help.

On a different level, I'm not at all sure that this collation needs to
be defined on octet strings. This is also in conflict with section
4.1, where it says "The i;ascii-numeric (Section 9.1) operation operates
on numbers." The best way to word things is to have this collation work
on characters, but sort them as numbers. The relevant sentence in
section 4.1 can just be removed. The first paragraph of 9.1 should be
reworded as follows:

   The "i;ascii-numeric" collation is a simple collation intended for
   use with arbitrary sized unsigned decimal integer numbers.
   These numbers are represented by digits from the US-ASCII range
   (U+0030 to U+0039).  Before converting from string to integer, the input
   string is truncated at the first non-digit character.  All input is
   valid; strings which do not start with a digit represent positive
   infinity. Leading zeroes are allowed but ignored.

>OK, this may be inadvertantly funny - are "naive" and "Llwyd" supposed to include a non-ascii character, or is that sentence saying something else? (Welcome to the world of the RFC Editor)

A separate point: "to to" in the first line should be fixed.
I'll get back to this in response to Lisa.

>From Brian: Surely this needs to be done?

Yes, this needs to be done. We should replace the reference with
draft-ietf-ltru-registry-14.txt (and maybe also draft-ietf-ltru-matching-15.txt,
both of these together will be BCP 47; don't know whether/how we can
refer to half of a BCP :-(.

>From Spencer: I'm thinking that the "checking the SP SP "1" SP SP string for correctness" also needs to be done pretty soon :-0

I'd be glad to check, but I'm not sure what needs to be checked.
Everything looks good, except that:
- In the string that is being searched, spaces seem to be collapsed,
  i.e. the draft says " 1 " (SP SP "1" SP SP) where it should say
  "  1  " (SP SP "1" SP SP). This may be an effect of XML2RFC
  or some other production effect. In the case of XML2RFC, I guess
  there should be a workaround. If we don't find a workaround at
  this stage, we have to make a note to the RFC editor.
  (and we need a note to ourselves to make sure we check this again,
   too).
- I'd personally change "the substring operation could" to
  "the substring operation should", but that may be too late
  (and lowercase 'should' is better avoided in RFCs).

Regards,    Martin.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 16 Aug 2006 13:22:46 +0200

Spencer Dawkins writes:
> Spencer-Reply: That would work for me - I know Brian needed help, too, 
> and he's the one that needs to be happy. But we're within an RFC 
> Editor note of finished on this.

I would wish more readers than just Brian to understand the document ;)

Brian?

> Some current-day protocols are defined such that 'x contains y' 
> returns true/false. Servers implementing such protocols should still 
> be able to use collations.
>
> Spencer-reply: If you stuck "In this way, servers can use collations 
> to support protocols that are defined such that 'x contains y' 
> returns true-false." at the end of the paragraph, I'd "get it" 
> without help.

Fine, except that I'll try to find some wording that avoids 'contains'. 
Reviewer request, some months ago. "is a substring of", perhaps.

...
> Spencer-reply: Again, your explanation helped a lot and I'd like to 
> hijack it into the document. Something like "In IMAP, the default 
> collation is i;ascii-casemap, since its operations are understood to 
> match IMAP's built-in operations."?

Fine.

> decade and implemented in many products.

But maybe, just maybe, section 9 needs to be clarified as such:

    This section describes an initial set of collations for the collation
    registry. These collations were originally specied in RFC 2244.

> Spencer-reply: I guess my point was that this was extremely subtle for 
> those of us who don't work with i18n comparison all day long. Perhaps 
> 'Welsh names such as "L1wyd", when the Welsh consider "ll" a separate 
> letter and sort it between "1" and "m"'? But you're going to have to 
> figure out how to get "naïve" into an RFC... Perhaps your AD can step 
> in front of this speeding bullet?

Lisa suggested rewording to use bullet points and some parenthetical 
elaboration. That's ok with me. I'll look at some RFCs and make the 
bullets look the conventional way.

If the RFC editor quibbles about naïve, it can also be deleted. There 
are enough points in that list that any single one can be deleted 
without harm. People are just as easily persuaded by five reasons as by 
six.

Arnt

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 17 Aug 2006 10:41:37 +0200

> I would wish more readers than just Brian to understand the document ;)

I can understand the new sentence :-)

     Brian
>> to support protocols that are defined such that 'x contains y' returns

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 18 Aug 2006 19:36:18 +0200

I apparently forgot to reply back in March. Sorry.

Spencer Dawkins <[email-masked]>
> I was thinking about this from the client's perspective (if the collation 
> doesn't report "error", how does the client know to report anything to 
> someone who might be able to fix the input?).

There's no point in doing that. Take my home, IMAP. If an IMAP client
tells the server to sort the messages by subject, and one message's
subject contains an illegal ISO-2022-JP sequence, the collation will
return "error" a few times. That's not a bug in the client or server,
it's just another broken mail message. Usually it's spam. Routine.

> No, I'm sorry that I wasn't clearer. I was trying to say that if the 
> designated expert fumbles the evaluation, perhaps talking to an APPS-area 
> AD would be sufficient to resolve most appeals, without involving the rest 
> of the IESG, unless the APPS-area AD fumbles the first-level appeal.

I made this change.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 18 Aug 2006 19:51:56 +0200

Cc'ers: Just Hit Delete.

Spencer: This lists what I change in -14 and what not.

> I'm not sure what the sentence is saying.

Fixed. There were missing words.

> What does the last sentence mean? (Brian Carpenter asked me, so he 

I changed it as discussed.

> EDITORIAL: I'm guessing that the previous paragraph should be moved up one?

Not moved.

> i;ascii-casemap and IMAP's built-in operations or is talking about 

New text is as suggested by you: «In IMAP, the default collation is 
i;ascii-casemap, because its operations are understood to match's 
IMAP's built-in operations.»

> something else? (Welcome to the world of the RFC Editor)

Changed as per Lisa's suggestion.

> string for correctness" also needs to be done pretty soon :-0

I don't think I'm going to use xml2rfc again. I hate seeing "In section 
3.5" when I don't see 3.5 in the source. Having to redo this spacery 
time and time again is also painful.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 18 Aug 2006 19:53:03 +0200

I noticed this one too late to mention in the other answer.

> I'm just struggling a bit here.

Appended text: «In this way, collations can be used with protocols that 
are defined such that "x is substring of y" returns true-false.»

Arnt

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 18 Aug 2006 12:59:17 -0500

Hi, Arnt,

These look good.

Thanks,

Spencer

----- Original Message ----- 
From: "Arnt Gulbrandsen" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; "General Area Review 
Team" <[email-masked]>; "Lisa Dusseault" <[email-masked]>
Sent: Friday, August 18, 2006 12:51 PM
Subject: Re: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt

Cc'ers: Just Hit Delete.

Not moved.

Arnt

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 18 Aug 2006 12:59:43 -0500

This also looks good.

Thanks,

Spencer

----- Original Message ----- 
From: "Arnt Gulbrandsen" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; "General Area Review 
Team" <[email-masked]>; "Lisa Dusseault" <[email-masked]>
Sent: Friday, August 18, 2006 12:53 PM
Subject: Re: [Gen-art] Gen-ART Review of draft-newman-i18n-comparator-13.txt

Spencer Dawkins writes:
> returned"), and that's OK. I'm just struggling a bit here.

Arnt

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 18 Aug 2006 11:19:40 -0700

On Aug 18, 2006, at 11:03 AM, Spencer Dawkins wrote:

> If you guys are OK here, I'm OK.

I'm OK Enough here.  It's hard to make things truly general.  We  
really need to solve the IMAP need, and the authors have done a good  
job of trying to make it useful elsewhere.  Most of the work of  
making the registry useful elsewhere would belong in other documents  
anyway.  For example, if IRIS wanted to use collations to sort  
registry entries and report errors, the reporting of errors during  
sorting would have to be part of the IRIS proposal anyway.

Then if that fails there's always new RFC numbers for later versions  
of this work.

Lisa

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 18 Aug 2006 13:03:35 -0500

Hi, Arnt,

>> of the IESG, unless the APPS-area AD fumbles the first-level appeal.

Likewise (and I noted it when I reviewed -13 - thanks!)

Spencer

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 19 Aug 2006 12:33:05 +0200

Lisa Dusseault writes:
> really need to solve the IMAP need,
> ...

And the Sieve need. That's arguably even more pressing. 3028bis blocks 
on draft-newman-i18n-comparator and it's difficult to do much more than 
«hello world» in sieve without using a collation.

Arnt