Subject: [VCARDDAV] Questions about vCard 4.0 (draft rev-11)

From: Daisuke Miyakawa <[email-masked]>
Date: Tue, 22 Jun 2010 11:27:54 +0900

Hi,

(Please tell me when I'm saying something inappropriate in this ML. I'm new
to the community)
I'm a Japanese interested in vCard 4.0, and have 3 questions for
implementing parser/composer.

1) case sensitiveness
>   Property names and parameter names are case insensitive (e.g., the
>   property name "fn" is the same as "FN" and "Fn").

1-a) Rationale
I'd like to know why this draft changed the behavior around
case-sensitiveness of property name.
My understanding is that vCard 2.1 and 3.0 implicitly preferred upper cases
(though there seems no mention).
When developing a vCard parser for some embedded devices, I found using
case-sensitive comparison was a little costly.
At least, I think it is worth mentioning that upper cases would be better,
as a lot of old vCard files (both vCard 2.1 and vCard 3.0) seem to use that
form.

1-b) How about BEGIN and END?
There's no mention about how BEGIN/END should be emitted. Is it ok to use
"begin:vcard"?

2) VERSION

> Special notes: The property MUST be present in the vCard object.
>                The value MUST be "4.0" if the vCard corresponds to this
>                specification.

When parsing vCard files, I found it should be better to have "VERSION" line
"just after" "BEGIN:VCARD" line, as parsers cannot know which version should
be used.
My understanding is vCard 2.1 even allows vCard without "VERSION:2.1" line.
When parsing some complicated text with backslash and Quoted-Printable (only
allowed in vCard 2.1),
the parser cannot determine how a target property is treated (error or not).

e.g.
BEGIN:VCARD
FN:Daisuke;Miyakawa
AGENT:BEGIN:VCARD\nFN:Cool!\nEND:VCARD
VERSION:3.0  <-- The parser found that this vCard is NOT in vCard 4.0 but in
vCard 3.0, so AGENT is valid.
END:VCARD  <-- The parser found there's no N, which is valid in vCard 4.0
but not in vCard 3.0
(error?)

I think it would be better to explicitly mention that "VERSION:4.0 MUST be
just after BEGIN:VCARD line" for better comfortability of actual
implementation.

3) SORT-STRING
Some Japanese people said that we sometimes use organization (ORG) for
sorting, but we need some sort string for it.
However, the current draft says "SORT-STRING" is just for FN or N, not for
ORG.
Is there any discussion about that kind of topic?

Thanks,

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: Simon Perreault <[email-masked]>
Date: Tue, 22 Jun 2010 09:05:05 -0400

On 2010-06-21 22:27, Daisuke Miyakawa wrote:
> (Please tell me when I'm saying something inappropriate in this ML. I'm
> new to the community)

Good! This is the right place to ask such questions.

> My understanding is that vCard 2.1 and 3.0 implicitly preferred upper
> cases (though there seems no mention).

See RFC 2425 section 5.8.2:

   Type names and parameter names are case insensitive (e.g., the type
   name "fn" is the same as "FN" and "Fn").

The preference is irrelevant. Your parser needs to be able to understand
both forms.

> case-sensitive comparison was a little costly.

You must have been playing with some REALLY REALLY SMALL embedded
devices! ;) I've never encountered or heard of a platform where
case-insensitive comparison would make any measurable difference.

> At least, I think it is worth mentioning that upper cases would be
> better, as a lot of old vCard files (both vCard 2.1 and vCard 3.0) seem
> to use that form.

I'm sorry, I think this would be wrong. The right thing to say would be
that parsers need to understand both upper and lower case, but this is
already part of the text.

> There's no mention about how BEGIN/END should be emitted. Is it ok to
> use "begin:vcard"?

No. Unless specified otherwise, property values are case sensitive.
Therefore "begin:VCARD", "BEGIN:VCARD", and "Begin:VCARD" would be
valid, but "begin:vcard" would not. The ABNF in section 6.1.1 makes this
clear.

> When parsing vCard files, I found it should be better to have "VERSION"
> line "just after" "BEGIN:VCARD" line, as parsers cannot know which
> version should be used.
> My understanding is vCard 2.1 even allows vCard without "VERSION:2.1"
> line. When parsing some complicated text with backslash and
> Quoted-Printable (only allowed in vCard 2.1),
> VERSION:3.0  <-- The parser found that this vCard is NOT in vCard 4.0
> but in vCard 3.0, so AGENT is valid.
> END:VCARD  <-- The parser found there's no N, which is valid in vCard
> 4.0 but not in vCard 3.0
> (error?)
> 
> I think it would be better to explicitly mention that "VERSION:4.0 MUST
> be just after BEGIN:VCARD line" for better comfortability of actual
> implementation.

I tend to agree. Unless people disagree, this will be part of the next
revision.

> However, the current draft says "SORT-STRING" is just for FN or N, not
> for ORG.

You just started the discussion! ;)

I think this is valid and important. However, I would really like to see
a real-life example to ensure we stay grounded. Could you please write
one? I suppose it will be hard to translate to English, but please try...

Simon
-- 
NAT64/DNS64 open-source --> http://ecdysis.viagenie.ca
STUN/TURN server        --> http://numb.viagenie.ca
vCard 4.0               --> http://www.vcarddav.org

---

From: "Javier Godoy" <[email-masked]>
Date: Tue, 22 Jun 2010 18:16:31 -0300

Hi,

> valid, but "begin:vcard" would not. The ABNF in section 6.1.1 makes this

I only found the following statement in section 3.2 "Parameter values MAY be 
case sensitive or case insensitive, depending on their definition.", but it 
refers to parameter values, not to property values. I found nothing about 
property values, but this is one of the few cases where a case-sensitive 
string is specified as literal , so I think we do not need a general 
interpretation, but we need to be explicit about case sensitivity..

In section 3.2, ABNF is given as
 vcard = "BEGIN" ":" "VCARD" CRLF

And in section 6.1
 BEGIN-value = "VCARD"

>From section 2.3 of RFC 5234
 ABNF strings are case insensitive .
 To specify a rule that is case sensitive, specify the characters 
individually.
 For example:
   rulename    =  %d97 %d98 %d99

So the ABNF above should be

BEGIN-value = %d86 %d67 %d65 %d82 %d68; uppercase "VCARD"

(and the same in section 3.2).

Maybe we also want to review the ABNF for "Z" and "T" in zone and timestamp.

> I tend to agree. Unless people disagree, this will be part of the next

+1

Regards

Javier

---

From: Simon Perreault <[email-masked]>
Date: Tue, 22 Jun 2010 18:59:17 -0400

> BEGIN-value = %d86 %d67 %d65 %d82 %d68; uppercase "VCARD"

Wow! I didn't know that! Thanks!

Ok so we have two options:

1. Leave the ABNF as is but state that the value is case insensitive.
2. Change the ABNF and state that the value is case sensitive.

I think I would prefer 1.

> Maybe we also want to review the ABNF for "Z" and "T" in zone and

Definitely!

Simon

---

From: Daisuke Miyakawa <[email-masked]>
Date: Wed, 23 Jun 2010 11:03:27 +0900

Hi,

Thanks for your quick response!

> I think I would prefer 1.

I'll honor your decisions about this topic. Personally +1 to 1.

> one? I suppose it will be hard to translate to English, but please try...

I'm asking my friends knowing better about Japanese Contacts apps for the
examples (he actually talked about this topic!).
Stay tuned..

Personally, I suppose it might be better to accept multiple SORT-STRING
(0,n) and ask vCard composers to explicitly add PREF when they want to add
preference information to them.

e.g.

BEGIN:VCARD
VERSION:4.0
name.N;宮川;大輔;;;
name.FN:宮川 大輔
org.ORG:日本の組織                # "Japanese organization"
birthday.BDAY:19820616
name.SORT-STRING;PREF=90:みやかわ;だいすけ;;;
org.SORT-STRING;PREF=80;にほんのそしき
birthday.SORT-STRING;PREF=70:1982;06;16
END:VCARD

The examples above is not real; what I'm actually developing cannot utilize
those SORT-STRINGs, but theoretically developers may be able to utilize them
without any confusion.
Even when they cannot recognize multiple SORT-STRINGs, they can pick up one
with the highest PREF value.
Some applications should be able to sort contact list using each SORT-STRING
without any X- something.

Thanks,

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: "Javier Godoy" <[email-masked]>
Date: Wed, 23 Jun 2010 02:05:55 -0300

> I think I would prefer 1.

Allowing values to be case insensitive may work for BEGIN:VCARD and END:VCARD.
I don't have preference between either approach.

About case sensitiviy for "T" and "Z", they are defined in section 3.4.3
"Characters used as designators" of ISO 8601:2004 as: «
In representations the following characters are used as designators:
   [Z] is used as UTC designator.
   [T] is used as time designator
»

It seems they are specified as uppercase (at least, there is no example with
lowercase designators), so in this case we should the ABNF.

utc-designator = "Z"
zone   = utc-designator / utc-offset

time-designator = "T"
date-time = date-noreduc  time-designator time-notrunc
timestamp = date-complete time-designator time-complete
date-and-or-time = date-time / date / time-designator time

Best Regards

Javier

---

From: "Javier Godoy" <[email-masked]>
Date: Wed, 23 Jun 2010 04:43:19 -0300

> preference information to them.

Since ORG is (0,n) now it makes sense to define SORT-STRING as (0,n), but each
ORG should be related to at most one SORT-STRING, thus preference is not
required (the preferred organization SORT-STRING is that of the preferred ORG
property)

>with the highest PREF value.

BEGIN:VCARD
VERSION:4.0
name.N;宮川;大輔;;;
name.FN:宮川 大輔
org1.ORG;LANGUAGE=ja;PREF=80:日本の組織  # "Japanese organization"
org1.ORG;LANGUAGE=en;PREF=80:Japanese organization
org2.ORG;LANGUAGE=ja;PREF=70: もう一つの日本の組織
org2.ORG;LANGUAGE=en;PREF=70:Another Japanese organization
name.SORT-STRING:みやかわ;だいすけ
org1.SORT-STRING:にほんのそしき
org2.SORT-STRING:もう ひとつ の にっぽん の そしき
NOTE: I don't understand Japanese...
 I'm just doing my best to understand this issue.
END:VCARD

Note that the case of ORG is different from N, because language dependent
values may be specified. Maybe SORT-STRING should have a LANGUAGE parameter,
so that several SORT-STRING may be provided (when displaying English, it has
no sense to sort "Japanese organization" under "にほんのそしき", even if the
implementation supports it)

Besides, how are non-Japanese strings sorted among Japanese ones?

Does it help to interoperability if romanized SORT-STRINGs are provided
along/instead kana? Is it redundant? Is it ambiguous?
 SORT-STRING;にほんのそしき
 SORT-STRING;nippon no soshiki

What would happen if some vCard authors use one approach and others use the
other?

Best Regards

Javier

---

From: Daisuke Miyakawa <[email-masked]>
Date: Wed, 23 Jun 2010 17:32:15 +0900

> Besides, how are non-Japanese strings sorted among Japanese ones?

My understanding is "not defined formally".

- What icu library does and what my phone is doing looks different toward
ascii handling (ascii < kana in icu while kana < ascii in my phone).
- China and Japan uses same unicode codepoints with different
pronounciation.
- The comparator (collator, in icu term) would become huge if we define one
unified sort order toward every characters.

I'd like to know some authorities' answer.

> Does it help to interoperability if romanized SORT-STRINGs are provided
along/instead kana?
Hmm, sorry I don't think so.

e.g.

- かんべ (Kambe)
- べっしょ (Bessyo)
- よこはま (Yokohama)

The list above should be sorted as かんべ, べっしょ, よこはま from the view of how they
are phonetically expressed, while the order would become べっしょ (Bessyo), かんべ
(Kambe), よこはま (Yokohama) in alphabets. I suppose there's no relevance. Also,
Hepburn system may not be understood by some Japanese.

My understanding is that software/devices supporting vCard 4.0 should be
able to handle UTF-8, thus (at least) Unicode, so they should be able to
handle languages they want to support, but no software/device can understand
and correctly sort ALL languages in Unicode.
I don't think we need to prepare for cases other than kana, as for Japanese.
I'm not sure about the other languages.

> Is it redundant?
>From the view of one Japanese engineer, it looks redundant.
It may help for English speakers, but it should be simply "Japanese
organization" (Each organization would have English name).

> Is it ambiguous?
If we mix two types of SORT-STRINGs, it would become ambiguous. If we have
consistency which should be prefered, it should become clear.

> other?

I'm not 100% sure, but Japanese vCard authors implicitly understand that
SORT-STRING is same as X-PHONETIC- something used in some softwares and
"SOUND;X-IRMC-N", which has been used in vCard 2.1 to expressing phonetic
names or "Yomigana" (読み仮名, よみがな).

Japanese are so accustomed to phonetic sort that I could not understand
first why vCard 4.0 chose the name "SORT-STRING" instead of "PHONETIC-NAME"
(then, I learnt other countries often have other way of sorting =)

I'd like to know the other people's opinion.

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: Simon Perreault <[email-masked]>
Date: Wed, 23 Jun 2010 08:32:19 -0400

On 2010-06-23 03:43, Javier Godoy wrote:
> Since ORG is (0,n) now it makes sense to define SORT-STRING as (0,n),

I disagree. FN is (1,n). We're still in the same situation.

Suppose you have multiple FN properties. Which one are you going to use
for sorting? The first one? It wouldn't make sense to pick e.g. the
second one because there's nothing guaranteeing that all second FN
properties in all vCards have anything in common. Also, the order of
properties can be changed by any intermediate system since order doesn't
matter.

Since we're in the same situation with ORG, I don't see why we wouldn't
solve it exactly in the same way with a separate property, e.g.
ORG-SORT-STRING.

Now, one thing that I like about your example is the use of the LANGUAGE
tag. We need to at least add an example where the LANGUAGE tag is also
applied to the SORT-STRING property.

Example:

ORG;LANGUAGE=ja:日本の組織  # "Japanese organization"
ORG;LANGUAGE=en:Japanese organization
ORG-SORT-STRING;LANGUAGE=ja:にほんのそしき

(I'll figure out an equivalent example in French since we can't have
Japanese characters in RFCs...)

Simon

---

From: Daisuke Miyakawa <[email-masked]>
Date: Wed, 23 Jun 2010 21:52:33 +0900

ORG-SORT-STRING looks nice as for this discussion, which should solve what
I'm concerning now.

According to the image bellow:
http://itpro.nikkeibp.co.jp/article/Watcher/20070925/282926/?SS=imgview&FD=1213653566&ST=win

Active Directory has at least 5 custom values for phonetic name (almost same
as sort string in vCard 4.0).
- for first name
- for given name
- for display name
- for company name
- for department name

SORT-STRING includes the first two, and ORG-SORT-STRING includes the last
two.
I'm not sure how extent phonetic name for display name is important
nowadays.

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 23 Jun 2010 11:39:38 -0400

Hi Daisuke,

--On June 23, 2010 9:52:33 PM +0900 Daisuke Miyakawa <[email-masked]> 
wrote:

> ORG-SORT-STRING looks nice as for this discussion, which should solve
> what I'm concerning now.
>
>
> Active Directory has at least 5 custom values for phonetic name (almost
> same as sort string in vCard 4.0).
> - for first name
> - for given name
> - for display name
> - for company name
> - for department name
>
>

I worry that having multiple XXX-SORT-STRING properties is just going to be 
confusing and that we may need more - one for each field a client sorts on. 
The alternative would be to define a "SORT-AS" property parameter that 
could be applied to each property that could be sorted on.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 23 Jun 2010 18:04:10 +0200

On 23.06.2010 00:59, Simon Perreault wrote:
> ...

You can also say:

   BEGIN-value = %d86.67.65.82.68; uppercase "VCARD"

if you are concerned about verbosity...

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 23 Jun 2010 12:37:28 -0400

Hi Julian,

> if you are concerned about verbosity...

I think we should just allow any case for "VCARD". Otherwise, if we are 
concerned with case-sensitivity of this "property", then we would need to 
look at all the other similar ones - ones with an "enumerated" value like 
KIND. Would we have to define those case-sensitively in the ABNF too?

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 23 Jun 2010 18:43:33 +0200

On 23.06.2010 18:37, Cyrus Daboo wrote:
> Hi Julian,
>
> --On June 23, 2010 6:04:10 PM +0200 Julian Reschke
> <[email-masked]> wrote:
>
> concerned with case-sensitivity of this "property", then we would need
> to look at all the other similar ones - ones with an "enumerated" value
> like KIND. Would we have to define those case-sensitively in the ABNF too?

I have no opinion whether they should allow case-insensitive matching; 
that's something for you, the subject matter experts...

Best regards, Julian

---

From: "Javier Godoy" <[email-masked]>
Date: Wed, 23 Jun 2010 14:52:50 -0300

> I disagree. FN is (1,n). We're still in the same situation.

I though that FN was (1,n) because the name of the same person may be written 
in different languages (or scripts), and not because one person could have 
different names in the same language. On the other side, ORG is (0,n) not only 
because the name of the same organization may be given in different languages, 
but also because the same individual may be related to several organizations.

> properties can be changed by any intermediate system since order doesn't

I would use the preferred one, if the PREF parameter is provided. In case of 
tie, or if no SORT-STRING is provided,  would choose any of them (i.e. 
undefined)
But, why did we allow FN to be (0,n)?

> solve it exactly in the same way with a separate property, e.g.

If one wouldn't need to store the kana for every ORG, that would be fine. As 
explained below, a conservative guess would be that it might not be the case.

org1.ORG;LANGUAGE=ja;PREF=80:日本の組織  # "Japanese organization"
org1.ORG;LANGUAGE=en;PREF=80:Japanese organization
org2.ORG;LANGUAGE=ja;PREF=90: もう一つの日本の組織
org2.ORG;LANGUAGE=en;PREF=90:Another Japanese organization
name.SORT-STRING:みやかわ;だいすけ
org1.SORT-STRING:にほんのそしき
org2.SORT-STRING:もう ひとつ の にっぽん の そしき

Would collate by にほんのそしき (Kana for "Japanese organization"). If it is 
changed to:

It would collate by もう ひとつ の にっぽん  の そしき (kana for "Another 
Japanese organization")

> applied to the SORT-STRING property.

LANGUAGE is not a parameter for SORT-STRING. Adding a LANGUAGE would allow 
different SORT-STRINGs for different languages while preserving (0,1) 
cardinality.

> Japanese characters in RFCs...)

I don't think there is an equivalent example in US-ASCII.

As I understand this issue, ORG and FN use unsortable ideograms (kanji), while 
SORT-STRING uses a syllabic script (kana) which is appropriate for collation. 
Personal names are represented in kanji in normal, everyday writing, while 
their collation sequence in a directory (or a catalog) follows kana 
representations [1], and both are required when you fill a form (though some 
algorithms exist, maybe converting properly from kanji to kana is not a 
straighforward task, I don't know)

I think we should provide a layman's description of kanji/kana, so that we can 
render the example as:

ORG;LANGUAGE=ja:<kanji for Japanese organization>
ORG;LANGUAGE=en:Japanese organization
ORG-SORT-STRING;LANGUAGE=ja:<kana for Japanese organization>

[1] http://www.dcmipubs.org/ojs/index.php/pubs/article/viewFile/22/13

Best Regards

Javier

---

From: Simon Perreault <[email-masked]>
Date: Wed, 23 Jun 2010 17:29:03 -0400

On 06/23/2010 11:39 AM, Cyrus Daboo wrote:
> The alternative would be to define a "SORT-AS" property
> parameter that could be applied to each property that could be sorted on.

+1

Simon

---

From: Simon Perreault <[email-masked]>
Date: Wed, 23 Jun 2010 17:35:42 -0400

On 06/23/2010 01:52 PM, Javier Godoy wrote:
> I though that FN was (1,n) because the name of the same person may be
> written in different languages (or scripts), and not because one person
> could have different names in the same language.

No. See section 5.1:

   Properties with different LANGUAGE parameters that represent the same
   data count as 1 toward cardinality and MUST have the same PID value
   if the PID parameter is used.  This is because there is logically a
   single property which is expressed in multiple languages.

FN is plural because there may be multiple valid ways of representing
one's name. Some could be preferred.

> On the other side, ORG
> is (0,n) not only because the name of the same organization may be given
> in different languages, but also because the same individual may be
> related to several organizations.

ORG is (0,n) *only* because the same individual may be related to
several organizations.

> I would use the preferred one, if the PREF parameter is provided. In
> case of tie, or if no SORT-STRING is provided,  would choose any of them
> (i.e. undefined)

Yes, that's a valid algorithm. I suppose we'll leave this to the
implementors...

> LANGUAGE is not a parameter for SORT-STRING. Adding a LANGUAGE would
> allow different SORT-STRINGs for different languages while preserving
> (0,1) cardinality.

Yup, that's exactly what I meant. But now I prefer Cyrus's suggestion
better. Let's make SORT-STRING a parameter.

> I don't think there is an equivalent example in US-ASCII.

Then you'll be surprised by what I come up with. ;)

Thanks,
Simon

---

From: Daisuke Miyakawa <[email-masked]>
Date: Thu, 24 Jun 2010 09:31:22 +0900

+1 to Javier's plan

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: Daisuke Miyakawa <[email-masked]>
Date: Sun, 27 Jun 2010 11:24:45 +0900

Hi,

I'm wondering who will have final decision on these kind of discussion, as
I'm a newbie in this ML.
Could someone tell me about it, or just point me some general guide line?

Thanks,

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]

---

From: Simon Perreault <[email-masked]>
Date: Mon, 28 Jun 2010 08:36:29 -0400

On 2010-06-26 22:24, Daisuke Miyakawa wrote:
> I'm wondering who will have final decision on these kind of discussion,
> as I'm a newbie in this ML.

Ideally there should be no decision. We discuss until we all agree.

See here for more info, especially section 5.2:
http://tools.ietf.org/html/rfc4677

Simon

---

From: Daisuke Miyakawa <[email-masked]>
Date: Tue, 29 Jun 2010 07:33:25 +0900

Understood. Thanks!

-- 
Daisuke Miyakawa (宮川大輔)
[email-masked]