Subject: [VCARDDAV] Format of various date fields

From: "Darryl Champagne" <[email-masked]>
Date: Wed, 7 May 2008 22:04:19 -0400

Hi,

One recurring issue with mobile device implementations is trying to keep the
processing simple when converting from the vCard format into the native data
format (and back).  Things like "BDAY;VALUE=text:circa 1800", and in fact,
any variations at all (such as 1996-08-11T12:34:56Z vs. 19960811T123456Z)
make this more difficult.

For mobile devices (and simplicity of implementation in general), it would
help if a single format were adopted (at least at the SHOULD level, to allow
for users hand editing fields), rather than leaving things wide-open.  Note
that I don't especially care what format is adopted, just that as few as
possible formats are allowed.

Note that there are a number of ways possible to allow for the equivalent
information to be conveyed:

For example:
1) Unknown values could be explicitly referenced with unique values, such as
"x" (or 0, or whatever).  Therefore a birthday with an unknown year could be
represented by "xxxx-06-16"

2) Ranges of values could be explicitly allowed - "Circa 1800" is roughly
equivalent to "1800-01-01 +- 0100-01-01" or "1800-xx-xx +- 0050-xx-xx" (the
year 1800 plus or minus 50 years) or "1750-xx-xx - 1850-xx-xx".  

Again - I do not care which format is selected, but I am trying to point out
that exactly specified formats would be much easier to deal with in the
mobile world than nearly freeform text, and also would work better for
comparison purposes (e.g. if during a Sync I am comparing 1800+-50 years
from one side, and 1820 from the other, I can reasonable choose the 1820
value, but I am unlikely to know how to deal with "circa" or "~" or "just
before", but any user interface that allows circa can easily pick a
reasonable range).

Thanks,
dgc
--
Darryl G. Champagne
Technology Evangelist
Funambol
OMA-DS Vice Chair
[email-masked]          http://www.funambol.com

_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Mark Paterson <[email-masked]>
Date: Thu, 08 May 2008 10:15:03 -0400

Agreed. It would also make sense I think to update the text concerning 
valid date time values to be the same as iCal. There are cases where 
mobile devices will reject perfectly valid date time stamps. Being more 
restrictive as iCal is would help interoperability as well.

Mark

-- 

Oracle Email Signature Logo
Mark Paterson | Product Manager, UC & Mobile Services | 514.905.8649
Oracle Server Technologies - Collaboration
600 Blvd. de Maissoneuve West, Suite 1900, Montreal, Quebec, H3A 3J2
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Simon Perreault <[email-masked]>
Date: Thu, 8 May 2008 08:48:05 -0400

On Wednesday 07 May 2008 22:04:19 Darryl Champagne wrote:
> Again - I do not care which format is selected, but I am trying to point
> out that exactly specified formats would be much easier to deal with in the

Here's my 2¢:

- I think that the DATE, TIME, and DATE-TIME value types are *very well 
defined* and that converting to/from them will not be ambiguous at all. Do 
you disagree?

- Resetting a DATE/TIME/DATE-TIME-valued property to freeform text implies 
that any comparison is futile. This is opaque text and you're not supposed to 
interpret it. Just store it as opaque text. If you can't, drop it, and accept 
that you're not implementing vCard fully (for potentially valid reasons like 
lack of storage, etc.)

It would be much harder on the implementation if we would specify a date 
format for expressing uncertainty and precision than just allowing the 
property to be reset to freeform text (which is not supposed to be 
interpreted).
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Thu, 8 May 2008 21:28:45 -0400

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Simon Perreault
> Sent: Thursday, May 08, 2008 8:48 AM

> defined* and that converting to/from them will not be ambiguous at all. Do

They are relatively well defined, but not as simple as they could be -
Primarily in the optional "-" and ":" characters (and that each one is
individually optional per the abnf, rather than either use them for all or
not as per ISO).  Do we really need both basic and extended format, and
should a mix (e.g. 2008-0612) be allowed, as per the abnf?

I have to ask what value is gained by this.  Clearly it is not difficult to
grab the year, and then check for a "-" and skip it if present, and so on,
but it prevents simple clients from just doing a blind string compare, which
does happen during synchronization (checking for duplicates, and checking
for rev changes).

My other concern relates to issue ID 236 - the request to allow for month
and day without year.  If this is allowed, which would presumably require
the leading dashes, why not just require dashes uniformly?

> that you're not implementing vCard fully (for potentially valid reasons

My concern is that for certain clients (I would expect a pretty decent sized
set), freeform text just means there is no meaningful data supplied, yet the
field and its useless data is present and being transferred.

> property to be reset to freeform text (which is not supposed to be

Why? I didn't state that uncertainty support is required. If one does not
choose to support uncertainty, then consider the plus or minus part as just
text - how would that be any different (other than having some additional
information)?  You would have a date that could be recognized, plus some
extra stuff that would be discarded (or stored).  Isn't that better than
having nothing?  

Is there some other need for freeform text?  

If vCard isn't supposed to support genealogy (as per issue 156), then why
support circa and all that?  If it's not required, then let's avoid having
the text option.  If reduced precision is desired (issue 236 & 156), then
just define it (and using a date plus a duration of uncertainty, or a start
and end date can also work).  For that matter, if genealogy support might be
desirable, since it is part of the information about an individual, which
can go into a "data format for representing and exchanging a variety of
information about an individual", then having the items 156 may as well be
defined (or reserved, like a GEN- prefix).

---

From: Simon Perreault <[email-masked]>
Date: Fri, 9 May 2008 08:09:53 -0400

On Thursday 08 May 2008 21:28:45 Darryl Champagne wrote:
> > defined* and that converting to/from them will not be ambiguous at all.
> > Do you disagree?
>

Oh, sure, purely syntactic issues are easy to fix. Consider it done.

   date = date-fullyear ["-" date-month ["-" date-mday]]

   time = time-hour [":" time-minute [":" time-second [time-secfrac]]]
           [time-zone]

> extra stuff that would be discarded (or stored).  Isn't that better than

Well, "circa 1800" is really not the same as "1800". In many cases dropping 
the uncertaincy indicator could be worse than dropping the whole thing. I'm 
not sure making support for uncertainty optional would be a good idea.

> and end date can also work).  For that matter, if genealogy support might
> be desirable, since it is part of the information about an individual,
> which can go into a "data format for representing and exchanging a variety
> of information about an individual", then having the items 156 may as well
> be defined (or reserved, like a GEN- prefix).

Now let's not get into a false dichotomy.

I'd like to hear other people voice their opinion on this issue. What should 
we do about uncertainty in date/time fields?

- Have it be resettable to a text value.
- Extend the format with "± ..."
- Do both.
- Do nothing.
- Do something else: ...
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 9 May 2008 10:31:16 -0400

> Sent: Friday, May 09, 2008 8:10 AM
> To: [email-masked]
> Subject: Re: [VCARDDAV] Format of various date fields
> 

Thanks,
dgc

---

From: Mark Paterson <[email-masked]>
Date: Fri, 09 May 2008 09:31:57 -0400

>    time = time-hour [":" time-minute [":" time-second [time-secfrac]]]
That unfortunately does not fix the greater issue Darryl is trying to solve.

And it doesn't answer the question, "Do we really need both basic and 
extended format?", iCal doesn't and most of the IOP issues between iCal 
implementations are not problems regarding the date format. This is 
because iCal is far more precise about what is a valid date-time value. 
Having both iCal and vCard agree with each other seems like something 
that would  help reduce interoperability issues

--

---

From: "Darryl Champagne" <[email-masked]>
Date: Thu, 29 May 2008 19:24:08 -0400

Hi,

In the current issues list at vcarddav.org, issue 171 Per Property UID is
marked as closed, with a last status of "This bug is closed until someone
says that what was written doesn't work."

I would like to clearly state that the current UID proposal does not work
for what OMA-DS requested, and issue 171 should not be marked as closed.  If
you need a formal liaison statement to reiterate this, I can provide it (but
it takes several weeks).  Note that I am not commenting about Cyrus's
request - UIDs may work for him, or perhaps both UIDs and PIDs are required
- just that the majority in OMA-DS desire PIDs (essentially, anyone who
develops OMA-DS clients - UIDs are OK for servers).

While the link for the last OMA-DS contribution
(http://www3.ietf.org/proceedings/08mar/slides/vcarddav-1.ppt on
http://www.vcarddav.org/wiki, labeled "OMA input to vcard-rev") is dead, the
same issues can be seen as problem #3, on slide #6 in the public file:
http://www.openmobilealliance.org/ftp/Public_documents/DS/DS_DO/2008/OMA-DS-
DS_DO-2008-0003-INP_VCARD_status.zip

The reasons UIDs do not meet the needs of OMA-DS are:

- OMA-DS needs to work in environments where the client devices do not store
the actual vCard - instead they translate the data in and out of a native
data format.  In this environment the requirements to store additional data
need to be minimized (or completely removed for minimalist devices).  For
example, DS can work where the client device stores just a couple of words
of additional data per data source (This is not typical, and it requires
hard coded ID mappings, dirty bits or a slow sync every time, and so on, but
it is supported).  Imposing a requirement for a UID adds additional data of
comparable size to the data being stored, which is not practical for limited
devices (e.g. EMAIL;UID="urn:uuid:69531f4a-c34d-4a1e-8922-bd38a9476a53":
[email-masked] - 17 characters of signal, and an additional 45 characters
of identifier)

- Diffs do not convey sufficient information (which is why PIDs/UIDs were
requested), because there is no ordering requirement for the data within a
vCard, the vCards need to be recreated from an internal data format, and
diffs do not hold all the relevant state, such as the entire history of
which items were deleted, re-created, or moved.  I will provide detailed
examples in a following email, since examples would be too long for here.

Additionally, it should be noted that the requirements related to "... MUST
be present if synchronization is to be used" are not especially practical,
since vCards can be used for a storage format, in which the desire to
synchronize usually comes along much later than the creation/storage of the
data (It is only after I have a large set of contacts that it matters if I
sync them).  I would suggest a SHOULD, as DS does expect to (and essentially
is required to) be able to synchronize data without UIDs/PIDs (as it
currently does for pre-vCard 4.0 formats).

Thanks,
dgc
--
Darryl G. Champagne
Technology Evangelist
Funambol
OMA-DS vice-chair

---

From: Kepeng Li <[email-masked]>
Date: Fri, 30 May 2008 09:38:16 +0800

Hi all,

    I agree that the current UID proposal does not work for what OMA-DS
requested.

    And I agree with Darryl's proposal.

BR,
Kepeng Li
Huawei
OMA-DS Chair

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Darryl Champagne
Sent: Friday, May 30, 2008 7:24 AM
To: [email-masked]
Subject: [VCARDDAV] UID/PID problems

Hi,

Thanks,
dgc

---

From: "Javier Godoy" <[email-masked]>
Date: Fri, 30 May 2008 06:25:04 -0300

Hi Darryl,

> for what OMA-DS requested, and issue 171 should not be marked as closed.

I agree..

> [email-masked] - 17 characters of signal, and an additional 45 

If a native data format were used, the uuid could be stored as an integer, 
thus the application would store "only" 16 bytes of identifier instead of 
45 (it is overkilling too, but the signal lenght is nearly the same as the 
identifier length). However, this optimization cannot be applied to other 
URN namespaces or URI schemes (such as tag:) and the application should have 
to deal with that case or resign not to conform the standard. Do we really 
need to allow *any* URI as PID? (IMHO, we don't)..

Other issue we have been discussing was about the PID lenght. I think we 
have not reached consensus because everyone is thinking about different 
scenarios and we tried to stick with a single solution. 128-bit identifiers 
are successful in the worst case, when a single vCard is replicated through 
thousand of devices, and we expect any of them to synchronize with any other 
with no collision risk. Of course this works, but it is too much for smaller 
cases: if syncronization were performed among few devices, a shorter PID 
would be enough.

> which items were deleted, re-created, or moved.

As you pointed in your other mail, the problem with diffs is that old 
information must to be stored too (all the changes history) or we have no 
means for distinguish edits from deletes. Besides, we had a (still 
unresolved) issue with devices which forget some properties.

Suppose there *IS* an ordering requirement. Instead of PIDs, the application 
could store the line number of each property and use it for recreating the 
plain text vCard.

In the following example (from your previous mail, with some additional 
steps) I introduce an ORDERED:yes property and two pseudo-properties DELETED 
and MISSING for representing properties which were deleted and properties 
which were omitted (e.g. because the device has no slots for them, or 
because it does not support that type of property). This is highly 
controversial, but IMHO the overhead is bounded

The proposed approach not only addresses the "forgetful device" problem, but 
also yields (counting the UID) 86 bytes of overhead for the plain text 
vCard, and 56 bytes (plus something proportional to the number of other 
properties) using a hypothetical "native data format".

The motto would be "if you want to synchronize, you must either provide PIDs 
or keep things ordered".

------------

The phone and/or server has additional information associated with each TEL
entry, such as the speed-dial number, that is not allowed to be shared (or 
it is not useful for other devices).

For this example, I will reference that additional info as:
SpeedDial numbers 11 - 15, which happens to match the line numbers.

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=5555
TEL;TYPE=work:tel:+1-555-555-5555;ext=4444
TEL;TYPE=work:tel:+1-555-555-5555;ext=3333
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
ORDERED:yes

Now I make the following changes:

1) Delete ext 4444, SpeedDial #12.

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=5555
DELETED:1
TEL;TYPE=work:tel:+1-555-555-5555;ext=3333
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
ORDERED:yes

2) Change the preferred work number, extension 5555 with SpeedDial #11, to
extension 3333, and SpeedDial #13 to extension 5555.

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=3333
DELETED:1
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
ORDERED:yes

3) Change SpeedDial #11 to extension 2222, and SpeedDial #14 to 5555.

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=3333
DELETED
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
ORDERED:yes

At this point... which SpeedDial number goes with which entry?
#11 ext=3333
#12 (deleted)
#13 ext=2222
#14 ext=5555
#15 ext home:tel

4) Save the vCard in other device which only supports 3 phone numbers

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=3333
DELETED
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
MISSING
...Usual vCard stuff...
ORDERED:yes

5) Add a new work number +1-555-555-4444 (in the limited device).
Since the limited device only supports 3 phone numbers, one of the work 
entries should be removed (in this example ext=5555 is removed).
We are marking this deletion as "MISSING" instead of "DELETED" because it 
should NOT propagate when syncronizing.

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=3333
DELETED
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
MISSING
MISSING
...Usual vCard stuff...
TEL;TYPE=work:tel:+1-555-555-4444
ORDERED:yes

6) Now delete (with propagation) ext=3333 and ext=2222 (in the limited 
device)

...10 lines of usual vCard stuff...
DELETED
DELETED
DELETED
MISSING
MISSING
...Usual vCard stuff...
TEL;TYPE=work:tel:+1-555-555-4444
ORDERED:yes

7) Synchronize back.

The first device has the vCard from step 3:

...10 lines of usual vCard stuff...
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=3333
DELETED
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
ORDERED:yes

After synchronizing, it will have

DELETED
DELETED
DELETED
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
TEL;TYPE=work:tel:+1-555-555-4444
ORDERED:yes

or, if we "fold" the DELETED pseudo-properties:

...10 lines of usual vCard stuff...

DELETED:3
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
TEL;TYPE=work:tel:+1-555-555-4444
ORDERED:yes

8) Assign work:tel:+1-555-555-5555;ext=9999 to SpeedDial #13

...10 lines of usual vCard stuff...
DELETED:2
TEL;TYPE=work:tel:+1-555-555-5555;ext=9999
TEL;TYPE=work:tel:+1-555-555-5555;ext=5555
TEL;TYPE=home:tel:+33-01-23-45-67
...Usual vCard stuff...
TEL;TYPE=work:tel:+1-555-555-4444
ORDERED:yes

(SpeedDial #11 and #12 have been deleted).

---------

The overhead (as plain text) of this approach is only 13 bytes for the 
one-time "ORDERED:yes" property and 11 bytes per each deleted or missing 
block (13 bytes if there is a chunk of  +100 properties). We should also 
count a 51-byte UID (urn:uuid) property.

For the example above, folding the MISSING and DELETE pseudo-properties when 
possible, the overhead is

Initial vCard: 64 bytes (ORDERED:yes + UID)
Step 1 to 3: 75 bytes  (ORDERED, UID and DELETED block)
Step 4 to 6: 86 bytes  (ORDERED, UID and DELETED+MISSING blocks)
Step 7 and 8: 75 bytes (ORDERED, UID and DELETED block)

Let suppose we have implemented a näive "native data format" (*) where each 
property is stored as a pair of fields:
- property-name identifier and data-length:  fixed length: N bytes
- value and parameters: variable-length

(*) This is not intended to be a production-grade format: it is only an 
example! Issues such as memory alignments, fixed-length formats and whatever 
else are not discussed.

Mapping from plain text vCard to this format include the following steps:
- Write the line number of each actual property. Surely we can expect less 
than 2^16 lines thus each property adds 2 bytes of overhead.
- Represent the UID:urn:uuid as a 128-bit integer property, and the ORDERED 
property as an empty property (fixed overhead = 16+2N bytes)
- Do not represent missing blocks (we know where they are because the other 
properties have line numbers) (overhead = 0 bits per block).
- Represent each DELETED block of length 1 as a property with no data 
(overhead = N bytes per block).
- Represent each DELETED block of length greater than 1 as a property whose 
value is the block length (overhead = N+1 or N+2 bytes per block).

For the vCard example before step 1, the overhead is
- first 10 lines of "usual vCard stuff" (BEGIN:VCARD + 9 properties): 9*2 = 
18 bytes.
- trailing "usual vCard stuff" (U/2 properties): U bytes.
- ORDERED:yes and UID (not counted above): 16+2N bytes.
- 5 TEL properties: 5*2 = 10 bytes.

Total overhead of the initial vCard (U+34+2N)+10

Steps 1 to 3 : (U+34+2N) +8+N
Steps 4 and 5 : (U+34+2N) +6+N
Step 6 : (U+34+2N) +2+3N or (U+34+2N) +4+N
Step 7:  (U+34+2N) +6+3N or (U+34+2N) +8+N
Step 8: (U+34+2N) +8+2N or (U+34+2N) +10+N

If N=4, we have at most (U+56) bytes of overhead. Compare this with the 45 
characters of urn:uuid.

Hope that helps.

Javier

---

From: Simon Perreault <[email-masked]>
Date: Fri, 30 May 2008 08:45:53 -0400

> says that what was written doesn't work."

Oh please don't take that too seriously. Of course it has been shown that it 
doesn't work, the issue just needs to be reopened. *clicky*

> http://www.vcarddav.org/wiki, labeled "OMA input to vcard-rev") is dead,

There, I updated it. Someone never heard of the "cool URIs don't change" 
proverb.

> - OMA-DS needs to work in environments where the client devices do not
> store the actual vCard - instead they translate the data in and out of a
> native data format.  In this environment the requirements to store
> additional data need to be minimized (or completely removed for minimalist
> devices).  For example, DS can work where the client device stores just a
> couple of words of additional data per data source (This is not typical,
> and it requires hard coded ID mappings, dirty bits or a slow sync every
> time, and so on, but it is supported).  Imposing a requirement for a UID
> adds additional data of comparable size to the data being stored, which is
> not practical for limited devices (e.g.
> EMAIL;UID="urn:uuid:69531f4a-c34d-4a1e-8922-bd38a9476a53":
> [email-masked] - 17 characters of signal, and an additional 45
> characters of identifier)

I fully agree with you. Even while drafting the current proposal I was 
thinking "This is going to get shot down so quickly it won't even be funny." 
but we needed a starting point.

> examples in a following email, since examples would be too long for here.

Here I don't follow you. The two examples you've shown have been so long (in 
the amount of text, literally) that the point gets lost (at least I couldn't 
follow).

> sync them).  I would suggest a SHOULD, as DS does expect to (and
> essentially is required to) be able to synchronize data without UIDs/PIDs
> (as it currently does for pre-vCard 4.0 formats).

Yes, I agree.

---

From: Simon Perreault <[email-masked]>
Date: Fri, 30 May 2008 08:47:15 -0400

>     And I agree with Darryl's proposal.

Which is?

If I remember correctly, he provided arguments as to why the current proposal 
and diffs don't work, but there was no new proposal.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Mark Paterson <[email-masked]>
Date: Fri, 30 May 2008 10:35:35 -0400

> and diffs don't work, but there was no new proposal.
Nope, but there were some pretty good arguments and I believe the 
gentleman was simply trying to state that he agreed with them. Let's try 
to foster participation from individuals allover the world, even those 
whose English skills may not be overly strong.

Best Regards,
Mark Paterson

--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 30 May 2008 10:46:10 -0400

> whose English skills may not be overly strong.

I'm sorry that I seemed rude. I just wanted to point out that there was no new 
proposition and that we actually needed one.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 30 May 2008 10:52:04 -0400

I referred back to the PID="small integer" approach as being strongly
preferred over a UID.

dgc

> Sent: Friday, May 30, 2008 8:47 AM
> To: [email-masked]
> Subject: Re: [VCARDDAV] UID/PID problems
>

---

From: Simon Perreault <[email-masked]>
Date: Fri, 30 May 2008 11:16:41 -0400

> I referred back to the PID="small integer" approach as being strongly

Yes I think everyone now agrees that UIDs are "too big".

But the *enormous* issue with the small integer approach is: how do you make 
any use of that? It seems to be so prone to collisions as to be useless.

That's the proposition that's missing.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 30 May 2008 11:21:40 -0400

> Sent: Friday, May 30, 2008 8:46 AM
> To: [email-masked]

I couldn't tell that - lack of communication is not the same as agreement
(Not trying to be rude, but I just did not know the state, so I reasserted
by prior arguments).

Your last emails about UID/PIDs were about storing the original plus a diff,
and using a distributed source code revision system, and I checked the issue
list after a month and it was still marked closed.... I can only go with
what I see.

> the amount of text, literally) that the point gets lost (at least I

I can't tell if the Diff/UID/PID email, and the example showing SpeedDial
numbers from a month ago answered this or not.  

I could trace through what it would look like during a sync between a server
and two clients, with new data created on both clients, but that is rather
long.  

Unique PIDs are a total non-issue unless a third party is involved, with
data being created on at least 2 devices, so there is a clear limit to how
much can be simplified.  The order that the data is passed between all three
parties matters, but not all "owner" information is lost (you don't just
create new data and have a conflict).

---

From: Simon Perreault <[email-masked]>
Date: Fri, 30 May 2008 11:35:30 -0400

> numbers from a month ago answered this or not.

It did show that diffs alone are not sufficient.

It did not show that small integer PIDs are any good (what about collisions?).

> much can be simplified.  The order that the data is passed between all
> three parties matters, but not all "owner" information is lost (you don't
> just create new data and have a conflict).

You're right, but isn't it simplifying too much? Doesn't a third party 
creating informating happen very often?

e.g. PC, PDA, cell phone (or even PC, cell1, cell2)

You're much more qualified than I to answer that question. But we need to put 
these limitations in writing, as well as the consequences in case of a 
conflict.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 30 May 2008 11:40:26 -0400

> Sent: Friday, May 30, 2008 11:17 AM
> To: [email-masked]

That's not really the case.  In the real world collisions are generally only
an issue during a Sync, and during a sync you have additional knowledge -
you know which side changed something since the last time.  Thus a new PID=3
on both sides is NOT a collision - you know that both sides have a new PID=3
that are potentially different, and have to be treated as both potentially
new.

It's also only possible to create that limited collision case with at least
3 parties being involved, with new data being created on at least 2 of them
between successive syncs.  In the real world, because we are talking about
actual people's data, that situation will generally only occur when someone
tries to enter a new data item in multiple places.  Generally it actually is
the same data, just perhaps with a difference in format.

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 30 May 2008 12:14:50 -0400

Hi,
Ordering, and deleted and missing is an intriguing possibility.

I need to think about it a bit more, but I see a couple of issues.

If you send the missing/deleted as just the change from the last sync state,
that would appear to work fine, but I don't see a good way to recover (do a
slow sync because of some problem).  Keep in mind that one scenario for a
recovery sync is that you are syncing a device with a server, and stop using
the device for a while, then start again.  The server will eventually want
to reclaim that space (you might no longer even have the device), and toss
out all data specific to that client (it keeps your address book, since you
are syncing with other devices like your PC).  If you start syncing again
with the device, then you have to reanalyze everything (match client
identifiers with server identifiers, frequently without a UID).

On the other hand, if you keep the Missing/Deleted in the vCard forever, it
seems like you have an unbounded growth problem as you accumulate changes.
It seems pretty unlikely, as you are more likely to just keep increasing a
count, and probably not a huge issue in the real world (but it is an
implementation issue), except for those poor QA people who run tests.  I can
just imagine stress testing accumulating thousands of add/deletes in each
vCard.

I guess the other thing is that it seems to be just like using a virtual PID
that equals the Line Number, plus a way to get rid of the overhead of long
lists.  I'm not sure how it is any different than the PID for the collision
scenario (3+ devices with 2+ adds).

<more embedded>

> From: Javier Godoy [mailto:[email-masked]]
> Sent: Friday, May 30, 2008 5:25 AM
> To: [email-masked]; Darryl Champagne
> > store
> > characters

But how realistic is this thousands of devices scenario?  The only
situations I know of where more than a handful sync is for things like
global address books (corporate/university directories), and those are
almost always one-way - at most a couple of people have write access,
everyone else only has read access.

> could store the line number of each property and use it for recreating the

Yep, plus with a simple device, you just stick with unordered if you have
to.

> The motto would be "if you want to synchronize, you must either provide
...
> If N=4, we have at most (U+56) bytes of overhead. Compare this with the 45

Thanks, good analysis.

---

From: "Javier Godoy" <[email-masked]>
Date: Fri, 30 May 2008 18:18:39 -0300

----- Original Message ----- 
From: "Darryl Champagne" <[email-masked]>
To: "'Javier Godoy'" <[email-masked]>; <[email-masked]>
Sent: Friday, May 30, 2008 1:14 PM
Subject: RE: [VCARDDAV] UID/PID problems

> Ordering, and deleted and missing is an intriguing possibility.

Just a note for clarifying things: the MISSING feature can be achived with 
PIDs too, but it is somehow more expensive in that case, because missing 
PIDs cannot be folded unless PIDs were assigned sequentially (and sequential 
PIDs increase the risk of collision).

> I need to think about it a bit more, but I see a couple of issues.

Just "a couple" ??? I would have expected tons of them ;-)

> the device for a while, then start again.

If you stopped using the device, you would probably want to synchronize 
newer data *from* the server (e.g. getting the vCard instance by UID), 
instead of a mutual synchronization... but we should consider this 
situation.

> The server will eventually want

Reclaiming space is a bit tricky, since (with the ordered-deleted-missing 
approach) this information is meaningful.
It would require a way for identifiying the original from which deletions 
are tracked, and that introduces more overhead.

Furthermore, there could be different versions of a vCard instance, which 
have been ordered from different initial states.

1) Device A creates an unordered vCard
2) A syncs with B (it is unordered, then B can rearrange the properties).
3) A declares the instance as ORDERED.
4) B declares the instance as ORDERED.
5) A syncs with B.

The problem in this case arises from the synchronization of unordered 
instances in step 2.

> just imagine stress testing accumulating thousands of add/deletes in each

You are right (it is theoretically possible), but since n consecutive DELETE 
pseudo-properties fold into a single DELETE:n the unbounded growth problem 
requieres deletions to be performed in an interleaved fashion (e.g. delete 
properties on even  lines and keep properties in the odd ones). For 
instance, after you have added and deleted (NOT modified) twelve emails, you 
will have

DELETE:12
EMAIL:[email-masked]

Of course one could argue that the number of consecutive deletions could 
overflow (in my previous e-mail I have supposed 2^16 lines...). Again, how 
many properties are to be expected in a vCard instance lifetime?

On the other hand, MISSING stands for properties which cannot be stored in 
the device, thus you require an actual property from the other side for 
generating a missing and it is harder to make an unbounded list..

sync(missing,nothing) = missing
sync(missing,missing) = missing
sync(missing,deleted) = deleted
sync(missing,datum) = data (or missing, if datum is not supported)

> scenario (3+ devices with 2+ adds).

With exactly 3 devices, I cannot set up a conflicting condition (assuming 
each device implements a "this is my new property, and I haven't told anyone 
about it" policy when synchronizing).

1)  in device A
VCARD:BEGIN
ORDERED:yes
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=1111
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
VCARD:END

2) sync device B and device C with device A.

3) add TEL;TYPE=work:tel:+1-555-555-5555;ext=3333 in device B

--device B
VCARD:BEGIN
ORDERED:yes
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=1111
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=work:tel:+1-555-555-5555;ext=3333
VCARD:END

4) add TEL;TYPE=work:tel:+1-555-555-5555;ext=4444 in device C
--device C
VCARD:BEGIN
ORDERED:yes
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=1111
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
TEL;TYPE=work:tel:+1-555-555-5555;ext=4444
VCARD:END

5) Sync device B with device A. There is no problem, A understand ext=3333 
is a new property.

6) Sync device C with device A. C should tell A (by means of the 
synchronization protocol) that ext=4444 is not a modification for the third 
value, but a new one. For instance, C could fetch the instance from A, then 
it learn about ext=3333, and modify the vCard to

VCARD:BEGIN
ORDERED:yes
TEL;TYPE=work,pref:tel:+1-555-555-5555;ext=1111
TEL;TYPE=work:tel:+1-555-555-5555;ext=2222
MISSING
TEL;TYPE=work:tel:+1-555-555-5555;ext=3333
VCARD:END

which can be safely synchronized.

However, the conflict arise if we do 6) sync device C with device D, and 7) 
sync device D with device A.

>> scenarios and we tried to stick with a single solution. 128-bit
>> identifiers are successful in the worst case, when a single vCard is 
>> replicated
>> through thousand of devices, and we expect any of them to synchronize 
>> with any
>> other with no collision risk. Of course this works, but it is too much 
>> for
>> smaller cases: if syncronization were performed among few devices, a 
>> shorter PID
>> would be enough.
>

It is no realistic at all (besides, such a scenario would require a 
mechanism for representing modification times on a per property basis, let 
alone trust issues...) I wrote it in that way for emphasizing on the 
"smaller cases" which do really happen (I though consensus was toward 
128-bit PIDs for some reason).

I hope we can extract something in clear from this...

Best Regards

Javier