Subject: [VCARDDAV] carddav-04 questions/comments

From: Jari Urpalainen <[email-masked]>
Date: Wed, 23 Apr 2008 14:15:23 +0300

Hi !

I don't have many comments on this i-d, since it looks pretty decent as 
is already. Just wandering what stops it from progressing ;-). That 
said, I've implemented both a client based on Evolution (-data-server) 
and an Apache server module, and both were pretty straightforward to do 
since  the whole idea is simple and imo worth doing.

There are some nits I'd still like to emphasize, first if it helps 
progressing, I could easily ditch the descriptions of the "competitive" 
protocols in the introduction, at least I don't need any  justifications 
why this shouldn't be done (is it too easy/simple?).

Secondly, there can be multiple param- and prop-filters (in dtds) and it 
seems not be clearly described (or at all) that it probably means ANDing 
multiple conditions (when there multiple similar elements). I'm not 
proposing about adding and/or/nots or whatever though some of the 
competing protocols have them. If i'm not mistaken the same applies to 
CalDAV though it has an example, so probably here an example would 
suffice (with some normative text).

Lastly, on the client side I'm doing syncing at some regular intervals. 
I'm aware of the Cyrus's i-d  dav-sync proposal, but I'm looking for a 
simpler (intermediate) solution, i.e. that you could do conditional 
reports based on ETags. Currently, only conditional retrievals of 
resources with http get is specified, so why couldn't that be done with 
reports as well ? So in a typical use-case there aren't necessarily any 
changes and reports tend to be fairly large, though you are requesting 
only ETag values and resource names. So it would be fairly easy to 
implement it as well (at least naive ones). Also as it is fully optional 
to the server implementations it doesn't break any clients, but could it 
be documented somewhere ? (well, similar stuff have been done in sip 
related wgs... Had you versioning in reports you could even patch 
reports, but I'm not proposing that, just this _simple_ thing)

thanks, Jari    

_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Julian Reschke <[email-masked]>
Date: Wed, 23 Apr 2008 13:33:12 +0200

> is already. Just wandering what stops it from progressing ;-). That 

...I'd like to see somebody reply to my comments I posted some time 
ago... ;-)

> ...

Well, a REPORT can be executed conditionally as well, but of course it 
only helps if you have useful ETags on the request URI.

I've made a proposal to make this more useful for WebDAV land; did you 
see that? -> 
<http://greenbytes.de/tech/webdav/draft-reschke-http-get-location-01.html>

> ...

BR, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 23 Apr 2008 10:15:12 -0400

Hi Jari,

--On April 23, 2008 2:15:23 PM +0300 Jari Urpalainen 
<[email-masked]> wrote:

> is already. Just wandering what stops it from progressing ;-).

Sorry that's my fault. Been pretty busy recently. I've got some stuff to 
finish off for calsify first, then I will be back onto carddav full time. 
There were a couple of open questions from the IETF meeting last month that 
need some resolution: the question as to what DAV level is required, and 
whether DAV ACL support is a MUST.

> That

Its certainly easy to do if you've already done a CalDAV client or server!

> why this shouldn't be done (is it too easy/simple?).

My plan has always been to get rid of the "competitive" analysis portion. I 
put it in more to justify the initial interest in the spec. Now that we 
have a formal WG shepherding this, I don't think its needed. At the very 
least it ought to be an Appendix. Any other opinions on this? If it stays 
in it really needs more detail in a number of areas.

> suffice (with some normative text).

OK, so this is wrong. In CalDAV the prop-filter element descriptions states 
that it matches if ALL child param-filter elements also match - i.e. an AND 
operation. But in CardDAV it states that prop-filter matches if ANY 
param-filter matches - i.e. an OR operation. For consistency I think that 
should be changed in CardDAV.

One other issues is the top-level filter element which can contain multiple 
prop-filter elements. There it is not stated at all whether all or any must 
match. This is actually a little different from CalDAV in that CalDAV also 
has comp-filter elements which are not needed in CardDAV.

The issue then is should filter/prop-filter be an AND or OR? For 
consistency AND would be the choice. However, I think a very common query 
that clients will do is an "address expansion" operation that will attempt 
to search for some text appearing in any of a set of properties. e.g. I 
type "cyrus" and would like completion on full name, nickname or email 
address on that. In that case an OR operation is needed.

We deliberately did not try to implement a boolean query syntax for CalDAV 
as we felt that would be too complicated from the outset. However for 
CardDAV we might want to implement a simple option:

- define an attribute for the <filter> and <prop-filter> elements called 
"test" with two values "allof" and "anyof". The former implements an AND 
operation for all child element tests, the later an OR operation. This is 
actually "borrowed" from the SIEVE language which implements similar simple 
boolean test constructs for filtering email headers.

> reports, but I'm not proposing that, just this _simple_ thing)

As we are finding with CalDAV deployments, the reliance on polling by 
clients is a big scalability problem - its exacerbated in CalDAV in that 
calendar users typically want immediate notification of invitations and 
therefore nearly always turn the clients polling interval down to the 
smallest value (maybe 1 minute). 20K users doing PROPFINDs every minute 
starts to be the biggest burden for a server.

To alleviate that Apple implemented a "ctag" extension for calendar 
collections 
(<http://trac.macosforge.org/projects/calendarserver/browser/CalendarServer/trunk/doc/Extensions/caldav-ctag.txt>) 
in our CalDAV server. The ctag changes whenever a resource in the 
collection is added, removed or changed. That means clients can do Depth:0 
PROPFINDs on the calendar collection first, and if the ctag is the same as 
before, can skip doing the full PROPFIND Depth:1 to get resources and their 
ETags. That certainly cuts down the load on the server a lot given that the 
collections themselves don't change all that often. This is a slightly 
different approach than Julian's get-location extension which could also be 
used for something like this. The ctag behavior is trivial to implement.

Really the proper solution to this is push notifications from server to 
client, combined with a sync report to only fetch changes from the server. 
Once CardDAV is done (and we finish the CalDAV scheduling spec) I think 
notifications is the next important piece to work on. There has already 
been some discussions on this in the past with SIP, XMPP, HTTP based 
solutions proposed. Plus the IETF has recently discussed the same problem 
for email push notifications. A generic push notification protocol/service 
seems needed here.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 23 Apr 2008 16:51:25 +0200

> in our CalDAV server. The ctag changes whenever a resource in the 

Wouldn't the Last-Modified date of the collection resource work the 
same? In which case you could just do a PROPFIND with If-Modified-Since 
request header...

BR, Julian

---

From: Jari Urpalainen <[email-masked]>
Date: Thu, 24 Apr 2008 10:15:11 +0300

ext Cyrus Daboo wrote:
> Sorry that's my fault. Been pretty busy recently. I've got some stuff 
> to finish off for calsify first, then I will be back onto carddav full 
> time. 
ok. I was just wandering...
> There were a couple of open questions from the IETF meeting last month 
> that need some resolution: the question as to what DAV level is 
> required, and whether DAV ACL support is a MUST.
>
FWIW I've no problem with a MUST ACL requirement for the server, though 
many practical servers could easily live without it (e.g read-only 
versions). But as it is MUST in CalDAV and with my implementation 
experience ;-) it isn't that hard it should imo stay since I'd like to 
get some more deployment on it (ok, there are some gripes in it but i 
like the general idea).
> Its certainly easy to do if you've already done a CalDAV client or 
right, this _is_ much easier indeed. Though a decent vcard library for 
parsing is required with some interfaces.
> My plan has always been to get rid of the "competitive" analysis 
> portion. I put it in more to justify the initial interest in the spec. 
> Now that we have a formal WG shepherding this, I don't think its 
> needed. At the very least it ought to be an Appendix. Any other 
> opinions on this? If it stays in it really needs more detail in a 
> number of areas.
>
just remove the whole stuff, please... this is one alternative (and a 
reasonable one imo) and let the deployments speak.
> OK, so this is wrong. In CalDAV the prop-filter element descriptions 
> states that it matches if ALL child param-filter elements also match - 
> i.e. an AND operation. But in CardDAV it states that prop-filter 
> matches if ANY param-filter matches - i.e. an OR operation. For 
> consistency I think that should be changed in CardDAV.
>
> One other issues is the top-level filter element which can contain 
> multiple prop-filter elements. There it is not stated at all whether 
> all or any must match. This is actually a little different from CalDAV 
> in that CalDAV also has comp-filter elements which are not needed in 
> CardDAV.
>
right.
> consistency AND would be the choice. However, I think a very common 
> query that clients will do is an "address expansion" operation that 
> will attempt to search for some text appearing in any of a set of 
> properties. e.g. I type "cyrus" and would like completion on full 
> name, nickname or email address on that. In that case an OR operation 
> is needed.
>
> We deliberately did not try to implement a boolean query syntax for 
> CalDAV as we felt that would be too complicated from the outset. 
> However for CardDAV we might want to implement a simple option:
>
> - define an attribute for the <filter> and <prop-filter> elements 
> called "test" with two values "allof" and "anyof". The former 
> implements an AND operation for all child element tests, the later an 
> OR operation. This is actually "borrowed" from the SIEVE language 
> which implements similar simple boolean test constructs for filtering 
> email headers.
>
This is fine by me as it doesn't require any rocket-science and it seems 
easily understandable right from the syntax.
> clients is a big scalability problem - its exacerbated in CalDAV in 
> that calendar users typically want immediate notification of 
> invitations and therefore nearly always turn the clients polling 
> interval down to the smallest value (maybe 1 minute). 20K users doing 
> PROPFINDs every minute starts to be the biggest burden for a server.
>
right and i agree it's more evident with calendaring than here.
> collection is added, removed or changed. That means clients can do 
> Depth:0 PROPFINDs on the calendar collection first, and if the ctag is 
> the same as before, can skip doing the full PROPFIND Depth:1 to get 
> resources and their ETags. That certainly cuts down the load on the 
> server a lot given that the collections themselves don't change all 
> that often. This is a slightly different approach than Julian's 
> get-location extension which could also be used for something like 
> this. The ctag behavior is trivial to implement.
I don't much care about the solution as long as it is really easy to 
implement, doesn't break naive clients but allows also more efficient 
stuff on a server side (so just described conditional ETags with simple 
or not so simple report requests ;-)).  I'll also have to look at 
Julian's proposal, however but this _is_ imo something that must be 
done, since it is much easier than the notifications. As an example I'll 
have a list of ~ 2000 vcards and the report response is ~500 kbytes, i 
don't definitely want to do polling once a minute in a mobile terminal 
(and probably not even with a desktop).
Server side headache can hardly be avoided (but that's only a minor ;-) 
implementation  scaling issue as one might say).
>
> Really the proper solution to this is push notifications from server 
> to client, combined with a sync report to only fetch changes from the 
> server. Once CardDAV is done (and we finish the CalDAV scheduling 
> spec) I think notifications is the next important piece to work on. 
> There has already been some discussions on this in the past with SIP, 
> XMPP, HTTP based solutions proposed. Plus the IETF has recently 
> discussed the same problem for email push notifications. A generic 
> push notification protocol/service seems needed here.
>
right, but it introduces other challenges depending on the transport 
etc., i've been involved with sip event notifications of xcap resources 
(includes patching of resources but no metadata retrievals other than 
ETags), which is almost a "generic" similar stuff and it _is_ really 
challenging stuff to implement in reality. Agreed that it is useful 
sometimes, but I'm much more contented with the somewhat optimized 
polling reports than these notifications.

thanks Cyrus,
Jari