Subject: [VCARDDAV] Performing an initial sync with the server?

From: Nick Zitzmann <[email-masked]>
Date: Wed, 18 Nov 2009 13:55:43 -0700

I'm making a CardDAV client. I tried searching around, and reading the latest draft, and I don't understand one very basic thing: How do I perform an initial sync with the server, downloading every vCard in an address book?

I was able to successfully get the addressbook-home-set property from the server's principal URL for the server. But when I try to send it a report like this:

REPORT /addressbooks/__uids__/ABEB6FF7-E708-4387-A2D5-053F9A9DBA2E/ HTTP/1.1
Authorization: (yes, we authorize okay)
Host: example.org:8843
Accept: */*
Depth: 1
Content-Type: text/xml
Content-Length: 221

<?xml version="1.0" encoding="utf-8"?><r1:addressbook-query xmlns:r0="DAV:" xmlns:r1="urn:ietf:params:xml:ns:carddav"><r0:prop><r0:getetag/><r1:address-data><r1:allprop/></r1:address-data></r0:prop></r1:addressbook-query>

I'd expect it to return a multi-status containing the contents of every card in the address book, plus each of their etags, but instead I get an error 400 in return. The server on port 8843 is Mac OS X's CardDAV server.

I already tried changing the "<r1:address-data><r1:allprop/></r1:address-data>" to "<r1:address-data/>" and it didn't make a difference. The CalDAV equivalent query, calendar-query, works just fine on the CalDAV server running on the same system...

Nick Zitzmann
<http://www.chronosnet.com/>

---

From: Simon Perreault <[email-masked]>
Date: Wed, 18 Nov 2009 18:48:18 -0500

> I'm making a CardDAV client. I tried searching around, and reading the latest draft, and I don't understand one very basic thing: How do I perform an initial sync with the server, downloading every…

Why not do it the simple way and GET each vCard?

People have been mirroring web sites for a long time using this method...

Simon
-- 
DNS64 open-source   --> http://ecdysis.viagenie.ca
STUN/TURN server    --> http://numb.viagenie.ca
vCard 4.0           --> http://www.vcarddav.org

---

From: Helge Hess <[email-masked]>
Date: Thu, 19 Nov 2009 01:00:19 +0100

> <?xml version="1.0" encoding="utf-8"?><r1:addressbook-query xmlns:r0="DAV:" xmlns:r1="urn:ietf:params:xml:ns:carddav"><r0:prop><r0:getetag/><r1:address-data><r1:allprop/></r1:address-data></r0:prop…

I think the XML is wrong.

<?xml version="1.0" encoding="utf-8"?>
<r1:addressbook-query xmlns:r0="DAV:"
   xmlns:r1="urn:ietf:params:xml:ns:carddav"
  <r0:prop>
    <r0:getetag/>
    <r1:addressdata>
      <r1:allprop/>
    </r1:address-data>
  </r0:prop>
</r1:addressbook-query>

shouldn't that be just:

<?xml version="1.0" encoding="utf-8"?>
<r1:addressbook-query xmlns:r0="DAV:"
   xmlns:r1="urn:ietf:params:xml:ns:carddav"
  <r0:prop>
    <r0:getetag/>
    <r1:addressdata/>
  </r0:prop>
</r1:addressbook-query>

?

What I recommend (for a full sync) is doing the PROPFIND depth:1 to discover the resources, and then do CardDAV multiget REPORTs to retrieve the items in batches (128 at a time seems to be a good value).
Obviously check for WebDAV sync support in the same run and support that too (to avoid traffic in BIG collections).

BUT: doing individual GETs also performs very well on persistent connections against Apache. So you might to support that too (GroupDAV fallback).

Helge

---

From: Nick Zitzmann <[email-masked]>
Date: Wed, 18 Nov 2009 17:28:11 -0700

>   xmlns:r1="urn:ietf:params:xml:ns:carddav"

I tried that (and "addressdata" should be "address-data" if I read correctly), and it didn't work, either. I've been trying different things for the last few hours, and all of them have returned error 400s or 500s, so maybe I did something wrong, I don't know, but I'm going to try rewriting my code to do a PROPFIND and an addressbook-multiget report.

Still, I find it a little strange that this report isn't working, whereas the calendar-query report on the CalDAV side of the same server software works just fine. I hope it's something I'm doing wrong and not a server bug, because the server is supposed to support this report.

Thanks to the people who responded.

Nick Zitzmann

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 18 Nov 2009 20:21:07 -0500

Hi Nick,

--On November 18, 2009 5:28:11 PM -0700 Nick Zitzmann <[email-masked]> 
wrote:

> Still, I find it a little strange that this report isn't working, whereas
> the calendar-query report on the CalDAV side of the same server software
> works just fine. I hope it's something I'm doing wrong and not a server
> bug, because the server is supposed to support this report.
>

If you really want to get all the vCards then do a PROPFIND Depth:1 on the 
addressbook collection and list just the resource names. Then use the 
multiget REPORT to fetch the resources in batches, requesting the 
DAV:getetag and CARDDAV:address-data elements as properties - you need to 
keep track of the etag->resource mapping in your client for later 
synchronization. When using multiget you should limit the total number of 
resources requested in each REPORT because servers may have a limit.

In regard to the actual report your tried, the XML you used is:

<?xml version="1.0" encoding="utf-8"?>
<r1:addressbook-query xmlns:r0="DAV:" 
xmlns:r1="urn:ietf:params:xml:ns:carddav">
<r0:prop>
  <r0:getetag/>
  <r1:address-data>
    <r1:allprop/>
  </r1:address-data>
</r0:prop>
</r1:addressbook-query>

However, the addressbook-query is defined as:

       <!ELEMENT addressbook-query ((DAV:allprop |
                                     DAV:propname |
                                     DAV:prop)?, filter, limit?)>

So note that you are missing the CARDDAV:filter element. That must be 
present, even if it is empty.

-- 
Cyrus Daboo

---

From: Nick Zitzmann <[email-masked]>
Date: Wed, 18 Nov 2009 18:43:58 -0700

On Nov 18, 2009, at 6:21 PM, Cyrus Daboo wrote:

> If you really want to get all the vCards then do a PROPFIND Depth:1 on the addressbook collection and list just the resource names. Then use the multiget REPORT to fetch the resources in batches, requesting the DAV:getetag and CARDDAV:address-data elements as properties - you need to keep track of the etag->resource mapping in your client for later synchronization. When using multiget you should limit the total number of resources requested in each REPORT because servers may have a limit.

Thanks Cyrus. And thanks for CardDAV, CalDAV, and Mulberry.

One more question. I noticed that, when Mac OS X's CardDAV server is accessed by a user for the very first time, the server automatically creates an "addressbook" collection in the user's addressbook-home-set. I don't see any requirement in the draft that says that this "addressbook" collection must exist for all users, so I'm guessing it's just a Snow Leopard Server thing.

But the confusion sets in when I found that the Snow Leopard Address Book application's CardDAV support apparently does not work if that "addressbook" folder is MIA, and it doesn't seem to create it if it doesn't exist. Is this a bug, or is there an undocumented requirement that the "addressbook" collection must exist? If it's not Address Book that's making that collection, I wonder if server makers are going to have to ensure that the collection always exists in order to keep one of its biggest clients happy...

Just wondering if I can assume then that the "addressbook" collection will always exist or not... My client will only use one collection in the home set, and I want to make my client as interoperable with other clients as possible.

Nick Zitzmann

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 18 Nov 2009 21:44:43 -0500

Hi Nick,

--On November 18, 2009 6:43:58 PM -0700 Nick Zitzmann <[email-masked]> 
wrote:

> One more question. I noticed that, when Mac OS X's CardDAV server is
> accessed by a user for the very first time, the server automatically
> creates an "addressbook" collection in the user's addressbook-home-set. I
> don't see any requirement in the draft that says that this "addressbook"
> collection must exist for all users, so I'm guessing it's just a Snow
> Leopard Server thing.

Both CalDAV and CardDAV servers on OS X create a default calendar or 
address book collection when a user's account is provisioned. This avoids 
the need for the client or user to have to do that step.

> But the confusion sets in when I found that the Snow Leopard Address Book
> application's CardDAV support apparently does not work if that
> "addressbook" folder is MIA, and it doesn't seem to create it if it
> doesn't exist. Is this a bug, or is there an undocumented requirement
> that the "addressbook" collection must exist? If it's not Address Book
> that's making that collection, I wonder if server makers are going to
> have to ensure that the collection always exists in order to keep one of
> its biggest clients happy...

AddressBook.app currently requires an addressbook collection to be present 
in the user's address book home.

> Just wondering if I can assume then that the "addressbook" collection
> will always exist or not... My client will only use one collection in the
> home set, and I want to make my client as interoperable with other
> clients as possible.

There is no requirement in the spec that an address book with that name 
exist, or even that any exist. The proper behavior is for a client to do a 
PROPFIND Depth:1 on the address book home and look for valid address book 
collections in there.

-- 
Cyrus Daboo

---

From: Nick Zitzmann <[email-masked]>
Date: Wed, 18 Nov 2009 20:10:18 -0700

On Nov 18, 2009, at 7:44 PM, Cyrus Daboo wrote:

> AddressBook.app currently requires an addressbook collection to be present in the user's address book home.

Then I take it that's a bug. I filed Radar #7407407 on this...

> There is no requirement in the spec that an address book with that name exist, or even that any exist. The proper behavior is for a client to do a PROPFIND Depth:1 on the address book home and look for valid address book collections in there.

Possibly dumb question #2: If that is true, then is the proper behavior for clients to also support all collections at once, or is it okay to work with one collection? I'm wondering because my client doesn't really have a concept of card collections beyond groups, but those groups have an optional-many-to-many relationship with their contacts, and I noticed that Address Book stores its groups as vCards on the server... The draft doesn't say anything about collection support for clients AFAICT.

Nick Zitzmann