Subject: Re: [VCARDDAV] Vcarddav: creating new resources

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 15:44:02 +0100

Hi,

Cyrus encouraged me to bring up this topic again.

Last month's discussion ended around 
<http://www.ietf.org/mail-archive/web/vcarddav/current/msg00650.html>, 
with me reporting back from the related discussion over at the HTTPbis 
WG's mailing list 
(<http://lists.w3.org/Archives/Public/ietf-http-wg/2008OctDec/0096.html>).

At this point I think this WG needs to decide whether the "server picks 
the URI" approach should be *allowed*.

If it's not, the carddav specification clearly should state this, and in 
particular rule out those workarounds we're seeing in CalDAV (which got 
discussed over here and on the HTTPbis mailing list).

If it should be allowed (and I think it should), we should add a way to 
do it. PUT does not work for the reasons that came up during the 
discussion (please read the discussion on the httpbis WG mailing list). 
My proposal would be to use "Using POST to add Members to Web 
Distributed Authoring and Versioning (WebDAV) Collections" 
(<http://greenbytes.de/tech/webdav/draft-reschke-webdav-post-latest.html>) 
-- an alternative would be a new verb as proposed earlier ("ADDMEMBER").

If this WG wants to use this proposal, I'm happy to hand it over to the 
WG for completion (I think there are few things left to do, mainly 
taking out the stuff that relates to the unfinished HTTP Link header 
draft, and potentially moving the elements into the "DAV:" namespace).

BR, Julian

Julian Reschke wrote:
> Julian Reschke wrote:
>> ...
>> POST is not a new method.
>>
>> PUT+Location definitively does *not* work everywhere; I'm sure of that 
>> because the WebDAV client code I've written in the past certainly 
>> won't understand it.
>>
>> If the catch-all semantics of POST are a problem, then one way to 
>> handle  this is to offer the client enough information about what POST 
>> semantics are implemented. For instance, AtomPub does this by 
>> requiring the more specific behavior for the Atom feed URIs it 
>> exposes. My proposal (for WebDAV) does the equivalent; it exposes the 
>> URI for "add-member" POSTs on the wire (PROPFIND or DAV:error), so a 
>> client can rely on the desired semantics.
>>
>> BR, Julian
>> ...
> 
> For the record, we *did* move the discussion over to the httpbis WG's 
> mailing list; see thread at 
> <http://lists.w3.org/Archives/Public/ietf-http-wg/2008OctDec/thread.html#msg44> 
> in general, and Roy's message at 
> <http://lists.w3.org/Archives/Public/ietf-http-wg/2008OctDec/0096.html> 
> in particular.
> 
> BR, Julian

_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Helge Hess <[email-masked]>
Date: Sun, 30 Nov 2008 15:55:50 +0100

> Cyrus encouraged me to bring up this topic again.

Its still my opinion that CardDAV itself should say nothing about that  
(neither MUST nor MUST NOT).

The rules of HTTP/WebDAV and potential extensions should apply. If a  
server supplies a POST based creation method, it should advertise  
that. A client might discover that additional option and use it (or  
receive a 405 or 403 or some other error if it attempts to create a  
resource using a regular WebDAV approach).

I have no idea what would justify to put such a generic HTTP extension  
directly into CardDAV (or CalDAV BTW).
Please preserve the proper protocol layering.

Thanks,
   Helge
-- 
http://zideone.com/

---

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 16:06:26 +0100

> (neither MUST nor MUST NOT).

Well, if "server-picks-the-URI" is an allowed option, than the spec 
should state that and explain how to do it.

> The rules of HTTP/WebDAV and potential extensions should apply. If a 

Of course.

> server supplies a POST based creation method, it should advertise that. 

Yes.

> A client might discover that additional option and use it (or receive a 
> 405 or 403 or some other error if it attempts to create a resource using 
> a regular WebDAV approach).

Yes.

> directly into CardDAV (or CalDAV BTW).

I didn't suggest to put it *into* the spec. My proposal is to make a 
decision (because it has led to confusion in CalDAV), and -- if the WG 
thinks it's a use case that should be dealt with -- to reference a 
document that explains how.

Furthermore, *if* the WG thinks my proposal is a good approach,
we should discuss where to finish it, so that CardDAV can indeed 
reference it.

 > Please preserve the proper protocol layering.

Yes, exactly the same way we're doing it with ext-mkcol.

Best regards, Julian

---

From: Helge Hess <[email-masked]>
Date: Sun, 30 Nov 2008 16:29:59 +0100

> should state that and explain how to do it.

Why? Whats the gain in that? Either there is an HTTP extension or not.  
The client is allowed to use any WebDAV enhancement if the server  
supports it?!

You suggest to standardize the POST as _the_ variant to create  
resources, and I'm definitely -1 on that. Its too early and not  
necessary. The addition can fight on its own to become a relevant  
standard :-)

> Yes, exactly the same way we're doing it with ext-mkcol.

Its exactly not the same. Ext-mkcol adds functionality which is  
impossible with plain WebDAV (creation of typed collections).

In contrast the creation of resources is already standardized (in  
HTTP!, with the known and documented drawbacks). Your draft is just an  
optional enhancement.

Now I wouldn't hardcode the core HTTP behaviour in CardDAV by saying  
that a CardDAV server MUST be able to manage URLs choosen by the  
client. Precisely to allow for additional create-options, either yours  
_or others_. Let us resolve that topic at the HTTP level, not in xyzDAV.

(eg I would very much prefer additional, well specified, HTTP response  
header(s) to PUT, though I have no specific proposal ready).

Thanks,
   Helge

---

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 17:35:23 +0100

> Why? Whats the gain in that? Either there is an HTTP extension or not. 

Because so far there isn't one in the context of WebDAV.

The purpose of a spec is to allow interoperability, isn't it?

Of course.

On the other hand, the purpose of a spec is to state how clients and 
servers are supposed to interoperate. So if there are multiple ways to 
achieve a specific functionality, and all of them are optional, you 
haven't really helped implementers.

> necessary. The addition can fight on its own to become a relevant 

I disagree that it's not necessary, as we have proof that CalDAV 
implementers use hacks to achieve the same functionality.

> In contrast the creation of resources is already standardized (in HTTP!, 
> with the known and documented drawbacks). Your draft is just an optional 
> enhancement.

HTTP currently has exactly one way to create a new resource where the 
server picks the URI, and that is POST.

So far, POST isn't used in WebDAV protocols for several reasons. I think 
the most important one is that there's no way for a client to find out 
what exactly a POST request will cause.

We could standardize POST on WebDAV collection resources to have exactly 
the desired semantics. This doesn't work however as it would break 
existing servers that already do something else for POST on collection URIs.

The proposed extension is a lightweight mechanism for a server to opt-in 
into POST-for-create, and also allows to have a *separate* URI for that 
purpose.

> that a CardDAV server MUST be able to manage URLs choosen by the client. 

But that's what the spec currently does. I agree that we shouldn't 
require it if we know that servers won't be able to do it.

> Precisely to allow for additional create-options, either yours _or 
> others_. Let us resolve that topic at the HTTP level, not in xyzDAV.

It already *is* resolved on the HTTP level, and the answer is "just use 
POST".

> header(s) to PUT, though I have no specific proposal ready).

That would be an incompatible change to the semantics of PUT. A 
"success" response PUT means that a server has stored the enclosed 
entity at the specified URI, not somewhere else. A new response header 
can't change that semantics; existing clients will ignore it.

Best regards, Julian

---

From: Helge Hess <[email-masked]>
Date: Sun, 30 Nov 2008 19:17:32 +0100

On 30.11.2008, at 17:35, Julian Reschke wrote:
>> (eg I would very much prefer additional, well specified, HTTP  
>> response header(s) to PUT, though I have no specific proposal ready).
> entity at the specified URI, not somewhere else. A new response  
> header can't change that semantics; existing clients will ignore it.

As I said, I have no good proposal at this time. As discussed before I  
question the real world relevance. With the exception of LOCKs, the  
client will need to deal with evaporating resources anyways.
So its just fine that existing clients will ignore it, in fact thats  
the beauty of it, it doesn't require any changes to existing code.

BTW: there aren't that many interop issues in CalDAV due to this  
issue?! The 'real' CalDAV servers support client side names. And its  
not that hard to add to legacy servers either (its harder, but not  
hard).

The 'hacks' issue usually crops up with GroupDAV, which  
(intentionally) advertises the hacks for IMHO reasonably OK reasons ;-)

Thanks,
   Helge
-- 
Helge Hess
http://zideone.com/

---

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 19:29:26 +0100

> client will need to deal with evaporating resources anyways.

There really is no way to for a client to deal with that case. If the 
resource disappears between the successful PUT and a subsequent 
operation, there's nothing the client can really do, except for retrying 
the operation.

> So its just fine that existing clients will ignore it, in fact thats the 
> beauty of it, it doesn't require any changes to existing code.

Existing code breaks, for instance when it issues a sequence of PUT and 
PROPPATCH.

> BTW: there aren't that many interop issues in CalDAV due to this issue?! 
> The 'real' CalDAV servers support client side names. And its not that 
> hard to add to legacy servers either (its harder, but not hard).

If that's the case, and people agree that the situation will be the same 
for CardDAV, then yes, there's nothing this WG needs to do and we can 
just leave the spec as it is (mandating "proper" support for PUT).

BR, Julian

---

From: Helge Hess <[email-masked]>
Date: Sun, 30 Nov 2008 19:52:06 +0100

On 30.11.2008, at 19:29, Julian Reschke wrote:
> Helge Hess wrote:
>> ...
>> As I said, I have no good proposal at this time. As discussed  
>> before I question the real world relevance. With the exception of  
>> LOCKs, the client will need to deal with evaporating resources  
>> anyways.
> > ...
> There really is no way to for a client to deal with that case. If  
> the resource disappears between the successful PUT and a subsequent  
> operation, there's nothing the client can really do, except for  
> retrying the operation.

Why would the UA ever retry the operation? It got a 200 OK which means  
the PUT was successful. If the resource is missing, it can only mean  
that a concurrent user moved or deleted the resource (by current HTTP  
semantics).
This is exactly what the client should then report to the user.

BTW: this behaviour is not exactly uncommon. Just consider a drop box  
folder, where you have creation permissions, but no read or update.

>> So its just fine that existing clients will ignore it, in fact  
>> thats the beauty of it, it doesn't require any changes to existing  
>> code.
> Existing code breaks, for instance when it issues a sequence of PUT  
> and PROPPATCH.

I know of no CalDAV or CardDAV client which does this (except for  
collections, which is a different thing anyways). And for the reasons  
stated above the client would need to LOCK the collection - if the PUT 
+PROPPATCH is supposed to be atomic.

I'm aware that the issue exists with b0rked legacy file oriented  
WebDAV clients, but thats no good reason to add enhancements to  
*CardDAV* :-)

>> issue?! The 'real' CalDAV servers support client side names. And  
>> its not that hard to add to legacy servers either (its harder, but  
>> not hard).
> If that's the case, and people agree that the situation will be the  
> same for CardDAV, then yes, there's nothing this WG needs to do and  
> we can just leave the spec as it is (mandating "proper" support for  
> PUT).

That would be good and summarizes my point, thanks.

As mentioned this shouldn't stop anyone from working on good solutions  
which enhance the base spec. IMHO it would be precipitous to select/ 
hardcode the POST proposal at this point of time.

Greets,
   Helge

---

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 20:10:55 +0100

> This is exactly what the client should then report to the user.

OK, it can *either* report the problem (and fail the request), or retry. 
I do agree that retrying will not help in this case.

> folder, where you have creation permissions, but no read or update.

Yes, scenarios like this will suffer from the same problem.

> stated above the client would need to LOCK the collection - if the 
> PUT+PROPPATCH is supposed to be atomic.

It may not be a problem for CalDAV and CardDAV, where properties aren't 
used a lot. It is a problem with generic clients using WebDAV for remoting.

And no, locks usually are not required. A server that "immediately" 
moves a resource away is breaking the contract, so it's an edge case, 
and clients definitively do not need to worry about it in general.

Furthermore, the client can already achieve a level of protection 
against simultaneous changes by making the subsequent operation 
conditional (in case PUT returned an Etag).

Finally, for servers that *do* rename after PUT -- how would a WebDAV 
LOCK help here?

> I'm aware that the issue exists with b0rked legacy file oriented WebDAV 
> clients, but thats no good reason to add enhancements to *CardDAV* :-)

I do not agree these clients are broken.

If a resource moves away immediately after PUT, this is either an edge 
case, or, if it occurs *always*, a broken server.

>> same for CardDAV, then yes, there's nothing this WG needs to do and we 
>> can just leave the spec as it is (mandating "proper" support for PUT).
> 
> which enhance the base spec. IMHO it would be precipitous to 
> select/hardcode the POST proposal at this point of time.
> ...

That really doesn't work. If the base spec *requires* the 
client-picks-URI pattern, then a new server-picks-URI proposal doesn't 
help any server implementer, as client-picks-URI needs to be implemented 
anyway. So the key question really is: Should CardDAV require it?

Best regards, Julian

---

From: Helge Hess <[email-masked]>
Date: Sun, 30 Nov 2008 21:12:58 +0100

On 30.11.2008, at 20:10, Julian Reschke wrote:
> It may not be a problem for CalDAV and CardDAV

Thats the topic here, thanks :-)

> And no, locks usually are not required.

Locks are required to make such a operation atomic. How can you  
possibly say its not?

> A server that "immediately" moves a resource away is breaking the  
> contract

My example was not about the server moving/deleting/modifying a record  
but about concurrency with another user/client.
I'm just saying that this is no different to the server-move case from  
a client side perspective.

> so it's an edge case, and clients definitively do not need to worry  
> about it in general.

Its not an edge case in a shared folder environment. If you have a  
single company calendar/contact folder accessed by 50 or more users  
and synchronized in an automatic way, concurrency is a BIG issue. And  
with HTTP technology we want to scale much higher.

Especially an issue when we start to have XMPP push which tell the  
clients to resync in the background ASAP.

> conditional (in case PUT returned an Etag).

Yes, a client which actually does that will also properly detect a 404  
and report that properly to the user. Which is fine.
The etag obviously doesn't make the operation atomic. The PROPPATCH  
can still fail (in the 'edge' case) and lead to inconsistent state.

> Finally, for servers that *do* rename after PUT -- how would a  
> WebDAV LOCK help here?

We are going circles here. My sole point is that it doesn't  
(shouldn't) matter for any existing WebDAV software.

Breaks in edge cases are unacceptable in automated sync and proxying  
scenarios - especially if we are talking about an Internet standard  
documents.

IMHO:

A header based solution has advantages because it still works with  
existing clients, they just ignore the header and treat it like any  
other concurrent user edit. Its very easy to add to legacy or new  
software. For a working POST solution all clients and servers must be  
changed to make it work.
(the header solutions still works with a lot of old software and  
scenarios 'well', the POST solution will work with no old software at  
all)

BUT: I'm not really pro headers or pro POST or pro extra-method. All  
I'm saying is that its to early to decide for one approach, because  
there is no clear winner yet. Just specify the variants as separate  
RFCs and let the implementors decide if they implement one (or all!).

Greets,
   Helge

---

From: Julian Reschke <[email-masked]>
Date: Sun, 30 Nov 2008 22:20:23 +0100

> Thats the topic here, thanks :-)

PUT followed by PROPPATCH may not be the issue (because both don't use 
WebDAV properties a lot), but that doesn't mean it's a problem anyway. 
There are other operations that could follow (such as ACL, 
VERSION-CONTROL). Also, in general, a client assumes that something will 
keep it's URL unless somebody moves it away on purpose.

> Locks are required to make such a operation atomic. How can you possibly 
> say its not?

It's not required in the real world. Servers do not move away resources 
once they were created. If they do that, they are broken, sorry.

It's much more efficient not to use a lock (in particular as locks are 
totally optional), and just to be optimistic instead.

> I'm just saying that this is no different to the server-move case from a 
> client side perspective.

The difference is between an edge case (where a client can just retry) 
and it happening all the time.

> single company calendar/contact folder accessed by 50 or more users and 
> synchronized in an automatic way, concurrency is a BIG issue. And with 
> HTTP technology we want to scale much higher.

That sounds as if in this case, we really want to support the 
server-picks-the-URI behavior.

I think mandating WebDAV locks will not fly at all, as it will kill 
concurrency (unless you apply the lock only to the resource itself).

> clients to resync in the background ASAP.

Will that resync involve creating new calendar entries?

> The etag obviously doesn't make the operation atomic. The PROPPATCH can 
> still fail (in the 'edge' case) and lead to inconsistent state.

The PROPPATCH at least would fail if the previously written resource was 
replaced by a different resource.

> We are going circles here. My sole point is that it doesn't (shouldn't) 
> matter for any existing WebDAV software.
> 

If "breaks in edge cases" is unacceptable, then I don't see how 
client-picks-URI will work, *unless* the server rejects PUT-to-create if 
it isn't protected by "If-None-Match: *".

If that *is* the case, clients can just rely on the PUT/201 response and 
continue to use the URI.

> software. For a working POST solution all clients and servers must be 

a) A new response header for PUT has the same problem as your proposal 
to re-use "Location". It breaks the semantics of PUT defined in RFC 
2616. I'd recommend you go ahead and write down a detailed proposal and 
try to get IETF support for it.

b) What "working" clients and servers are you referring to? I thought 
we're discussing CardDAV?

> scenarios 'well', the POST solution will work with no old software at all)
> 
> BUT: I'm not really pro headers or pro POST or pro extra-method. All I'm 
> saying is that its to early to decide for one approach, because there is 
> no clear winner yet. Just specify the variants as separate RFCs and let 
> the implementors decide if they implement one (or all!).

Looking forward to these other proposals.

However: if CardDAV changes to allow servers not to support 
client-picks-URI (rejecting PUT on unmapped URIs), it *will* need to 
state what to do instead, otherwise it'll be underspecified.

BR, Julian