Subject: RE: [nfsv4] Directory delegations and notifications

From: "Khan, Saadia" <[email-masked]>
Date: Thu, 21 Apr 2005 14:27:04 -0700

> -----Original Message-----
> From: Brian Wickman [mailto:[email-masked]] 
> Sent: Thursday, April 21, 2005 11:40 AM
> To: Khan, Saadia
> Cc: [email-masked]
> Subject: Re: [nfsv4] Directory delegations and notifications
> 
> 
> 
>   I would like to make a few remarks about implications and 
> caveats with both Saadia's draft and my personal draft.  I 
> agree with Carl and the rest that directory write delegations 
> are probably not feasible on a v4.1 time-scale because of 
> issues that Dave brought up.  I should point out that 
> directory write delegations are an optional feature with my 
> draft (in fact, all delegations are optional features in my 
> draft, because the server can simply either return 
> ERR_NOTSUPP or just deny all write delegation requests.)
> 
>   There are a few snippets from Saadia's draft about which I'd like to
> remark:
> 
>  > These extensions allow improved namespace cache 
> consistency to be  > achieved through delegations and 
> synchronous recalls alone without  > asking for 
> notifications. In addition, if time-based consistency is  > 
> sufficient, asynchronous notifications can provide 
> performance  > benefits for the client, and possibly the 
> server, under some common  > operating conditions such as 
> slowly-changing and/or very large  > directories.
> 
>   As Carl mentioned, and as some saw at IETF, in the two case 
> studies that I analyzed, notification traffic (in both bytes 
> and moreso in ops) far outweighed the traffic necessary to 
> maintain the directory cache -- even in slowly changing and 
> large directories.  However, this isn't to say that the 
> problem may exist under certain understood operating 
> conditions at their extremes.  Under these circumstances, 
> Dave's idea of READDIR_CHANGES makes a lot more sense.  
> According to my numbers, on the average, < 5% of directory 
> entries change between the read directory delegation 
> revocation and the time the client touches its directory 
> cache again.  In general, this corresponds to only one 
> directory entry.  I believe this information is in my IETF slides.

I wonder why you say a lot of notification traffic when only one
directory entry changed in a large directory? There should only be one
notification sent. Did your prototype actually implement notifications
in addition to delegations or did you look at the network traffic to
guesstimate the amount of notifications that would be sent for certain
workloads? Do you have a paper on that?

> 
>  > ...asynchronous notifications provide a way to reduce 
> network traffic  > as well as improve client performance in 
> certain conditions.  > Notifications would not be requested 
> when the goal is just cache  > consitency.
> 
>   First point: The last sentence here makes me pose the 
> question of when notifications /would/ be requested by the 
> client.  That is, what are the client best practices 
> regarding requesting notifications?  Also, under what 
> circumstances will the client also be wanting to hold a 
> directory read delegation on the same directory?

The client can ask for notifications when it finds itself sending
readdirplus calls to the server for large directories and can keep the
directory contents and attributes updated without having to poll the
server every few seconds.

> 
>   Second point: When you want notifications yet don't want 
> notifications on all possible events, then you get 
> effectively a delegation on the remaining events (i.e., 
> synchronous recall).  For example, if I request to get 
> notifications when entries are added, deleted or renamed 
> within the directory, my notification will be revoked when 
> the directory's attributes change, say, via a SETATTR.  It 
> makes no sense that notifications should be recalled by a 
> delegation-like event.  If I'm going to ask for 
> notifications, I want the ability to choose which 
> notifications I receive to be meaningful without having to 
> worry about my notifications going away (being "revoked") 
> from an event that I do not care about.

Notifications are never revoked, the delegation is recalled if the
directory changes in a way that is not covered by a notification. And
attribute changes do not cause a delegation to be recalled. However if
you ask for notifications on create and not for removes, then a remove
will cause the delegation to be recalled. 

> 
>    Instead, notifications should be recalled when the server 
> feels the traffic being generated by the notification is far 
> too much for merit. They shouldn't be tied to the idea of 
> delegation, because it makes it difficult if not impossible 
> to acquire just notifications.  The most useful case is to 
> specify no notifications, in which case you get a true 
> delegation.  In this case, it'd be equivalent to using the 
> operation in my draft.

I am a bit confused about what you mean by recalling notifications, do
you mean recalling the delegation? The delegation can be recalled if the
server decides it is sending way too many notifications on a certain
directory. 

> 
>   Lastly, delegations are a construct that improve cache 
> consistency.  I demonstrated this to be true in practice.  
> Notifications, in the common case, cause far too much traffic 
> and don't improve upon v2/v3 time-based consistency.  
> However, they clearly have use cases but in the uncommon 
> case, when a given application genuinely would like to see 
> such events and be notified in user-land about such events 
> (e.g., GUI browser or producer/consumer system of sorts.)  As 
> such, notifications should be treated differently from 
> delegations and whether or not they are granted should be 
> governed by a policy different than that of delegation 
> granting policy.
> 
>   There are no drawbacks of simply having separate 
> GET_DIR_DELEGATION and GET_DIR_NOTIFICATIONS operation.  It 
> should make the camp that wants directory delegations happy.  
> It should make the camp that wants directory notifications 
> happy.  Having the two constructs tied together reduces the 
> utility of both, and virtually breaks the utility of 
> notifications altogether.  Implementing one or the other is 
> optional and you don't need to implement half of anything to 
> get the utility that you would like.
> 

I don't think I fully understand your approach here. If having two
separate operations now allows you to add notifications on top of
delegations, then that is something that the current proposal already
does. If that is not the case, then having a delegation as well as
notifications for one directory would cause that directory to receive
the notification "and" the delegation to be recalled in the event of a
change. I don't see any benefits in that, actually now the network
traffic has increased between the server and the client, which is what
you were trying to reduce in the first place.

Saadia

> My two cents.
> Cheers,
> Brian

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Brian Wickman <[email-masked]>
Date: Fri, 22 Apr 2005 10:56:13 -0400 (EDT)

> workloads? Do you have a paper on that?

I implemented delegations and notifications within a simulator and
replayed NFSv3 traffic within this environment using liberal server
policies with respect to granting delegations and notifications.

I measured client behavior under a scenario with delegations +
notifications and under a scenario with just delegations.  In the
notifications case, one workload exhibited 500x more notifications than
actual directory queries (e.g., LOOKUP, GETATTR).  The most conservative
workload still had 20x more notifications.  When you discard attribute
change notifications, there are still 56x and 12x more notifications than
queries in those respective workloads.  What is often happening here is
other clients sharing the same directories (think mail spools) are
creating and removing short-lived entries before the delegated clients
inspect the directory contents.  If the notification timescale is
sufficiently large, many of these notifications would not need to be
delivered.  However, presumably the timescale will be similar to that
used in v3 time-based consistency, which is short enough that these
notifications would be sent.

> will cause the delegation to be recalled.

By "notifications are never revoked," are you saying that notifications
will still be delivered if the delegation state is recalled?  I do not get
that impression from the draft if that is the case.

> > There are no drawbacks of simply having separate GET_DIR_DELEGATION
> > and GET_DIR_NOTIFICATIONS operation.  It should make the camp that
> > wants directory delegations happy.  It should make the camp that wants
> > directory notifications happy.  Having the two constructs tied
> > together reduces the utility of both, and virtually breaks the utility
> > of notifications altogether.  Implementing one or the other is
> > optional and you don't need to implement half of anything to get the
> > utility that you would like.
>

No, having two separate operations allows you to have notifications
without delegations.  Furthermore, it allows server and client
implementors to have a clearer separation of functionality and would make
the implementation of directory read delegations more straightforward (by
effectively treating them as read file delegations on the directory
object.)  In your approach, notifications are tied to delegations and are
strictly a way to maintain caches with v3-like semantics (even if
notifications are optional).  I've been finding in my case studies that as
a cache maintenance technique, notifications are a poor fit.  Instead, I
see notifications as useful when they are visible to the user as an
application-specific tool such as a file browser.  However, in this case,
a delegation would /not/ be desirable.  They could still be used to
maintain a weak v3-style time-based consistency, but at least delegations
would not be implied baggage.  If the delegation is implied, then you need
to select notifications on /all/ events even if they are uninteresting to
the client so as to guarantee persistence in the case that the
uninteresting event occurs.

Hope that clarifies some of my points.

Cheers,
Brian