Subject: RE: [nfsv4] Preview of new locking chapter

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 3 Aug 2006 10:59:21 -0400

> It seems that there needs to continue to be the stateid/credential 
> association to protect against the denial of service that is present.

The original denial-of-service concern was with respect to the 
owner sequence value which no longer exists, at least in part.
Given that v4.1 removes the owner sequence and the stateid sequence 
as denial-of-service targets, I think we should rethink this 
issue.  It still may be we want to go in this direction, but the
landscape has changed a bit, and I think with Mike's changes for
session, I don't think we have a denial of service issue.

> As long as we don't limit the size of COMPOUNDs too severly with 
> respect to the ERR_RESOURCE issue. :-)

Good point.  I think the right solution is to have TEST_STATEID accept
an array of stateids and give you the status of each one.  I have an
op description for that op which I just have to give a final once over.

> Did you mean to say that we rename FREE_BADLOCK to FREE_STATEID and 
> it would be used for the BAD stateid recovery as described and add 
> the ability to free/release a lock-stateid that was associated with 
> a valid open-stateid but did not have any byte-range locks currently 
> active?  Yes, I like that idea.

Then I'll write up the op description on that basis and updated the
spec chapter to use the new names.

Just to clarify, the stateid lifetime for a lock stateid will not 
termininate as a result of LOCKU but will exist until either the
OPEN of which it is a part is closed or there is FREE_STATEID.  This
would limit the need for open_lock_owner4.  I'll insert clarifying
language in the locking chapter.

-----Original Message-----
From: Spencer Shepler [mailto:[email-masked]] 
Sent: Wednesday, August 02, 2006 9:46 PM
To: [email-masked]
Subject: Re: [nfsv4] Preview of new locking chapter

On Wed, Noveck, Dave wrote:
> 
> > The reason for this is that these three areas are intermixed because

> > of the structure we had in rfc3530 and was never cleaned up.
> 
> I think they are still intermixed but it would certainly be good if we

> could figure out how to clearly separate them.  I need to think about 
> how this can be done.

Since this is more editorial in nature, let's take it off-list.

> > The first is the issue of having the principal used at OPEN be the 
> > same one that is used for later use of the resultant stateid. 
> > Simply, does the same user have to use the stateid on READ/WRITE as 
> > did the OPEN.
> 
> There seemed to be a general agreement that we want to go in the 
> direction of tying credentials to owner state but there wasn't 
> sufficient discussion to determine exactly how far in this direction 
> we want to go.  There are a lot of issues here about the interaction 
> of state-owners and stateids.
> 
> We can start with your statement about credentials that must be used 
> when a stateid returned from open is used by READ, WRITE and 
> presumably truncate.

Okay.

> 
> Then there is a related question about whether open-owners are tied to

> credentials.  Can the same owner do two different opens with different

> credentials?

Yes. Especially improtant for clients that would like to minimize the
number of open-owners.

> What about LOCK requests?  Must they use the same credential?  If not,

> can READs and WRITEs using the locking stateid, use the lock 
> credential or the open credential?  Similarly can the same lock-owner 
> use different credentials on different open files?

Again, for the client that minimizes the number of open-owners the
ability to have a different credential on LOCK would be good.

It seems that there needs to continue to be the stateid/credential
association to protect against the denial of service that is present.

> What about using delegation stateids?
> 
> I think someone has to come up with a proposal in this area.

True.

> > Is there a scalability issue, that we care about, with the 
> > introduction of TEST_STATEID and FREE_BADLOCK?  If the client 
> > receives the error indication on the SEQUENCE op results, how does 
> > it go about clearing the condition?  At the moment, it seems that 
> > the client must wonder through all of its state using TEST_STATEID 
> > until the error in SEQUENCE goes away.  I suppose that client's 
> > could essentially ignore that condition until it received and error 
> > on the revoked state and then did the FREE_BADLOCK.
> 
> I think many clients will follow that approach, but I think we have to

> have reasonable facilities for those that do want to get an immediate 
> indication of revoked locks.  Right now the V4.0 spec says you are 
> supposed to do this, even when timers show you might possibly be 
> subject to a lease expiration.
> 
> So I think that what exists is reasonably scalable, because a client 
> can package a large number of TESTLOCKs in a single COMPOUND and can 
> issue these COMPOUNDs in parallel.  He will then have a list of 
> revoked locks

As long as we don't limit the size of COMPOUNDs too severly with respect
to the ERR_RESOURCE issue. :-)

> and as each COMPOUND finishes he can do a FREE_BADLOCK on any that 
> show up as revoked.  I think that is reasonable but if it people don't

> think it is, we could implement an op to get all the revoked stateids 
> for a client.  Then you need a cookie if the list is too big to come 
> back and then you have issues of how long such state is kept, what to 
> do about changes in the list in the interim etc, so I'd like to stay 
> away from that if we can.

I agree.  I would prefer to avoid the enumeration feature as well.

I am okay with the idea that most clients will be happy to "ignore"
the SEQUENCE error until it stumbles into the stateid that is causing
the problem.  So maybe some text to that effect in the discussion would
be helpful.  I am also okay with the client bundling TESTLOCK requests.
Just need to ensure that the TESTLOCK operation is defined such that it
doesn't error out so that a longer COMPOUND would have a chance to
succeed.

One final nit is the names.  Can we have TEST_STATEID and
FREE_BAD_STATEID -or RELEASE_BAD_STATEID to generalize a litte?

> > Also, RELEASE_LOCKOWNER should not be made mandatory not to
implement.
> > The LOCK request with the open_to_lock_owner4 argument served two 
> > purposes.  The first was to establish the sequence number (which we 
> > do not need with the use of sessions) but it also associates the 
> > lock_owner with the open_owner.
> 
> Indirectly.  Actually it associates the open-owner/lock-owner/file 
> triple with a stateid.  Once you have that stateid, you can release or

> get additional locks by using stateid when you call LOCK.

Sorry for being imprecise, yes, it is the triple.

> > If RELEASE_LOCKOWNER is not available, the client will need to deal 
> > with error returns for regular LOCK requests or make every call with

> > the open_to_lock_owner4 type.
> 
> Certainly he doesn't have to do this if he has some lock for that 
> lock-owner/open-owner/file triple.
> 
> Whether he has to do it when he doesn't have a lock depends on the 
> lifetime rules for a stateid.  If your assumption is that a stateid 
> for a given lock-owner subordinate to given open can go away as soon 
> as the last lock goes away (even when the open still exists), then you

> would have to use open_to_lock_owner4, which doesn't seem all that bad

> to me.

I was assuming the shortest duration of the lock-stateid guessing that
there would be an implementation that would make that choice.

My concern with the continual use of the open_to_lock_owner4 branch was
the potential size of the lock_owner4.  Didn't seem that we should
punish the client becaused it chose a larger value for lock_owner4.
Also thought it would simplify the client implementation if it could
count on the particular encoding and not have to worry too much about an
error coming back from the server when it was in the middle of doing
file locking.

> However, my assumption (not written down) is that the stateid lifetime

> would not end until the open goes away.  If that were the case, then 
> you could continue to use the stateid until the file was closed.
> If you had a case in which a file was open forever and you continued 
> to use a large set of lockowner then perhaps there might be an issue 
> but I kind of feel that is unlikely.  If we did have this case, it

I am not so sure.  We know that AIX uses a small number open-owners and
may continue to do so going forward (and other clients may choose this
path as well).  Seems prudent to account for such use.

> would simpler to give people a way to get rid of lock stateid's that 
> had no locks assigned to them.  Perhaps FREE_BADLOCK could be rename 
> so it would allowed to free any stateid for which there was actual 
> associated valid lock?

Did you mean to say that we rename FREE_BADLOCK to FREE_STATEID and it
would be used for the BAD stateid recovery as described and add the
ability to free/release a lock-stateid that was associated with a valid
open-stateid but did not have any byte-range locks currently active?
Yes, I like that idea.

> > The RELEASE_LOCKOWNER operation should remain and used by the client

> > when it knows that the lockowner is no longer pursuing locks or 
> > makes the determination itself (because of resource limits) that it 
> > can be released.
> 
> We could put this back but it seems odd when the real issue is stateid

> lifetime to control this at the lockowner level.

Agreed.  Let's insert the ability to release the lock-stateid from the
server's bookkeeping.

Spencer

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Mike Eisler <[email-masked]>
Date: Thu, 3 Aug 2006 09:14:08 -0700 (PDT)

> owner sequence value which no longer exists, at least in part.

Correct.

> session, I don't think we have a denial of service issue.

I agree.

To solve the issue in NFSv4.0 one has to implement
the server to bind the cred that OPENs the file
to that which uses the open owner's sequence and the
lock owner sequences.

As it stands now, the sequencing is exclusively in the SEQUENCE
operation. With the introduction of the SSV concept,
I've analyzed several ways an attacker could
manipulate the slot table and disrupt other users of the
session, and my conclusion is that connection hijacking is
the only means possible. Even in NFSv3, connection hijacking can
disrupt
other users, even if RPCSEC_GSS with privacy is used. 
Connection hijacking can only be foiled with IPsec.

Some form of stateid/credential association is nonetheless useful.
Let's
say user A opens a file, and then invokes a setuid
program that takes the open file descriptor. The setuid program
should be using A's credential in order to ensure that any
I/O or lock operations it issues continue to work.

This is a client problem though. It needs to use the cred of the
file opener, not that of the process doing the I/O.
I believe most vnoded kernels do this anway.

Therefore, stateid/cred association comes for free. It also
doesn't require the client maintain a separate open_owner
for each cred; and open_owner can be shared across files and
creds. The only requirement is that the cred of a file's opener,
be used on the operations of that file.

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 03 Aug 2006 13:44:08 -0400

> I believe most vnoded kernels do this anway.

How would a proposal for stateid/credential association work in a pnfs
context? Did you really mean stateid/principal association?

Cheers,
  Trond

---

From: Mike Eisler <[email-masked]>
Date: Thu, 3 Aug 2006 11:52:25 -0700 (PDT)

> context? Did you really mean stateid/principal association?

I'm not sure what you mena by credential (but it was apparently
different than what I meant). But stateid/principal is what
I meant.

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 3 Aug 2006 10:26:30 -0500

> language in the locking chapter.

Yes.