Subject: Re: [nfsv4] Another question, this one for RFC 7530

From: David Noveck <[email-masked]>
Date: Thu, 1 Oct 2015 10:38:40 -0400

> At least some version(s) of Linux client (RHEL 6.4) has demonstrated an
issue, running cthon04 lock test8.

> Apparently the client loses track of the stateid during the repeated
lock/unlock cycles this test does.

It would make sense for a client to assume that it is free to forget about
a lock stateid if it has no
associated locks.  If that is what is happening, this issue boils down to
the question of whether the client
is within its rights to do so.

The first paragraph of 8.2.4 is relevant here:

Stateids must remain valid until either a client restart or a server

restart or until the client returns all of the locks associated with
the stateid by means of an operation such as CLOSE or DELEGRETURN.
If the locks are lost due to revocation, as long as the client ID is
valid, the stateid remains a valid designation of that revoked state
until the client frees it by using FREE_STATEID.  Stateids associated
with byte-range locks are an exception.  They remain valid even if a
LOCKU frees all remaining locks, so long as the open file with which
they are associated remains open, unless the client frees the
stateids via the FREE_STATEID operation.

As I read the above, it is saying that the server is not free to forget
about such
stateids and that if the client remembers and uses the value, the server is
obligated to remember the associated  open stateid and lockowner. The
question which is not specifically addressed and which is relevant to
Frank's
issue, is whether the client is free to forget about such stateids. I would
argue
that the implication is that it is free to forget such information, since
there is
nothing explicitly saying it needs to be remembered.

Given that the server is explicitly told to remember the stateid, I'd say
there is no
basis for the server cancelling this stateid and issuing a new one in the
case in
which the client specifies a new lockowner which not truly "new" for the
current
open. That had been suggested previously as an option in this case.

> The Ganesha server currently ends up returning an NFS4ERR_RESOURCE,

Why?

> which certainly needs to change whatever the decision in this case is.

Agreed.

On Wed, Sep 30, 2015 at 7:29 PM, Frank Filz <[email-masked]> wrote:

> > Frank Filz wrote:
> > > It's not clear whether a client is allowed to send a LOCK request with
> > > new_lock_owner true when the client has previously done so for the
> > > same lock owner and file (and thus the server has an active stateid
> > > for that owner/file pair). If it is allowed, should the server re-use
> > > the same stateid, or discard the old stateid and issue a new one?
> > >
> > I don't believe that the RFCs state this explicitly, but the combination
> of
> > "new" and no obvious answer to your above question (how to handle it, if
> > allowed) made me decide to consider it an error.
> >
> > The FreeBSD server will return NFS4ERR_INVAL for this and I'll be
> curious to
> > see what others think?
> >
> > I have not heard of this causing a problem for extant clients.
> >
> > Fyi, the FreeBSD server does allow the same lock stateid to be used for
> > multiple file handles, but logs that it encountered this and to the best
> of my
> > knowledge, this has never been logged.
> >
> > As such, I believe extant client use the open_to_lock_owner case to
> create a
> > lock stateid for each (FH, lock_owner) tuple and only if it doesn't
> already
> > have a lock stateid for the tuple.
> >
> > Hopefully others can clarify this for you more definitively, rick
>
> At least some version(s) of Linux client (RHEL 6.4) has demonstrated an
> issue, running cthon04 lock test8. Apparently the client loses track of the
> stateid during the repeated lock/unlock cycles this test does. The Ganesha
> server currently ends up returning an NFS4ERR_RESOURCE, which certainly
> needs to change whatever the decision in this case is.
>
> Frank
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: "Frank Filz" <[email-masked]>
Date: Thu, 1 Oct 2015 09:51:38 -0700

> From: David Noveck [mailto:[email-masked]]
> Sent: Thursday, October 1, 2015 7:39 AM
> To: Frank Filz <[email-masked]>
> Cc: Rick Macklem <[email-masked]>; [email-masked]
> Subject: Re: [nfsv4] Another question, this one for RFC 7530
> 
> It would make sense for a client to assume that it is free to forget about a
> lock stateid if it has no associated locks.  If that is what is happening, this issue
> boils down to the question of whether the client is within its rights to do so.
> 
> Stateids must remain valid until either a client restart or a server restart or
> until the client returns all of the locks associated with the stateid by means of
> an operation such as CLOSE or DELEGRETURN.
> If the locks are lost due to revocation, as long as the client ID is valid, the
> stateid remains a valid designation of that revoked state until the client frees
> it by using FREE_STATEID.  Stateids associated with byte-range locks are an
> exception.  They remain valid even if a LOCKU frees all remaining locks, so
> long as the open file with which they are associated remains open, unless the
> client frees the stateids via the FREE_STATEID operation.
> 
> As I read the above, it is saying that the server is not free to forget about
> such stateids and that if the client remembers and uses the value, the server
> is obligated to remember the associated  open stateid and lockowner.  The
> question which is not specifically addressed and which is relevant to Frank's
> argue that the implication is that it is free to forget such information, since
> there is nothing explicitly saying it needs to be remembered.
> 
> Given that the server is explicitly told to remember the stateid, I'd say there
> is no basis for the server cancelling this stateid and issuing a new one in the
> case in which the client specifies a new lockowner which not truly "new" for
> the current open.  That had been suggested previously as an option in this
> case.

Ok, then we just need to fetch out the original stateid and re-use it.

> > The Ganesha server currently ends up returning an NFS4ERR_RESOURCE,

It's the result of error translation because we were creating a new stateid and then trying to hash it by owner/file and getting a collision with the old stateid. We just need to up front determine if there IS an old stateid to re-use.

> > > It's not clear whether a client is allowed to send a LOCK request
> > > with new_lock_owner true when the client has previously done so for
> > > the same lock owner and file (and thus the server has an active
> > > stateid for that owner/file pair). If it is allowed, should the
> > > server re-use the same stateid, or discard the old stateid and issue a new
> one?
> > >
> > I don't believe that the RFCs state this explicitly, but the
> > combination of "new" and no obvious answer to your above question (how
> > to handle it, if
> > curious to see what others think?
> >
> > Fyi, the FreeBSD server does allow the same lock stateid to be used
> > for multiple file handles, but logs that it encountered this and to
> > the best of my knowledge, this has never been logged.
> >
> > create a lock stateid for each (FH, lock_owner) tuple and only if it
> > doesn't already have a lock stateid for the tuple.
> >
> At least some version(s) of Linux client (RHEL 6.4) has demonstrated an issue,
> running cthon04 lock test8. Apparently the client loses track of the stateid
> during the repeated lock/unlock cycles this test does. The Ganesha server
> currently ends up returning an NFS4ERR_RESOURCE, which certainly needs to
> change whatever the decision in this case is.
> 
> Frank

---

---

From: Rick Macklem <[email-masked]>
Date: Thu, 1 Oct 2015 19:06:45 -0400 (EDT)

Frank Filz wrote:
> > a
> > restart or
Hate to say it Dave, but you quoted from the wrong RFC. Here's the para from 7530:
9.1.4.4.  Stateid Lifetime and Validation

   Stateids must remain valid until either a client restart or a server
   restart, or until the client returns all of the locks associated with
   the stateid by means of an operation such as CLOSE or DELEGRETURN.

When I read the above sentence, I don't see anything that says the "must remain valid"
only applies to the server and not the client.

   If the locks are lost due to revocation, as long as the client ID is
   valid, the stateid remains a valid designation of that revoked state.
   Stateids associated with byte-range locks are an exception.  They
   remain valid even if a LOCKU frees all remaining locks, so long as
   the open file with which they are associated remains open.

This notes that the lock stateid remains valid even after all the locks are LOCKU'd
and, again, doesn't seem to apply to server only to me.

Sorry, but I think the above says that the client is supposed to remember a lock
stateid until the related Open is CLOSE'd. For DELEGRETURN, I believe the above statement
refers to the delegation stateid and not a lock stateid, since the lock stateid isn't
related to a delegation, but is related to an Open via the open_to_lock_owner4 case
of the LOCK operation.

If you do allow a client to "reuse" the lock_owner via open_to_lock_owner4, then
what is done with the open_seqid and lock_seqid?
- Does the server...check the open_seqid using the open_stateid and then set
  the seqid for the lock_owner to lock_seqid.
OR
- Does the server check the lock_seqid against the lock_owner it already has
  (and was required to retain) and fail the operation with NFS4ERR_BAD_SEQID
  if it isn't correct.
OR ???
Since this isn't specified by the RFC, I think the safe answer is to require a
client to obey the "must remain valid" until it does a CLOSE on the associated
open stateid.

Also, since extant servers will return errors for this case, I think it would
be prudent for client implementors to follow the "must remain valid" edict.

rick

> > about
> > there
> It's the result of error translation because we were creating a new stateid
> and then trying to hash it by owner/file and getting a collision with the
> old stateid. We just need to up front determine if there IS an old stateid
> to re-use.
> 
> > issue,

---

From: "Frank Filz" <[email-masked]>
Date: Thu, 1 Oct 2015 16:24:26 -0700

> From: Rick Macklem [mailto:[email-masked]]
> 
> Frank Filz wrote:
> > > It would make sense for a client to assume that it is free to forget
> > > about a lock stateid if it has no associated locks.  If that is what
> > > is happening, this issue boils down to the question of whether the
> > > client is within its rights to do so.
> > >
> > > until the client frees it by using FREE_STATEID.  Stateids
> > > associated with byte-range locks are an exception.  They remain
> > > valid even if a LOCKU frees all remaining locks, so long as the open
> > > file with which they are associated remains open, unless the client
> > > frees the stateids via the FREE_STATEID operation.
> > >
> This notes that the lock stateid remains valid even after all the locks are
> LOCKU'd and, again, doesn't seem to apply to server only to me.
> 
> Sorry, but I think the above says that the client is supposed to remember a
> lock stateid until the related Open is CLOSE'd. For DELEGRETURN, I believe
> the above statement refers to the delegation stateid and not a lock stateid,
> since the lock stateid isn't related to a delegation, but is related to an Open
> via the open_to_lock_owner4 case of the LOCK operation.
> 
> If you do allow a client to "reuse" the lock_owner via open_to_lock_owner4,
> then what is done with the open_seqid and lock_seqid?

We already have to handle re-use of lock owner since the same lock owner could be used for a different file, so the client had better be keeping proper track of the owner seqid (this is one of the places that's seriously cleaned up in NFS v4.1...).

Frank

> Also, since extant servers will return errors for this case, I think it would be
> prudent for client implementors to follow the "must remain valid" edict.
> 
> rick
> 
> > > As I read the above, it is saying that the server is not free to
> > > forget about such stateids and that if the client remembers and uses
> > > the value, the server is obligated to remember the associated  open
> > > stateid and lockowner.  The question which is not specifically
> > > addressed and which is relevant to Frank's issue, is whether the
> > > client is free to forget about such  stateids.  I would argue that
> > > the implication is that it is free to forget such information, since
> > > Given that the server is explicitly told to remember the stateid,
> > > I'd say there is no basis for the server cancelling this stateid and
> > > issuing a new one in the case in which the client specifies a new
> > > lockowner which not truly "new"
> > > for
> > > the current open.  That had been suggested previously as an option
> > > in this case.
> >
> > It's the result of error translation because we were creating a new
> > stateid and then trying to hash it by owner/file and getting a
> > collision with the old stateid. We just need to up front determine if
> > there IS an old stateid to re-use.
> >
> > > On Wed, Sep 30, 2015 at 7:29 PM, Frank Filz
> > > <[email-masked]>
> > > wrote:
> > > > Frank Filz wrote:
> > > > > It's not clear whether a client is allowed to send a LOCK
> > > > > request with new_lock_owner true when the client has previously
> > > > > done so for the same lock owner and file (and thus the server
> > > > > has an active stateid for that owner/file pair). If it is
> > > > > allowed, should the server re-use the same stateid, or discard
> > > > > the old stateid and issue a new
> > > one?
> > > > >
> > > > combination of "new" and no obvious answer to your above question
> > > > (how to handle it, if
> > > > Fyi, the FreeBSD server does allow the same lock stateid to be
> > > > used for multiple file handles, but logs that it encountered this
> > > > and to the best of my knowledge, this has never been logged.
> > > >
> > > > As such, I believe extant client use the open_to_lock_owner case
> > > > to create a lock stateid for each (FH, lock_owner) tuple and only
> > > > if it doesn't already have a lock stateid for the tuple.
> > > >
> > > At least some version(s) of Linux client (RHEL 6.4) has demonstrated
> > > an issue, running cthon04 lock test8. Apparently the client loses
> > > track of the stateid during the repeated lock/unlock cycles this
> > > test does. The Ganesha server currently ends up returning an
> > > NFS4ERR_RESOURCE, which certainly needs to change whatever the
> > > decision in this case is.
> > >
> > > Frank
> >
> >
> >
> > ---

---