Subject: RE: downgrade/upgrade (was Re: Spec-related discussions at the Ba ke-off)

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 6 Nov 2000 11:13:25 -0800

Mike Eisler writes:
> 
> "Carl Beame" <[email-masked]> writes:
> 
> > > When the process closes one of the file descriptors on "A", the NFSv4
> > > file system needs to downgrade the SHARE to either READ or WRITE on
> > > the
> > > server. It cannot issue a CLOSE, because this removes all the SHARE
> > > state, so would leave a window for another client to OPEN the file
> > > with
> > > a DENY=~NONE, and so prevent the client from doing another OPEN to
> > > re-open the file. So an OPEN_DOWNGRADE operation is needed to deal
> > > with
> > > this.
> > 
> > You say it cannot issue a close, because close closes all opens to the
> > same file, because the stateid returned by open for the same
> > file/lockowner is the same. What I am saying is make it different, and
> > it all works. Are you saying that your CAN'T have a different stateid
> > for each opening of the same file/lockowner?
> 
> As the spec is currently written, yes.

I agree that the spec does not allow that.

I'm not sure about whether you CAN do otherwise (i.e. whether defining
things otherwise causes other difficulties).
 
> 
> > We have two possible solutions to implement downgrade. 
> > 
> > 1) Close the currently open file on the open upgrade and reopen the
> > file. 
> > 
> > 	Problem: All Locks would be lost
> 
> but you could restablish them on the new open correct?

You could try (and you might or might not succeed).

> 
> > 
> > 2) Open a second file handle with the new sharing mode and on the open
> > downgrade close the corresponding file handle.
> > 
> > 	Problem: While both are open and a lock request comes in, which file
> > do we lock? We may lock the file handle which gets closed in the
> > downgrade and loss the lock which should remain.
> 
> You would lock them on the second file handle.

If you made that choice, then if a downgrade resulted in a closing
of that file handle, you might have locks that would have to be 
moved to the first file handle and the server has no reliable way
to do that.
 
> 
> > It seems to me that the Unix client is converting a "second" open to
> > an open upgrade and then converting the "first" close to an open
> > downgrade and then the last close to a CLOSE. Why not let the server
> 
> Yes.

The unix client is doing an OPEN and the it is really the server who
is converting it to an open upgrade.

Yes, it is the client who converts the close to an open downgrade.

> 
> > handle this by using multiple opens and multiple closes?
> 
> Here is the situation folks were concerned with.  Let's
> suppose the client issues two OPENs on the same file and
> nfs_lockowner, each with same initial sequence #.
> Distinguishing the OPENs, especially if they have same
> creds, same args, is hard.  There was also the worry abount
> the Byzantine Router re-send an OPEN, though I think the
> open confirmation stuff that was added after the
> upgrade/downgrade discussion mitigates this.

I don't see this.  The proposal is for stateid (which is now
either per-lockowner or per-lockowner+file, depending on your
point of view), to be per-open-file-instance.  I have not
heard of any proposal to change seqid from being what it is
in the spec, per-lockowner.

> 
> You could certainly take the existing .x file, write a new
> internet-draft to match it (though I think that 64 bit
> stateids will be too small) to a model that assigns a
> unique sequence per open/nfs_lockowner pair. Open
> confirmation would be needed to distinguish OPEN with the
> same signature. Such an approach will also consume more
> server memory.

I've always thought 64-bit stateids were a bit small.  I
never understood why we didn't expand them when we discussed
this at the first bake-off.  Now I realize that Mike
was thinking stateid=lockowner when he asserted 64-bit was 
fine, so the mystery is partly explained.

On the other hand, Robinson was definitely thinking 
stateid=lockowner+file.  I thought he was also pro-64-bit,
but maybe it was just that he wasn't zealously pro-128-bit.

If your reference point on server memory is the stateid=
lockowner, then certainly.  If your reference point is
stateid=lowckowner+file, then going to stateid=open-file-
instance, then the change will be modest.  I'm assuming 
that the client is free to collapse multiple instances
of open for the same file, lockowner, and access and deny
bits and reference count.

My concern with making this change is different.  My worry
is that it requires the client to keep track of locks
in terms of the specific open file instances with which 
they are associated and to act on this knowledge on the
*protocol* level.  Will this information be available
across the VOP interface?  If not, how are UNIX clients
going to implement this?

I think we need those planning UNIX and UNIX-like client
implementations to consider this carefully (and to comment). 

> 
> But that's not the protocol we have a pending proposed
> standard for.

We have a spec that defines stateid in two different ways.

So perhaps it would be more correct to say that it is not
one of the protocols that we have a pending proposed standard
for.

> 
> So, do we want to kill the current pending proposed standard.

Can someone who understands all this IETF stuff explain what
is actually happening here.

Here's my problem.  As I understood things, the working group
had gone as far as it could in defining an NFS-v4, without 
actually trying to implement it.  Although I am generally 
pleased with our work, it never occurred to me to suppose that
we are going to implement it and find nothing that needs to 
be changed.

This is separate from the issue of whether Carl's downgrade
issue is important enough to merit a change (and whether we
have a change that will work).  Opinions will differ on that.

Even if you think that none of the issues that implementors
brought up at the bake-off merit any change, don't assume 
that the spec is perfect as written.  I don't.  No testing 
of server failure has been done.  Nor of network partitions,
delegation, migration, and .. intersections of many of the
above.  Yes, these things have been thought about so I
don't expect the spec to fall to pieces when confronted with
these challenges, but I don't expect perfection, either.
If we've made mistakes (which I don't doubt), we will have
to fix them.  I had understood that was part of the purpose
of the period from Proposed Standard to Draft Standard.

With regard to the specifics of the downgrade issue, I 
would ask the following questions:

     Is this important enough to change the spec?

     Do we have a change that we are confident about (Won't
     cause other problems, etc.)?

As far as the first question is concerned, I am ambivalent.
I've always felt that trying to implement a file server within
a non-fixable API was like playing the piano with boxing gloves;
you shouldn't be surprised when it doesn't sound so good.  On the
other hand, we have made significant efforts to accommodate
such servers (volatile file handles and other things I can't
remember).  If we've done that, it seems kind of strange to
say that now we won't.

On the second question, I continue to have my doubts.  I think
the issues of existing VOP interfaces being a problem on 
clients are real.  (Of course, you could apply the boxing
glove analogy to them as well).  I wish we had a solution 
that didn't have that problem, but right now I don't know of
one.

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 6 Nov 2000 17:47:11 -0600

> is actually happening here.

From http://www.ietf.org/rfc/rfc2026.txt, Section 4.1.1

"A Proposed Standard specification is generally stable, has resolved
 known design choices, is believed to be well-understood, has received
 significant community review, and appears to enjoy enough community
 interest to be considered valuable.  However, further experience
 might result in a change or even retraction of the specification
 before it advances."

- further on in section 4.1.1 -

"Implementors should treat Proposed Standards as immature
 specifications.  It is desirable to implement them in order to gain
 experience and to validate, test, and clarify the specification.
 However, since the content of Proposed Standards may be changed if
 problems are found or better solutions are identified, deploying
 implementations of such standards into a disruption-sensitive
 environment is not recommended."

> we are going to implement it and find nothing that needs to 

I have assumed that there may need to be clarifications between
Proposed and Draft.  However, if changes in the .x file were needed
between Proposed and Draft, then this would result in the first minor
version of NFSv4.

<...snip...>

> of the period from Proposed Standard to Draft Standard.

Right.  Quoting from http://www.ietf.org/rfc/rfc2026.txt, section 4.1.2

"A specification from which at least two independent and interoperable
 implementations from different code bases have been developed, and
 for which sufficient successful operational experience has been
 obtained, may be elevated to the "Draft Standard" level."
...
"The requirement for at least two independent and interoperable
 implementations applies to all of the options and features of the
 specification.  In cases in which one or more options or features
 have not been demonstrated in at least two interoperable
 implementations, the specification may advance to the Draft Standard
 level only if those options or features are removed."
...
"A Draft Standard is normally considered to be a final specification,
 and changes are likely to be made only to solve specific problems
 encountered.  In most circumstances, it is reasonable for vendors to
 deploy implementations of Draft Standards into a disruption sensitive
 environment."

Even with the statements about what environments should be targeted
with implementations based on Proposed and Draft standards, the real
world has shown that as soon as a protocol reaches the Proposed state
implementations are shipped to customers.  There are numerous examples
of this occurring.  Even with NFSv4, there will be a full Linux
implementation available shortly after there is an NFSv4 RFC at
Proposed.

> remember).  If we've done that, it seems kind of strange to

The real question should be "Is the WG willing to pull the current
Proposed Standard back and re-work it?"  I say Proposed Standard here
since the IESG has provided its approval and the only action left is
the final edits of draft-07 before it becomes and RFC.  Is the WG
willing to pull the draft back and delay the process?  Are we able to
resolve the issues in a way that wouldn't require updating the pending
RFC immediately?  What I mean by this is to have the NFSv4 RFC
published and the WG turns out the first draft representing the Draft
Standard that includes clarifications (without .x changes)?

-- 

- Spencer -

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 07 Nov 2000 16:19:45 -0500

>Standard that includes clarifications (without .x changes)?

        Process-wise, another option is to let the current document
go out as Proposed Standard, then immediately start work on its
replacement, which could go out as a replacement also at Proposed
Standard.  IETF specs can cycle at the same state of standards
progress repeatedly before moving forward to the next standards state.

        So letting the current document go forward as Proposed
Standard RFC does NOT mean that the next chance to fix the document
is to wait until it is ready for Draft Standard.

        Which path to take is up to the WG of course.  As a user,
my main concern is that we have clear and accurate specifications
so that I have wide interoperability of products.

Cheers,

Ran
[email-masked]