Subject: Re: [nfsv4] (old) comments regarding draft-ietf-nfsv4-scsi-01

From: Christoph Hellwig <[email-masked]>
Date: Sat, 3 Oct 2015 19:09:07 +0200

Hi David,

On Tue, Sep 29, 2015 at 10:29:27AM -0400, David Noveck wrote:
> > To be coherent with other NFSv4 documents
> > this probably should read open, lock or delegation stateid.
> 
> I believe that it is coherent with other documents.  It is just that it
> is not the same as those other documents and I believe that is for good
> reasons.
> 
> I believe those other documents deal with stateids that one presents in
> an NFSv4 operation. On the other hand, this text deals with stateids
> that one must posess to assure that you have an open file that the MDS
> knows about.  Since, you are using an IO protocol that has no concept
> of stateids, the situation is different making the text seem discordant
> even though it isn't.

All of that applies to RFC5663, RFC5664 and the flexfiles document in
the loosely coupled NFSv3 or NFSv4.0 storage device case as well.

> > Second this doesn't match existing implementations which are heavy on
> > use of special stateids.
> 
> I'm not sure exactly what you mean by "heavy on the use of special
> stateids" given that the actual requests sent do not contain stateids.
> The text in question refers, not to the stateids actually sent (since
> there aren't any), but to the stateids that the client actually possesses.
> In other words, the issue is whether these clients are (knowingly or not)
> doing IO using pNFS on files that have not been opened on the MDS.  Let's
> put aside RFC2219 terms for a bit. Can we agree that such clients are
> BROKEN?

That's true, but then again not specific to this layout in any way.

> > Again this is a generic issue for any loosely coupled layout.
> 
> This applies to some layouts but not all.  Not sure how to deal with issues
> that deal with loosely coupled layouts as a class.  Maybe this should be
> dealt with in draft-ietf-pnfs-layouts.

Tom, do you agree this should go into draft-ietf-pnfs-layouts?  I think
it probably should.

> 
> > I also
> > actively disagree with this condition: given that we need an
> > open/lock/delegation stateid AND a layout stateid there is no need to
> > revoke the layout stateid when revoking the o/l/d stateid.
> 
> Actually there is a need.  The reason is that when you revoke an open, the
> situation is different from recalling a layout/delegation.  When you have
> a recall, the client must acknowledge the action, by returning the object
> in question.  In the case of layouts, the MDS has fencing available to
> make sure that the revocation is adhered to, any communications
> problems notwithstanding.   One option, which I didn't pursue because it
> seemed too complicated, would be to allow the MDS not to recall the
> layouts if the client acknowledged the revocation by using FREE_STATEID.
> The MDS would have to delay IO's done by the client in question, from the
> period at which the revocation happened until it was acknowledged.

Or we could fence directly if a recall is not acknowledged.  Either way
seems fine to me, but I strongly disagree with adding these generic pNFS
requirements for loosely coupled layouts to the scsi-layout document.
They need to go into one place for all the four different loosly couple
layours, otherwise we get into a nightmare of needing all kinds of layout
specific behaviors instead of a single set of common rules.

---

From: David Noveck <[email-masked]>
Date: Sun, 4 Oct 2015 13:33:37 -0400

I think we are in danger of talking past one another due to losing the
context of the discussion.

I'm going to try to re-establish the context here, but to do so I'm not
going to produce an enormous email including lines with  5+ greater-than
signs.

This all started when I sent you a proposed re-organization of the material
originally in your "Security Considerations" section.  I did this as a way
of trying to better understand what might be necessary to address potential
requirements that Tom put forward in draft-ietf-nfsv4-pnfs-layouts, which
are that it is the job of each layout to respect the overall semantics of
nfsv4.  I think the idea was that the distribution of responsibilities for
ensuring this (among MDS, storage device, client) might be different for
different layout types.

I sent this to you privately, as opposed to posting it to the list for two
reasons:

   - I wasn't sure that I understood the block layout type well enough to
   propose a replacement.
   - I didn't want the group to get into a big discussion of these issues,
   which might derail the scsi layout document, which despite the issues with
   the "Security Considerations" section, is an improvement over RFC5663.

At the time, I felt that it was the job of each layout type definition
document to explain how it would meet the semantic requirements for the
layout type it was defining.  I never considered the possibility of putting
this material in another document, or to re-organize the treatment of pnfs
layouts in a more hierarchical way, focusing on the distinction between
loosely-coupled and tightly-coupled layouts.

When you asked me to post my comments, I figured that my contribution might
be helpful but I reaffirmed my belief that my proposed changes should not
be considered requirements/recommendations for changes in the scsi layout
document.  Again, I didn't want to interfere with that document going
forward.

Now we've discussed my comments a bit and you asked me some specific
questions about why I said certain things and I addressed those questions.
Your response was that my answers were not specific to the scsi layout and
might well apply to other layouts, which is true, at least in broad
outline. When I wrote my original proposed text, I was not excluding any
text that might apply to other layouts.  I was only writing for the scsi
layout.  The same applies to my responses to your questions.  I was
answering your questions about my text which was addressing a potential
enhancement to the scsi layout document and I could not have done so
without saying something that applied to other layout a well.

I think we should distinguish two things:

   - Is what I said correct?
   - If it is correct, what does it imply about what various documents
   should say?

My impression is that you have no substantive objection to my comments.
For example, I think we agree that if there are pNFS (scsi, block, object)
clients that did IO on files that were not open, they would be broken.  In
each case, the fact that other clients with  different layout types might
be similarly broken, does not means that the client with the problem is OK.

As far as the various documents:

   - With regard to the scsi layout, I think this would be helpful to add
   but the decision is yours.
   - With regard to RFCs 5663 and 5664, they probably don't say anything on
   the matter, but it might well be argued that this is obvious/understood.
   - With regard to flexfiles, this is probably not mentioned in my most
   recent comments on flexfiles-06.  All of the stuff on stateids was in such
   an unclear state I may have just waited for the document to be put in a
   consistent state with regard to stateids for the loosely coupled cases
   before addressing issues like this.
   - With regard to draft-ietf-nfsv4-pnfs-layouts, it is up to Tom, but if
   I were he, I'd be put off by the fact that taking this approach means that
   the document would have to update RFCs 5661, 5663, 5664 and the scsi layout
   document.  That sounds like a real drag for the author and the shepherd.

SImilar issues would apply to layout recall upon open revocation, etc.

With regard to the supposed nightmare of each layout type having
layout-specific requirements, there are no such requirements.  The only
requirement is that the semantics of NFSv4.1 are to be obeyed and currently
each layout type document indicates how to do that.   The fact that that
may not have  been done particularly well in the case of some earlier RFCs
does not change that.

As I've said, the decision about what to do about the scsi layout is up to
you, but I don't want us to consider that this sets a precedent.  I want it
to be clear that if a future loosely-coupled layout specification is up for
consideration and has a problem, the fact that other layouts may have the
same issue does not somehow mean that the problem does not have to be
fixed.  A fix could involve a normative reference to another document that
shows how the problem would be addressed

I was trying to suggest a means by which the scsi layout document could
explain to prospective implementers how those semantic requirements might
be met.  The fact that the same techniques would apply to other similar
layouts does not invalidate that.  While it is possible we could have done
something different to structure the loosely coupled layouts so that a
common treatment would be possible, we are where we are.  If all of these
layouts needed different approaches to meeting semantic requirements that
might be a troubling situation but not a nightmare.  The fact that the same
approach works for all of them makes things easier and simpler, and not
more difficult and nightmarish.

If we have a situation in which a number of layout types adopt the same
approach to solving a problem, that isn't necessarily wrong and it is
likely that the responsibility for explaining how semantic requirements are
to be met will stay with the layout type specification, at least for a
while.  If there were someone already working on
draft-ietf-nfsv4-pnfs-loose-coupling, things might be different, but it is
too late to change the current situation.

.

---

From: Christoph Hellwig <[email-masked]>
Date: Mon, 5 Oct 2015 08:34:51 +0200

Hi David,

thanks for restarting the discussion.  Just to clarify my point of view:
I don't really care if we derail and delay the SCSI layout document, I'd
rather get it right than get it quick.  I also agree with your comments
in general but I really don't want to single out this document for them
as I strongly believe they apply to a whole class of layouts.  In fact
to all layouts except for the file layout, and the tightly coupled variant
of flexfiles which I think is so far unimplemented.

That's also why I really wanted to drag you into this public discussion of
your points as I think they apply much more broadly and should be brought
to attention of the whole WG.  I have to admit that I'm a little
disappointed that no one else chimed in so far.

> My impression is that you have no substantive objection to my comments.

Agreed.

>    document.  That sounds like a real drag for the author and the shepherd.

I agree about that process being painful, on the other hand we need a
document that explains the requirements for these loosely coupled layouts,
and a documented titled "Requirements for pNFS Layout Types" seems like the
next best place to do that given the RFC5661 is out of the door.

> shows how the problem would be addressed

Which is another argument for getting it right now and documenting it
either in draft-ietf-nfsv4-pnfs-layouts or at least another document that
is not the scsi-layout draft.  One option might be to make this a standalone
document that does not update any existing rfc, but which will be referenced
by scsi-layout and flex-files as well as any future new loosly coupled
layout.

---

From: David Noveck <[email-masked]>
Date: Mon, 5 Oct 2015 09:57:24 -0400

> I have to admit that I'm a little

I'm not disappointed.  Two people having a productive discussion is better
than lager number having
a discussion that doesn't go anywhere.  I've seen productive discussions
with three or four participants
but never more than that.

It is possible to interpret the lack of other participants as indicating a
lack of interest.  On the other hand,
if people thought we were not properly addressing the issues, they are
capable of saying so.

I don't think it is needed but I do agree that it is desirable.

> next best place to do that given the RFC5661 is out of the door.

Unfortunately, the document with that name,  d
raft-ietf-nfsv4-layout-types-03.txt is now expired.

I think we need to discuss this with Tom, but my impression is that he will
be busy with
flexfiles and v4.2 for a while, besides doing implementation work.

> > A fix could involve a normative reference to another document that

> is not the scsi-layout draft.

I don't think it is an  argument for that, but I agree that, given that you
feel
time is not an issue, I do think the idea of a new document is viable and
worth pursuing.

> One option might be to make this a standalone
> by scsi-layout and flex-files as well as any future new loosely coupled
> layout.

I think you need to co-ordinate with Tom on that for two reasons:

   - Since you don't want a document that only scsi-layout references, he
   would have to be wiling to wait for this new document rather than
   addressing these issues in flexfiles directly.
   - You need to come to an agreement regarding how to address the issues
   that were to be addressed in draft-ietf-nfsv4-pnfs-layouts.

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 5 Oct 2015 15:48:58 -0400 (EDT)

On Mon, 5 Oct 2015, David Noveck wrote:

> Unfortunately, the document with that name,
> draft-ietf-nfsv4-layout-types-03.txt is now expired.

Reviving an expired internet-draft is simply a matter of submitting a new
version of it.

>From an IETF process point of view, that is not an obstacle.  Once a
document is a WG document, it ceases to be the authors' document and is
under the control of the WG.  A single document editor is usually needed
to actually collect and make the needed change to the text, submit new
versions of the document, etc., but that editor need not be one of the
original authors.  If there is someone who has the time and interest in
performing the editor role for this document, the chairs can appoint that
person as document editor and the document can still move forward.

The chairs may decide that they don't want to split the WG's efforts and
detract from the main goals, but "Tom is busy" is no reason why
"Requirements for pNFS Layout Types" can't move forward.

-Ben