Subject: [nfsv4] Benjamin Kaduk's Discuss on draft-ietf-nfsv4-mv0-trunking-update-03: (with DISCUSS and COMMENT)

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 09 Jan 2019 11:17:42 -0800

Benjamin Kaduk has entered the following ballot position for
draft-ietf-nfsv4-mv0-trunking-update-03: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-mv0-trunking-update/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

First off, thanks for the work on this document; it's important to get this
behavior clarified and functional even for NFSv4.0.

That said, this document (along with the pieces of 7530 and 7931 that I
read along the way) still leave me uncertain about how some things are
supposed to work.  (If it's clarified in parts of those documents that I
didn't read, I'll happily clear and apologize for the disruption, of
course.)

To start with, I'm still lacking a clear high-level picture of why a client
needs to care about trunking detection vs. just treating all listed
addresses as replicas.  There are some parts in the body where we talk
about, e.g., lock state and similar maintenance, but I don't have a clear
picture of what the risks and benefits of (not) tracking trunking are, and
this would be a fine opportunity to add some text.  Specifically, in
Section 5.2.1, we just say that "[a] client may use file system location
elements simultaneously to provide higher-performance access to the target
file system"; most of the focus of this document makes me think that this
statement was intended to apply only to trunking, but I also think there
are supposed to be replication-only scenarios that provide performance
gains.  I'm not sure if we need to clarify the distinction in that location
as well as the high-level overview.

It's also unclear to me what parts of migration flows are under the control
of the client vs. the server.  It's clear that the server has to initiate
migration via NFS4ERR_MOVED, but my current understanding is just that this
prompts the client to look at fs_locations, and the client has control over
which alternate location to move to.  But there's also a lot of discussion
in all three documents about the servers migrating state along with
migration, so it seems like the server should be controlling where the
client goes.  Is this just supposed to be by limiting the fs_locations data
to the specific migration target chosen by the server?  (If so, this would
probably have potential for poor interaction with the implicit filesystem
discovery described in Section 5.3.)  On the other hand, Section 5.2.6
talks about the server putting entries "that represent addresses usable
with the current server or a migration target before those associated with
replicas", which seems to imply that there is some other way to know what
the migration target is.

Section 5.2.6 also tells the client to rely on that ordering:

                                   To keep this process as short as
   possible, Servers are REQUIRED to place file system location entries
   that represent addresses usable with the current server or a
   migration target before those associated with replicas.  A client can
   then cease scanning for trunkable file system location entries once
   it encounters a file system location element whose fs_name differs

but I don't think a client actually can do so, since the client has no way
to know that the server implements this document as opposed to stock
7530+7931 (at least, no way that I saw).

Finally, removing the last paragraph of Section 8.5 of RFC 7530 could have
negative operational impact if updated clients interact with non-updated
servers/environments that are misconfigured in the described fashion.  It's
probably worth stating in the top-level Section 5 that such misconfigured
servers are believed to no longer exist (if that's in fact true, of
course; if not, we'd need to reconsider the change).

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Section 1

   As part of addressing this need, [RFC7931] introduces trunking into
   NFS version 4.0 along with a trunking detection mechanism.  This
   enables a client to determine whether two distinct network addresses
   are connected to the same NFS version 4.0 server instance.
   Nevertheless, the use of the concept of server-trunkability is the
   same in both protocol versions.

Er, what are the two protocol versions in question?  (I assume 4.0 and 4.1,
but you don't say 4.1 anywhere.)

   o  To provide NFS version 4.0 with a means of trunking discovery,
      compatible with the means of trunking detection introduced by
      [RFC7931].

We haven't yet mentioned that the distinction between "detection" and
"discovery" is important, so it's probably worth a forward reference to the
text below.

Section 5.1

   The fs_locations attribute (described as "RECOMMENDED" in [RFC7530])

If you're going to describe this section as "replacing Section 8.1 of
[RFC7530]", then it needs to stand on its own without reference to the
current Section 8.1 of RFC 7530.  That is, if the "RECOMMENDED" nature is
to remain, then it should be described as such de novo in this text.

      Clients use the existing means for NFSv4.0 trunking detection,
      defined in [RFC7931], to confirm that such addresses are connected
      to the same server.  The client can ignore addresses found not to
      be so connected.

nit: I would suggest phrasing this as "use the NFSv4.0 trunking detection
mechanism [RFC7931] to confirm [...]", as temporal refernces like
"existing" may not age well.
not-nit: "ignore" is pretty strong; does this imply that a client is free
to ignore things like migration, replication, and referrals?

      location entries.  If a file system location entry specifies a
      network address, there is only a single corresponding location
      element.  When a file system location entry contains a host name,
      the client resolves the hostname, producing one file system
      location element for each of the resulting network addresses.
      Issues regarding the trustworthiness of hostname resolutions are
      further discussed in Section 7.

nit(?) this is confusing if we read "Section 7" as being "Section 7 of RFC
7530", which is a tempting reading since this text is supposed to replace
text in that document.  Perhaps "Section 7 of [[this document]]" would make
more sense (but I also forget the RFC Editor's policy on such
self-references).

Section 5.2.1

                                                                 The
   client utilizes trunking detection and/or discovery, further
   described in Section 5.2.2 of the current document, to determine a

nit(?) perhaps s/the current document/[[this document]]/ as above (for
update by the RFC Editor).  I'll stop commenting this construction, though
of course if such changes are made they should be done globally.

Section 5.2.3

   Because of the need to support multiple connections, clients face the

What need?  Where is this need articulated?

                    As a result, clients supporting multiple connection
   types need to attempt to establish a connection on various connection
   types allowing it to determine which connection types are supported.

nit: maybe describe this as a "trial and error" approach to connection type
support determination?

   To avoid waiting when there is at least one viable network path
   available, simultaneous attempts to establish multiple connection
   types are possible.  Once a viable connection is established, the
   client discards less-preferred connections.

It's probably worth referencing the "happy eyeballs" technique used
elsewhere (e.g., RFC 8305) as being analogous.

Section 5.2.5

   Such migration can help provide load balancing or general resource
   reallocation.  [...]

side note: is this load balancing generally going to be just of a "move a
filesystem or ten to a different server when load gets too high" or are
people also doing "send different clients to different replicas for the
same filesystem" live load-balancing?

Section 5.2.6

   When the set of network addresses designated by a file system
   location attribute changes, NFS4ERR_MOVED might or might not result.
   In some of the cases in which NFS4ERR_MOVED is returned migration has
   occurred, while in others there is a shift in the network addresses
   used to access a particular file system with no migration.

I got pretty confused when I first read this, thinking there was some
implication that a server could introduce a fleeting NFS4ERR_MOVED as a
notification that addresses changed, even if the server could otherwise
continue handling the client's requests.  Perhaps:

% When the set of network addresses on a server change in a way that would
% affect a file system location attribute, there are several possible
% outcomes for clients currently accessing that file system.  NFS4ERR_MOVED
% is returned only when the server cannot satisfy a request from the client,
% whether because the file system has been migrated to a different server, is
% only accessible at a different trunked address on the same server, or some
% other reason.

Similarly, we may want to clarify that (e.g.) case (1) is not going to
result in an NFS4ERR_MOVED.

   2.  When the list of network addresses is a subset of that previously
       in effect, immediate action is not needed if an address missing
       in the replacement list is not currently in use by the client.
       The client should avoid using that address in the future, whether
       the address is for a replica or an additional path to the server
       being used.

"avoid using that address in the future" needs to be scoped to this
filesystem; it's not going to work if clients treat it as a global
blacklisting.

   Although significant harm cannot arise from this misapprehension, it
   can give rise to disconcerting situations.  For example, if a lock
   has been revoked during the address shift, it will appear to the
   client as if the lock has been lost during migration, normally
   calling for it to be recoverable via an fs-specific grace period
   associated with the migration event.

I think this example needs to be clarified more or rewritten to describe
what behavior fo which participant that normally happens does not happen
(specifically, the "normally ..." clause).

   from the current fs_name, or whose address is not server-trunkable
   with the one it is currently using.

nit: does it make more sense to put the address clause first, since fs_name
is only valid within the scope of a given address/server?

Section 5.3

   As mentioned above, a single file system location entry may have a
   server address target in the form of a DNS host name that resolves to
   multiple network addresses, while multiple file system location
   entries may have their own server address targets that reference the
   same server.

nit: I'm not sure that "while" is the right word here.  Perhaps "and
conversely"?

   When server-trunkable addresses for a server exist, the client may
   assume that for each file system in the namespace of a given server
   network address, there exist file systems at corresponding namespace
   locations for each of the other server network addresses.  It may do

Pretty sure you need to say "trunkable" here, too.

   this even in the absence of explicit listing in fs_locations.  Such

I may be confused, but we're talking about different file systems within a
single server's single-server namespace, right?  So there is not even a way
for them to be listed in the fs_locations for queries on FHs in the current
filesystem (unless the server exports the same filesystem under different
paths in its namespace for some reason).  So, we should probably be saying
more about how these are fs_locations results returned for queries against
different filesystems hosted on the same server...

   corresponding file system locations can be used as alternative
   locations, just as those explicitly specified via the fs_locations
   attribute.

... (and possibly some related tweaks in this part too).

Section 7

We probably need to reiterate the privacy considerations inherent in the
UCS approach, mentioned at the end of Section 5.6 of RFC 7931.

      o  When DNS is used to convert NFS server host names to network
         addresses and DNSSEC [RFC4033] is not available, the validity
         of the network addresses returned cannot be relied upon.
         However, when the client uses RPCSEC_GSS [RFC7861] to access
         NFS servers, it is possible for mutual authentication to detect
         invalid server addresses.  Other forms of transport layer

nit: It seems to only sort-of be the case that the mutual authentication
detects invalid addresses.  I tend to think of the property involved as
ensuring that I am talking to who I think I am, which encompasses both the
intended network address and the stuff on the other end.  On the other
hand, one could imagine some bizzare deployments that share kerberos keys
across servers where GSS could succeed (if the acceptor didn't have strict
host name checking in place) but the address would still be unintended.
If I had to rephrase this (unclear that it's really necessary), I might go
with something like "to increase confidence in the correctness of server
addresses", but there are lots of valid things to say here and it's not a
big deal.

      o  Fetching file system location information SHOULD be performed
         using RPCSEC_GSS with integrity protection, as previously

I forget if we have to say "integrity protection or better" or if this
phrasing also includes the confidentiality protection case.

      When a file system location attribute is fetched upon connecting
      with an NFSv4 server, it SHOULD, as stated above, be done using
      RPCSEC_GSS with integrity protection.

It looks like this is now three places where this normative requirement is
stated (7530's security considerations, and earlier in this section).
Usually we try to stick to just one, to avoid risk of conflicting
interpretations, and restate requirements non-normatively when needed.
(It's not even clear that this duplication is needed, though.)

                                                               For
      example, if a range of network addresses can be determined that
      assure that the servers and clients using AUTH_SYS are subject to
      appropriate constraints (such as physical network isolation and
      the use of administrative controls within the operating systems),
      then network adresses in this range can be used with others
      discarded or restricted in their use of AUTH_SYS.

I'd strongly suggest adding a comma or something here to avoid the
misparsing of "used with others".

      To summarize considerations regarding the use of RPCSEC_GSS in
      fetching file system location information, consider the following
      possibilities for requests to interrogate location information,
      with interrogation approaches on the referring and destination
      servers arrived at separately:

I don't understand what this is trying to say, especially in light of the
following bullet points being essentially recommendations for behavior (in
one case, limited to a specific situation where disrecommended behavior is
unavoidable).

I do appreciate the good discussions about the provenance and reliability
of location data -- it seems to be pretty complete, so thank you!

---

From: David Noveck <[email-masked]>
Date: Fri, 18 Jan 2019 12:47:42 -0500

The following is the response to your DISCUSS.   The response to your
additional COMMENTs
wIll be sent soon.

> supposed to work.

That's unfortunate.  It shouldn't happen.

> (If it's clarified in parts of those documents that I

There is no disruption.   This discussion seems to me fully in line with
the way that the process needs to work.

I believe there are some relevant clarifications that derive from existing
documents, but it does not appear to me, that you missed anything.   The
relevant text does not, on its own, provide sufficient clarification, so it
appears that we will need to clarify things so that future readers do not
run into the difficulties that you saw.

> this would be a fine opportunity to add some text.

The issues that arise when you are not aware of the trunking relationship
between two paths do have to with management of locking state.   RFC7530
(following on from similar text in RFC3530) cited the confusions regarding
locking that would arise if a client thought it was talking to two
different servers (i.e. replicas) when it was in fact talking to a single
system over two different paths.   Basically, a client who is unaware of
this distinction, might work successfully  sometimes, but in most cases you
have to know about trunking relationships that exist and it would be
extremely hard to write locking code that worked without being aware of
trunking relationships.

In fact, this is why:

   - RFC5661 added a means of trunking detection to NFSv4.1
   - RFCs7530 and 3530 advised NFSv4.0 clients to actively prevent the
   occurrence of trunking by adopting what is described in RFC7931 as the
   "non-uniform client string model".
   - RFC7931 provided a means of trunking detection usable by NFSv4.0

-04 will need to clearly reference the importantance of state/locking
considerations in making it necessary that the client and server agree as
to trunking relationships.  I anticipate we will do this by replacing the
third paragraph of the introduction by the following two paragraphs;

As part of addressing this need, [RFC7931] introduces trunking into
NFS version 4.0 along with a trunking detection mechanism.  A trunking
detection mechanism enables a client to determine whether two distinct
network addresses are connected to the same NFS version 4.0 server
instance.  This knowledge is necessary since, without it, a client
unaware of a trnking relation ship between paths it is using
simultaneous is likely to become confused in ways described in
[RF7530].

NFSv4.1 was defined with an integral means of trunking detection
described in [RFC5661], while NFSv4.0 initially did not have one, with
it being added by [RFC7931].  Nevertheless, the use of the concept of
server-trunkability is the same in both protocol versions.

> Specifically, in
> statement was intended to apply only to trunking,

That was the intention and given that this is a trunking-motivated
document, it seemed reasonable.  However, as you point out, there are cases
in which those those words could reasonably be read as applying to
replication.

> but I also think there

There are such scenarios but they are not as important because:

   - They only apply to a subset of workloads, basically read-only file
   systems, and very low-change read-write filesystems where the application
   takes special care to propagate updates to all replicas.
   - The client (or the application) needs to be aware of the fact that
   spcial state mangement actions need to be taken.  Foe example, if you are
   to stripe your read from a given file between/amon a set of replicas, there
   needs to be.

It's also relevant to consider the original purpose of the replication
feature in NFSv4.0.   The intention was to provide alternative locations
that could be used if the primary location became unavailable.   This has
not prevented its use to provide parallel acces to multiple replicas to
increase performance, in a limited set of contexts, primarily to provide
read-only access, but it is important in deciding how to describe this
option in -04.

I believe we can address this issue in -04 by replacing the current fourth
paragraph of section 5.2.1 by the following two paragraphs:

A client may use file system location elements simultaneously to
provide higher-performance access to the target file system.  This is
most simply be done using trunling although the use of multiple
replicas simultaneously is possible. To enable this simultaneous
acces, the client utilizes trunking detection and/or discovery,
further described in Section 5.2.2 of the current document, to
determine a set of network paths that are server-trunkable with the
one currently being used to access the file system.   Once this
determination is made, requests may be routed across multiple paths,
using the existing state management mechanism

Multiple replicas may also be used simultaneously, typicalls in
handling read-only file systems.   In this case, each replica has its
own state management which the client needs to be aware of, doin
multiple file opens to enable a file to be read simultaneoudly from
multiple replicas.

> migration via NFS4ERR_MOVED,

The server has to precipitate the client's response to the migration event
by informing it of the unusability of the (previously) current instance by
returning NFS4ERR_MOVED.  However, in almost all cases, this is not the
start of the server's work.

The server must, before informing the client of the shift, make sure that
when the client acts on it, the successor server (or servers) are prepared
to respond appropriately:

   - The destination server needs to have a fully up-to-date copy of the
   source file system.  Where the file system is read-write, this means that
   all update made to the sotrce file system must be propagated to the
   destination(s).
   - When transparent state migration is implemented, locking data needs to
   be similarly propagated.

> but my current understanding is just that this
> which alternate location to move to.

When multiple locations are provided, the client has this choice.

> But there's also a lot of discussion
> to the specific migration target chosen by the server?

As a practical matter there are difficulties propagating locking state and
other updates to multiple desinations so that I expect most implementations
to designate a single successor, at least for a while.  However, the
protocol does allow multiple choices to be provided.  If the server does
so, then it would have to propagate information to multiple targets and be
prepared for any valid choice the client might make.

> (If so, this would
> discovery described in Section 5.3.)

I'm not sure of what problems you foresee.  Perhaps I need to better understand
your interpretation of section 5.3.

> On the other hand, Section 5.2.6

Not sure what you mean by "some other way".

I think the server has to know what addresses are usable with current
server because it is the one that provides the appropriate service.  With
regard to migration targets, as we have seen, the sending server has to
know this because he is typically involved in preparing (through
information progation) the destination server to provide the needed
service.

> 7530+7931 (at least, no way that I saw).

The basic assumption behind this paragraph as written is that no previous
document has provided guidance as to the use of the location attribute to
provide information relevant to trunking discovery.   However, in
considering interactions with previous servers as you have done (nice
catch!) it appears that it would be possible for a server to present two
trunkable aaddresses even without any awareness of the fact that they are
trunkable, so that a client following the guidance above might deny himself
knowledge of some piece of information related to trunking discovery.

In light of this, it appears that -04 will have to rewrite the material in
the last paragraph of section 5.2.6 to be something like the following:

Because a file system location attribute may include entries relating
to the current server, the migration destination, and possible
replicas to use, scanning for available network addresses thst might
be trunkable with addresses already seen could potentially be a long
process.  In order keep this process as short as possible, Servers
that provide infornation about trunkable network paths when the exist
are REQUIRED to place file system location entries that represent
addresses usable with the current server or a migration target before
those associated with replicas.

This ordering allows a client to cease scanning for trunkable file
system location entries once it encounters a file system location
element whose fs_name differs from the current fs_name, or whose
address is not server-trunkable with the one it is currently using.
While the possibility exists, that a client might prematurely cease
scanning for trunkable addresses when receiving a location attribute
from a an older server not following the order constaint above, the
harm is expected to be limited since such servers would not be
expected to present infomation about trunkble server access paths.

> course; if not, we'd need to reconsider the change).

It turns out this deletion was inadvertant.  The paragraph will be added
back in -04.

---

From: David Noveck <[email-masked]>
Date: Thu, 24 Jan 2019 16:22:28 -0500

Following are the responses to your non-DISCUSS comments.

> but you don't say 4.1 anywhere.)

This was addressed by the introductory paragraphs about locking prompted by
your DISCUSS.

The paragraphs were:

As part of addressing this need, [RFC7931] introduces trunking into NFS
version 4.0 along with a trunking detection mechanism.  A trunking
detection mechanism enables a client to determine whether two distinct
network addresses are connected to the same NFS version 4.0 server
instance.  This knowledge is necessary since, without it, a client unaware
of a trunking relationship between paths it is using simultaneous is likely
to become confused in ways described in [RF7530].

NFSv4.1 was defined with an integral means of trunking detection described
in [RFC5661], while NFSv4.0 initially did not have one, with it being added
by [RFC7931].  Nevertheless, the use of the concept of server-trunkability
is the same in both protocol versions

> "discovery" is important, so it's probably worth a forward reference to
the

I think we can revise this bullet to read as follows in -04:

   - To provide NFS version 4.0 with a means of finding addresses trunkable
   with a given address. i.e., trunking discovery, compatible with the means
   of trunking detection introduced by [RFC7931].  For an explanation of
   trunking detection and trunking discovery see Section 3.

> to remain, then it should be described as such de novo in this text.

The problem we have to deal with is the fact that the term “RECOMMENDED” is
not in accord with RFC2119.  .  RFC7530 deals with this by making an
exception in its Section 1.1.   To avoid us having to do the same thing, we
could simply delete the parenthetical material and render the sentence as
follows:

The fs_locations attribute allows specification of file system locations
where the data corresponding to a given file system may be accessed.

> "existing" may not age well

OK.  Will fix in -04.

.
> to ignore things like migration, replication, and referrals?

That needs to be rephrased.   The intention is that they can ignore the
non-confirmed

trunking relationship.

I think the following should appear in -04:

Clients use the means for NFSv4.0 trunking detection, defined in
[RFC7931],  to confirm that such addresses are connected to the same
server.  The client can ignore non-confirmed trunking relationships and
treat the corresponding addresses as connected to different servers.

> more sense (but I also forget the RFC Editor's policy on such

I don't think I ever knew that policy.

-04 will say "Section 7 of the current document" and if that is not OK,
with the RFC editor, we'll find out about it during the RFC editing process.

> of course if such changes are made they should be done globally.

We'll kick the can down the road (gently) toward the RFC Editor.

Part of the problem here is the need to replace "connections'" by
"connection types".

I believe we should add a short introductory paragraph at the start of
Section 5.2.3,

reading as follows in-04:

NFS Version 4 may be implemented using a number of different types of
connections:

   - Stream connections may be used to provide RPC service as described in
   [RFC5531].
   - RDMA-capable connections may be used to provide RPC service using
   RPC-over-RDMA, as described in [RFC8166].

> nit: maybe describe this as a "trial and error" approach to connection
type

We could say:

As a result, clients supporting multiple connection types need to attempt
to establish a connection on various connection types allowing it to
determine, via a trial-and-error approach,  which connection types are
supported.

> elsewhere (e.g., RFC 8305) as being analogous.

Possibly, but I’m not familiar enough with this RFC to reference it is this

context.

> filesystem or ten to a different server when load gets too high"

That’s one thing that can be done.

> or are

I don’t know of a current implementation but believe this can be done.

> When the set of network addresses on a server change in a way that would
> affect a file system location attribute, there are several possible
> outcomes for clients currently accessing that file system.  NFS4ERR_MOVED
> is returned only when the server cannot satisfy a request from the client,
> whether because the file system has been migrated to a different server,
is
> only accessible at a different trunked address on the same server, or some
> other reason.

Your version is clearer.  We can pick this up in -04.

> result in an NFS4ERR_MOVED.

I think we can do this in the introductory paragraph by adding the
following sentence at the end:

In the cases 1 and 2 below, NFS4ERR_REMOVED not returned.

> filesystem; it's not going to work if clients treat it as a global

Ok.  Will replace “avoid using that address in the future” by the following;

avoid using that address to access that filesystem in the future

> (specifically, the "normally ..." clause).

We could drop the “normally ..” clause and in its place add the following
sentence:

When such a lock is lost, it is responsibility of the destination server to
provide for its recovery

through use of an fs-specific grace period.

> is only valid within the scope of a given address/server?

I don’t think so.

> nit: I'm not sure that "while" is the right word here.  Perhaps "and

Don’t think “and conversely” is right.  Open to other suggestions.

> Pretty sure you need to say "trunkable" here, too.

Ok.  Will change “other server network addresses” to “other
server-trunkable  network addresses”.

> single server's single-server namespace, right?

Yes, these file systems exist within the containing server single-server
namespace.

> So there is not even a way
> paths in its namespace for some reason).

But they could appear in fs-locations entries for other file systems.  The
text in question is

basically saying that they don’t have to.

> So, we should probably be saying

> ... (and possibly some related tweaks in this part too).

Not sure what would be needed.  Open to concrete suggestions.

> UCS approach, mentioned at the end of Section 5.6 of RFC 7931.

Given the length of this portion of Section 5.6 of RFC7931,
reiterating this material as part of

the security considerations of this document could result in undue
attention to something that,

while worthy of note, will be, for most implementations, not a major
concern.  After all, Section 5.6

of RFC7931 closes by stating “How to balance these considerations
depends on implementation

goals” and this material does not appear in the security
considerations section of RFC7931.   I think

we could reference this material without duplicating it in this document.

> ensuring that I am talking to who I think I am,

I do too.

> which encompasses both the

It  seems to me that what you are describing as a “bizzare deployment” is
in fact a broken one,

in that the putative authentication of the server would be useless.

> addresses", but there are lots of valid things to say here and it's not a

I think we should avoid entering that swamp.   I might be imagining things,
but I worry that

alligators might be present.

> phrasing also includes the confidentiality protection case.

I don’t know either but I believe this formulation follows the current
practice for NFSv4 documents.

> (It's not even clear that this duplication is needed, though.)

I believe that restating this non-normatively would contribute to confusion
as it would seem inconsistent

I don’t see any danger of conflicting interpretations given that there is a
clear reference to the original requirement, which is treated as primary.

> misparsing of "used with others".

Not sure where you want a comma or what other somethings might be added.
Please make a concrete suggestion.

>following bullet points being essentially recommendations for behavior

These are, as you suppose, recommendations for behavior.  I’m not clear
exactly how that fact

make these security-oriented recommendations difficult to understand

> (in

This is a consequence of the fact that use of AUTH_SYS, although not a very
good idea, is currently

allowed, and realistically will never be disallowed.  As a result, we can’t
prevent its use and have to

give the best guidance we can.

On Fri, Jan 18, 2019 at 12:47 PM David Noveck <[email-masked]> wrote:

> As part of addressing this need, [RFC7931] introduces trunking into NFS version 4.0 along with a trunking detection mechanism.  A trunking detection mechanism enables a client to determine whether two distinct network addresses are connected to the same NFS version 4.0 server instance.  This knowledge is necessary since, without it, a client unaware of a trnking relation ship between paths it is using simultaneous is likely to become confused in ways described in [RF7530].
>
> NFSv4.1 was defined with an integral means of trunking detection described in [RFC5661], while NFSv4.0 initially did not have one, with it being added by [RFC7931].  Nevertheless, the use of the concept of server-trunkability is the same in both protocol versions.
>
>
> > Specifically, in
> A client may use file system location elements simultaneously to provide higher-performance access to the target file system.  This is most simply be done using trunling although the use of multiple replicas simultaneously is possible. To enable this simultaneous acces, the client utilizes trunking detection and/or discovery, further described in Section 5.2.2 of the current document, to determine a set of network paths that are server-trunkable with the one currently being used to access the file system.   Once this determination is made, requests may be routed across multiple paths, using the existing state management mechanism
>
> Multiple replicas may also be used simultaneously, typicalls in handling read-only file systems.   In this case, each replica has its own state management which the client needs to be aware of, doin multiple file opens to enable a file to be read simultaneoudly from multiple replicas.
>
>    - When transparent state migration is implemented, locking data needs
>    to be similarly propagated.
>
> Because a file system location attribute may include entries relating to the current server, the migration destination, and possible replicas to use, scanning for available network addresses thst might be trunkable with addresses already seen could potentially be a long process.  In order keep this process as short as possible, Servers that provide infornation about trunkable network paths when the exist are REQUIRED to place file system location entries that represent addresses usable with the current server or a migration target before those associated with replicas.
>
> This ordering allows a client to cease scanning for trunkable file system location entries once it encounters a file system location element whose fs_name differs from the current fs_name, or whose address is not server-trunkable with the one it is currently using.  While the possibility exists, that a client might prematurely cease scanning for trunkable addresses when receiving a location attribute from a an older server not following the order constaint above, the harm is expected to be limited since such servers would not be expected to present infomation about trunkble server access paths.
>
>

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 26 Jan 2019 14:30:28 -0600

> unaware of a trnking relation ship between paths it is using

nit: "trunking relationship"

> server-trunkability is the same in both protocol versions.

That would be a big help; thanks.

> most simply be done using trunling although the use of multiple

nit: "is most simply done using trunking"

> acces, the client utilizes trunking detection and/or discovery,

nit: "access"

> Multiple replicas may also be used simultaneously, typicalls in

nit: "typically"

> own state management which the client needs to be aware of, doin

nit: "doing"

> multiple file opens to enable a file to be read simultaneoudly from

SGTM.

> When multiple locations are provided, the client has this choice.

Okay, thanks for confirming that.

> your interpretation of section 5.3.

I don't have a detailed description of a problematic scenario, here, so my
"probably" is to be treated as speculative.

Basically, my reading of 5.3 is that it provides a way for a client to
know, implicitly, that server Y serves file system B because server Y and
server X are trunked, and the client knows that server X serves file system
B (because the client was already talking to server X for file system A and
got back a list of filesystems on X).  So if the client is currently
talking to server Z for file system B but then B tries to initiate
migration to server X, the client could well be within its rights to talk
to server Y instead.  This would be problematic if Z has only propagated
locking state to X and not Y.

I'm happy to learn what misunderstandings are present in the above :)

> Not sure what you mean by "some other way".

I'm not sure either :)

The point being that if 5.2.6 allows a server to include both migration
targets and (non-migration-target) replicas in the filesystem location
attribute, a client trying migration would probably want to know where the
boundary between migration targets and non-migration-targets is.  From the
discussion above, it sounds like the client doesn't have "some other way"
of knowing this, and having the server list non-migration-targets is a bad
idea, at least when initiating migration.

> expected to present infomation about trunkble server access paths.

Works for me.

> It turns out this deletion was inadvertant.  The paragraph will be added

That works too :)

-Benjamin

---

From: Benjamin Kaduk <[email-masked]>
Date: Sat, 26 Jan 2019 14:58:20 -0600

>    trunking detection and trunking discovery see Section 3.

Thanks.

> exception in its Section 1.1.   To avoid us having to do the same thing, we

Ah, I had completely missed that disclaimer in RFC 7530, whoops.

> where the data corresponding to a given file system may be accessed.

We might also be able to point to Section 5.2 of 7530 for the RECOMMENDED
definition.  But since Section 2.2.7 also refers to it as "the fs_locations
RECOMMENDED attribute" as well as the main treatement in the (now-replaced)
Section 8.1, it seems we can probably get away with the minimalistic
version.

> treat the corresponding addresses as connected to different servers.

SGTM

> with the RFC editor, we'll find out about it during the RFC editing process.

I'd consider adding an RFC Editor note calling out the question, since this
"current document" construction appears in several places and we'd want to
be consistent about it.

> Part of the problem here is the need to replace "connections'" by

Ah, yes, that would help.

>    RPC-over-RDMA, as described in [RFC8166].

As would this (though I think this part didn't bother me, since I lurk on
the nfsv4 list and remembered the different transports being used).

> determine, via a trial-and-error approach,  which connection types are

Sure.

> Possibly, but I’m not familiar enough with this RFC to reference it is this

Okay; it's not going to be a critical omission if it doesn't happen.

> > whether because the file system has been migrated to a different server, is

Okay.

> avoid using that address to access that filesystem in the future

Thanks.

> through use of an fs-specific grace period.

Thanks.

> > is only valid within the scope of a given address/server?

Okay.

> Don’t think “and conversely” is right.  Open to other suggestions.

Rereading, I agree that "conversely" is not right.
All I can come up with right now is to split with a semicolon: "addresses;
it is also possible for multiple file system location entries to have"

> basically saying that they don’t have to.

Okay.

> Not sure what would be needed.  Open to concrete suggestions.

It's probably okay to leave as-is; no concrete suggestions are coming to
me.

> we could reference this material without duplicating it in this document.

So maybe something like "the privacy considerations relating to uniform vs.
non-uniform client strings, discussed in Section 5.6 of RFC 7931, are also
applicable to their usage for trunking detection in NFS v4.0"?

> alligators might be present.

Okay.  "It's not a big deal."

> practice for NFSv4 documents.

that works for me

> clear reference to the original requirement, which is treated as primary.

It's your call; this is a non-blocking comment.

> Please make a concrete suggestion.

I'm looking for "then network addresses in this range can be used, with
others discarded".

> make these security-oriented recommendations difficult to understand

I was trying to contrast "possibilities" with "recommendations"; even just
replacing the one word with the other might help me out.

-Benjamin

---

From: Chuck Lever <[email-masked]>
Date: Mon, 28 Jan 2019 10:51:55 -0500

> On Jan 26, 2019, at 3:58 PM, Benjamin Kaduk <[email-masked]> wrote:
> 

[ ... remaining comments snipped ... ]

If there are no more comments to resolve, I will assemble and submit -04 this
week based on these e-mail threads.

--
Chuck Lever