Subject: Re: [nfsv4] No WG meeting for IETF 91

From: David Noveck <[email-masked]>
Date: Wed, 1 Oct 2014 10:50:48 -0400

i agree that Tom's request for use cases is reasonable.  My issue/dismay is
limited to some of the other things (i.e. the cetera in "et cetera").

I think we should be moving toward making this a full wg item.  Given what
Tom has been saying about moving to non-monolithic specs, and to avoid
tying features to particular minor versions, it seems that we should be
turning the draft into a working group standards-track document.

On Tue, Sep 30, 2014 at 12:22 PM, Spencer Shepler <[email-masked]>
wrote:

>
>
>
>
> *From:* nfsv4 [mailto:[email-masked]] *On Behalf Of *Thomas Haynes
> *Sent:* Monday, September 29, 2014 5:51 PM
> *To:* Marc Eshel
> *Cc:* J. Bruce Fields; [email-masked]; nfsv4
>
> *Subject:* Re: [nfsv4] No WG meeting for IETF 91
>
>
>
>
>
> On Sep 29, 2014, at 7:50 PM, Marc Eshel <[email-masked]> wrote:
>
>
>
>
> Tom Haynes <[email-masked]> wrote on 09/29/2014 08:50:37
> AM:
>
> > From: Tom Haynes <[email-masked]>
> > To: Marc Eshel/Almaden/IBM@IBMUS,
> > Cc: "J. Bruce Fields" <[email-masked]>, nfsv4 <nfsv4-
> > [email-masked]>, "[email-masked]" <[email-masked]>
> > Date: 09/29/2014 08:50 AM
> > Subject: Re: [nfsv4] No WG meeting for IETF 91
> >
> > On Sep 29, 2014, at 11:17 AM, Marc Eshel <[email-masked]> wrote:
> >
> > <Mail Attachment.gif>
> >
> > "nfsv4" <[email-masked]> wrote on 09/29/2014 07:51:46 AM:
> >
> > > From: "J. Bruce Fields" <[email-masked]>
> > > To: David Noveck <[email-masked]>,
> > > Cc: "[email-masked]" <[email-masked]>
> > > Date: 09/29/2014 07:52 AM
> > > Subject: Re: [nfsv4] No WG meeting for IETF 91
> > > Sent by: "nfsv4" <[email-masked]>
> > >
> > > On Mon, Sep 29, 2014 at 10:37:56AM -0400, David Noveck wrote:
> > > > I think we're in agreement about the direction the working group
> needs to
> > > > move.   However, I don't see some of the obstacles to going forward
> that
> > > > you seem to.
> > > >
> > > > > Me to, hence my position advocating a more modular and extensible
> > > > protocol that allows > add-on features to be added, discovered,
> > > >
> > > > In fact. NFSv4 is already such a protocol.  It is designed to allow
> > > > optional features to be added (and discovered by seeing if NOTSUPP is
> > > > returned or not).  The only big problem is some versioning rulesthat
> have
> > > >  been interpreted as requiring us to batch features together.  It
> may be
> > > > time for the working group to step through the open door (or it
> > may be time
> > > > to do that at IETF92 in Dallas).
> > > >
> > > > > and version-negotiated dynamically on top of the base protocol.
> > > >
> > > > I don't see that there is any need for negotiation here.  What I see
> > > > happening is the server saying what features it supports and the
> client
> > > > deciding to use them or not.    If the client doesn't like that set
> of
> > > > features, it can try using a different server but "negotiation" is
> not
> > > > really happening and using that word makes it seem that this is a
> more
> > > > difficult issue than it in fact is.  Real negotiation is a difficult
> > > > problem and we just need optional features and feature
> > discovery, which are
> > > > things already in NFSv4.
> > > >
> > > > > I believe that specifying these protocol features (that may
> > include, e.g.
> > > >  data structure
> > > > > definitions, operations,and new attributes, flags, or options
> > to existing
> > > > operations)
> > > >
> > > > These are precisely the things that can be added in normal (i.e. not
> like
> > > > 4.1) minor versions.  The protocol already has the wherewithal
> > to have such
> > > > optional features added.  Tom provided the only missing piece by
> allowing
> > > > new switch cases or flags to be added to existing operations in
> v4.2. and
> > > > beyond.
> > > >
> > > > > could be done rapidly with relatively little marginal effort
> -therefore
> > > > shortening the
> > > > > innovation turnaround.
> > > >
> > > > The effort depends on the size of the feature.  Small features could
> be
> > > > done quite quickly while larger features could still take up
> considerable
> > > > time.   The major difficulty we have had with large features has been
> > > > assigning them to a specific minor version and then having them hold
> up
> > > > other features that are not so big.
> > > >
> > > > Let me try to make this all more concrete by offering two proposals:
> > > >
> > > >    - A proposal for a last-minute addition to the v4.2 spec to enable
> > > >    extensions within v4.2.  This is a straw-man proposal which I
> > > expect to be
> > > >    shot down, but it will show the wg what is possible, if we
> > > think this is a
> > > >    critical problem that must be resolved ASAP.
> > > >    - A proposal that we produce a v4.3 spec which consists of
> material to
> > > >    enable and further clarify the process of doing further
> incremental
> > > >    additions *and nothing else *.  The goal would be to get sucha
> spec to
> > > >    WGLC in one year, i.e by 11/2015 (IETF 94?).  Development of
> individual
> > > >    features could proceed in parallel.
> > >
> > > I've got no passionate objection, but absent an example it feels a bit
> > > like, um, meta-feature-creep?
> > >
> > > Is there some specific feature that, say, we *could* deliver to users
> in
> > > 2015, but won't be able to because of the batching requirements?
> >
> > Yes, xattr.
>
> >
> > I thought we still hadn’t accepted xattr as something we were going to
> pursue?
>
> I am not sure why not, I did not hear any objection after presenting it 3
> times, other than some comments on how to do it right.
>
>
>
> I have consistently asked for the use cases, i.e., why do we want to do it
> other than it is available?
>
>
>
> I have heard others ask for that as well.
>
>
>
> Your current draft details uses of current xattr implementations, but it
> does not motivate why we want
>
> to take this into NFS nor what are the requirements.
>
>
>
> As I've pointed out in the past, we had a problem getting Labeled NFS
> going until we provide a requirements
>
> document and use cases for labels over the wire. The 3-4 cases (most
> notably seVirt for selinux) really
>
> allowed us to sell the need, even though we ended up removing them from
> the final requirements document.
>
>
>
> The other thing missing from your document is how different clients (and
> by abstraction the applications which would
>
> use xattrs) are going to be able to interoperate with the provided xattrs.
>
> *[Spencer] *
>
>
>
> *Tom’s request for use cases, etc. is reasonable – it will provide the
> appropriate motivation and help with bringing the work in as a full WG
> item.*
>
>
>
> *Spencer*
>
>
>
>
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>
>

---

From: Tom Haynes <[email-masked]>
Date: Wed, 1 Oct 2014 11:39:38 -0400

On Oct 1, 2014, at 10:50 AM, David Noveck <[email-masked]> wrote:

> i agree that Tom's request for use cases is reasonable.  My issue/dismay is limited to some of the other things (i.e. the cetera in "et cetera”).

Heh, the document has been getting better. :->

There seems to be a strand that everything which applies to other opaque strings, internationalization, security, etc (there it is again!) applies to similar constructs in xattrs. These need to be called out.

> 
> I think we should be moving toward making this a full wg item.  Given what Tom has been saying about moving to non-monolithic specs, and to avoid tying features to particular minor versions, it seems that we should be turning the draft into a working group standards-track document.

Accepting it without having the use cases makes that a moot point.

None of the NFSv4.2 items were accepted as standard track documents on their own.

I’d prefer we not accept this until we establish the need. I’ve been asking for use cases for at least a year.

> 
> On Tue, Sep 30, 2014 at 12:22 PM, Spencer Shepler <[email-masked]> wrote:
>  
> 
>  
> 
> From: nfsv4 [mailto:[email-masked]] On Behalf Of Thomas Haynes
> Sent: Monday, September 29, 2014 5:51 PM
> To: Marc Eshel
> Cc: J. Bruce Fields; [email-masked]; nfsv4
> 
> 
> Tom Haynes <[email-masked]> wrote on 09/29/2014 08:50:37 AM:
> 
> > > > I think we're in agreement about the direction the working group needs to
> > > > move.   However, I don't see some of the obstacles to going forward that
> > > > you seem to.
> > > > 
> > > > returned or not).  The only big problem is some versioning rulesthat have
> > > >  been interpreted as requiring us to batch features together.  It may be
> > > > happening is the server saying what features it supports and the client
> > > > deciding to use them or not.    If the client doesn't like that set of
> > > > features, it can try using a different server but "negotiation" is not
> > > > really happening and using that word makes it seem that this is a more
> > > > These are precisely the things that can be added in normal (i.e. not like
> > > > optional features added.  Tom provided the only missing piece by allowing
> > > > new switch cases or flags to be added to existing operations in v4.2. and
> > > > beyond.
> > > > 
> > > > > could be done rapidly with relatively little marginal effort -therefore
> > > > shortening the
> > > > > innovation turnaround.
> > > > 
> > > > The effort depends on the size of the feature.  Small features could be
> > > > done quite quickly while larger features could still take up considerable
> > > > assigning them to a specific minor version and then having them hold up
> > > >    - A proposal that we produce a v4.3 spec which consists of material to
> > > >    enable and further clarify the process of doing further incremental
> > > >    additions *and nothing else *.  The goal would be to get sucha spec to
> > > >    WGLC in one year, i.e by 11/2015 (IETF 94?).  Development of individual
> > > Is there some specific feature that, say, we *could* deliver to users in
> > I thought we still hadn’t accepted xattr as something we were going to pursue? 
> 
> I am not sure why not, I did not hear any objection after presenting it 3 times, other than some comments on how to do it right.
> 
>  
> 
> I have consistently asked for the use cases, i.e., why do we want to do it other than it is available?
> 
>  
> 
> Your current draft details uses of current xattr implementations, but it does not motivate why we want
> 
> As I've pointed out in the past, we had a problem getting Labeled NFS going until we provide a requirements
> 
> document and use cases for labels over the wire. The 3-4 cases (most notably seVirt for selinux) really
> 
> allowed us to sell the need, even though we ended up removing them from the final requirements document.
> 
>  
> 
> The other thing missing from your document is how different clients (and by abstraction the applications which would
> 
> [Spencer]
> Tom’s request for use cases, etc. is reasonable – it will provide the appropriate motivation and help with bringing the work in as a full WG item.
> 
>  
> 
> Spencer
> 
>  
> 
>  
> 
>

---

From: David Noveck <[email-masked]>
Date: Wed, 1 Oct 2014 13:25:17 -0400

> Accepting it without having the use cases makes that a moot point.

Let me clarify.  I believe your request for use cases is reasonable.  When
I proposed moving forward with this, my assumption was that the requested
use cases would be provided, and i wasn't suggesting that we short-cur that
process.

> None of the NFSv4.2 items were accepted as standard track documents on
their own.

I think that was precisely the problem.  We had a monolithic spec with
numerous features and that made it hard to deal with the fact that all were
not progressing at the anticipated pace. if all the features are in a
single spec, the approval process, as you have noted, becomes more
difficult, and it becomes hard to defer one feature for  a moderate amount
of time (like one year)., without deferring features that are ready to go.
If it's in a separate spec, that spec's approval can be appropriately
deferred.  It's more troublesome to try to pull one feature out of a
monolithic spec.

It hurts when we do that, so let's not do that.

> I’d prefer we not accept this until we establish the need.

I think the need is clear but I recognize that others might not agree.  I
think we should only move forward with this once there is a working group
consensus.

> I’ve been asking for use cases for at least a year.

If this is the one thing that is  holding up going forward on this, my
guess is that you'll get what you want with dispatch.

If you seem to be asking for a solution to the internationalization problem
in addition, you blur the message.

On Wed, Oct 1, 2014 at 11:39 AM, Tom Haynes <[email-masked]>
wrote:

> i agree that Tom's request for use cases is reasonable.  My issue/dismay
> is limited to some of the other things (i.e. the cetera in "et cetera”).
>
>
> There seems to be a strand that everything which applies to other opaque
> strings, internationalization, security, etc (there it is again!) applies
> to similar constructs in xattrs. These need to be called out.
>
>
> I’d prefer we not accept this until we establish the need. I’ve been
> asking for use cases for at least a year.
>
>
>

---

From: Tom Haynes <[email-masked]>
Date: Wed, 1 Oct 2014 12:08:39 -0700

> in addition, you blur the message.

Does i18n need to be solved? no

As I stated below, the issue has to be called out.

The whole premise of the xattr approach is to allow different implementations
to work with a common infrastructure via NFS.

And in my mind, that is where the requirements need to come in.

What is a case insensitive client supposed to do when presented with case sensitive names?

What is a client which has a max xattr size of 20k supposed to do when the server
has 40k of xattrs?

I know you've provided an approach for the first, but these are the
"etc" that need to be taken care of here.

If there were a posix standard for xattr, this would be much easier. But since
there is not, IMHO this is where the hard work exists for the proposal.

BTW, I agree 100% that I am an individual and that the WG may reach rough
consensus to proceed. But, I'm pretty sure the IESG review will track the
same set of questions that I am asking here. :-)

---

From: Marc Eshel <[email-masked]>
Date: Thu, 2 Oct 2014 16:21:33 -0700

"nfsv4" <[email-masked]> wrote on 09/29/2014 05:51:00 PM:

> From: Thomas Haynes <[email-masked]>
> To: Marc Eshel/Almaden/IBM@IBMUS
> Cc: "J. Bruce Fields" <[email-masked]>, "[email-masked]" 
> <[email-masked]>, nfsv4 <[email-masked]>
> Date: 09/29/2014 05:52 PM
AM:
> > > > optional features to be added (and discovered by seeing if NOTSUPP 

is
may be
client

not
more
> > > > > could be done rapidly with relatively little marginal 
effort-therefore
incremental
> > I thought we still hadn?t accepted xattr as something we were 
> going to pursue? 
> 
> I am not sure why not, I did not hear any objection after presenting
> it 3 times, other than some comments on how to do it right. 
> 

Section 1: "... results in data loss if one copies, over the NFS protocol, 

a file with xattrs from one file system to another that also supports 
xattrs".
Isn't that motivation enough?

> 
> As I've pointed out in the past, we had a problem getting Labeled 
> NFS going until we provide a requirements

I am not sure it's fair to compare Labeled NFS requirements with those of 
xattrs. Based on the past discussions, I believe we've established there 
is a consensus that adding xattrs to the protocol is a good thing. Xattrs 
are out there and people use them - every file system and platform 
supports them. We can't argue why they are needed - as somebody said on 
the list earlier, that ship has sailed. We can argue about why we need 
this in NFS - and we did in this mailing list for a long time. We 
certainly need to get the requirements listed and we believe we've done 
that in the draft (Section 5.3). 

> 
> The other thing missing from your document is how different clients 
> (and by abstraction the applications which would

What do you mean by interoperate? The protocol only provides a way to get 
and set xattrs, so the issue arises if one client (hence the application 
uses it) supports xattrs and the other does not. This problem exists today 

with a local application that understands xattrs and the one using NFS.

Marc.

> 
> Besides my earlier question on client max size being larger than the
> size of all attributes for a file, you can explore issues with 
> character sets (i18n/case sensitivity/etc) in both keys and values, 
> which of the four namespaces are supported (an earlier draft had a 
> requirement, it is gone now), etc.
> 
> I know the draft states:
> 
>    Future versions of this document or other related IETF documents may
>    define specific values for xattr key names, or mechanisms for
>    encoding namespace in xattrname4. 
> But I was pretty sure there were security implications on using 
> namespaces other than "user".
> 
> ________________________
_______________________
> nfsv4 mailing list
> [email-masked]

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 2 Oct 2014 23:06:30 -0400

> Isn't that motivation enough?

That's kind of obvious and it doesn't really advance the discussion.  We
can't judge whether anyone's likely to actually care about losing those
xattrs without evidence that they're actually using them in the first
place.

I think what you wrote that's more helpful is:

	http://tools.ietf.org/html/draft-naik-nfsv4-xattrs-00#section-2

	Examples of application-specific xattrs include storing metadata
	that tracks what application created the file, a tag to indicate
	when the file was last verified by a data integrity scrubber, or
	a tag to hold a checksum/crypto hash of the file contents along
	with the date of that signature. Xattrs can also be used for
	decorations or annotations. For example, a file downloaded from
	a web server can be tagged with the URL, which can be convenient
	if its source has to be determined in the future. Likewise, an
	email attachment can when saved be tagged with the message-id of
	the email. This will make it possible to trace the original
	message. Xattrs can be retrieved and set through system calls or
	shell commands and generally supported by user-space tools that
	preserve other file attributes.

It shouldn't be to hard to expand.  I don't know exactly what sort
people are looking for, but random ideas:

Find some systems, scan them for files with xattrs set.  Google for the
names of those xattrs, find out who's using them and why, contact the
developers to get details about what having/not having those xattrs
means to their applications.

I did a quick look at my laptop:

	getfttr -h -d -R / 2>/dev/null|grep -v '#'|grep -v '^$'|cut -d= -f1|sort|uniq

and that turned up:

	user.crtime_usec
		on files in /var/log/journal/

	user.xdg.origin.url
	user.yum.checksum.sha
	user.yum.checksum.sha256
		on files in /var/cache/yum/

So might be worth looking at journal and yum?  Googling for "xdg" and
xattrs turns up:

	http://www.freedesktop.org/wiki/CommonExtendedAttributes/

Maybe that has some useful ideas.

Maybe do similar digging around on other operating systems that
implement extended attributes?

Is there some big set of open-source code that you could search for
references to the libattr functions?  Or is there a way to search
rpm/dpkg databases to find out what depends on libattr?

Do you have some particular motivation for working on this?  Do you have
users asking for it?  Have you talked to them to find out what they plan
to do with xattrs?

--b.

---

From: Thomas Haynes <[email-masked]>
Date: Fri, 3 Oct 2014 13:47:20 -0400

> xattrs without evidence that they're actually using them in the first

Something else to look at:

http://blog.cloudera.com/blog/2014/06/why-extended-attributes-are-coming-to-hdfs/

Also note it talks about size limits and case sensitivity.

And

http://namei.org/presentations/linuxcon09_nfsv3xattrs.pdf

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 3 Oct 2014 14:00:14 -0400

> rpm/dpkg databases to find out what depends on libattr?

The following are for F20 and Ubuntu 14.04.  I haven't tried to
investigate any of them.

--b.

[bfields@pad ~]$ repoquery --whatrequires libattr
acl-0:2.2.52-4.fc20.x86_64
aide-0:0.15.1-6.fc20.x86_64
attr-0:2.4.47-3.fc20.x86_64
bsdcpio-0:3.1.2-7.fc20.x86_64
bsdtar-0:3.1.2-7.fc20.x86_64
coreutils-0:8.21-18.fc20.x86_64
coreutils-0:8.21-21.fc20.x86_64
dar-0:2.4.10-3.fc20.x86_64
dar-0:2.4.15-1.fc20.x86_64
gnome-vfs2-0:2.24.4-14.fc20.i686
gnome-vfs2-0:2.24.4-14.fc20.x86_64
kdelibs-6:4.11.3-9.fc20.i686
kdelibs-6:4.11.3-9.fc20.x86_64
kdelibs-6:4.14.1-1.fc20.i686
kdelibs-6:4.14.1-1.fc20.x86_64
kdelibs3-0:3.5.10-54.fc20.i686
kdelibs3-0:3.5.10-54.fc20.x86_64
krusader-0:2.4.0-0.10.beta3.fc20.x86_64
libacl-0:2.2.52-4.fc20.i686
libacl-0:2.2.52-4.fc20.x86_64
libarchive-0:3.1.2-7.fc20.i686
libarchive-0:3.1.2-7.fc20.x86_64
libattr-devel-0:2.4.47-3.fc20.i686
libattr-devel-0:2.4.47-3.fc20.x86_64
libcap-0:2.22-7.fc20.i686
libcap-0:2.22-7.fc20.x86_64
libdar-0:2.4.10-3.fc20.i686
libdar-0:2.4.10-3.fc20.x86_64
libdar-0:2.4.15-1.fc20.i686
libdar-0:2.4.15-1.fc20.x86_64
libguestfs-1:1.24.1-3.fc20.i686
libguestfs-1:1.24.1-3.fc20.x86_64
netatalk-4:2.2.3-9.fc20.x86_64
nfs4-acl-tools-0:0.3.3-11.fc20.x86_64
pam_openshift-0:1.9.0.1-1.fc20.i686
pam_openshift-0:1.9.0.1-1.fc20.x86_64
pam_openshift-0:1.10.1.1-1.fc20.i686
pam_openshift-0:1.10.1.1-1.fc20.x86_64
patch-0:2.7.1-7.fc20.x86_64
patchelf-0:0.6-7.fc20.x86_64
pyxattr-0:0.5.1-4.fc20.x86_64
samba-libs-2:4.1.1-1.fc20.i686
samba-libs-2:4.1.1-1.fc20.x86_64
samba-libs-2:4.1.12-4.fc20.i686
samba-libs-2:4.1.12-4.fc20.x86_64
scpio-0:1.5.2-8.fc20.x86_64
shadow-utils-2:4.1.5.1-8.fc20.x86_64
spax-0:1.5.2-8.fc20.x86_64
star-0:1.5.2-8.fc20.x86_64
vim-X11-2:7.4.027-2.fc20.x86_64
vim-X11-2:7.4.417-1.fc20.x86_64
xfsdump-0:3.1.3-2.fc20.x86_64

root@pepper:/home/bfields# apt-cache rdepends libattr1
libattr1
Reverse Depends:
  libarchive1
  libarchive12
  libkio5
  samba-libs
  python-samba
  libkio5
  syrep
  nfs4-acl-tools
  netatalk
  lighttpd
  libfile-extattr-perl
  libeiskaltdcpp2.2
  libdar64-5
  libapache2-mod-mime-xattr
  ciopfs
  aide-dynamic
  xfsdump
  samba-libs
  python3-pyxattr-dbg
  python3-pyxattr
  python-samba
  python-pyxattr-dbg
  python-pyxattr
  libkio5
  libattr1-dev
  libarchive13
  libacl1
  coreutils
  attr

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 3 Oct 2014 14:07:46 -0400

> 	preserve other file attributes.
...
> 	http://www.freedesktop.org/wiki/CommonExtendedAttributes/

(And, apologies, I missed that a) you already reference that, b) the
most recent draft has some more use cases beyond the paragraph quoted
above.)

--b.

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 3 Oct 2014 15:00:09 -0400

>   libapache2-mod-mime-xattr

Most of those are for the Linux POSIX ACL and SeLinux support, so are
not applicable as use-cases for NFS. I'm also confused as to how we
would use xfsdump over NFS...
It would therefore be nice if you could sort out which ones are
actually using xattrs. I suspect the list will boil down to 'tracker'
(not listed above), 'beagle' (also not listed above), 'cpio', 'tar',
'cp' and 'samba'.

Cheers
  Trond

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 3 Oct 2014 15:04:19 -0400

> (not listed above), 'beagle' (also not listed above), 'cpio', 'tar',

Yeah, and stuff like patch and vim are likely also just copying xattrs?

Still, there might be something interesting in there.  But I'm not
volunteering to look!

--b.

---

From: Marc Eshel <[email-masked]>
Date: Fri, 3 Oct 2014 12:59:52 -0700

Thanks for trying to help, but I believe that the focuses of the proposal 
is on user applications and not Linux internal use or Linux utilities that 
already make use of the system/trusted/secure attributes. The attributes 
will not be tagged and Linux Server implementation will probably just use 
the user attributes type. So to prove that it is needed we need to name 
some customer applications and describe how they use xattr. I probed one 
customer and  they described their use of xattr as a way to tag files for 
migration policy to decide where and when to move files. Another big one 
is  swift (the object protocol) that uses xattr and when implemented over 
a regular file system that than need to be backed up or accessed using 
different protocol (like NFS) it need to retain the xattr. 

I am not sure why we are wasting time arguing about the obvious, xattr are 
out there and used and for NFS to stay relevant we need to make it support 
all file system features and xattr is supported by most file systems. We 
should focuses and use the limited time that the WG member can spend here 
(like was said before, it is no one full time job) on how to implement the 
right support for xattr.

Again, at some point we need to reach consensus and move on, I though that 
we passed that point but if some one object to the need of xattr to be 
part on the NFS protocol please speak up and if not lets just talk about 
how to do it. 

From:   "J. Bruce Fields" <[email-masked]>
To:     Trond Myklebust <[email-masked]>, 
Cc:     Marc Eshel/Almaden/IBM@IBMUS, "[email-masked]" <[email-masked]>
Date:   10/03/2014 12:04 PM
Subject:        Re: [nfsv4] No WG meeting for IETF 91

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Fri, 3 Oct 2014 20:21:26 +0000

At some point in any proposal someone asks for it to be included as an official working item.

>From this message (and others) it appears that your opinion is that this proposal has been made implicitly.

To make it formal, there seems to enough interest in discussion that we should go ahead and make it a WG item (objections welcome if they exist).

Note that this does presume that the work should be included in a minor version but we are on that path.

Spencer

Sent from my Windows Phone
________________________________
From: Marc Eshel<mailto:[email-masked]>
Sent: ‎10/‎3/‎2014 1:00 PM
To: J. Bruce Fields<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [nfsv4] No WG meeting for IETF 91

--b.

---

From: Thomas Haynes <[email-masked]>
Date: Sat, 4 Oct 2014 00:24:30 -0400

On Oct 3, 2014, at 3:59 PM, Marc Eshel <[email-masked]> wrote:

> out there and used

Are they? How much?

The argument being presented is that we should support xattrs
because they are there. 

What drives us to take the work? Where do we lose "customers"
if we do not do xattrs?

Is it a feature that will be used once it is delivered?

Here is a link which talks about them being not used as much:

http://www.lesbonscomptes.com/pages/extattrs.html

And while it helps the draft because it blames the lack of NFS support,
the tone on various different online articles I've read today is that
xattrs while present are not used that much.

Labeled NFS was in somewhat the same boat until the seVIrt use
case was presented to the group.

> and for NFS to stay relevant we need to make it support 
> all file system features and xattr is supported by most file systems.

Yes, it is a filesystem. It is not just a transport protocol. 

There were two things that enabled Labeled NFS to finally proceed:

1) The spec detailed how different MAC implementations could co-exist.

2) The requirements document broke down the responsibilities of the client
and the server.

As it currently stands, the xattr draft either implicitly assumes that the
only clients will be Linux based or ignores the interaction between different
client implementations.

Also note that it helped that each of the MAC implementations that were
considered for LNFS had well defined semantics (and in some cases
actual specifications).

> We 

If we only visit it every 4 months, nothing will be done.

> part on the NFS protocol please speak up and if not lets just talk about 

I've been objecting since it was first proposed.

What is the use case that makes us want to do it?

How do different client OSes interact?
a) In namespaces
b) Securely
c) With the limits

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 4 Oct 2014 09:11:14 +0200

> What drives us to take the work? Where do we lose "customers"

That didn't seem to matter when adding named attribute support to
NFSv4 :)  Not that I'm arguing for adding extattrs, especially in
the form of the draft, but it seems like the standards fortunately
are higher now.

> How do different client OSes interact?

As someone who has worked with Linux xattrs for a long time, I think
the Linux concept of the text prefixes for nightmares which have
semantic meanings is a nightmare.  I don't really understand why and
how it made it into Linux, as the original IRIX version had binary
namespaces, as does FreeBSD.

The current version of draft-naik-nfsv4-xattrs-01 totally ignores
the issue of access control to different namespaces.  To be actually
interoperable between Linux, FreeBSD, MacOS and Window (and more theoretically
OS/2 and IRIX) it should be defined to operate on only what Linux calls
"user." xattrs, FreeBSD calls EXTATTR_NAMESPACE_USER extattrs, IRIX calls
user namespace attrs, and the the only one available to the rest.

If we actually expose the Linux prefixes we create a giant inteoperability
problem with unspecified mappings, including an expectation of Linux clients
that they could store the attribute types that have security implications
on servers that can not enforce anything like that.

Note that all of the kernel-implemented Linux xattrs in these namespaces
are a blatant abuse of the xattr user interface and should have used
different system calls as they don't have binary blob semantics, so if
we plant to support them in NFSv4 they should probably be different
operations.  All the important cases are already handled in a way more
(labels) or less (ACLs) suitable already.

---

From: Thomas Haynes <[email-masked]>
Date: Sat, 4 Oct 2014 10:35:36 -0400

> the form of the draft, but it seems like the standards fortunately

One would hope that we could learn from our mistakes.

---

From: Chuck Lever <[email-masked]>
Date: Sat, 4 Oct 2014 12:04:54 -0400

> user namespace attrs, and the the only one available to the rest.

I thought we had already agreed that only the “user.” namespace would
be exposed via NFS. If that’s not in the draft specification, it really
needs to be added.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 4 Oct 2014 18:36:22 +0200

> be exposed via NFS. If that’s not in the draft specification, it really

The draft has wording contrary to that.  From section 3:

This document does not propose any language to restrict the key names
of extended attributes. Future versions, or other related IETF
documents, may include additional text to enforce namespace prefix to
key names, formalize names of some well-defined xattrs, or impose
additional restrictions on the allowed namespaces to user-managed
metadata only, in order to prevent the development of non-
interoperable implementations. This document, however, does
require that the attribute key/value MUST not be interpreted by
the NFS clients and servers.

To me that wording is one of the blocker, the other is the cumbersome
XDR that is contrary to all existing xattr implementations.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Sat, 4 Oct 2014 15:03:39 -0400

> be exposed via NFS. If that’s not in the draft specification, it really

Section 3 does say that the "user" namespace is the only one for which
xattrs operations "can be considered interoperable across platforms and
vendor implementations", so it seems strange if they're trying to keep
the others.

I think they were trying to keep NFS xattr names the same as local
attribute names instead of requiring implementations to strip the
"user.", and also maybe hoping to leave open the possibility of future
standardization of behavior in other namespaces without requiring new
xdr.  But I think all the feedback has been that that's a quagmire.

So I'm not sure what the appropriate language is.  Maybe something
roughly like:

	Some operating systems define multiple "namespaces" for xattrs,
	with the system enforcing different policies for different
	xattrs, and giving some xattrs special meaning to the system.
	(For example, some systems use xattr operations to set and query
	ACLs.)

	NFS extended attributes are not organized into namespaces and do
	not have special meaning to the server.  Several operating
	systems that define multiple namespaces have a "user" namespace
	with these semantics.  Such systems will want to keep NFS
	extended attributes under the "user" namespace by translating
	any NFS xattr name "X" to a local extended attribute name
	"user.X".

--b.

---

From: Manoj Naik <[email-masked]>
Date: Sat, 4 Oct 2014 13:37:25 -0700

"nfsv4" <[email-masked]> wrote on 10/04/2014 12:03:39 PM:

> To: Chuck Lever <[email-masked]>
> Cc: Christoph Hellwig <[email-masked]>, "[email-masked]" <[email-masked]>
> Date: 10/04/2014 12:03 PM

Definitely not trying to keep the others. But, based on what I thought was
a request, leaving open the option of allowing xattrs from other namespaces
to be added in the future if they could be made interoperable.

> xdr.  But I think all the feedback has been that that's a quagmire.

Yes.

>    any NFS xattr name "X" to a local extended attribute name

I can go with this.

Manoj.

---

From: Manoj Naik <[email-masked]>
Date: Sat, 4 Oct 2014 13:40:38 -0700

"nfsv4" <[email-masked]> wrote on 10/04/2014 09:36:22 AM:

> From: Christoph Hellwig <[email-masked]>
> Cc: "J. Bruce Fields" <[email-masked]>, Christoph Hellwig
> <[email-masked]>, "[email-masked]" <[email-masked]>
> Date: 10/04/2014 09:36 AM

Yup - I added this to leave open the option of adding non-user namespace
xattrs in the future, if they could be well-defined and made to be
interoperable. Perhaps not something this draft should address - I'll
remove it.

Manoj.

---

From: Christoph Hellwig <[email-masked]>
Date: Sun, 5 Oct 2014 09:28:04 +0200

> So I'm not sure what the appropriate language is.  Maybe something

Looks reasonable to me, although I'm not sure the last sentence belongs
into the standards document.

---

From: David Noveck <[email-masked]>
Date: Sun, 5 Oct 2014 08:21:10 -0400

>    (For example, some systems use xattr operations to set and query

Given that NFSv4 has ACL's, this statement might lead to a discussion that
is really beside-the-point for user xattr's.  Can we say "security-related
attributes".

---

From: Marc Eshel <[email-masked]>
Date: Mon, 6 Oct 2014 12:34:38 -0700

Hi Spencer 

We are formally requesting that the xattr draft be made a WG item. 

Thanks, Marc.

From:   Spencer Shepler <[email-masked]>
To:     Marc Eshel/Almaden/IBM@IBMUS, "J. Bruce Fields" 
<[email-masked]>, 
Cc:     "[email-masked]" <[email-masked]>
Date:   10/03/2014 01:21 PM
Subject:        RE: [nfsv4] No WG meeting for IETF 91

At some point in any proposal someone asks for it to be included as an 
official working item. 

From this message (and others) it appears that your opinion is that this 
proposal has been made implicitly. 

To make it formal, there seems to enough interest in discussion that we 
should go ahead and make it a WG item (objections welcome if they exist). 

Note that this does presume that the work should be included in a minor 
version but we are on that path. 

Spencer 

From: Marc Eshel
To: J. Bruce Fields
Cc: [email-masked]

right support for xattr.

--b.

---

From: Tom Haynes <[email-masked]>
Date: Mon, 6 Oct 2014 15:48:27 -0400

One thing I got from the recent set of discussions on the WG alias is that
there is not consensus on this issue.

---

From: Manoj Naik <[email-masked]>
Date: Mon, 6 Oct 2014 13:36:59 -0700

Tom,

The recent discussion on the mailing list is about specifics of the
protocol. That discussion will continue and will result in updates to the
draft, but is a complete specification (final draft), that is done and
baked, a prerequisite for being a WG item?

We need agreement on whether xattrs should be added to the NFSv4 protocol
or not, before getting into the operation details. You've argued that there
isn't a need. Can others chime in?

Manoj.

"nfsv4" <[email-masked]> wrote on 10/06/2014 12:48:27 PM:

> Date: 10/06/2014 12:48 PM
> Subject: Re: [nfsv4] request for xattr as a WG item

We
> >> On Fri, Oct 3, 2014 at 2:00 PM, J. Bruce Fields <[email-masked]>

---

From: Nico Williams <[email-masked]>
Date: Mon, 6 Oct 2014 15:58:51 -0500

The earlier discussions showed that there was consensus for an xattrs
extension, or at least that named attrs have the wrong semantics.  It seems
likely to me that a call for consensus will show consensus for adoption.
FWIW, I'm in favor.

Nico
--

---

From: "Matt W. Benjamin" <[email-masked]>
Date: Mon, 6 Oct 2014 17:00:00 -0400 (EDT)

Hi,

That was my interpretation of the foregoing discussion, and I would also support.

Matt

----- "Nico Williams" <[email-masked]> wrote:

> extension, or at least that named attrs have the wrong semantics. It
> seems likely to me that a call for consensus will show consensus for
> adoption. FWIW, I'm in favor.
> 
> Nico
> -- 

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309

---

From: Trond Myklebust <[email-masked]>
Date: Mon, 6 Oct 2014 17:25:23 -0400

Hi Manoj,

On Mon, Oct 6, 2014 at 4:36 PM, Manoj Naik <[email-masked]> wrote:
> We need agreement on whether xattrs should be added to the NFSv4 protocol or
> not, before getting into the operation details. You've argued that there

I don't believe that Tom has made a blanket statement that there is no
need. I've read him as saying that a well formulated use case should
be a requirement before we make this a full blown WG item. Why is that
unreasonable?

Put differently: we've all managed to live just fine without an
implementation in NFS for more than 10 years of existence of the xattr
api in Linux filesystems, so why must we consider it now?

Cheers
  Trond

---

From: Marc Eshel <[email-masked]>
Date: Mon, 6 Oct 2014 14:25:42 -0700

You are the only one that objects to the need for xattr as part of the NFS 
protocol, the reset of the discussions are constructive criticism on how 
the draft can improve. I am still not sure how we reach consensus in this 
WG so maybe Spencer can clarify on how we will decide if xattr is a WG 
item or not.
Marc. 

Cc:     Spencer Shepler <[email-masked]>, "[email-masked]" 
<[email-masked]>

---

From: Tom Haynes <[email-masked]>
Date: Mon, 6 Oct 2014 17:37:11 -0400

> WG so maybe Spencer can clarify on how we will decide if xattr is a WG 

Marc,

Please reread my posts - I do not object to working on xattrs. I do
object to taking an item without use cases and requirements. It does
not matter what the item under consideration is, I actually do not care
one way or another about xattrs.

Also, you seem to be stating that I have not contributed constructive
criticism on how to improve the draft. Which is not true.

Enjoy,
Tom

---

From: Manoj Naik <[email-masked]>
Date: Mon, 6 Oct 2014 14:41:40 -0700

Trond Myklebust <[email-masked]> wrote on 10/06/2014 02:25:23 PM:

> From: Trond Myklebust <[email-masked]>
> To: Manoj Naik/Almaden/IBM@IBMUS
> Cc: Tom Haynes <[email-masked]>, "[email-masked]"
> <[email-masked]>
> Date: 10/06/2014 02:25 PM

See Section 2 in the current draft. Perhaps this can be expanded
(volunteers?), but most users (our customers too) don't describe _how_ they
use extended attributes. For good reason, why should they? I can give you a
list of customers that are asking for it, but I think that isn't the point.
We've talked in the mailing list in the past about giving them alternatives
to xattrs, but isn't this something that is best left to the application to
decide? Every file system and platform supports xattrs - I would be hard
pressed to find another universal fs feature that the protocol does not
support ;)

> api in Linux filesystems, so why must we consider it now?

And I can probably go back 10 years since I've heard requests for xattrs
over NFS ;) I am starting to understand why - it's a long arduous process.

Manoj.

> >> > is on user applications and not Linux internal use or Linux
utilities
> >> > will not be tagged and Linux Server implementation will probably
just
> >> > use
> >> > customer and  they described their use of xattr as a way to tag
files
> >> > for
> >> > I am not sure why we are wasting time arguing about the obvious,
xattr
> >> > are
> >> > (like was said before, it is no one full time job) on how to
implement
> >> > the
> >> >
> >> > right support for xattr.
> >> >
> >> > Again, at some point we need to reach consensus and move on, I
though

---

From: Nico Williams <[email-masked]>
Date: Mon, 6 Oct 2014 16:51:39 -0500

Not having had xattrs in the protocol even though some clients/servers have
then locally is not evidence of lack of need, but it might be evidence of
WG aloofness :).  Strictly speaking "need" is difficult to establish, and
in this case seems like a fetch-a-rock exercise.  Need is also not the
standard for adoption at the IETF in general (interest, authors, reviewers,
impact on Internet architecture -- these are far more important).
Well-defined requirements, OTOH, should be a pre-req for publication here,
largely because there isn't a standard to follow as to xattrs, but ISTM
that this need only be agreed to to get started.

(I do wish we'd found a way to use named attrs for xattrs.  Maybe we still
could?  Atomicity could be had with rename, for example.  Anyways, they
have distinct namespaces and security semantics, and that's the key.)

Nico
--

---

From: Trond Myklebust <[email-masked]>
Date: Mon, 6 Oct 2014 18:00:42 -0400

On Mon, Oct 6, 2014 at 5:41 PM, Manoj Naik <[email-masked]> wrote:
> use extended attributes. For good reason, why should they?

They should because that's how feature requests work.

Your list is full of potential use cases, and would have been a good
starting point if there were no xattr implementations, however 10
years down the road after the interface was defined and implemented,
it should be possible to find applications. I just tried GNOME, and
applied an icon to a directory on my browser. The result of 'getattr
-d ~/Desktop/for_david' was still empty. This makes sense for an
application that wants to be portable across POSIX platforms...

> I can give you a

It is if you can't motivate why they are needed.

---

From: Marc Eshel <[email-masked]>
Date: Mon, 6 Oct 2014 15:11:25 -0700

Tom Haynes <[email-masked]> wrote on 10/06/2014 02:37:11 
PM:

> Cc: "[email-masked]" <[email-masked]>, Spencer Shepler 
> <[email-masked]>
> Date: 10/06/2014 02:37 PM

> one way or another about xattrs.

We added a section to the draft based on your request to explain why it is 
needed, but you are still not convinced. I think that you are looking for 
a killer application that will prove the need for xattr and I don't know 
of one. all I know is what we added to the draft and what customers tell 
us, but it is not enough for you. So I don't know how to convince you, 
others on the WG seem to agree that it is needed, I assume based on their 
knowledge and experience. I can not give you a name of an application that 
a customer is running because I don't know the name and even if I did it 
would mean nothing, all I can say that for example one application is used 
to tag files using xattr for future migration, it is like a policy for 
file movement among the different storage levels. Maybe you should talk to 
your customers and see if they needed it. 

> criticism on how to improve the draft. Which is not true.

You did give some useful information on how to implement it but it is 
useless information if it how to implement something that we don't need.

> >> is on user applications and not Linux internal use or Linux utilities 

> >> will not be tagged and Linux Server implementation will probably just 

> >> customer and  they described their use of xattr as a way to tag files 

xattr 
We 
implement 
> > the 
> >> 
> >> right support for xattr.
> >> 
> >> Again, at some point we need to reach consensus and move on, I though

---

From: Manoj Naik <[email-masked]>
Date: Mon, 6 Oct 2014 15:20:49 -0700

Trond Myklebust <[email-masked]> wrote on 10/06/2014 03:00:42 PM:

> Cc: "[email-masked]" <[email-masked]>, Tom Haynes
> <[email-masked]>
> Date: 10/06/2014 03:00 PM
> >> > protocol. That discussion will continue and will result in updates
to
> They should because that's how feature requests work.

No they shouldn't. Extended attributes is a file system feature that is,
for all practical purposes, well defined for the platform/file system - if
not interoperable. (Which is why I agree we need to state _requirements_ in
the protocol). Customers are free to use them as they wish, they have no
obligation to tell you what they put in there and why.

Today, I can cp (or rsync) a file from one filesystem (say XFS) to another
(say ext4) across two different Linux systems and it'll preserve xattrs.
But I can't do the same over NFS. This should be sufficient to establish a
"need" IMO. Obviously, you disagree.

Manoj.

> >> >> One thing I got from the recent set of discussions on the WG alias
is
> >> >> > At some point in any proposal someone asks for it to be included
as
> >> >> > an
> >> >> > official working item.
> >> >> >
> >> >> > From this message (and others) it appears that your opinion is
that
> >> >> > this
> >> >> > To make it formal, there seems to enough interest in discussion
that
> >> >> > we
utilities
just
> >> >> > use
> >> >> > the user attributes type. So to prove that it is needed we need
to
> >> >> > name
> >> >> > some customer applications and describe how they use xattr. I
probed
> >> >> > one
files
> >> >> > for
> >> >> > migration policy to decide where and when to move files. Another
big
> >> >> > one
> >> >> > is  swift (the object protocol) that uses xattr and when
implemented
> >> >> > over
> >> >> > out there and used and for NFS to stay relevant we need to make
it
> >> >> > support
> >> >> > should focuses and use the limited time that the WG member can
spend
> >> >> > here
though
> >> >> > that
> >> >> >
> >> >> > we passed that point but if some one object to the need of xattr
to
> >> >> > be
> >> >> > part on the NFS protocol please speak up and if not lets just
talk
> >> >> > about
> >> >> >>>> Is there some big set of open-source code that you could
search
> >> >> >> Most of those are for the Linux POSIX ACL and SeLinux support,
so

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 6 Oct 2014 22:22:53 +0000

Manoj,

It looks like you have a set of feedback that can be incorporated into the I-D (you acknowledge on another thread I believe).

Would be helpful to get that into an update.  In the interim, if anyone has text to contribute for use case, that would be great.

Use case information would be helpful as we move the content through the rest of the IETF process and for the eventual reader of the work that has not been involved with this particular discussion.

Spencer

From: nfsv4 [mailto:[email-masked]] On Behalf Of Manoj Naik
Sent: Monday, October 6, 2014 2:42 PM
To: Trond Myklebust
Cc: [email-masked]
Subject: Re: [nfsv4] request for xattr as a WG item

Trond Myklebust <[email-masked]<mailto:[email-masked]>> wrote on 10/06/2014 02:25:23 PM:

> From: Trond Myklebust <[email-masked]<mailto:[email-masked]>>
> Cc: Tom Haynes <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>"
> <[email-masked]<mailto:[email-masked]>>
> On Mon, Oct 6, 2014 at 4:36 PM, Manoj Naik <[email-masked]<mailto:[email-masked]>> wrote:
> > Tom,
> >

See Section 2 in the current draft. Perhaps this can be expanded (volunteers?), but most users (our customers too) don't describe _how_ they use extended attributes. For good reason, why should they? I can give you a list of customers that are asking for it, but I think that isn't the point. We've talked in the mailing list in the past about giving them alternatives to xattrs, but isn't this something that is best left to the application to decide? Every file system and platform supports xattrs - I would be hard pressed to find another universal fs feature that the protocol does not support ;)

> api in Linux filesystems, so why must we consider it now?

And I can probably go back 10 years since I've heard requests for xattrs over NFS ;) I am starting to understand why - it's a long arduous process.

Manoj.

>
> Cheers
>   Trond
>
>
> > Manoj.
> >
> > "nfsv4" <[email-masked]<mailto:[email-masked]>> wrote on 10/06/2014 12:48:27 PM:
> >
> >> From: Tom Haynes <[email-masked]<mailto:[email-masked]>>
> >> Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
> >> Sent by: "nfsv4" <[email-masked]<mailto:[email-masked]>>
> >
> >
> >>
> >> On Oct 6, 2014, at 3:34 PM, Marc Eshel <[email-masked]<mailto:[email-masked]>> wrote:
> >>
> >> > Hi Spencer
> >> >
> >> > From:   Spencer Shepler <[email-masked]<mailto:[email-masked]>>
> >> > <[email-masked]<mailto:[email-masked]>>,
> >> > Cc:     "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
> >> > From:   "J. Bruce Fields" <[email-masked]<mailto:[email-masked]>>
> >> > To:     Trond Myklebust <[email-masked]<mailto:[email-masked]>>,
> >> > Cc:     Marc Eshel/Almaden/IBM@IBMUS, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
> >> >> On Fri, Oct 3, 2014 at 2:00 PM, J. Bruce Fields <[email-masked]<mailto:[email-masked]>>
> >> > wrote:
> >> > [email-masked]<mailto:[email-masked]>
> >> > [email-masked]<mailto:[email-masked]>
> >> [email-masked]<mailto:[email-masked]>
> > [email-masked]<mailto:[email-masked]>

---

From: "Erasani, Pranoop" <[email-masked]>
Date: Tue, 7 Oct 2014 03:23:46 +0000

I'm yet to fully appreciate the latest changes in the draft, but based on the last WG discussion, I'm in favor of this proposal and support moving forward with xattr as a WG item.

I see a number of use cases, where customers ask for "tagging" NFS files with user defined metadata. There is a whole new class of storage appliances (object stores) being built with object protocols whole value is in enhancing file with user defined metadata. This allows user-defined metadata optimized placement (e.g. path independence), metadata driven search and retrieval etc. and allows deeper application integration with the NFSv4.x protocol. And as you probably know, that is a good thing.

Talking with customers, such object-store protocols became popular because, NFS is too much oriented towards POSIX compliance. While I agree that majority of applications run over POSIX, we have seen marked amount of adoption on a number of clients that don't follow POSIX standard. We need to be accommodative of non-POSIX use cases and innovate to cater NFS towards such use cases. As you may also know, a number of requirements that eventually ended up in NFSv4.2 were from non-POSIX use cases.

That said, I'm happy to contribute a section on use cases for this document.

On the logistical front, I hope we are not making it a norm to have a separate "use cases document" for each proposal. That should be in the personal draft itself, especially as we are looking at more and more optional features in the spirit of "DOT" minor versions.

- Pranoop

-----Original Message-----
From: nfsv4 [mailto:[email-masked]] On Behalf Of Matt W. Benjamin
Sent: Monday, October 06, 2014 2:00 PM
To: Nico Williams
Cc: NFSv4
Subject: Re: [nfsv4] request for xattr as a WG item

Hi,

Matt

http://linuxbox.com

tel.  734-761-4689
fax.  734-769-8938
cel.  734-216-5309

---

From: Benjamin Kaduk <[email-masked]>
Date: Mon, 6 Oct 2014 23:49:32 -0400 (EDT)

> obligation to tell you what they put in there and why.

Well, interoperable is kind of the core of what we do at the IETF.  We
make standards that can have independent implementations which are
interoperable.

> "need" IMO. Obviously, you disagree.

That is hardly compelling to me at all.  Our goal is to permit
interoperable implementations, and if that means that we have to impose
limits on the functionality between two peers of the same implementation
(compared to local filesystems on that implementation), then that is the
price we must pay.  There should not inherently be an expectation that all
features available on all local filesystems should be also available on
network filesystems; this may not be technically feasible.

We will need to expend technical effort to find a way to make Linux xattrs
interoperable with Solaris ones interoperable with everything else, over
the NFS protocol.  My impression is that what is being asked for is a
justification for why we collectively should expend that effort.  An
answer that boils down to "because it's there" or "because we can" does
not seem sufficient, to me.  Many of the examples of xattr usage given in
the -01 are things that may not necessarily make sense when applied to
files on a network filesystem -- a file from an NFS share that is
controlled by someone else may inherently need to be less trusted in the
same way that a file downloaded from the internet is; using Dropbox to
synchronize files to a NFS share doesn't seem to make much sense.

I believe that the question you should answer is: what is/are the
NFS-specific use cases that justify spending the WG's time.  If a large
portion of the WG are willing to contribute their time without further
justification, then the answer need not be as strong as it would be
otherwise, but there should still be an answer.

-Ben

---

From: Tom Haynes <[email-masked]>
Date: Tue, 7 Oct 2014 00:05:14 -0400

> I see a number of use cases, where customers ask for "tagging" NFS files with user defined metadata. There is a whole new class of storage appliances (object stores) being built with object protoco…

Why don’t named attributes meet these needs?

---

From: "Erasani, Pranoop" <[email-masked]>
Date: Tue, 7 Oct 2014 06:08:57 +0000

The reasons are mentioned here (http://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01#page-5).

Apart from that, consider a simple case of setting multiple user-defined attributes at once. It requires multiple NFS operations in named attributes to accomplish  what a single setxattr is expected to do. Why would I want to use a sub-standard mechanism to accomplish user defined metadata.

Moreover, as aptly described by Manoj/Marc, subjecting user-defined attributes to close-to-open semantics is an overhead that could clearly be avoided. For the use cases that I'm tracking scale is important part of the consideration and clearly named attributes for these reasons won't be my choice.

HTH.

- Pranoop

-----Original Message-----
From: Tom Haynes [mailto:[email-masked]] 
Sent: Monday, October 06, 2014 9:05 PM
To: Erasani, Pranoop
Cc: Matt W. Benjamin; Nico Williams; NFSv4
Subject: Re: [nfsv4] request for xattr as a WG item

> I see a number of use cases, where customers ask for "tagging" NFS files with user defined metadata. There is a whole new class of storage appliances (object stores) being built with object protoco…

Why don't named attributes meet these needs?

---

From: Christoph Hellwig <[email-masked]>
Date: Tue, 7 Oct 2014 11:41:21 +0200

> api in Linux filesystems, so why must we consider it now?

FYI I'm in full agreement with Tom and Trond that you need to specify
the actual use cases and thus prove the need.  I suspect it won't be
too hard, but it needs to be done.

---

From: Christoph Hellwig <[email-masked]>
Date: Tue, 7 Oct 2014 11:44:09 +0200

> "need" IMO. Obviously, you disagree.

Two Linux filesystems aren't too interesting.  You need to demonstrate
interoperability between say XFS on Linux and FFS on FreeBSD exported over
NFS.

---

From: Christoph Hellwig <[email-masked]>
Date: Tue, 7 Oct 2014 12:00:12 +0200

> Why don’t named attributes meet these needs?

They are much higher overhead, both on disk and on the wire.  For example
an xattr on XFS consumes just the size of the key + attr in the best
case, and some additional btree linkage in the worse case.  A named attribute
(which is just a horrible name for a file that is hidden from the normal
namespaces) needs a full inode for the stat data, a directory entry, and
often a block/extent to store the actual data.

Similarly on the wire you need CTO semantics, explicit OPEN / CLOSE calls,
GETATTRS and so on.

---

From: Manoj Naik <[email-masked]>
Date: Tue, 7 Oct 2014 12:00:46 -0700

Christoph Hellwig <[email-masked]> wrote on 10/07/2014 02:44:09 AM:

> Cc: Trond Myklebust <[email-masked]>, "[email-masked]"
<[email-masked]>
> Date: 10/07/2014 02:44 AM

I don't have access to FreeBSD, but I can preserve xattrs with rsync (-aX)
between Linux/ext4 and OSX/hfs. rsync will correctly strip off the Linux
"user." prefix and copy all user xattrs.

Manoj.

---

From: Saul Tamari <[email-masked]>
Date: Tue, 7 Oct 2014 18:13:13 -0400

Hi,

Maybe it is possible to add a new and simplified named-attribute model
for NFSv4.0 while keeping the NFSv4.0 XDR?
This simplified model will not support GETATTR/SETATTR for the
attributes and remove the filesystem inode requirement.

With regard to the OPEN/CLOSE over the wire overhead, these are only
required when creating/modifying an attribute, while READ (which I
assume is the most common call) can take an all 1's stateid without
OPENing the file.

Saul

---

From: Dave Quigley <[email-masked]>
Date: Tue, 07 Oct 2014 20:46:07 -0400

> on servers that can not enforce anything like that.

I agree with this and I would pretty vocal in my stance against 
exporting security.* The security.* and trusted.* namespaces shouldn't 
be seen outside of the local box.

---

From: Dave Quigley <[email-masked]>
Date: Tue, 07 Oct 2014 21:08:15 -0400

> your customers and see if they needed it.

The thing is that just having a section justifying the feature is not 
enough. I learned that when I started Labeled NFS. I started with a 
requirements document. I then was asked to a use case document, an 
impact study, a requirements document, and finally the actual 
specification which got merged into 4.2. Compared to the varied 
semantics of xattr implementations on different systems security labels 
were easy. We punted on syntax and semantics and abstracted that out 
into a separate document. We punted on establishing a security model and 
said it was out of scope since its really a platform specific 
implementation detail.

If you want a way of laying out your case for xattrs you can try 
following what I started for Labeled NFS and what Tom continued and 
finally shepherded through the 4.2 adoption process. We have specific 
use cases in there for security labels. I'm sure you can find real 
concrete examples for the equivalent of the user namespace style xattrs 
from Linux.

Dave

---

From: Marc Eshel <[email-masked]>
Date: Tue, 7 Oct 2014 18:33:17 -0700

"nfsv4" <[email-masked]> wrote on 10/07/2014 06:08:15 PM:

> From: Dave Quigley <[email-masked]>
> To: [email-masked], 
> Date: 10/07/2014 06:08 PM

> concrete examples for the equivalent of the user namespace style xattrs 

I agree that we have to add more to the section on justifying the need for 
xattr, and we will, and invite again members of the WG to contribute to 
this section, like Pranoop and others did. I don't think that the fact 
that you and Tom had a hard time with Labeled NFS should imply that every 
new feature now must have a separate document to justify it. Where is this 
requirement coming from? I believe that we need to reach consensus in the 
WG and that should be enough to add a new feature. Am I wrong with this 
assumption?

---

From: Dave Quigley <[email-masked]>
Date: Wed, 08 Oct 2014 01:32:00 -0400

> WG and that should be enough to add a new feature. Am I wrong with this

I'm not saying all of the documents are necessary but if you look at the 
nfsv4 working group documents you see that many of the features have 
requirements documents. We settled finally on two documents for Labeled 
NFS: the actual spec in the v4.2 document and this requirements document 
(https://datatracker.ietf.org/doc/rfc7204/). You have people who are 
pushing back on having specific use cases and a document like the second 
one which spells out your requirements is a good place to put that 
information. To be honest I can't tell if your specification actually 
meets whatever requirements you're hoping to meet because they aren't 
spelled out anywhere. I can't tell if the requirements you're trying to 
meet actually address the uses of xatters appropriately across all the 
existing implementations that exist out there. I'm just sharing a bit of 
helpful advice since I'm seeing you get similar push back to what we 
did. It doesn't take much work to have a requirements document with what 
you hope to accomplish and what usecases (with some concrete ones) you 
hope to support. Based on what I've read in this thread it might address 
people's concerns.

Dave

---

From: Benny Halevy <[email-masked]>
Date: Wed, 8 Oct 2014 17:19:16 -0400

A requirement for a separate requirements doc justifying a new feature
really depends on its complexity and overall intrusiveness to the protocol.

I don't see a compelling reason for a full blown document for small and
independent features. In this case, for the sake of brevity a requirements
and use cases section should do.

Benny
On Oct 7, 2014 10:32 PM, "Dave Quigley" <[email-masked]> wrote:

>>>> We added a section to the draft based on your request to explain why
>>>>
>>> it is
>>
>>> needed, but you are still not convinced. I think that you are looking
>>>>
>>> for
>>
>>> a killer application that will prove the need for xattr and I don't
>>>>
>>> know
>>
>>> of one. all I know is what we added to the draft and what customers
>>>>
>>> tell
>>
>>>> others on the WG seem to agree that it is needed, I assume based on
>>>>
>>> their
>>
>>> knowledge and experience. I can not give you a name of an application
>>>>
>>> that
>>
>>> a customer is running because I don't know the name and even if I did
>>>>
>>> it
>>
>>> would mean nothing, all I can say that for example one application is
>>>>
>>> used
>>
>>>> file movement among the different storage levels. Maybe you should
>>>>
>>> talk to
>>
> NFS: the actual spec in the v4.2 document and this requirements document (
> https://datatracker.ietf.org/doc/rfc7204/). You have people who are
> information. To be honest I can't tell if your specification actually meets
> whatever requirements you're hoping to meet because they aren't spelled out
> anywhere. I can't tell if the requirements you're trying to meet actually
> address the uses of xatters appropriately across all the existing
> implementations that exist out there. I'm just sharing a bit of helpful
> advice since I'm seeing you get similar push back to what we did. It
> doesn't take much work to have a requirements document with what you hope
> to accomplish and what usecases (with some concrete ones) you hope to
> support. Based on what I've read in this thread it might address people's
> concerns.
>
> Dave
>

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 8 Oct 2014 21:23:51 +0000

No, it is definitely not a requirement to have a separate requirements document.

Many features have started out as a separate document that have included the “why” and then once the WG took on the item officially those sections or items have been subsumed by other work.

Where separate documents have been required are instances where the WG charter has expanded or where the work will garner broader attention within either the IESG or the IETF.

In the case of xattr, I have not seen anything yet that calls for a standalone document.

Spencer

From: nfsv4 [mailto:[email-masked]] On Behalf Of Benny Halevy
Sent: Wednesday, October 8, 2014 2:19 PM
To: Dave Quigley
Cc: nfsv4; [email-masked]
Subject: Re: [nfsv4] request for xattr as a WG item

A requirement for a separate requirements doc justifying a new feature really depends on its complexity and overall intrusiveness to the protocol.

I don't see a compelling reason for a full blown document for small and independent features. In this case, for the sake of brevity a requirements and use cases section should do.

Benny
On Oct 7, 2014 10:32 PM, "Dave Quigley" <[email-masked]<mailto:[email-masked]>> wrote:
On 10/7/2014 9:33 PM, Marc Eshel wrote:
"nfsv4" <[email-masked]<mailto:[email-masked]>> wrote on 10/07/2014 06:08:15 PM:
From: Dave Quigley <[email-masked]<mailto:[email-masked]>>
To: [email-masked]<mailto:[email-masked]>,
Date: 10/07/2014 06:08 PM
Subject: Re: [nfsv4] request for xattr as a WG item
Sent by: "nfsv4" <[email-masked]<mailto:[email-masked]>>

Dave

I'm not saying all of the documents are necessary but if you look at the nfsv4 working group documents you see that many of the features have requirements documents. We settled finally on two documents for Labeled NFS: the actual spec in the v4.2 document and this requirements document (https://datatracker.ietf.org/doc/rfc7204/). You have people who are pushing back on having specific use cases and a document like the second one which spells out your requirements is a good place to put that information. To be honest I can't tell if your specification actually meets whatever requirements you're hoping to meet because they aren't spelled out anywhere. I can't tell if the requirements you're trying to meet actually address the uses of xatters appropriately across all the existing implementations that exist out there. I'm just sharing a bit of helpful advice since I'm seeing you get similar push back to what we did. It doesn't take much work to have a requirements document with what you hope to accomplish and what usecases (with some concrete ones) you hope to support. Based on what I've read in this thread it might address people's concerns.

Dave

---

From: Spencer Shepler <[email-masked]>
Date: Wed, 8 Oct 2014 21:25:00 +0000

Marc,

Do you have an idea of when an update to the I-D may be available?

Spencer

> From: nfsv4 [mailto:[email-masked]] On Behalf Of Marc Eshel
> Sent: Tuesday, October 7, 2014 6:33 PM
> To: Dave Quigley
> Cc: [email-masked]; nfsv4
> > Subject: Re: [nfsv4] request for xattr as a WG item Sent by: "nfsv4"
> > <[email-masked]>
> >
> > > us, but it is not enough for you. So I don't know how to convince
> > > you, others on the WG seem to agree that it is needed, I assume
> > > based on
> their
> > > to tag files using xattr for future migration, it is like a policy
> > > for file movement among the different storage levels. Maybe you
> > > should
> talk to
> > semantics of xattr implementations on different systems security
> > labels were easy. We punted on syntax and semantics and abstracted
> > that out into a separate document. We punted on establishing a
> > security model and
> 
> > concrete examples for the equivalent of the user namespace style
> > xattrs from Linux.
> 
> I agree that we have to add more to the section on justifying the need for xattr,
> and we will, and invite again members of the WG to contribute to this section,
> like Pranoop and others did. I don't think that the fact that you and Tom had a
> hard time with Labeled NFS should imply that every new feature now must
> have a separate document to justify it. Where is this requirement coming
> from? I believe that we need to reach consensus in the WG and that should be
> enough to add a new feature. Am I wrong with this assumption?
> 
> >
> > Dave
> >

---

From: Marc Eshel <[email-masked]>
Date: Wed, 8 Oct 2014 15:28:44 -0700

Hi Spencer,

We have enough proposed changes from the feedback in the last few days 
that a new update to the draft is needed, but after working on it for more 
than a year and watching several rounds of similar discussions at 
different times on the list and at IETF, I would like to hear some 
assurance from the chairs that the WG will take it up as an item. We take 
it as an action item to update the draft, but this is not a personal 
mission, and needs support from the community. I just want confirmation 
that this is not a waste of our time.

Thanks, Marc.

From:   Spencer Shepler <[email-masked]>
To:     Marc Eshel/Almaden/IBM@IBMUS, Dave Quigley 
<[email-masked]>, 
Cc:     "[email-masked]" <[email-masked]>, nfsv4 <[email-masked]>
Date:   10/08/2014 02:25 PM
Subject:        RE: [nfsv4] request for xattr as a WG item

Marc,

Spencer

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 9 Oct 2014 00:54:19 +0000

Marc,

I have not observed consensus as of yet -- I see positive comments and engagement and with use-case clarity we should be able to move forward.

The use-case item does not need to land in an I-D if you are looking for minimal overhead.  A email to the WG with a draft of what would land in the I-D would be sufficient to get feedback and to demonstrate consensus.

BTW: just to state the obvious, when an item becomes a WG I-D does not guarantee forward movement...

Spencer

> From: Marc Eshel [mailto:[email-masked]]
> Sent: Wednesday, October 8, 2014 3:29 PM
> To: Spencer Shepler
> Cc: Dave Quigley; Marc Eshel; [email-masked]; nfsv4
> We have enough proposed changes from the feedback in the last few days that
> a new update to the draft is needed, but after working on it for more than a
> year and watching several rounds of similar discussions at different times on
> the list and at IETF, I would like to hear some assurance from the chairs that
> the WG will take it up as an item. We take it as an action item to update the
> draft, but this is not a personal mission, and needs support from the
> community. I just want confirmation that this is not a waste of our time.
> 
> Thanks, Marc.
> 
> 
> 
> > > The thing is that just having a section justifying the feature is
> > > not enough. I learned that when I started Labeled NFS. I started
> > > with a requirements document. I then was asked to a use case
> > > document, an impact study, a requirements document, and finally the
> > > actual specification which got merged into 4.2. Compared to the
> > > varied semantics of xattr implementations on different systems
> > > security labels were easy. We punted on syntax and semantics and
> > > abstracted that out into a separate document. We punted on
> > > establishing a security model and
> >
> > > finally shepherded through the 4.2 adoption process. We have
> > > specific use cases in there for security labels. I'm sure you can
> > > find real concrete examples for the equivalent of the user namespace
> > > style xattrs from Linux.
> >
> > hard time with Labeled NFS should imply that every new feature now
> > must have a separate document to justify it. Where is this requirement
> > coming from? I believe that we need to reach consensus in the WG and
> > that
> should be

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 9 Oct 2014 13:55:59 -0400

> assume is the most common call) can take an all 1's stateid without

The servers also need to fake up filehandles and readdir cookies and
turn truncates and writes into get-modify-sets.  It can probably all be
made to work with sufficient cleverness but it would be much simpler
with the appropriate protocol.

--b.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 9 Oct 2014 14:54:01 -0400

> The argument being presented is that we should support xattrs

That's overstating it; section 2

	http://tools.ietf.org/html/draft-naik-nfsv4-xattrs-01#section-2

*does* have examples, though though not with a lot of detail about any
particular example.

Personally my main worry was the confusion between xattr support and acl
and selinux support.  Because that's almost universally what people have
given me as use cases in casual interactions.

I'm mostly reassured that there are in fact people using them for other
things, though absent more evidence I'm still sort of neutral on whether
it's worth doing.

I guess what would really make it easy for me would be *one* example
with evidence and details showing how somebody's life would be improved
with xattr support.  Ideally a real "somebody" who would be sufficiently
motivated to actually test our implementation, so we'd have a pretty
good idea whether we'd met requirements or not.

So the reason I was suggesting e.g. scanning Linux filesystems or
looking at data dependencies would be that we might be able to scare up
some developer with a good use case.  And, I don't know, maybe that
could be easier than getting some customer to describe their internal
use.

--b.