Subject: [nfsv4] State of NFSv4 implementations ( who is active )

From: spencer shepler <[email-masked]>
Date: Sun, 27 Oct 2019 23:01:55 -0700

As a working group and care taker of the proposed standards within our
purview, I think it would be fruitful to have a collectively working group
view of what implementations are in-market and which are under support only
and which are under active development.

As we work through various bis work, we need to understand whom to have
contact with within the broader community, who is represented directly
within the working group and which implementations are not.

The other use of this knowledge is if, big if, our documents move from
proposed standard, they will need implementation reports and this type of
information will be helpful.

If individuals can self report here for those organizations (or individuals
- Hi, Rick!) that are active, etc., that would be great.  If there are
implementations not represented here based on alias membership, please ask
for permission and then post those activities as well.

Thanks,
Spencer

---

From: Daniel Gryniewicz <[email-masked]>
Date: Mon, 28 Oct 2019 08:49:51 -0400

On 10/28/19 2:01 AM, spencer shepler wrote:
> purview, I think it would be fruitful to have a collectively working 
> group view of what implementations are in-market and which are under 
> support only and which are under active development.
> 
> proposed standard, they will need implementation reports and this type 
> of information will be helpful.
> 
> If individuals can self report here for those organizations (or 
> individuals - Hi, Rick!) that are active, etc., that would be great.  If 
> there are implementations not represented here based on alias 
> membership, please ask for permission and then post those activities as 
> well.
> 
> Thanks,
> Spencer
> 

NFS-Ganesha (https://github.com/nfs-ganesha/nfs-ganesha) is a user-space 
NFS server under active development, and sold in currently supported 
products by at least IBM and Red Hat.

Daniel

---

From: Chuck Lever <[email-masked]>
Date: Mon, 28 Oct 2019 11:09:37 -0400

Hi Spencer-

> On Oct 28, 2019, at 2:01 AM, spencer shepler <[email-masked]> wrote:
> 
> 
> As a working group and care taker of the proposed standards within our purview, I think it would be fruitful to have a collectively working group view of what implementations are in-market and which are under support only and which are under active development.
> 
> As we work through various bis work, we need to understand whom to have contact with within the broader community, who is represented directly within the working group and which implementations are not.
> 
> The other use of this knowledge is if, big if, our documents move from proposed standard, they will need implementation reports and this type of information will be helpful.
> 
> If individuals can self report here for those organizations (or individuals - Hi, Rick!) that are active, etc., that would be great.  If there are implementations not represented here based on alias membership, please ask for permission and then post those activities as well.

Oracle Solaris has an NFS client and server that is still active.
Oracle also sells the ZFS Storage Appliance. These implementations
support NFSv2 through NFSv4.1 and NFS/RDMA. Does not support pNFS.

Oracle has an elastic cloud data center NFS server implementation.
It currently supports only NFSv3 and is available to Oracle Cloud
customers.

Not sure if Amazon is on this list, so I will mention that they
also have an elastic cloud data center NFS server implementation
known as EFS. It supports NFSv4.x, but I believe that pNFS is not
supported.

--
Chuck Lever

---

From: David Noveck <[email-masked]>
Date: Mon, 28 Oct 2019 11:40:37 -0400

i suppose I should add that Netapp has an NFSv4 server implementation
supporting v3, v4.0, and v4.1 that is shipped and in use for in on-premises
environments and also available on Azure, AWS, Google Cloud, and perhaps
some others.   We are working on v4..2 but have not shipped any support.
 We are also working on RDMA suport for NFS. We do support pNFS file.

On Mon, Oct 28, 2019 at 2:02 AM spencer shepler <[email-masked]>
wrote:

> there are implementations not represented here based on alias membership,
> please ask for permission and then post those activities as well.
>
> Thanks,
> Spencer
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Rick Macklem <[email-masked]>
Date: Mon, 28 Oct 2019 16:59:02 +0000

Chuck Lever wrote:
[good stuff snipped]
>known as EFS. It supports NFSv4.x, but I believe that pNFS is not
Yes, it would be nice if someone could make contact with them. I posted
here a year or two ago when I had an issue and never got any response.

It works fine with the FreeBSD client, except it has one fairly significant
flaw (that other client implementors might want to know about).
It stores session/open/lock state per cluster (I have no idea what constitutes
a cluster for EFS) and not globally.
As such, when the client establishes a new TCP connection, it sometimes
connects to a different cluster and then the client ends up doing a full
session/open/lock state recovery like it would for a traditional server reboot.
(It exposed a bunch of problems with the FreeBSD client's recovery code,
 so it served as a good debugging tool for the FreeBSD client.)

The other group that would be nice to get more involved (I haven't seen
posts and don't know if they attend Bakeathons?) is the VMware NFSv4.1
(don't know if they are working on NFSV4.2) client.
The first release (ESXi 6.5) had serious issues, but most of these were fixed
for ESXi 6.7, from what I saw. I never made contact with the group, but
someone who worked elsewhere within VMware posted my list of
problems on their internal bug tracker. That seemed to work?

As for FreeBSD:
- NFSv2->4.1 client and server released. Support for File and Flexible File
   layouts for pNFS for client and server. (No striping support in the server
   or Flexible File client code.)
   No support for RDMA or migration.
- NFSv4.2 client/server code nearing completion. Should be released in
   FreeBSD13 (and maybe FreeBSD12.2). Figure 6months->1year from now.
   - Available for testing in the meantime.
  Includes: intra-server copy, allocate, seek, advise (WILLNEED/DONTNEED),
     layouterror. It also has RFC-8276 xattr in it.
   Might have some MAC label support, but I haven't coded that yet.
Only RPCSEC_GSS_V1 at this time (part of the reason I am hesitant w.r.t. MAC
labelling).

rick (Hi Spencer:-)

---

From: "van der Linden, Frank" <[email-masked]>
Date: Mon, 28 Oct 2019 17:06:50 +0000

I know that team - let me forward your mail to them. I don't believe they are on this list.

Frank

On 10/28/19, 10:00 AM, "nfsv4 on behalf of Rick Macklem" <[email-masked] on behalf of [email-masked]> wrote:

Chuck Lever wrote:
[good stuff snipped]

As for FreeBSD:

rick (Hi Spencer:-)

---

From: Rick Macklem <[email-masked]>
Date: Mon, 28 Oct 2019 17:16:15 +0000

In fairness, I called the issue a "flaw" when it might more correctly be called
a quirk because, so long as your client recovers from a NFS4ERR_BAD_SESSION
correctly, it all works. (At the time, there was a FreeBSD bug that resulted in
the kernel RPC creating a new TCP connection when it didn't really need to.
This bug then exposed the "session/open/lock" recovery issue. Once I fixed
the recovery issues and the unnecessary TCP connection bug in FreeBSD, it
seems to work fine.)

rick

________________________________________
From: van der Linden, Frank <[email-masked]>
Sent: Monday, October 28, 2019 1:06 PM
To: Rick Macklem; Chuck Lever; spencer shepler
Cc: NFSv4
Subject: Re: [nfsv4] State of NFSv4 implementations ( who is active )

Frank

Chuck Lever wrote:
[good stuff snipped]

As for FreeBSD:

rick (Hi Spencer:-)

---

From: "Mkrtchyan, Tigran" <[email-masked]>
Date: Mon, 28 Oct 2019 18:19:38 +0100 (CET)

Hi Spencer,

dCache.org (DESY) has NFSv4.0 and 4.1 with pNFS that heavily used by many
research labs around the world. Our server support nfsv41_file and flexfile layouts.

We have prototypes for:
  - xattr
  - rpc-over-tls

Future plans:
  - v2 server-to-server copy

Best regards,
   Tigran.

----- Original Message -----
> From: "spencer shepler" <[email-masked]>
> To: "NFSv4" <[email-masked]>
> Sent: Monday, October 28, 2019 7:01:55 AM
> Subject: [nfsv4] State of NFSv4 implementations ( who is active )

---

From: David Noveck <[email-masked]>
Date: Mon, 28 Oct 2019 13:26:32 -0400

>The other use of this knowledge is if, big if, our documents move from
proposed standard, they will need
> implementation reports and this type of information will be helpful.

Given that Draft Standard has gone away, it is an enormous if.   Still,
that information has many interesting uses and it is good that we are
getting it.

---

From: Ulka Vaze <[email-masked]>
Date: Wed, 30 Oct 2019 10:36:51 +0000

Hi,
Vmware is active on NFS4.1 client side implementation. My name is Ulka Vaze ([email-masked]<mailto:[email-masked]>)

Thanks,
-Ulka Vaze

From: David Noveck <[email-masked]>
Date: Monday, October 28, 2019 at 10:51 PM
To: Ulka Vaze <[email-masked]>
Subject: Fwd: [nfsv4] State of NFSv4 implementations ( who is active )

---------- Forwarded message ---------
From: spencer shepler <[email-masked]<mailto:[email-masked]>>
Date: Mon, Oct 28, 2019 at 2:02 AM
Subject: [nfsv4] State of NFSv4 implementations ( who is active )
To: NFSv4 <[email-masked]<mailto:[email-masked]>>

Thanks,
Spencer

_______________________________________________
nfsv4 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nfsv4<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fnfsv4&data=02%7C01%7Cuvaze%40vmware.com%7C42ee5ceeaf4945cf457f08d75bcb4193%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637078800850873294&sdata=qMDkalnyRWI8iEXMBXgTAszhDsRBH9M9wY6Itke2L50%3D&reserved=0>

---

From: Chuck Lever <[email-masked]>
Date: Wed, 30 Oct 2019 11:29:35 -0400

> On Oct 28, 2019, at 11:09 AM, Chuck Lever <[email-masked]> wrote:
> 
> Hi Spencer-
> 

Oracle also has a user space NFS client implementation that is built
into the Oracle database product. The library is known as dnfs. It is
known to support NFSv3 on TCP and RDMA. There are rumors that it also
supports NFSv4.1, but I haven't confirmed that.

Note that this is not a POSIX-based client. It is focused on making
READ and WRITE operations perform well and behave consistently without
regard to the OS platform where the database is running.

--
Chuck Lever

---

From: Chuck Lever <[email-masked]>
Date: Wed, 30 Oct 2019 11:33:33 -0400

> If individuals can self report here for those organizations (or individuals - Hi, Rick!) that are active, etc., that would be great.  If there are implementations not represented here based on alia…

Bruce and Trond can correct me if I'm wrong:

Linux in-kernel NFS client supports:

- NFSv2 and NFSv3 on UDP, TCP, and RDMA
- NLM/NSM/MNT on UDP and TCP
- NFSv4.x family on TCP and RDMA

pNFS layout types:
File
Block / SCSI
Flex file

Linux in-kernel NFS server supports:

- NFSv2 and NFSv3 on UDP, TCP, and RDMA
- NLM/NSM/MNT on UDP and TCP
- NFSv4.x family on TCP and RDMA

pNFS layout types:
Block / SCSI

--
Chuck Lever