Subject: [nfsv4] Slides from the pNFS BoF we delivered at LISA 11

From: "Sorin Faibish" <[email-masked]>
Date: Thu, 08 Dec 2011 19:40:19 -0500

Attached please find the slides that we presented yesterday at USENIX LISA  
11
conference in Boston. We would like to get more detailed input on other OS
that will support pNFS server/client (this is for Rick as well) that were
not mentioned in this slide set.

/Sorin

-- 
Best Regards
Sorin Faibish
Corporate Distinguished Engineer
Unified Storage Division

        EMC²
where information lives

Phone: 508-435-1000 x 48545
Cellphone: 617-510-0422
Email : [email-masked]

---

From: Rick Macklem <[email-masked]>
Date: Thu, 8 Dec 2011 21:53:20 -0500 (EST)

Thanks for the slides, Sorin.

Just fyi, it was correct to not mention a FreeBSD client, since the
4.1 client code hasn't made it into any release yet. It is available
as a patch against the FreeBSD9.0 kernel, in case anyone is interested:
   http://people.freebsd.org/~rmacklem/nfsv4.1-client/nfsv41.patch

I am just starting to work on the logistics of getting this into the
FreeBSD source tree, so it may be in a FreeBSD9.n release, where n > 0.

Also, the above does not include any pNFS support. I am about ready to
start coding pNFS client stuff, which brings me to...
- It will be difficult for me to make much progress without any
  pNFS servers to test against. (The only NFSv4.1 server I have to
  test against is Fedora15 and I don't think it knows how to do pNFS?
  If there is a way to make Fedora15/16 (or any other freebie Linux
  distro) do pNFS, please let me know.

If others have suggestions on how I might gain access to pNFS servers
for testing, I would appreciate hearing. (However, I am not funded to
do any of this, so the suggestions can't include my spending $$$. I can
make it to a Bakeathon when it happens to be close to where I am, so I
can show up "on the cheap", like I did last June in Ann Arbor.)

rick

----- Original Message -----
> On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <[email-masked]>
> wrote:
> 
> > Attached please find the slides that we presented yesterday at
> > USENIX
> > LISA 11
> > conference in Boston. We would like to get more detailed input on
> > other
> > OS

---

From: "Sorin Faibish" <[email-masked]>
Date: Thu, 08 Dec 2011 22:11:42 -0500

>   distro) do pNFS, please let me know.
One suggestion is to try to use the pNFS for Lustre. I can send you the
patches or you can get them from Lustre tree. Let me know if this will be
good for you. We are already using it to test Linux file client as well as
Windows. Whamcloud demo it during SC11 and demo wire speed.

What I would also suggest is for you maybe to come to the Connectathon in
February and test the client against the other servers that you cannot have
as they are not products but prototypes.

> can show up "on the cheap", like I did last June in Ann Arbor.)
Sure. The next BAT is at CITI so maybe you bring a client prototype
there.

/Sorin

        EMC²
where information lives

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Fri, 9 Dec 2011 07:55:51 +0100

Hi Sorin,

slides are missing.

Tigran.
On Fri, Dec 9, 2011 at 1:40 AM, Sorin Faibish <[email-masked]> wrote:
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email-masked]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Fri, 9 Dec 2011 08:48:51 +0100

On Fri, Dec 9, 2011 at 3:53 AM, Rick Macklem <[email-masked]> wrote:
> for testing, I would appreciate hearing.

Hi Rick,

we have user space pNFS server written in java. It's part of dCache
project but can be used stand alone for testing. Requires JVM and disk
space to store data blocks:

1. checkout and build

$ hg clone http://www.dcache.org/~tigran/cgi-bin/hgwebdir.cgi/jpnfs dcache-nfs
$ cd dcache-nfs
$ mvn clean package

2. tweak nfs.properties:
put you host ip instead of 192.168.2.108

current setup will start mds and ds on the same host on ports 2049 and 2052.
File system is re-initialized on restart.

3. start it:
$ java -jar target/jpNFS-0.0.4-SNAPSHOT-jar-with-dependencies.jar oncrpcsvc.xml

4. test your client

Regards,
   Tigran.

 (However, I am not funded to
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Boaz Harrosh <[email-masked]>
Date: Fri, 9 Dec 2011 13:08:57 +0200

> can show up "on the cheap", like I did last June in Ann Arbor.)

One way to go is the Linux pnfs project at linux-nfs.org start here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree

It is an NFSD linux Kernel server with pNFS extensions destined to
be part of Linux at some future release.

You will need to compile your own Kernel, or alternatively this Kernel
is available in rpm form  from Steve Dickson for some Fedoras:
http://steved.fedorapeople.org/repos/pnfs/

The easiest is to first use the what is called pnfs-exp which is an MDS
plus a single DS exporting any local file system.

Then you can also export a GFS2 read-only and an EXOFS which is a full
fixture pnfs-objects based cluster filesystem.

The Luster patch suggested to you is also based on above Benny's tree.

I have on my todo list a full osd (pnfs-objects) stack port to FreeBSD
But that's in the far future. 
Meanwhile I'd suggest the pnfs-client-core to layout-driver API be closely
crafted to match that of the Linux LD API so it could be kept the same
code base. For example the pnfs-objects LD is mostly pure C code relying
on the ORE (Objects Raid Engine). The ORE is multy licensed library that
has ports to Many platforms including Linux-Kernel 
Linux/FreeBSD/windows User-Mode and more of which only the Linux variants
actually run because of the need of the iscsi stack to support BiDi and
large CDB commands, Only supported by Linux.

Cheers
Boaz

---

From: "Sorin Faibish" <[email-masked]>
Date: Thu, 08 Dec 2011 19:48:43 -0500

On Thu, 08 Dec 2011 19:40:19 -0500, Sorin Faibish <[email-masked]> wrote:

> Attached please find the slides that we presented yesterday at USENIX  
> LISA 11

        EMC²
where information lives

---

From: Ric Wheeler <[email-masked]>
Date: Fri, 09 Dec 2011 14:57:23 -0500

> Tigran.

Steve posted them at:

http://people.redhat.com/steved/Lisa-11/LISA-11-pNFS-BoF-final.pdf

Ric

---

From: Steve Dickson <[email-masked]>
Date: Mon, 12 Dec 2011 08:08:43 -0500

> http://steved.fedorapeople.org/repos/pnfs/
These are a bit crusty... Since pNFS is enabled in F16 kernels and
beyond, I would suggest for a recent kernel, either download one
from the rawhide repo or just grab one from here
     http://koji.fedoraproject.org/koji/packageinfo?packageID=8

> plus a single DS exporting any local file system.
Yes, this is by far the easiest... I've done some "hello world" testing
with this code and it seems fairly stable.. The code is on the 
pnfsd-lexp branch in Benny's tree
    git://linux-nfs.org/~bhalevy/linux-pnfs.git

Its a fairly simple process to port that code into a Fedora kernel rpm.
If that would help... just let me know...

steved.

---

From: Rick Macklem <[email-masked]>
Date: Tue, 13 Dec 2011 22:20:31 -0500 (EST)

Boaz Harrosh wrote:
[good stuff snipped for brevity]
> code base.

Am I correct to assume that fs/nfs/nfsfilelayout* implements this API
for the file layout case? If so, I'll try and build glue that can use
that code unchanged (or at least minimally changed).

And Peter (I see you're on the cc list), is it possible to get a copy
of these files without any GPL? It looks like they have the U of Michigan
copyright, which I think should be fine for the FreeBSD folks, but then
there is

   MODULE_LICENSE("GPL");

in it, which I assume means it's GPL'd. (Basically, exactly the same file,
but released without this line, would make the FreeBSD folks happy, I think.)
(No rush on this, I can use GPL'd code, I just can't put it in the FreeBSD
source tree easily.) The same would apply to any other layout driver that
others, like Panasas or EMC might be willing to open source for FreeBSD.

And Boaz, thanks for the useful information. I'll probably start with
Fedora plus a patched kernel for a server to test against. (It sounds
like Steve is willing to help me out, since I haven't used "rpm" in
years and have never typed "git" in my life.:-)

Thanks everyone for your help with this, rick

> For example the pnfs-objects LD is mostly pure C code
> relying
> >>> that will support pNFS server/client (this is for Rick as well)
> >>> that
> >>> were

---

From: Benny Halevy <[email-masked]>
Date: Thu, 15 Dec 2011 17:42:53 +0200

On Wed, Dec 14, 2011 at 5:20 AM, Rick Macklem <[email-masked]> wrote:

> for the file layout case?

True.

> If so, I'll try and build glue that can use

Not up to me, but it's more likely you'd be able to get the Windows code
over the university license more easily...

Benny