Subject: [nfsv4] Current implementation status of NFSv4.1

From: Spencer Shepler <[email-masked]>
Date: Mon, 18 Oct 2010 20:30:13 +0000

I am curious if anyone is interested and able to do a 
presentation at the upcoming WG meeting on 4.1 implementation
status?  Given the nature of the interop testing (NDAs) I am
not asking for an industry report but more of an experience
so far and status for single implementations.  Covering
things like: functionality implemented, issues found, issues
unresolved, areas of concern, lessons learned for future work.
That type of thing.

Anyone interested?

Thanks,
Spencer

p.s. Obviously, this could include NFSv4.0 as well. :-)

---

From: sfaibish <[email-masked]>
Date: Mon, 18 Oct 2010 16:45:07 -0400

> unresolved, areas of concern, lessons learned for future work.
Very good proposition. I vote for it. Thanks for proposing

/

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>
>

-- 
Best Regards

Sorin Faibish
Corporate Distinguished Engineer
Unified Storage Division
         EMC²
where information lives

Phone: 508-249-5745
Cellphone: 617-510-0422
Email : [email-masked]

---

From: sfaibish <[email-masked]>
Date: Mon, 18 Oct 2010 16:48:13 -0400

> unresolved, areas of concern, lessons learned for future work.
Very good idea. I vote for it and I will send my issues found
at BAT. Thank you

/Sorin

-- 
Best Regards

Sorin Faibish

Phone: 508-249-5745
Cellphone: 617-510-0422
Email : [email-masked]

---

From: "William A. (Andy) Adamson" <[email-masked]>
Date: Mon, 18 Oct 2010 17:01:51 -0400

> unresolved, areas of concern, lessons learned for future work.

I'd be willing to give a Linux implementation status.  How long a time
slot were you thinking of?

-->Andy

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 18 Oct 2010 21:27:52 +0000

> -----Original Message-----
> From: William A. (Andy) Adamson [mailto:[email-masked]]
> Sent: Monday, October 18, 2010 2:02 PM
> To: Spencer Shepler
> Cc: [email-masked]
> Subject: Re: [nfsv4] Current implementation status of NFSv4.1
> 
> > I am curious if anyone is interested and able to do a presentation at
> > the upcoming WG meeting on 4.1 implementation status?  Given the
> > nature of the interop testing (NDAs) I am not asking for an industry
> > report but more of an experience so far and status for single
> > implementations.  Covering things like: functionality implemented,
> > issues found, issues unresolved, areas of concern, lessons learned for
> > future work.
> > That type of thing.
> >
> > Anyone interested?
> 

Interestingly, I didn't have a duration in mind -- if there is material
to cover that would be helpful in moving our work forward, we will
make time for it.

Spencer

---

From: Thomas Haynes <[email-masked]>
Date: Mon, 18 Oct 2010 22:43:39 -0500

> to cover that would be helpful in moving our work forward, we will

Could we put it at the end of the agenda, after the 4.2 discussion?

I.e., I was unhappy that the 4.2 discussion fell off of the end in Maastricht.

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Tue, 19 Oct 2010 09:34:29 +0200

We did extensive pNFS test last couple of mounts which we present this week
at Conference on Computing in High Energy and Nuclear Physics in Taipei.
I can provide numbers and slides if you need.

Regards,
   Tigran

---

From: "William A. (Andy) Adamson" <[email-masked]>
Date: Tue, 19 Oct 2010 09:16:16 -0400

> I can provide numbers and slides if you need.

Hi Tigran

Very interesting - please forward them to me.

Thanks

-->Andy

---

From: Marc Eshel <[email-masked]>
Date: Tue, 19 Oct 2010 08:44:13 -0700

Hi Tigran,
We are all interested, can you post it to this email list.
Thanks, Marc.

From:
"William A. (Andy) Adamson" <[email-masked]>
To:
Tigran Mkrtchyan <[email-masked]>
Cc:
[email-masked]
Date:
10/19/2010 06:16 AM
Subject:
Re: [nfsv4] Current implementation status of NFSv4.1
Sent by:
[email-masked]

Hi Tigran

Thanks

-->Andy

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 19 Oct 2010 17:07:00 +0000

Tom,

It appears that a good portion of agenda (half?) will be
dedicated to proposals targeted towards NFSv4.x.

I appreciate the desire to discuss NFSv4.2 and we should
but what exactly is to be covered in that slot if not proposals
for functionality?   As I have suggested about other topics,
we should do our best to start a discussion here on the WG alias
and then close or move those discussions forward dramatically
at the WG meeting.

Said another way, I want content behind agenda items and
this includes the NFSv4.2 discussion.  If there isn't predetermined
content then the item will be pushed to the end of agenda
with little time.

btw: I will be posting an updated agenda later today.

Spencer

From: Thomas Haynes [mailto:[email-masked]]
Sent: Monday, October 18, 2010 8:44 PM
To: Spencer Shepler
Cc: William A. (Andy) Adamson; [email-masked]
Subject: Re: [nfsv4] Current implementation status of NFSv4.1

Spencer

---

From: Tigran Mkrtchyan <[email-masked]>
Date: Wed, 20 Oct 2010 18:12:38 +0200

Here is the relevant information:

SETUP
 
server:
   sw: dCache-1.9.10-1 on MDS and DS
   hw: 5x DELL R510 2x4 core 2.27 GHz, 10 Gb/s NIC (intel), 12 GB RAM, 
15 TB RAID6 SATA

client:
  sw: RHEL 5.3 + 2.6.36-rc3-pnfs-all-latest + nfs-utils-1.2.1,
  hw: 32x DELL M600 blades  2x4 cores 2.5 GHz, 16 GB RAM, 1 Gb/s NIC
  mount options:/* */*/-t nfs4 -o minorversion=1,rsize=32768,wsize=32768/*

network:
   hw: Force 10 Gbit switch

In total: 256 cores vs. 5 DS + 1 MDS

We run 8248  typical analysis batch jobs processing 33 TB of data.
It took 4 days with ~330 MB/s. No crashes.

We written in 10 days ~13 TB data from Stanford University to Hamburg
with FTP
client on a single node. Each file was between 100-200 GB. No crashes.

With 50 parallel jobs we get up to network limit. With mo jobs performance
started to degrade. I suspect, that there are no enough io thread in our
server
( only 5 per DS ). Our admins say that we hit the disk limits on a
single server
as we do not stripe the files. This have to be sill investigated.

All clients committed a group suicide after attempt to write into a
single file
(this was triggered by operational mistake). As this is not supported by our
server I suspect that IOERROR or LAYOUT_TRYLATER error was returned.
Unfortunately this happened during my vacation and I was not able to
look at it.

We have declared test success full. In total we have seen quite a good
performance
and reasonable stability.  We was able to show our users that NFSv4.1 is
real and
can be used even if it's still experimental.

We look forward to do similar test with some of the vendor systems as well.

Regards,
   Tigran.

P.S: here is the link to original presentation, but it full of 'foreign'
words.
http://117.103.105.177/MaKaC/materialDisplay.py?contribId=217&sessionId=60&materialId=slides&confId=3

On 10/19/2010 05:44 PM, Marc Eshel wrote:
>> We did extensive pNFS test last couple of mounts which we present this 
>>     
> week
>