Subject: [nfsv4] FedFS Meeting Minutes, 11/20/2008

From: James Lentini <[email-masked]>
Date: Thu, 20 Nov 2008 15:41:21 -0500 (EST)

FedFS Meeting Minutes, 11/20/2008
---------------------------------

Attendees
---------

Craig Everhart (NetApp)
James Lentini (NetApp)
Renu Tewari (IBM/ARC)
Robert Thurlow (Sun)

Minutes
-------

+ IETF Note Well Agreement

  This is a reminder that our discussions are governed by the 
  IETF Note Well Agreement. See:

    http://www.ietf.org/NOTEWELL.html

  We will start each weeks meeting with this announcement.

+ IETF NFSv4 Meeting

  James sent draft slides for our presentation to the group. James 
  incorporated the feedback and has sent the final version to Mike Eisler 
  for presentation at tomorrow's meeting.

+ Review Action Items

  [AI] James Lentini will create a new version of the requirements draft next
       week

  James is 1/2 through the document. He expects to send the completed version 
  out by the end of the day.

  We discussed the format to present the changes in. Craig noted that it would
  be nice to see change bars. James will investigate if it is possible to add
  change bars.  

+ NSDB Replication Semantics

  Leif Johansson asked about our LDAP replication requirements. Leif pointed
  out that (a) LDAP replication is not standard and (b) different LDAP 
  implementations provided different semantics.

  Mario also raised similar concerns during our weekly meeting on 10/9/08.
  Mario also noted that replicating between LDAP implementations is not 
  universally supported. As a result, it would be difficult to construct 
  an NSDB comprised of instances of multiple LDAP implementations.

  We started by discussing if an NSDB should be replicated using LDAP.

  We agreed that the goal was to allow an NSDB to be replicated using 
  LDAP for two reasons: (a) to provide multiple copies of the root fileset
  (b) to distribute NSDB instances across a large, possibly geographically 
  distributed federation, and (c) to remove a single point of failure.

  We then discussed the FedFS requirements for NSDB replication.

  At a high level, FedFS requires that a change to the NSDB be reflected 
  at all instances of the NSDB.

  This raised two questions:

  - How long can an update take to propagate?
 
  - Do all NSDB instances need to be modifiable or can there be a subset of 
    NSDB instances (possibly 1) that are modifiable?

  One idea that had bee discussed in the past, but never written down, was
  that an FSN's NSDB name might map to multiple locations using DNS. The 
  closest NSDB instance to the source of the DNS query would be reported.
  In this scheme, all NSDB instances would be peer's of one another. Therefore
  there would be a requirement that all NSDB instances be modifiable. 
  
  Based on our current protocols, our current requirements are that 
  the LDAP replication mechanism provides:
	- ability to update all NSDB instances (multi-master)
	- changes are propagated to other instances over time, but there 
	  are no exact limits on how long this should take

  While these are the current requirements for an LDAP replication mechanism, 
  we suspect that single-master replication implementations are more 
  prevalent and therefore perhaps we need to augment our protocols so that a
  multi-master configuration is not required.

  We will investigate what replication mechanisms (single-master, multi-master,
  or token-based hopping-master) are supported by current implementations.

  [AI] Renu will research what replication mechanisms are supported by the 
       OpenLDAP and IBM LDAP server.

  [AI] James will ping Mario and see if he has an update on his investigation 
       of Active Directory and iPlanet replication mechanism.

  [AI] James will research what replication mechanisms are supported by 
       Red Hat Directory Service.

  What did AFS use for replication?
  AFS had a protocol called ubik.

  We should focus on the FedFS requirements and then determine what we need.
  Practically speaking, we are constrained by what is available in LDAP 
  implementations.

+ For next week
	- Access Control in the Federation
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Leif Johansson <[email-masked]>
Date: Thu, 20 Nov 2008 23:57:46 +0100

<snip>

>   an NSDB comprised of instances of multiple LDAP implementations.

Afaik there only case of cross-vendor support for replication is that samba4
can participate in AD replication. Thats it.

<snip>

>   - How long can an update take to propagate?

This is the critical question! Most LDAP vendors support some variation of 
Harald Alvestrands "eventually consistent" model. AFS ubiq on the other hand 
supports the always consistent model - i.e all updates are commits across all
db servers before the update is successful.

>   In this scheme, all NSDB instances would be peer's of one another.
> Therefore there would be a requirement that all NSDB instances be
> modifiable.

It sounds like they would also need to be equally consistent too, no?

> 	- ability to update all NSDB instances (multi-master)

Just so that y'all know: there is never (for some value of never) going to 
be a cross-vendor way of doing this. 

> 	  are no exact limits on how long this should take

So if it takes a couple of hours for an update to make it to every NSDB
instance clients will still experience a consistent view of the filesystem? 

>
>   While these are the current requirements for an LDAP replication
> mechanism, we suspect that single-master replication implementations are
> more prevalent and therefore perhaps we need to augment our protocols so
> that a multi-master configuration is not required.

Actually today most major vendors support multi-master today - only very
differently.

	Cheers Leif

---

From: James Lentini <[email-masked]>
Date: Fri, 21 Nov 2008 15:42:56 -0500 (EST)

Leif,

These are very useful data points. I have a few follow up questions 
below:

> can participate in AD replication. Thats it.

As you can see, we 

Does samba4 have its own LDAP implementation or does it use OpenLDAP?

I was under the impression that cross vendor replication was supported 
between Active Directory and iPlanet or Active Directory and OpenLDAP. 
One of our contributors was going to look into support for replication 
between iPlanet and OpenLDAP.

> It sounds like they would also need to be equally consistent too, no?

Yes, I think that would be another implication.

> instance clients will still experience a consistent view of the filesystem? 

The view would not be consistent, but the protocols do not to 
guarantee namespace consistency.

The Purpose section of the requirements document:

http://www.ietf.org/internet-drafts/draft-ietf-nfsv4-federated-fs-reqts-00.txt

has more information on this. 

For example, the protocols allow a client, c1, to traverse a junction, 
X, that is subsequently modified. When client, c2, traverses the 
junction X, c2 may see a different namespace than client c1. This is 
currently allowed.

> Actually today most major vendors support multi-master today - only very

During the NFSv4 working group meeting today (I was listening in on 
the audio stream), someone (was it you?) suggested an approach using 
LDAP single-master replication with replica acknowledgments. Did I 
get that right? Is there information on this available somewhere?
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Leif Johansson <[email-masked]>
Date: Fri, 21 Nov 2008 21:56:31 +0100

On Friday 21 November 2008 21:42:56 James Lentini wrote:
> > >   Mario also raised similar concerns during our weekly meeting on
> > > 10/9/08. Mario also noted that replicating between LDAP implementations
> > > is not universally supported. As a result, it would be difficult to
> > > construct an NSDB comprised of instances of multiple LDAP
> > > implementations.
> >
> > Afaik there only case of cross-vendor support for replication is that
> > samba4 can participate in AD replication. Thats it.
>
> As you can see, we
>

Its own.

> between iPlanet and OpenLDAP.

There may be partial support based on moving LDIF around but there is
no standardized replication. There are probably many 3:rd party plugins
out there. 

<snip>

> junction X, c2 may see a different namespace than client c1. This is

ok but is it ok that c1 and c2 see a different view of the fs depending on
which replica of the NSDB they look at?

> > > mechanism, we suspect that single-master replication implementations
> > > are more prevalent and therefore perhaps we need to augment our
> > > protocols so that a multi-master configuration is not required.
> >

No that wasn't me and I don't think it will help you much. Single-
master isn't standardized either.

	Cheers Leif