Subject: [nfsv4] Stringprep (or not) in existing NFSv4.0 implementations.

From: "Noveck, David" <[email-masked]>
Date: Wed, 31 Jul 2013 11:50:52 -0400

As part of dealing with the IESG issues about I18N in RFC3530bis, there is a question that keeps coming up.  It is whether anyone has supported stringprep in NFSv4.0 implementations, as required by RFC3530.  So far, I've said "I don't think so" but that doesn't seem to be doing the trick.  Can people help me get some more complete  information?

I guess we need to be able to tell people, to the degree we honestly can, that existing v4.0 implementations  do not do any of the following.  By the way, these are all stringprep MUST's, no matter how bizarre they might seem.

*         Servers (or server/filesystem pairs)  that reject names containing character values unassigned in Unicode 3.2.

*         Servers (or server/filesystem pairs)  that scan names for any of the following characters and simply delete them

o    00AD; SOFT HYPHEN

o    1806; MONGOLIAN TODO SOFT HYPHEN

o    200B; ZERO WIDTH SPACE

o    2060; WORD JOINER

o    FEFF; ZERO WIDTH NO-BREAK SPACE

o    034F; COMBINING GRAPHEME JOINER

o    180B; MONGOLIAN FREE VARIATION SELECTOR ONE

o    180C; MONGOLIAN FREE VARIATION SELECTOR TWO

o    180D; MONGOLIAN FREE VARIATION SELECTOR THREE

o    200C; ZERO WIDTH NON-JOINER

o    200D; ZERO WIDTH JOINER

o    FE00; VARIATION SELECTOR-1

o    FE01; VARIATION SELECTOR-2

o    FE02; VARIATION SELECTOR-3

o    FE03; VARIATION SELECTOR-4

o    FE04; VARIATION SELECTOR-5

o    FE05; VARIATION SELECTOR-6

o    FE06; VARIATION SELECTOR-7

o    FE07; VARIATION SELECTOR-8

o    FE08; VARIATION SELECTOR-9

o    FE09; VARIATION SELECTOR-10

o    FE0A; VARIATION SELECTOR-11

o    FE0B; VARIATION SELECTOR-12

o    FE0C; VARIATION SELECTOR-13

o    FE0D; VARIATION SELECTOR-14

o    FE0E; VARIATION SELECTOR-15

o    FE0F; VARIATION SELECTOR-16

*         Servers (or server/filesystem pairs)  that reject names because they do not match a particular normalization form (e.g. NKFC or NKFD).

Also, with regard to clients, we need to know if they:

*         Filter character values unassigned  in Unicode 3.2 or otherwise exclude names (e.g. by returning errors) that contain such characters.

*         In any other way, depend on servers doing any of the things in the previous list.

On the other hand, if you do know of nfsv4.0 clients or servers that do these things, please tell me.  I'd like to know now.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Wed, 31 Jul 2013 15:57:42 +0000

On Wed, 2013-07-31 at 11:50 -0400, Noveck, David wrote:
> As part of dealing with the IESG issues about I18N in RFC3530bis,
> there is a question that keeps coming up.  It is whether anyone has
> supported stringprep in NFSv4.0 implementations, as required by
> RFC3530.  So far, I’ve said “I don’t think so” but that doesn’t seem
> to be doing the trick.  Can people help me get some more complete
>  information?
> 
>  
> 
> I guess we need to be able to tell people, to the degree we honestly
> can, that existing v4.0 implementations  do not do any of the
> following.  By the way, these are all stringprep MUST’s, no matter how
> bizarre they might seem.
> 
>  
> 
> ·         Servers (or server/filesystem pairs)  that reject names
> containing character values unassigned in Unicode 3.2. 
> 
> ·         Servers (or server/filesystem pairs)  that scan names for
> any of the following characters and simply delete them
> 
> o    00AD; SOFT HYPHEN
> 
> ·         Servers (or server/filesystem pairs)  that reject names
> because they do not match a particular normalization form (e.g. NKFC
> or NKFD).
> 
>  
> 
> ·         Filter character values unassigned  in Unicode 3.2 or
> otherwise exclude names (e.g. by returning errors) that contain such
> characters.

Definitely not.

> 
> ·         In any other way, depend on servers doing any of the things
> in the previous list.

No.

> On the other hand, if you do know of nfsv4.0 clients or servers that
> do these things, please tell me.  I’d like to know now.
> 
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: Nico Williams <[email-masked]>
Date: Wed, 31 Jul 2013 12:32:16 -0500

On Wed, Jul 31, 2013 at 10:50 AM, Noveck, David <[email-masked]> wrote:
> As part of dealing with the IESG issues about I18N in RFC3530bis, there is a
> question that keeps coming up.  It is whether anyone has supported
> stringprep in NFSv4.0 implementations, as required by RFC3530.  So far, I’ve
> said “I don’t think so” but that doesn’t seem to be doing the trick.  Can
> people help me get some more complete  information?

I looked in http://src.illumos.org/source, and while ZFS does a bit of
string preparation, it does not do any of the things you list below
(see below).

I'll note that where any stringprep is applied by servers it tends to
be in the *filesystem*, not in the NFS server proper.  Considering
that NASes generally need to interop using multiple remote filesystem
protocols, with legacy filesystem *content*, and with POSIX and Win32
semantics, it seems really wrong to mandate any sort of stringprep in
the NFS protocol.

I think an updated RFC should do nothing more than RECOMMEND a
stringprep, and preferably a realistic one at that (with absolutely
minimal mapping tables).  I'm not sure that there's any need to
negotiate anything about this, though having a way for clients to
observe what stringprep applies to any given fs/directory would be
nice.

We may need to nail down some semantics:

 - normalize on create?  this has some POSIX (and other) compatibility
considerations

 - normalize on lookup?  this has some security considerations

 - don't normalize at all?  this has some security and user interface
considerations

All three suck, but normalize on lookup works reasonably well (it's
what ZFS does).  The client could only have a role in the first and
last cases, and, really, it should have no role.

Malicious clients could send filenames designed to make the server
spend a lot of time on normalization, so that arguably it'd be best to
do normalization on the client or not at all.

I think the correct design principles here are:

 - the filesystem makes right

 - client-side user-land run-times make right

Corollary: the actual protocol client and server have no real role to
play.  We should make recommendations and stop there.

> I guess we need to be able to tell people, to the degree we honestly can,
> that existing v4.0 implementations  do not do any of the following.  By the
> way, these are all stringprep MUST’s, no matter how bizarre they might seem.

Indeed.  And if we agree with the principles stated above then NFSv4.x
has no business requiring any of this.

> ·         Servers (or server/filesystem pairs)  that reject names containing
> character values unassigned in Unicode 3.2.

This sort of requirement has become antiquated by now anyways.

> ·         Servers (or server/filesystem pairs)  that scan names for any of
> the following characters and simply delete them

ZFS doesn't do this to my knowledge.

> ·         Servers (or server/filesystem pairs)  that reject names because
> they do not match a particular normalization form (e.g. NKFC or NKFD).

ZFS normalizes on lookup, preserving the originals as best it can.

> Also, with regard to clients, we need to know if they:

I doubt any such clients exist.

Nico
--

---

From: "J. Bruce Fields" <[email-masked]>
Date: Wed, 31 Jul 2013 14:06:42 -0400

> I guess we need to be able to tell people, to the degree we honestly can, that existing v4.0 implementations  do not do any of the following.  By the way, these are all stringprep MUST's, no matter…

The answers for the Linux server are all "no".

(With the caveat that it might in theory be possible to implement some
of these in the filesystem, but I haven't checked and would be surprised
if any filesystem actually does.)

--b.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Wed, 31 Jul 2013 18:15:44 +0000

> if any filesystem actually does.)

The only one I can think of that might do it on Linux is hfs+ and that would only be for MacOS compatibility. None of the POSIX-compatible filesystems should do this.

Cheers
  Trond

---

From: Saul Tamari <[email-masked]>
Date: Wed, 31 Jul 2013 14:26:34 -0400

Hi,

Are you looking for a server that rejects file names that are defined
in pynfs's get_invalid_utf8strings() and used by some pynfs tests ?

Thanks,
Saul

p.s. below is the relevant list of illegal strings:

def get_invalid_utf8strings():
    """Return a list of invalid ISO10646-UTF-8 strings"""
    # FIXME: More invalid strings.
    return ["\xc0\xc1", # starts two multibyte sequences
            "\xe0\x8a", # terminates a multibyte sequence too early
            "\xc0\xaf", # overlong character"
            "\xfc\x80\x80\x80\x80\xaf", # overlong character
            "\xfc\x80\x80\x80\x80\x80", # NULL
            "\xed\xa0\x80", # UTF-16 surrogate
            "\xed\xbf\xbf", # UTF-16 surrogate
            "\xef\xbf\xbe", # Invalid character U+FFFE
            "\xe3\xc0\xc0", # just mangled.
            "\xc0\x90", # overlong character
            # byte sequences that should never appear at start
            "\x80",
            "\xbf",
            "\xfe",
            "\xff",
            # starts with no ends
            "\xc0 ",
            "\xdf ",
            "\xe0 ",
            "\xef ",
            "\xf0 ",
            "\xf7 ",
            "\xf8 ",
            "\xfb ",
            "\xfc ",
            "\xfd "
            ]

On Wed, Jul 31, 2013 at 11:50 AM, Noveck, David <[email-masked]> wrote:
> ·         Filter character values unassigned  in Unicode 3.2 or otherwise
> exclude names (e.g. by returning errors) that contain such characters.
>
> ·         In any other way, depend on servers doing any of the things in the
> previous list.
>
>
>
> On the other hand, if you do know of nfsv4.0 clients or servers that do
> these things, please tell me.  I’d like to know now.
>
>

---

From: "Noveck, David" <[email-masked]>
Date: Wed, 31 Jul 2013 14:42:11 -0400

I'm interested in whether servers there are servers that either do or don't check for things like that, but that isn't the stringprep issue.  Stringprep would reject a lot of stuff that is valid utf-8 but isn't valid Unicode 3.2 for instance. 

RFC3530 made rejection of such strings in names a MUST.  RFC3530bis  makes their rejection a SHOULD.

There are some sorts of strings where it is still a MUST,  like principals (user and group names) and server names.

Look at RFC3530bis for the details of all this.

-----Original Message-----
From: Saul Tamari [mailto:[email-masked]] 
Sent: Wednesday, July 31, 2013 2:27 PM
To: Noveck, David
Cc: nfsv4 list ([email-masked])
Subject: Re: [nfsv4] Stringprep (or not) in existing NFSv4.0 implementations.

Hi,

Are you looking for a server that rejects file names that are defined in pynfs's get_invalid_utf8strings() and used by some pynfs tests ?

Thanks,
Saul

On Wed, Jul 31, 2013 at 11:50 AM, Noveck, David <[email-masked]> wrote:
> RFC3530.  So far, I've said "I don't think so" but that doesn't seem 
> to be doing the trick.  Can people help me get some more complete  information?
>
>
>
> following.  By the way, these are all stringprep MUST's, no matter how bizarre they might seem.
>
>
>
> *         Servers (or server/filesystem pairs)  that reject names containing
> *         Servers (or server/filesystem pairs)  that scan names for any of
> *         Servers (or server/filesystem pairs)  that reject names because
> *         Filter character values unassigned  in Unicode 3.2 or otherwise
> do these things, please tell me.  I'd like to know now.
>
>

---

From: "Noveck, David" <[email-masked]>
Date: Wed, 31 Jul 2013 14:46:54 -0400

Just realized I wasn't clear before.  Replace the last three lines of the previous messages by:

RFC3530 made rejection of strings which are not valid utf-8 as names a MUST.  RFC3530bis  makes their rejection a SHOULD.

There are some sorts of strings where it (rejection of strings that are not valid utf-8) is still a MUST,  like principals (user and group names) and server names.

-----Original Message-----
From: Saul Tamari [mailto:[email-masked]]
Sent: Wednesday, July 31, 2013 2:27 PM
To: Noveck, David
Cc: nfsv4 list ([email-masked])
Subject: Re: [nfsv4] Stringprep (or not) in existing NFSv4.0 implementations.

Hi,

Thanks,
Saul

---

From: Nico Williams <[email-masked]>
Date: Wed, 31 Jul 2013 14:01:39 -0500

> The only one I can think of that might do it on Linux is hfs+ and that would only be for MacOS compatibility. None of the POSIX-compatible filesystems should do this.

ZFS on Linux does what I described earlier.  Still, no ZFS
implementation, to my knowledge, does what David was asking about.