Subject: Re: [nfsv4] NFSv4 I18n string type summary

From: <[email-masked]>
Date: Sun, 1 Aug 2010 16:46:09 -0400

Dave,

Taking a quick pass through this:
- filenames (incl. component4 and linktext4) are "special" and we are
going to have to
	do something NFSv4-specific in order to cope with physical
filesystems that have
	different normalization rules.  I would still watch what precis
does (e.g., for
	what characters to prohibit - e.g., control characters are
usually prohibited in
	file names, although we may be able to punt that to the
underlying physical FS).
- domain names (both as server names and as part of a principal) are
handled by pointing
	at IDNAbis, and figuring out representation.  I still dislike
A-labels on the wire
	in a protocol that is fully UTF8-capable (A-label exist because
DNS isn't). 
- user names.  I would watch what precis does here, as they're likely to
do something
	useful for user names in general that we can point to.  I would
treat group names
	as a class of user names for i18n purposes.

Those three items may cover us, as:
- A principal is constructed from a user/group name plus a server name. 
- If tags are never compared in the protocol, they can be described as
"opaque".
- Stating that MIME types and svraddr4 are always ASCII removes i18n
issues.

I still want to take another run at A-labels vs. U-labels, because the
current direction allows A-labels to show up in config files and
associated management software - IMHO, that sort of abuse of
administrators is just plain wrong when the protocol and related
technology support Unicode.

Thanks,
--David

> -----Original Message-----
> From: Noveck, David
> Sent: Thursday, July 29, 2010 5:16 PM
> To: Black, David; [email-masked]'
> Subject: NFSv4 I18n string type summary
> 
> David Black asked me to provide information on all the i18n-related
types in NFSv4.  It turns
> out that there is a section devoted to that very purpose in
draft-ietf-nfsv4-rfc3530bis-04.txt.
> It is section 12.2 and starts on page 164.  The draft only expires on
January 8, 2011 so is
> available until that time or until a revision is published (I'm
guessing something like three
> months from now).
> 
> I've reproduced the text (minus heading crap) below and will try to
give people a brief summary
> of the summary here.  It will not fit in 144 characters, however.
> 
> There are a number of types of strings which are combinations of other
type such as users and
> groups which are the form principal@domain.  There is some preliminary
processing defined for
> such types that defines how to break these up into other types.  These
mixture type will not
> be further discussed here.  See table 6 in section 12.2 below.
> 
> Aside from these there are three classes of string types to deal with.
> 
> First are those that there is some existing internet specification on
which handling can be based.
> Life is good.
+-----------------+----------------------------------------------------+
> | Type            | Explanation and Discussion
|
+-----------------+----------------------------------------------------+
> | svrname4        | Server name as returned by server.  Not sent by
|
> |                 | client, except in VERIFY/NVERIFY.  Handled by
|
> |                 | IDNAbis rules with A-labels and UTF-8 strings
|
> |                 | all allowed.
|
> | prinsfx4        | Suffix part of principal, in the form of a domain
|
> |                 | name.  Handled by IDNAbis rules with A-labels and
|
> |                 | UTF-8 strings all allowed.
|
+-----------------+----------------------------------------------------+
> 
> Next are the more troublesome and ideologically contentious types
where the issue is basically that we
> are communicating with entities that exist and that we, in general,
cannot change.  In the case of
> component names and link text strings they are the file system, while
in the case of the principal
> names, it is whatever mechanism the implementations have agreed upon
to map between names and numeric
> ids.  We are communicating with the file system which has its own
rules for name processing.  This
> makes a directive approach whereby the details of all string
processing are to be specified a non-
> starter.  I perceive some reluctance to accept this fact and proceed
in that light.
+-----------------+----------------------------------------------------+
> | Type            | Explanation and Discussion
|
+-----------------+----------------------------------------------------+
> | component4      | Should be utf8 but clients may need to access file
|
> |                 | systems with a different name structure, possibly
|
> |                 | including non-utf8 names.
|
> | linktext4       | Should be utf8 since text may include name
|
> |                 | components. Because of the need to access existing
|
> |                 | existing file systems, this check may be
inhibited.|
> | prinpfx4        | Must match one of a list of valid users or groups
|
> |                 | for that particular domain.
|
+-----------------+----------------------------------------------------+
> 
> Finally, the following types, even though nominally UTF-8, have for
various no i18n-related processing
> defined in the protocol.
+-----------------+----------------------------------------------------+
> | Type            | Explanation and Discussion
|
+-----------------+----------------------------------------------------+
> | comptag4        | Tags are debugging strings that are in the
protocol|
> |                 | for the purpose of helping protocol debugging.
|
> |                 | There is no specification as to what should be in
|
> |                 | They are defined as UTF-8 but are not validated
for|
> |                 | UTF-8 correctness by the receiver.  What would you
|
> |                 | do if it is wrong?
|
+-----------------+----------------------------------------------------+
> | fattr4_mimetype | All mime types are validated by comparing them
|
> |                 | against a fixed list, all of whose members consist
|
> |                 | only of ascii characters, making other
i18n-related|
> |                 | processing unnecessary.
|
+-----------------+----------------------------------------------------+
> | svraddr4        | Denotes something already validated as an IP
|
> |                 | address, whether IPv4 or IPv6, which means it has
|
> |                 | to be all ascii.
|
+-----------------+----------------------------------------------------+
------------------------------------------------------------------------
> 
> 12.2.  String Type Overview
> 
> 12.2.1.  Overall String Class Divisions
> 
>    NFS version 4 has to deal with with a large set of diffreent types
of
>    strings and because of the different role of each,
>    internationalization issues will be different for each:
> 
>    o  For some types of strings, the fundamental internationalization-
>       related decisions are the province of the file system or the
>       security-handling functions of the server and the protocol's job
>       is to establish the rules under which file systems and servers
are
>       allowed to exercise this freedom, to avoid adding to confusion.
> 
>    o  In other cases, the fundamental internationalization issues are
>       the responsibility of other IETF groups and our jobis simply to
>       reference those and perhaps make a few choices as to how they
are
>       to be used (e.g.  U-labels vs. A-labels).
> 
>    o  There are also cases in which a string has a small amount of NFS
>       version 4 processing which results in one or more strings being
>       referred to one of the other categories.
> 
>    We will divide strings to be dealt with into the following classes:
> 
>    MIX  indicating that there is small amount of preparatory
processing
>       that either picks an appropriate modes of internationalization
>       handling or divides the string into a set of (two) strings with
a
>       different mode internationalization handling for each.  The
>       details are discussed in the section "Types with Pre-processing
to
>       Resolve Mixture Issues".
> 
>    NIP  indicating that, for various reasons, there is no need for
>       internationalization-specific processing to be performed.  The
>       specifics of the various string types handled in this way are
>       described in the section "String Types without
>       Internationalization Processing".
> 
>    INET  indicating that the string needs to be processed in a fashion
>       is goverened by non-NFS-specific internet specifications.  The
>       details are discussed in the section "Types with Processing
>       Defined by Other Internet Areas".
> 
>    NFS  indicating that the string needs to be processed in a fashion
is
>       goverened by NFSv4-specific consideration.  The primary focus is
>       on enabling flexibility for the various file systems to be
>       accessed and is described in the section "String Types with NFS-
>       specific Processing".
> 
> 12.2.2.  Divisions by Typedef Parent types
> 
>    There are a number of different string types within NFS version 4
and
>    internationalization handling will be different for different types
>    of strings.  Each the types will be in one of four groups based on
>    the parent type that specifies the nature of its relationship to
utf8
>    and ascii.
> 
>    utf8_should/SHOULD:  indicating that strings of this type should be
>       UTF-8 but clients and servers will not check for valid UTF-8
>       encoding.
> 
>    utf8val_should/VSHOULD:  indicating that strings of this type
should
>       be and generally will be in the form of the UTF-8 encoding of
>       Unicode.  Strings in most cases will be checked by the server
for
>       valid UTF-8 but for certain file systems, such checking may be
>       inhibited.
> 
>    utf8val_must/VMUST:  indicating that strings of this type must be
in
>       the form of the UTF-8 encoding of Unicode.  Strings will be
>       checked by the server for valid UTF-8 and the server should
ensure
>       that when sent to the client, they are valid UTF-8.
> 
>    ascii_must/ASCII:  indicating that strings of this type must be
pure
>       ASCII, and thus automatically UTF-8.  The processing of these
>       string must ensure that they are only have ASCII characters but
>       this need not be a separate step if any normally required check
>       for validity inherently assures that only ASCII characters are
>       present.
> 
> 12.2.3.  Individual Types and Their Handling
> 
>    The first table outlines the handling for the primary string types,
>    i.e. those not derived as a prefix or a suffix from a mixture type.
> 
>
+-----------------+---------+-------+-------------------------------+
>    | Type            | Parent  | Class | Explanation
|
+-----------------+---------+-------+-------------------------------+
>    | comptag4        | SHOULD  | NIP   | Should be utf8 but no
|
>    |                 |         |       | validation by server or
|
>    |                 |         |       | client is to be done.
|
>    | component4      | VSHOULD | NFS   | Should be utf8 but clients
|
>    |                 |         |       | may need to access file
|
>    |                 |         |       | systems with a different name
|
>    |                 |         |       | structure. files systems with
|
>    |                 |         |       | non-utf8 names.
|
>    | linktext4       | VSHOULD | NFS   | Should be utf8 since text may
|
>    |                 |         |       | include name components.
|
>    |                 |         |       | Because of the need to access
|
>    |                 |         |       | existing file systems, this
|
>    |                 |         |       | check may be inhibited.
|
>    | fattr4_mimetype | ASCII   | NIP   | All mime types are ascii so
|
>    |                 |         |       | no specific utf8 processing
|
>    |                 |         |       | is required, given that you
|
>    |                 |         |       | are comparing to that list.
|
+-----------------+---------+-------+-------------------------------+
> 
>                                   Table 5
> 
>    There are a number of string types that are compound in that they
may
>    consist of multiple conjoined strings with different utf8-related
>    processing for each.
> 
>
+---------+--------+-------+----------------------------------------+
>    | Type    | Parent | Class | Explanation
|
+---------+--------+-------+----------------------------------------+
>    | prin4   | VMUST  | MIX   | Consists of two parts separated by an
|
>    |         |        |       | at-sign, a prinpfx4 and a prinsfx4.
|
>    |         |        |       | These are described in the next table.
|
>    | server4 | VMUST  | MIX   | Is either an IP address (serveraddr4)
|
>    |         |        |       | which has to be pure ascii or a server
|
>    |         |        |       | name svrname4, which is described
|
>    |         |        |       | immediately below.
|
+---------+--------+-------+----------------------------------------+
> 
>                                   Table 6
> 
>    The last table describes the components of the compound types
>    described above.
> 
>
+----------+-------+------+-----------------------------------------+
>    | Type     | Class | Def  | Explanation
|
+----------+-------+------+-----------------------------------------+
>    | svraddr4 | ASCII | NIP  | Server as IP address, whether IPv4 or
|
>    |          |       |      | IPv6,
|
>    | svrname4 | VMUST | INET | Server name as returned by server.  Not
|
>    |          |       |      | sent by client, except in
|
>    |          |       |      | VERIFY/NVERIFY.
|
>    | prinsfx4 | VMUST | INET | Suffix part of principal, in the form
|
>    |          |       |      | of a domain name.
|
>    | prinpfx4 | VMUST | NFS  | Must match one of a list of valid users
|
>    |          |       |      | or groups for that particular domain.
|
+----------+-------+------+-----------------------------------------+
>                                   Table 7

---

From: <[email-masked]>
Date: Sat, 7 Aug 2010 14:19:21 -0400

Actually, we have more agreements than disagreements.  Where we have
disagreements, we have to decide whether they are just matters of
philosophy/approach or really affect the substance of RFC3530bis.  In
the former case, we should be able to wordsmith our way to something
generally acceptable.  I think we can all help the process by
concentrating on the question of what disagreements exist with the text
in draft-04.

> - filenames (incl. component4 and linktext4) are "special" 

Agreed.

>    and we are going to have to do something NFSv4-specific 
>    in order to cope with physical filesystems that have
>    different normalization rules.  

Agreed.  That is the approach taken in draft-04.

I'd appreciate your comments on specific issues you have with what is
there.

>     I would still watch what precis does (e.g., for what 
>     characters to prohibit - e.g., control characters are 
>     usually prohibited in file names, although we may be 
>     able to punt that to the underlying physical FS).

To follow your football metaphor, if there is to be any punting, it
would be the filesystems punting this to precis and I don't believe that
any punting is required.  My impression is that the model of making such
choices the responsibility of the file system, i.e. what has actually
been implemented, has not been faced with any fourth-down situations.  I
won't deal with the contentious issue of what the stringprep-NFSv4
scoreboard shows. 

This area has been under the control of the file system and there has
been no occasion (or reason) to transfer control of this.  Ignoring this
fact is the fundamental reason that RFC3530 diverged so far from reality
and I would hate for us to reproduce the same problem with precis in
RFC3530bis that RFC3530 had with stringprep.

As you have pointed out, the critical design issue here is that people
be able to open existing files and that they wind up opening the ones
that they intended to open.  This makes it critical to retain the
filesystems' role here and not punt it to precis or anyone else because
NFSv4.0 is an existing protocol that works with existing filesystems.
If a file system allows a particular character, then there can be file
names that include it, so the fact the precis excludes it is not
relevant.  If one were to have a file stored in a filesystem which
violated the rules precis has yet to formulate, one could not open it
using NFSv4, while it could be opened locally.  Aside from the ex post
facto fairness aspects, the point is that it is the job of NFSv4 to
prevent that from happening.   

You say we "may be able to" deal with this as I have suggested and is
done in draft-04.   Normally (although not in formal modal logic), this
implies that one might not be able to.  If you feel it is indeed
possible that we cannot do this, please explain why and suggest an
alternative that you think is better, and we can have the necessary
discussion.

handled by pointing
A-labels on the wire
DNS isn't).

I agree.  You mention taking another run at the A-vs.-U issue.  My
sympathies are with your position.  What is in draft-04 represents a
least-common-denominator consensus that I could get everybody to agree
on for a microsecond or so or at least not actively decry.  If you and
Nico (I assume) can discuss the issues and we wind up with a rough
U-labels-only consensus, I would have not have trouble making the next
draft reflect that (and your points make sense to me, although my
understanding of the area is not complete).
 
> - user names.  I would watch what precis does here, as they're likely
to do something
> 	useful for user names in general that we can point to.  

Perhaps but the issue we will face is if there are user names that
precis rules out that are sent by NFSv4 implementations.  For example,
it might decide that user names are to be in NFC, which is very
reasonable in the abstract, but if there are existing implementations
that have user names not in that class, they must not be rejected on
that account.  The issue is similar to filenames in draft-04, but the
solution has differences, primarily because n-i/n-p does not seem
possible for user names.  One could not ask a filesystem to store for
each file, in addition to the uid/gid, enough information to determine
which of the N canonically equivalent versions of each name are to be
returned for that particular user or group.  I guess one could ask :-)

Perhaps it is better to cross this bridge when and if we come to it.  It
certainly would be helpful if we could limit our disagreements with
those espousing the reigning IETF i18n philosophy to filenames.

>     I would treat group names

Draft-04 treats these as part of a common class although it does not
give precedence to users.  That's a very a minor difference but if you
feel it is important to address, we can do that.

-----Original Message-----
From: Black, David 
Sent: Sunday, August 01, 2010 4:46 PM
To: Noveck, David; [email-masked]'
Subject: RE: NFSv4 I18n string type summary

Dave,

Thanks,
--David

types in NFSv4.  It turns
January 8, 2011 so is
type such as users and
processing defined for
mixture type will not
|
|
|
|
|
|
|
|
in that light.
|
|
|
|
|
|
inhibited.|
|
|
|
protocol|
|
|
for|
|
|
|
|
i18n-related|
|
|
|
|
of
are
are
processing
a
to
is
and
utf8
should
for
in
ensure
pure
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
may
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|