Subject: Re: [ldapext] DBIS commentary

From: "Bannister, Mark" <[email-masked]>
Date: Thu, 26 Nov 2015 11:01:44 +0000

Hi Jordan,

This is all excellent feedback, thanks, I really appreciate the time you have taken to provide your
comments.  Many of the points you raise I did spend a long time considering myself when I was
writing the internet drafts, so I hope to provide you with more insight into my rationale and why
I decided to do things the way I did.

I hope you don't mind but I'm copying it to the ldapext mailing list, so that others can also
comment if they wish.  For others, Jordan is commenting from the DBIS/RFC2307
schema comparison and not directly from the internet drafts:

http://sourceforge.net/p/dbis/wiki/DBIS%20and%20RFC2307%20schemas/

Jordan Brown wrote:
> High-level comments:
> - In general, our direction is towards X.500 compatibility and Active Directory compatibility.  I would
>   resist new mechanisms that duplicate those.
> - As noted in more detail below, I consider case-insensitive processing to be the correct direction and
>   would resist moves towards case sensitivity.
> - Some of the mechanisms here (e.g. gecos mapping) seem to duplicate RFC 4876 profiles.

Thanks for the summary, I'll answer these points individually as I get to them below.

> 
> Footnote 1 says that published classes may not be redefined. Although not all changes seem permissible,
> some changes (e.g. addition of MAY elements) seem like they could be allowed.  Is there a reference for
> this restriction?  Note that between X.521-1993 and X.521-2012 the TelecommunicationAttributeSet had
> an attribute deleted.

I haven't looked for a reference to this restriction, I had learned it from seeing previous exchanges
on the ldapext mailing list with Kurt Zeilenga.  Here is one such exchange, regarding changing the
schema declaration of 'posixGroup' in RFC2307bis:

https://www.ietf.org/mail-archive/web/ldapext/current/msg02072.html

>
> posixAccount / posixUserAccount
> 
> en:  I consider the change from a case-sensitive name in NIS to a case-insensitive name in LDAP to be an
> improvement that should not be reversed.  It is telling that (a) there are no cases in the industry other
> than UNIX user names where user names are case sensitive, (b) that there are numerous examples in
> the industry of treating the left side of an e-mail address as case-insensitive, and (c) the X.520 "name"
> attribute, the base attribute for all naming, is case-insensitive.

NIS was case sensitive.  That led some firms to put entries in various maps that were the same except
for case.  I have examples here where this occurs in the passwd, group, netgroup and services maps.
Changing these entries would be a) very difficult indeed to the point of insurmountable as they've
been ingrained in many systems in a very large enterprise for over a decade and b) unnecessary had
the RFC2307 schema been 100% backwards compatible with NIS.  I explain this rationale further in my
DBIS paper:

http://ldapcon.org/2015/wp-content/uploads/2015/09/DBIS-Paper-2015.pdf

We had a few choices.  The one which we used for the past few years was to define our own custom
schema.  We had to invent three new custom attributes and seven new custom object classes to
get around this limitation.  It infuriated me that we couldn't use RFC2307 out-of-the-box because it
was not 100% compatible with the NIS schema, and this was one of the primary driving forces
behind me starting DBIS.

DBIS is my stake in the ground to say we MUST solve this problem.  I hear what you're saying,
but your arguments for case insensitivity disregard that UNIX is primarily a case-sensitive
operating system and that NIS was also case-sensitive.  Frankly, as RFC2307 was supposed to
be primarily getting NIS data into LDAP, it failed tremendously by bending to the Windows/AD
case insensitivity preferences.  UNIX will not change.  It will always be case sensitive.  Therefore
we need an LDAP schema that will respect that.

I hope this doesn't mean we end up with one schema for UNIX and one for Windows, perhaps
we can be creative, for example, encouraging Windows software to force all characters to
lowercase before adding entries to LDAP and constructing search filters, although this will
probably be prone to breakage.  Or perhaps we can define some special attributes where case
is handled differently depending on the capability of the client.  If the client claims it is
case sensitive, these special attributes are handled case sensitively, and vice versa if the client
claims it is case insensitive.  I see case sensitivity as an issue that will prevail forever, in the
same way that endianness will differ between architectures; they will never converge and
so we should not be accepting standards that do not provide a solution for both types.

>
> authPassword:  Imported from RFC2307bis-02 and thence from RFC 3112.  Partially, but only partially,
> duplicates userPassword (from X.509 via RFC2307).  Appears to completely duplicate userPwd (also
> from X.509).  May also duplicate encryptedUserPassword, mentioned in X.520 but not defined
> anywhere that I can find.

I added authPassword simply because I wanted those using RFC2307bis to be able to migrate
seamlessly to DBIS.  I made it clear in the drafts that there are better and more secure ways to store
passwords and these should be used in preference - authPassword was provided for backwards
compatibility only.

If you have an issue with authPassword being in RFC2307bis, now is the time to raise it, as it
is on the new LDAPEXT charter and the WG are planning to finalise RFC2307bis in the
next 12 months and give it its own RFC number.

>
> gecos:  Removed in DBIS with no explicit replacement.  DBIS recommends using its mapping capabilities
> to map gecos onto some other attribute, e.g. displayName.  While eliminating gecos seems desirable
> (as an unnecessarily UNIX-specific attribute), it seems desirable to explicitly specify a replacement
> rather than to rely on custom mapping.  (Note also that some implementations have encoded
> additional information, e.g. telephone number, into the gecos field.  This seems like an undesirable
> overloading of the schema.)

LDAP is so much better at representing all kinds of random information than NIS was, and the passwd
flat-text file.  Just because there was a random "throw in any information you like" including an
implied encouragement in Sun Microsystems' training courses to include telephone information
in the gecos field, does not mean that when you move to a much more structured system such
as LDAP you have to keep the general purpose nonsense.  If you want to encode a telephone number, 
there is an attribute for that.  If you want a surname, or a first name, there are attributes for that
too.  "Gecos" is not a unique piece of information, it is a container for an assortment of data, which
will differ from one firm to the next.  The equivalent to "gecos" is an LDAP object.

>
> description:  Removed from explicit mention in DBIS, but inherited from Person if inetOrgPerson is
> used as the base class.  Not clear how it fits into the UNIX/NIS schema.

I removed it because I saw no purpose for it on a passwd entry, and as you say you can get it
back again if you include another auxiliary class which you probably want to do
anyway to have something to map gecos to.

>
> Structural User Account:  changes from "account" (RFC 4524) to "inetOrgPerson" 
> (RFC 2798).  While this is an obvious simplification, it does not seem correct.  
> An account may be associated with more than one person, and one person may be associated with
> more than one account.  (However, associating it with "account" 
> leaves it unclear where to collect a gecos/display name, except perhaps
> "description".)

Quite, "account" doesn't have a field where you can sensibly put the descriptive
name of the user, whereas inetOrgPerson does.  In a previous project where we
were migrating NIS to AD, we used displayName to hold the contents of the
gecos field, and that is what swayed my choice of inetOrgPerson here.

> 
> posixGroup / posixGroupAccount
> 
> X.521 groupOfNames seems like a more appropriate basis for the definition of a group.

But that doesn't allow for GID number.

> en:  See above.

Reply above.

> 
> exactUser:  Ties into the case-sensitivity question.  If case-insensitive, "memberUid" is reasonably
> appropriate, because in the LDAP world "uid" means "login name".

"uid" meaning "login name" is a really bad choice.  I've always thought that, and I've pushed to get
away from it in DBIS.  UID is a number, it should always be a number, why anyone thought it would
be a good idea for that to actually be a name escapes me.  I've discussed the case sensitivity
issue already, and I called this "exactUser" to follow-on from the idea of exact case, and to
follow a similar line as "exactName" (en).

>
> manager:  although not inappropriate, X.520 "owner" seems more appropriate.  (And note that
> groupOfNames includes "owner".)

As "manager" was already in RFC2307/2307bis for host and network entries, I just expanded
the support to many other entries, as it seemed a sensible thing to do.  Having some entries
use "owner" and some use "manager" would be wrong, removing "manager" from
host and network entries would add an unnecessary step when migrating from an RFC2307
to a DBIS schema, and allowing both "owner" and "manager" seems excessive.

>
> nisNetgroup / netgroupObject
> 
> Although not a perfect match, groupOfNames seems like it might be a better basis for
> grouping computers.

netgroups are a specific construct, trying to squeeze them into something else that is
not a perfect match would be a suboptimal decision.  LDAP can represent netgroup
data better, so let's represent it better.

>
> All of the changes here seem to relate to issues described above.
> 
> ipService / ipServiceObject
> ipProtocol / ipProtocolObject
> 
> The changes here all appear to be similar to those described above, with the note that case
> sensitivity here seems especially pointless.

Not at all pointless given that we have real world examples of service names,
one in uppercase, one in lowercase, which have two different port numbers assigned.
If DBIS did not have an answer to that, we'd be back to using a custom schema,
which defeats the object of the exercise.  NIS was case sensitive, DBIS aims to be
100% compatible with the NIS schema, as previously discussed.

> 
> oncRpc / rpcObject
> 
> ONC RPC is only one kind of RPC, so leaving "ONC" in its name seems appropriate.

RFC5531 is titled "RPC" not "ONC RPC", and there is no other type of RPC defined in an
RFC as far as I am aware.  To me it was obvious what RPC referred to.  I needed a new
class anyway, and calling it oncRpcObject seemed unnecessary when rpcObject was
clear enough.

> 
> Again, the changes here all appear similar to those above.
> 
> ipHost / ipHostObject
> 
> This schema doesn't seem to allow for a host that has both IPv4 and IPv6 addresses, nor for one
> that has no addresses.

Mixing IPv4 and IPv6 should just be a case of assigning both the ipv4HostObject and ipv6HostObject
classes to the same entry.  A host with no IP address, if we need to support that, I suggest
we change ipHostObject to be STRUCTURAL rather than ABSTRACT.

> 
> While I could see using an LDAP directory as the back end for a DNS server, I could not see using
> it as the client's primary source for address<->name mapping.  
> In fact, ripping that capability out of our LDAP support is on my list of things to do - it adds essentially
> zero value and occasionally causes problems.

I agree, but DBIS must be backwards compatible.

> 
> ipNetwork/ipNetworkObject
> 
> As above.
> 
> automountMap / AutomountMapObject
> 
> As above.
> 
> automount / automountEntry
> 
> If we're going to change definitions, we should ensure that the new definition allows for SMB
> references in addition to NFS references. That suggests a URL form.

I actually added automountMapType to the draft but didn't update the comparison table.
This was so that we could also support amd-style entries.  I suppose that is where we
could also state that entries were smb-style entries too, and provide different
attributes that are SMB-friendly.

>
> Note that splitting automountInformation into automountLocation and automountOption means
> that you can't have multiple locations with different options.

You're right, but why would you do that?

>
> automountMulti
> 
> I don't yet understand what this class is for.

I've renamed this to automountMultiMount in the draft and didn't update the table.
This is for Solaris-style multi-mount entries.  I think they're being referred to as
hierarchical mounts now, but see some example entries here:

http://sourceforge.net/p/dbis/wiki/Map%20Entries/

Best regards,
Mark.

--------------------------------------------------------------------------------

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies; do not disclose, use or act upon the information; and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.

---

From: Steven Legg <[email-masked]>
Date: Fri, 27 Nov 2015 16:40:08 +1100

Hi Mark & Jordan,

On 26/11/2015 10:01 PM, Bannister, Mark wrote:
[snip]

> https://www.ietf.org/mail-archive/web/ldapext/current/msg02072.html

The rules of extensibility for object classes in X.519-2008, Clause 12.4 say:

     "Optional user attributes may be added to an existing object class without
      assigning a new object identifier."

Removing an attribute isn't covered, so by omission, it isn't allowed.

I don't recall the LDAP specifications having anything specific to say on the
matter, though I suspect the prevailing attitude is that you can't change an
object class without changing the OID.

The teletexTerminalIdentifier attribute was removed from TelecommunicationAttributeSet
because the service and/or the specifications were defunct, but it was a violation
of X.500's own extensibility rules. IMO the attribute should have just been made
obsolete. To get around possible interoperability problems I treat the new object
class definitions minus teletexTerminalIdentifier as the "original" definitions so
if I encounter a definition that includes teletexTerminalIdentifier it looks like
an allowed extension to the object class.

[snip]

Regards,
Steven

---

From: Simo Sorce <[email-masked]>
Date: Fri, 27 Nov 2015 10:06:42 -0500

On Thu, 2015-11-26 at 11:01 +0000, Bannister, Mark wrote:
> > - As noted in more detail below, I consider case-insensitive
> > processing to be the correct direction and
> > Footnote 1 says that published classes may not be redefined.
> > Although not all changes seem permissible,
> the RFC2307 schema been 100% backwards compatible with NIS.

NIS was just a poor job of taking the passwd and group files and
reproducing them verbatim in a networked environment, and it badly at
it.
Case sensitive names is something that may be wanted by some legacy
environments, but it is really a bad idea in modern environments and
should be discouraged.
I firmly believe you could have a way to allow case-sensitive names,
but insensitive names should be the default and recommended way, I
fully agree with Jordan on this.

Usernames are used by *users*, not just by machines, and most users
(and admins) will consider the Mark and mark to be the same thing, and
rightly so, it is. Any system that allows you to have two users with
substantially the same name is a failure and a security issue waiting
to happen, so that should not be encouraged.

>   I explain this rationale further in my
> DBIS paper:
> 

Unix passwd files are case sensitive, that doesn't mean LDAP has to be
at all. If you have the right pam/nss modules you can definitely behave
in both case preserving or case insensitive ways easily, and the Unix
system will not care. The kernel cares only about uidNumber/gidNumber
and names are just a representation.
There are unfortunately people that code their applications in a way
that user user/group names fro access control without first resolving
them to their numeric identifier, those are already broken wrt renames,
and may be broken if you return a random case from nsswitch plugins,
but you do not have to return a random case from a nsswitch plugin you
can simply always return all lowercase and your system is perfectly
consistent then.

Proof is that we've been using case-insensitive setups for more than a
decade (both LDAP, and Samba/Windows based source of user information)
and it works in 99.9% of setups.

> probably be prone to breakage.

Any new schema for LDAP that insist on being only case sensitive is
simply broken and cannot be a standard IMHO.

>   Or perhaps we can define some special attributes where case
> is handled differently depending on the capability of the client.

A schema that allows to define sensitivity *may* work, depending on the
details.

>   If the client claims it is
> case sensitive, these special attributes are handled case
> sensitively, and vice versa if the client

Not sure about that, we have experience that unix/linux system can work
just fine with case insensitive user names, only some broken/legacy
setup cannot cope because of existing broken data, but that is not a
Unix issue, it is a specific deployment issue. It is ok to let those
deployments have an option perhaps (I think they are just broken on
security grounds, but that's just my opinion) but I think your
characterization that Unix/Liunux clients can only cope with case
sensitive names is misleading, given we have experience that they can
cope just fine assuming the nsswitch and pam module used are properly
built to deal with that.

> next 12 months and give it its own RFC number.

I have the issue that in 2015 we cannot propose standards that assume
password hashes are distributed outside of the directory, and that
password storage should definitely be an implementation detail (it can
be standardized, but not necessarily with an identity information
standard). Clients should just use ldap binds and let the LDAP server
handle internally how authentication is handled (may not be a password
at all).

> > gecos:  Removed in DBIS with no explicit replacement.  DBIS
> > recommends using its mapping capabilities

I tend to agree with this sentiment.

> to a DBIS schema, and allowing both "owner" and "manager" seems

Actually owner and manager are quite different concepts, and allowing
both would not be bad idea.

> data better, so let's represent it better.

Netgroups are an abomination that should die. People should migrate off
of them, and we should not encode that abomination into a new standard
in 2015. This is one of the things that have me opposed to DBIS as a
standard, leave netgroups to RFC2307 and let them die there.

> 100% compatible with the NIS schema, as previously discussed.

NIS is dead Jim, dead! :-)

New standards should define stuff useful for the future, not backward
looking stuff that encodes and encroach mistakes done in the past.

For people that needs backwards compat with NIS what you should do is
simply define a generic NIS MAP object, and let individual sites store
what they want on these "maps" including crusty old concepts that
nobody should uses any more like the services and rpc maps.

> > In fact, ripping that capability out of our LDAP support is on my
> > list of things to do - it adds essentially

Amen

> I agree, but DBIS must be backwards compatible.

For you it may be, but from a Standards body it is not necessarily the
case, if your "proposal" is also "unchangeable" then there is no
discussion to be had in this group, no discussion, means also no
standard though (IMHO). As standards in IETF come from at least rough
consensus and here I do not see much consensus nor a willingness to
accept other points of view.

Simo.

> NOTICE: Morgan Stanley is not acting as a municipal advisor and the
> opinions or views contained herein are not intended to be, and do not
> constitute, advice within the meaning of Section 975 of the Dodd
> -Frank Wall Street Reform and Consumer Protection Act. If you have
> received this communication in error, please destroy all electronic
> and paper copies; do not disclose, use or act upon the information;
> and notify the sender immediately. Mistransmission is not intended to
> waive confidentiality or privilege. Morgan Stanley reserves the
> right, to the extent permitted under applicable law, to monitor
> electronic communications. This message is subject to terms available
> at the following link: http://www.morganstanley.com/disclaimers. If
> you cannot access these links, please notify us by reply message and
> we will send the contents to you. By messaging with Morgan Stanley
> you consent to the foregoing.
> _______________________________________________
> Ldapext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ldapext

---

From: Charlie <[email-masked]>
Date: Fri, 27 Nov 2015 12:36:46 -0500

Uids have not been numeric since RSTS/E and RSX.  The name is a contraction
of "user identifier" and in Unix they are the first field of the passwd
data object and are intended to be unique keys.  The numeric key associated
with a uid is the uidno or "user identifier NUMBER".

In modern deployments the taxon "login name" is imprecise and not unique; I
can and do "log in" using several different naming conventions (SAMname,
Kerberos principal, or LDAP distinguished name, which are all different)
and all of those identify me to the system as the unique user id keyed by
"uid".  POSIX file systems tag files with uidnos, and these are
just-in-time looked up to determine the user id currently associated with
the uidno.  That abstraction layer lets the names of groups and users be
changed without massive wholesale revision of filesystem metadata,
providing administrative scalability.

Thus, a uid is neither a login name nor a number; when cleanly implemented,
it is a unique alphanumeric key.  This is quite different from the
Microsoft SID-based system, but can be made compatible for most purposes by
making uids and SAMnames identical.

Netgroups are indeed an abomination and I  should rejoice if they were to
disappear.  Beers would be consumed.

Unix was intended by Ritchie et al to be case-sensitive, and significant
implementations exist that assume this.  However, I have converted a medium
sized enterprise (less than 200 named groups) to case insensitivity, by
renaming groups while retaining gidnos, during an LDAP conversion.  It's a
significant effort but nowhere near the complexity of converting to a
shared group and user namespace such as Microsoft AD implements;
duplication of names in the group and user namespaces is commonplace in
linux deployments.

Hoping that this was helpful and I'm not flogging any deceased equines,
--Charlie Brooks

---

From: Jordan Brown <[email-masked]>
Date: Mon, 30 Nov 2015 12:06:08 -0800

> https://www.ietf.org/mail-archive/web/ldapext/current/msg02072.html

He does seem to have a pretty strict stance on the matter.  Still, the X.519 
reference that Steven supplied suggests that there may be wiggle room there.  I 
don't know whether it's enough wiggle room to be helpful here, but it seems worth 
keeping in mind.

On the case-sensitivity front:  we have a pretty fundamental disagreement.  I 
believe that UNIX names must eventually become case-insensitive, that it is only a 
matter of time until they do. UNIX is not the whole world, and the entire rest of 
the world is clear that MARK, Mark, and mark are the same person.  I've got to 
believe that even in the UNIX world those differ-only-in-case instances are 
steadily on the decline - anybody who is coexisting with Windows or has adopted 
RFC 2307 is using case-insensitive names and it seems unlikely that anybody would 
create new instances of problematic names.  Even a purely UNIX environment isn't 
purely case-insensitive - sendmail treats user names as case-insensitive for at 
least some purposes.

Transition is indeed painful.  RFC 4876 profiles and in particular the 
attributeMap mechanism can assist; you can use an attributeMap entry to direct the 
clients to use an alternative, case-sensitive, attribute for the user name if that 
is what you need.

> next 12 months and give it its own RFC number.

I recommend looking pretty hard at whether a new attribute is needed, given that 
there seem to be three other existing attributes.

In general, it seems best to discourage making password hashes visible - perhaps 
even to the point of not standardizing them.

> will differ from one firm to the next.  The equivalent to "gecos" is an LDAP object.

Sorry, my side note ended up dominating the conversation.  I agree that 
overloading information into the gecos field is undesirable and that LDAP 
attributes should be used.

My key point here is that DBIS should specify what attribute should replace it, 
what attribute clients are expected to use to populate the pw_gecos field in 
struct passwd.

> gecos field, and that is what swayed my choice of inetOrgPerson here.

It seems like the right answer there would be a subclass of "account" with a "MAY 
displayName", to preserve the database normalization that lets you have accounts 
that are associated with more than one person (or with no person at all) and 
people who are associated with more than one account.

(But I see that inetOrgPerson includes "uid", suggesting that it was also intended 
to represent a computer account.)

> But that doesn't allow for GID number.

A subclass does.

> follow a similar line as "exactName" (en).

Again, UNIX is not the entire world.  RFC 2307 is not the primary reference for 
"uid" - RFC 2798 (inetOrgPerson) is, adopted and renamed from RFC 1274 "userid".  
(I presume that the earlier publication in RFC 2307 is an accident of the 
development timelines of the two documents.)

Yes, it's confusing for UNIX people, but it's the standard.

> to a DBIS schema, and allowing both "owner" and "manager" seems excessive.

No strong opinion.

> data better, so let's represent it better.

Yes, it can... and for most purposes it has, in the form of groupOfNames.  If you 
have users represented in the directory, the natural way to group them is with 
groupOfNames.  If you have hosts represented in the directory, the natural way to 
group them is with groupOfNames.  The only thing that there isn't an obvious way 
to group is "user X on host Y".  Not that I've seen a lot of concrete instances of 
netgroups, but I don't think I've seen one of those yet.

I believe Active Directory makes extensive use of groups of computers; 
interoperating seems desirable.

> 100% compatible with the NIS schema, as previously discussed.

Having two names that differ only in case lead to different port numbers seems 
like a spectacularly bad idea.  If there's an organization that has somehow found 
themselves in such a mess then I'm very sorry for them, but the real answer is 
that they need to extricate themselves from the mess rather than spreading it.

IANA is the reference for service names, and RFC 6335 says that they are 
case-insensitive.

> class anyway, and calling it oncRpcObject seemed unnecessary when rpcObject was

Although the title of RFC 5531 is "RPC: ...", the opening sentence says "This 
document specifies version 2 of the message protocol used in ONC Remote Procedure 
Call (RPC).".

XML-RPC seems to have somehow dodged getting an RFC, but is referenced by RFC 3529.
JSON-RPC seems to have at least something of a footprint, but no RFC.

> classes to the same entry.

You have ipv4HostObject and ipv6HostObject defined as structural classes, and a 
given object can have only one structural class.

> A host with no IP address, if we need to support that, I suggest

Yes.

We should also look at prior art.  Active Directory defines schemas for 
representing computers; it would seem remiss to not attempt to interoperate.

Again, that's if we need to include a "hosts" replacement at all.

(One reason to include one:  groups of hosts.)

> I agree, but DBIS must be backwards compatible.

When we're creating entirely new schema, backwards compatibility is not an 
absolute constraint.  Discarding evolutionary dead ends has substantial value.

> You're right, but why would you do that?

I blindly assumed that the existing schema made sense :-)

Indeed, the underlying file-based automount schema doesn't seem to allow different 
locations to have different options.