Subject: Re: Schema: encrypted 8-bit userPassword and SASLprep

From: John McMeeking <[email-masked]>
Date: Fri, 12 Sep 2003 09:27:50 -0500

To perform a bind, the client application gets a password in some form --
I'll assume some sort of string input -- and it has to create a binary
"credentials" to send to the server.  The input bytes might be US ASCII,
EBCDIC, some multi-byte (or 8-bit) character set, maybe a Java String.  So
in general, the client side must perform some sort of conversion from the
input string to that binary "credentials".  What is the target of the
conversion?  This is LDAP, so UTF-8 is a reasonable choice.  Assuming that
we don't want UTF-8, what other choices do we have?  In particular, what
choices do we have that don't require the client to know something about
the server implementation?  I don't see any.  And I don't think it is
reasonable, in general, for a client to need to know that the server is
using a Unix /etc/password file that contains hashed 8-bit code page
passwords (as an example).

I'll concede that there may be cases where a client needs to bind to a
server using /etc/password with some 8-bit code page instead of UTF-8.  Two
ways to handle that come to mind:
1) restrict passwords to the 7-bit character set so that UTF-8 conversion
is not a factor
2) provide some option in the client that allows an application that "knows
what it is doing" to provide binary credentials that the client wraps in an
octet string for the bind request.

I think those are compatible with the proposed "the application SHOULD
prepare..." wording.

I also think it is reasonable to extend the code page conversion discussion
to SASLprep, though I admit to a great deal of ignorance in that area.  My
understanding is that there are multiple ways of representing some
characters in Unicode.  Just like EBCDIC and ASCII produce different byte
representations for 'A', different input sources might produce different
Unicode representations of a given character.  SASLprep attempts to ensure
that those different representations are normalized so that users typing in
the same string from different sources will send the same password to the
server.  A server might be able to do this normalization for simple bind,
but not for something like DIGEST-MD5.

At least that's how I understand this.

John  McMeeking

                      Hallvard B Furuseth                                                                                   
                      <[email-masked]        To:       Michael Ströder <[email-masked]>                          
                      io.no>                      cc:       [email-masked]                                       
                      Sent by:                    Subject:  Re: Schema: encrypted 8-bit userPassword and SASLprep           
                      owner-ietf-ldapbis@O                                                                                  
                      penLDAP.org                                                                                           

                      09/12/2003 08:10 AM                                                                                   

Michael Ströder writes:
>Hallvard B Furuseth wrote:
>> [Schema] 2.41 (userPassword) says:
>>
>>    The application SHOULD prepare textual strings used as passwords
>>    by transcoding them to Unicode, applying SASLprep [SASLprep], and
>>    encoding as UTF-8.
>>
>> This is incompatible with passwords written in 8-bit character sets and
>> stored encrypted in files that cannot easily be decrypted, e.g. Unix
>> /etc/passwd files.  Since the server does not know the plaintext
>> passwords, it cannot prepare them as above.
>
> IMHO the password should be encoded as UTF-8 before applying the hash.
> Maybe the wording is somewhat misleading though I'm not sure.

No.  LDAP can't mandate how Unix should encode its /etc/passwd files,
and therefore also how the login, su and passwd programs should work.

> Note that hashed passwords are non-standard anyway... ;-)

No, they are irrelevant to the standard:
- The client binds with a plaintext password (hopefully inside TLS).
  So the hashed password doesn't affect the protocol.
- The server matches the password from the client against the hashed
  password, with the same algorithm as the login program does.

--
Hallvard

---

From: Hallvard B Furuseth <[email-masked]>
Date: Fri, 12 Sep 2003 17:44:30 +0200

Since [Schema] says userPassword must be stored unencrypted, I'll switch
to talk about this text in [Protocol] 4.2 (Bind Operation), and passwords
stored encrypted in the server in some other way than userPassword:

     To improve matching, applications SHOULD prepare textual strings
     used as passwords. Applications which prepare textural strings used
     as password are REQUIRED to prepare them by transcoding the string
     to [Unicode], apply [SASLprep], and encode as UTF-8.

If we do something about that text, we can align userPassword with it
later.

BTW, it's unclear whether or not the above text only applies to simple
auth, since the previous sentence is

     The simple form of an AuthenticationChoice specifies a simple
     password to be used for authentication.

That should be clarified.

Well.  Onwards ---
John McMeeking writes:

> input string to that binary "credentials".

True.

> What is the target of the conversion?  This is LDAP, so UTF-8 is a
> reasonable choice.

I didn't think of that.  You may be right, but I think not - not as the
only option, anyway.  That would have been nice, but the server still
cannot convert the stored password to UTF-8 if it only knows an
encrypted form of the password.  Which is probably is rather common.  So
if it receives an UTF-8 password from the client, it must instead
convert that to the encoded character set of the stored passwords and
then match the passwords.  Unfortunately I believe the mapping from
Unicode to some character sets/encodings is 1->many or many<->many, so
the resulting passwords may not match.

Applying Strprep before binding makes this worse, since Strprep e.g.
removes some characters from the password.  The server can't put them
back before matching.

> Assuming that we don't want UTF-8, what other choices do we have?

I think this will have to be chosen by the sysadmin, but:

Another option is to treat passwords as raw binary strings.  The user
can bind with the same password that he originally set, as long as he is
using the same character set, or at least no characters that differ in
the different character sets in use.  He can't bind from somewhere using
a totally different character set, but at least it will work properly on
campus.

With EBCDIC, I suppose a third option is to treat characters common to
EBCDIC and ASCII specially.  I don't want to think about that; maybe
someone who knows EBCDIC will say something:-)

> In particular, what choices do we have that don't require the client
> to know something about the server implementation?  I don't see any.

Me neither.  And that includes the current wording.

> And I don't think it is reasonable, in general, for a client to need
> to know that the server is using a Unix /etc/password file that
> contains hashed 8-bit code page passwords (as an example).

It doesn't need to know that.  What it needs to know in that case is
that passwords on the server side are encoded the same way as on the
client side.  (To be sure, they are hashed on the server side, but the
server takes care of that detail.)

> server using /etc/password with some 8-bit code page instead of UTF-8.
> Two ways to handle that come to mind:

Can only be done only be done if the passwd program, or whatever users
use to change their passwords, is written by someone with this problem
in mind - e.g. someone on campus.  Even so, it may not be compatible
with EBCDIC passwords.

Another option:

1.5) If one does have a local password-changing program, let it produce
both the hash that the system needs and the UTF-8 strprep'ed hash that
LDAP needs.

> 2) provide some option in the client that allows an application that
> "knows what it is doing" to provide binary credentials that the client
> wraps in an octet string for the bind request.

Yes, that's more or less what I was suggesting.  Don't say that
applications SHOULD translate passwords unless we also say that they
SHOULD provide an option not to do so, and/or to do some implementation-
defined conversion (like EBCDIC<->ASCII).

Though I'm not sure UTF-8/strprep is a good default.  It may be better
to just say that it should provide these options.

> I think those are compatible with the proposed "the application SHOULD

Formally, yes.  In practice, I'd expect that wording to lead to a lot of
clients that do prepare and don't provide an option not to do so.

-- 
Hallvard

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 12 Sep 2003 09:27:40 -0700

>stored encrypted in the server in some other way than userPassword:

I think we do need to switch this to a [Protocol]/[AuthMeth] discussion.

I think the WG has accepted (long ago) that simple bind may be used
with password attributes other than userPassword (e.g., authPassword)
and/or with external password stores.

Kurt, LDAP co-chair

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 12 Sep 2003 09:46:00 -0700

>     to [Unicode], apply [SASLprep], and encode as UTF-8.

s/application/client/

>That should be clarified.

I concur.  The statement above applies to the simple authentication method.
Other methods have their own requirements as stated in their technical
specification.

Kurt

---

From: Hallvard B Furuseth <[email-masked]>
Date: Tue, 23 Sep 2003 15:20:09 +0200

> I think we do need to switch this to a [Protocol]/[AuthMeth] discussion.

Hm.  It got very quiet here.  Should I mention that all of that message
was written in the spirit of addressing the [Protocol] paragraph about
saslprep, rather than userPassword?

I started this thread by confusing userPassword with auth passwords, and
I think I succeeded in confusing John about it too - or if not, his
message could at least be read as if I had:-) So I just went ahead and
changed the topic of the reply.

> and/or with external password stores.

Yes, that was what I was assuming this time.

-- 
Hallvard