Subject: draft-ietf-sasl-crammd5-01.txt

From: Simon Josefsson <[email-masked]>
Date: Mon, 01 Dec 2003 23:39:51 +0100

Hello, I have read the document.  Some comments:

,----
|      The client MUST prepare the user name and shared secret strings
|      using the [SASLPrep] profile of the [StringPrep] algorithm.  The
|      resulting values MUST be encoded as UTF-8 [UTF8].
`----

I suggest adding:

      The server MUST prepare the shared secret string using the
      [SASLPrep] profile of the [StringPrep] algorithm.  The resulting
      value MUST be encoded as UTF-8 [UTF8].  The server may store the
      prepared string instead of, or as well as, the unprepared
      string, so that it does not have to prepare it every time it is
      needed for computation.  However, if the original, unprepared
      string, is not stored, it may render the computed secret to be
      incompatible with a future revisions of SASLPrep that support
      currently unassigned code points (compare section 7 of
      [StringPrep]).  It is therefor recommended to store the
      unprepared string in the database.

Also, the example section is incorrect.  Suggested new version:

     S: * OK [CAPABILITY IMAP4rev1 STARTTLS LOGINDISABLED AUTH=CRAM-MD5]
     C: A0001 AUTHENTICATE CRAM-MD5
     S: + PDE4OTYuNjk3MTcwOTUyQHBvc3RvZmZpY2UuZXhhbXBsZS5uZXQ+
     C: am9lIDNkYmM4OGYwNjI0Nzc2YTczN2IzOTA5M2Y2ZWI2NDI3
     S: A0001 OK CRAM-MD5 authentication successful

     In this example, the shared secret is the string

          tanstaaftanstaaf

     Hence, the Keyed MD5 digest is produced by calculating

          MD5((tanstaaftanstaaf XOR opad),
               MD5((tanstaaftanstaaf XOR ipad),
               <[email-masked]>))

     where ipad and opad are as defined in [KEYED-MD5] and the string
     shown in the challenge is the base64 encoding of
     <[email-masked]>. The shared secret is
     null-padded to a length of 64 bytes. If the shared secret is longer
     than 64 bytes, the MD5 digest of the shared secret is used as a 16
     byte input to the keyed MD5 calculation.

     This produces a digest value (in hexadecimal) of

          3dbc88f0624776a737b39093f6eb6427

     The user name is then prepended to it, forming

          joe 3dbc88f0624776a737b39093f6eb6427

     Which is then base64 encoded to meet the requirements of the IMAP4
     AUTHENTICATE command (or the similar POP3 AUTH command), yielding

          am9lIDNkYmM4OGYwNjI0Nzc2YTczN2IzOTA5M2Y2ZWI2NDI3

Perhaps it would be useful to add an example for non-ASCII strings
too, using <U+4711> or similar to indicate non-ASCII characters.

Thanks,
Simon

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 01 Dec 2003 18:05:02 -0500

>       unprepared string in the database.

The correct way to handle this situation is to require that if the server 
stores the prepared string (or maybe, even if not), the server prepare the 
string with AllowUnassigned set to FALSE, thereby prohibiting unassigned 
code points.  If the server stores the prepared string, then it is properly 
a "storage string" and should not be permitted to contain unassigned code 
points.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA

---

From: Sam Hartman <[email-masked]>
Date: Tue, 02 Dec 2003 14:31:20 -0500

>>>>> "Jeffrey" == Jeffrey Hutzelman <[email-masked]> writes:

    Jeffrey> The correct way to handle this situation is to require
    Jeffrey> that if the server stores the prepared string (or maybe,
    Jeffrey> even if not), the server prepare the string with
    Jeffrey> AllowUnassigned set to FALSE, thereby prohibiting
    Jeffrey> unassigned code points.  If the server stores the
    Jeffrey> prepared string, then it is properly a "storage string"
    Jeffrey> and should not be permitted to contain unassigned code
    Jeffrey> points.

I agree.  I believe cram-md5 should always treat the secret as a
storage string when it is updating its database.

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 01:21:04 +0100

> storage string when it is updating its database.

I'm having trouble understanding what these statements imply for the
protocol.  Perhaps my trouble is because the discussion is now focused
on the database, and not on the protocol.

Do you mean that servers will be _required_ to use passwords prepared
with AllowUnassigned false in the HMAC-MD5 computation?

Regardless of whether the prepared string is ever stored in a
database?

If the answer is yes, I don't agree, because that would make CRAM-MD5
reject passwords containing characters introduced by Unicode 4.0.0.

Regards,
Simon

PS. Forward compatibility issues with StringPrep profiles are
discussed in the recent draft-faltstrom-unicode-synchronisation-00.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Tue, 02 Dec 2003 20:09:44 -0800

> Regardless of whether the prepared string is ever stored in a

First, I think "in a database" is a unnecessary qualification.
All that matters is that it is a "stored" string per stringprep.

In each of the password-based mechanisms (PLAIN, CRAM-MD5,
DIGEST-MD5, etc.), the user provided string is clear a "query"
string and the string which it is compared against (directly
or indirectly) is a "stored" string (regardless of whether
the server stores it locally or in some external systems).

> Perhaps my trouble is because the discussion is now focused

I do think that it should be considered an implementation detail
whether the server applies SASLprep with unassigned disallowed
before or after use storage.  What matters to the protocol/algorithm
is the server apply the SASLprep with disallowed before comparison
to the "stored" string.  Generally, doing it before storage is
more appropriate.  However, there is no harm (and there may be
advantage in some instances (e.g., external password stores))
of doing it after fetching it from storage (BUT before comparison).

So, maybe we're agreeing...

>reject passwords containing characters introduced by Unicode 4.0.0. 

As intended!   Until stringprep is updated to detail how these
newly assigned code points are to be handled, we need to exclude
them from appearing in the "stored" password string.

Kurt

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 12:21:28 -0500

>>>>> "Simon" == Simon Josefsson <[email-masked]> writes:

    Simon> I'm having trouble understanding what these statements
    Simon> imply for the protocol.  Perhaps my trouble is because the
    Simon> discussion is now focused on the database, and not on the
    Simon> protocol.

    Simon> Do you mean that servers will be _required_ to use
    Simon> passwords prepared with AllowUnassigned false in the
    Simon> HMAC-MD5 computation?

I mean that this should be a requirement for interoperability in the
long term.  If you don't have this requirement, then you may run into
cases where saslprep eventually maps out those characters in the
future.

Basically, if we require allowunassigned to be false, we guarantee
future interoperability at the cost of refusing to allow people to use
unassigned characters until saslprep is updated.

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 14:17:39 -0500

>>>>> "Simon" == Simon Josefsson <[email-masked]> writes:

    Simon> If SASLPrepX is guaranteed to behave the same as SASLPrep
    Simon> for currently assigned characters, there is no problem in
    Simon> storing SASLPrep prepared strings in the database, and my
    Simon> text is incorrect.  However, in that case, I believe the
    Simon> guarantee that SASLPrepX(SASLPrep(string)) =
    Simon> SASLPrepX(string) should be stated in the SASLPrep
    Simon> document.

I believe this guarantee is stated in stringprep.

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 20:35:33 +0100

> I believe this guarantee is stated in stringprep.

I'm sorry to be dense, but can you be more specific?  I thought future
stringprep profiles were allowed to use Unicode 4.0.0, including the
correct normalization tables, but if SASLPrep2 would do so, the above
guarantee doesn't hold.

Thanks,
Simon

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 14:43:29 -0500

>>>>> "Simon" == Simon Josefsson <[email-masked]> writes:

    Simon> I'm sorry to be dense, but can you be more specific?  I
    Simon> thought future stringprep profiles were allowed to use
    Simon> Unicode 4.0.0, including the correct normalization tables,
    Simon> but if SASLPrep2 would do so, the above guarantee doesn't
    Simon> hold.

The problem here being that normalization changed between 3.2 and 4.0
in incompatible ways?

Well, that's sort of an open issue within the IETF because that wasn't
supposed to happen.  Stringprep assumes normalization never changes;
we assume that future versions of stringprep onlly change in handling
of unassigned codepoints.  Unicode has created a mess for us and we
have not decided how to deal.

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 19:56:03 +0100

> unassigned characters until saslprep is updated.

I don't see a conflict with this and the text I proposed, but I'm not
sure if you argue for or against the text.  In fact, the problem of
forward compatible databases was the motivation for my text.  The text
would help if SASLPrepX(SASLPrep(string)) != SASLPrepX(string) where
SASLPrepX is any future revision of SASLPrep.

If SASLPrepX is guaranteed to behave the same as SASLPrep for
currently assigned characters, there is no problem in storing SASLPrep
prepared strings in the database, and my text is incorrect.  However,
in that case, I believe the guarantee that SASLPrepX(SASLPrep(string))
= SASLPrepX(string) should be stated in the SASLPrep document.

> the server stores it locally or in some external systems).

This is not the only interpretation.  It is possible to consider the
SASL library to be performing a query against the HMAC-MD5 oracle to
decide whether a Unicode string the library received from the
application (possibly ultimately from a database), after the library
has prepared it with SASLPrep with AllowUnassigned set to true, is the
same secret used by the client.

> them from appearing in the "stored" password string.

The behaviour of inputting Unicode 4.0.0 characters should still be
well defined.  The current text say nothing about what the server
should do, so rejecting the input isn't the only alternative.

Thanks,
Simon

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 21:34:08 +0100

> The problem here being that normalization changed between 3.2 and 4.0

That's one example of the problem, yes.  In general, I didn't find any
guarantee that a future version of a stringprep profile must be
backwards compatible to the previous version.  For SASLPrep, that
could be useful, but I haven't found any place that say it must be so.
If there isn't, storing the unprepared string in databases appear to
be the safe recommendation.  (It might be anyway, if the database is
shared with other password infrastructures that doesn't use SASLPrep,
but that's besides the point.)

Thanks,
Simon

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 15:36:37 -0500

>>  The problem here being that normalization changed between 3.2
    >> and 4.0 in incompatible ways?

    Simon> That's one example of the problem, yes.  In general, I
    Simon> didn't find any guarantee that a future version of a
    Simon> stringprep profile must be backwards compatible to the
    Simon> previous version.  For SASLPrep, that could be useful, but
    Simon> I haven't found any place that say it must be so.  If there
    Simon> isn't, storing the unprepared string in databases appear to
    Simon> be the safe recommendation.  (It might be anyway, if the
    Simon> database is shared with other password infrastructures that
    Simon> doesn't use SASLPrep, but that's besides the point.)

It's my understanding that the entire IETF internationalization
architecture as it depends on stringprep depends on this guarantee.
Sort of unfortunate that it is not stated.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 03 Dec 2003 13:05:45 -0800

>same secret used by the client.

Well, maybe the language I used was not clear enough.

My point is that the mechanism specification must clearly
state which strings are query strings and which strings are
stored strings and ensure that two query strings prepared
by different implementations are not (directly or indirectly)
compared.

The client copy of the user name and shared secret should
be defined as query strings and, hence, unassigned code points
are allowed to appear in the prepared output.  The server
copy of the user name and shared secret should be viewed
as stored strings and, hence, unassigned code points are not
allowed to appear in the prepared output.

The draft needs to be revised to make this clear.

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 03 Dec 2003 14:27:23 -0800

In consideration of the on-going CRAM-MD5 unassigned code point
discussions, I reviewed draft-ietf-sasl-plain-xx.txt handling
in this area and found that a few additional words would add
clarity in this area.

Hence, I'd like to make the following change:
@@ -151,10 +151,12 @@
 The presented authentication identity and password strings are not
 to be compared directly with stored strings.  The server SHALL first
 prepare authentication identity and password strings using the
 [SASLPrep] profile of the [StringPrep] algorithm.
+Per Section 7 of [StringPrep], unassigned code points may appear in
+prepared presented (query) strings but are prohibited in stored strings.
 If preparation fails or results in an empty string, verification SHALL
 fail.  If the server stores only the hash of expected string, the
 string MUST be prepared before generation of the hash.

(+ indicates new text)

Additionally (as previously discussed), I'll update the pseudo
code such that SASLprep() takes a second argument as follows:

+The second parameter of the SASLprep function, when true, indicates
+that the input string is a "query" string [StringPrep] and, hence,
+unassigned code points are allowed.  When the SASLprep function is
+called to prepared the password prior to computing the stored hash,
+the second parameter would be false as unassigned code points are
+not allowed in "stored strings" [StringPrep].

Kurt

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 03 Dec 2003 18:37:13 -0500

> SASLPrepX is any future revision of SASLPrep.

Yeah; that's supposed to be a guarantee.  Part of this is spelled out in 
RFC2454, at the end of section 3 and in section 7.1:

   Authors of profiles of this document need to consider the effects of
   changing the mapping of any currently-assigned character when
   updating their profiles.  Adding a new mapping for a currently-
   assigned character, or changing an existing mapping, could cause a
   variance between the behavior of systems that have been updated and
   systems that have not been updated.

   A subsequent version of a profile that references a newer version of
   a repertoire with new code points will inherently have some code
   points move from category U to either D, MN, or AO.  For backwards
   compatibility, a subsequent version of a profile MUST NOT move code
   points from any other category.  That is, current AO, MN, or D code
   points MUST NOT ever change to a different category.

The rest comes from Unicode itself, which guarantees that normalizations 
for existing code points will not change.  Sadly, it appears the UTC has 
been willing to violate that guarantee in some cases.

> same secret used by the client.

This critical issue is that for servers where a partial MD5 context is 
stored based on the shared secret, the secret MUST first be prepared 
according to SASLprep.  In this case, the secret needs to be treated as a 
storage string -- there's no way to reverse the context establishment, so 
if the string was not normalized correctly to begin with, you can't go back 
and fix it when you upgrade.  Thus, unassigned code points MUST NOT be 
permitted in such a case.

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 04 Dec 2003 19:07:19 -0800

> The draft needs to be revised to make this clear.

I'm having no luck coming up with text that explains this clearly. And 
I'm still not convinced that this should be discussed in the mechanism 
spec, since it applies much more widely than just CRAM. 2222bis seems 
to me to be a better place to lay this out.

That said, I have to rev the CRAM draft again for other (editorial) 
reasons, so if someone can come up with a diff to -01 I'll incorporate 
it. (Or is Kurt's paragraph above clear enough?)

--lyndon

---

From: Sam Hartman <[email-masked]>
Date: Fri, 05 Dec 2003 11:14:38 -0500

>>>>> "Lyndon" == Lyndon Nerenberg <[email-masked]> writes:

    Lyndon> I'm having no luck coming up with text that explains this
    Lyndon> clearly. And I'm still not convinced that this should be
    Lyndon> discussed in the mechanism spec, since it applies much
    Lyndon> more widely than just CRAM. 2222bis seems to me to be a
    Lyndon> better place to lay this out.

I disagree.  How to prepare authentication identities is entirely a
mechanism matter.  The base spec can point out the general concern but
cannot constrain what the mechanism does.

For example various GSSAPI mechanisms may do much more complicated
stringprep.  Kerberos certainly will.

---

From: Simon Josefsson <[email-masked]>
Date: Sun, 07 Dec 2003 05:30:47 +0100

Jeffrey Hutzelman <[email-masked]> writes:

> Yeah; that's supposed to be a guarantee.  Part of this is spelled out
> in RFC2454, at the end of section 3 and in section 7.1:
>

This is not much of a guarantee.  To me, the wording suggest that
stringprep profiles actually can be backwards incompatible, if the
designers feel the benefits outweigh the costs that are mentioned.

>    points MUST NOT ever change to a different category.

This is good text, and I thought it would take of this problem.
However, it doesn't appear to be that simple, because it continue to
say:

   Applications creating queries MUST treat U code points as if they
   were AO when preparing the query to be entered in the process
   described by a profile of stringprep.  Those applications MAY
   optionally have a preprocessor that provide stricter checks: treating
   unassigned code points in the input as errors, or warning the user
   about the fact that the code point is unassigned in the version of a
   profile that the software is based on; such a choice is a local
   matter for the software.

So the result is that if the profile is used with AllowUnassigned set
to true, the computed strings are not necessarily forwards compatible.

Furthermore, one of the three ways forward presented in
draft-faltstrom-unicode-synchronisation-00, to support new Unicode
versions, include breaking backwards compatibility in stringprep
profiles.

To summarize, I don't believe the SASLPrep2(SASLPrep1(string)) ==
SASLPrep2(string) guarantee hold universally, nor follow from the core
document.

This problem can be solved by adding a discussion in the SASLPrep
document that contain some guarantee that will hold for future
versions of the SASLPrep profile.  Perhaps even explicitly mention
which path in draft-faltstrom-unicode-synchronisation-00 SASLPrep will
follow.

Alternatively, you could recommend applications to store the
unprepared original passwords, which will be safe regardless of what
SASLPrep will behave in the future.  I prefer this approach, but
judging from the responses to my proposed text for CRAM-MD5, perhaps
there isn't consensus about that.

Finally, each mechanism using SASLPrep should also say when and how
AllowUnassigned is set.

Thanks,
Simon

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 08 Dec 2003 00:44:59 -0500

> designers feel the benefits outweigh the costs that are mentioned.

Yes, I believe that's exactly what it says.  It may not seem like much of a 
guarantee, but it's the best we can possibly have.  Designers of future 
versions of a profile will always be able to break backward compatibility, 
no matter what we do -- no "requirement" we write now can be effective 
against an IETF concensus in the future.

> to true, the computed strings are not necessarily forwards compatible.

That is correct - if you apply stringprep with AllowUnassigned set to TRUE, 
there is no guarantee that two implementations will produce the same result 
given the same input.  But that's OK for a query string that's going to be 
used "once", right now -- the worst thing that can happen is that it 
doesn't match when it "should".

However, for a string that's going to be stored long-term, preparing it 
incorrectly now is fatal; it will then _NEVER_ match a correctly prepared 
string.  If there are unassigned code points, then you can't know if you're 
preparing it correctly; the easiest solution to this problem is to raise an 
exception now, and refuse to store a string if you don't know how to 
prepare it correctly.  That's what setting AllowUnassigned to FALSE does. 
So I'll go back to my original statement -- IF a server is going to prepare 
strings in advance (which it cannot avoid if it's going to precompute a 
partial MD5 context), then it MUST set AllowUnassigned to FALSE and reject 
strings which contain unassigned code points.

> versions, include breaking backwards compatibility in stringprep

I've not had a chance to read that document yet, so I can't comment on it.

> which path in draft-faltstrom-unicode-synchronisation-00 SASLPrep will

We can't make an effective guarantee against a future IETF concensus.  But 
given the text in RFC2454, I do agree we need to address the issue, and 
express the intent that the mappings of code points not in class U will not 
change in future versions of the specification.

> there isn't consensus about that.

I get the impression that people like the idea of storing a partial MD5 
context, even though doing so doesn't offer much protection.  If servers 
are going to do this, I think they should be REQUIRED to prepare with 
AllowUnassigned set FALSE.  OTOH, if they're going to store passwords, I 
agree that storing unprepared passwords allows for the greatest chance of 
future interoperability.

> Finally, each mechanism using SASLPrep should also say when and how

Absolutely.

-- Jeff

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 01 Dec 2003 18:05:02 -0500

---

From: Sam Hartman <[email-masked]>
Date: Tue, 02 Dec 2003 14:31:20 -0500

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 01:21:04 +0100

Regards,
Simon

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Tue, 02 Dec 2003 20:09:44 -0800

Kurt

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 12:21:28 -0500

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 14:17:39 -0500

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 20:35:33 +0100

Thanks,
Simon

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 14:43:29 -0500

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 19:56:03 +0100

Thanks,
Simon

---

From: Simon Josefsson <[email-masked]>
Date: Wed, 03 Dec 2003 21:34:08 +0100

Thanks,
Simon

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Dec 2003 15:36:37 -0500

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 03 Dec 2003 13:05:45 -0800

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 03 Dec 2003 14:27:23 -0800

(+ indicates new text)

Kurt

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 03 Dec 2003 18:37:13 -0500

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 04 Dec 2003 19:07:19 -0800

--lyndon

---

From: Sam Hartman <[email-masked]>
Date: Fri, 05 Dec 2003 11:14:38 -0500

---

From: Simon Josefsson <[email-masked]>
Date: Sun, 07 Dec 2003 05:30:47 +0100

Thanks,
Simon

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Mon, 08 Dec 2003 00:44:59 -0500

Absolutely.

-- Jeff