Subject: RE: LDAP Certificate transfer syntax

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 03 Apr 2002 13:40:27 -0800

At 12:32 PM 2002-04-03, Sharon Boeyen wrote:
>>From the PKIX perspective, I firmly believe that backward compatibility 
>with the PKIX LDAP specs is a very important issue.

I think there is some confusion here as to regards to
"compatibility".  This is an update/revision of an LDAPv3
technical specification.  We are implicitly talking about
compatibility of (conformant) implementations of the
existing LDAPv3 technical specification and future
(conformant) implementations of a future LDAPv3 technical
specification.

LDAPv2 and LDAPv3 are distinct on-the-wire protocols.
Directory add, delete, modify, rename, or search operations
either has LDAPv2 syntax and semantics or has LDAPv3 syntax
and semantics.

Here we are talking about changes to LDAPv3 technical
specification and how they would or wouldn't affect LDAPv3
implementations.

These changes have ZERO impact on LDAPv2 implementations.

>I believe that what 
>David is proposing satisfies that important criteria and support the 
>proposal.
In David's table, he shows that existing LDAPv3 implementations
are "OK".   That is, LDAPv3 is not broken in this regard.  The
specification is clear and there multiple interoperable
LDAPv3 implementations.

Removing the "MUST" request and use certificate attributes using
;binary will cause interoperability problems and will force
unnecessary changes to BOTH LDAPv3 clients and LDAPv3 servers.
This is because LDAPv3 does not provide any mechanism for
an LDAP peer (client or server) to discover which options its peer
(server or client) to discover which attribute descriptions
it supports.

That is, this suggest change will implementations to updated to
support it.  This will include some clients (such as those which
as for "*" and expect userCertificate;binary) and most (if not
all) servers.  That's bad!

Hence, for LDAPv3 interoperability sake, the specification needs
to continue to MUST the use of ;binary for these attributes except
where the client and server have a prior agreement that the
OPTIONAL native encoding is to be used.

Of course, one ought to wonder why any LDAPv3 implementation
would support this OPTIONAL native encoding when the REQUIRED
encoding is more broadly implemented and provides the same
functionality.   It seems redundant to me.

Kurt

---

From: David Chadwick <[email-masked]>
Date: Wed, 03 Apr 2002 23:44:45 +0100

> These changes have ZERO impact on LDAPv2 implementations.

Agreed

> specification is clear and there multiple interoperable

But LDAP (v2 and v3) is deficient in that it has never specified a
workable "native" transfer encoding for certificates. The LDAPv2 attempt
was broken as we know, and v3 never replaced it - until my proposed text
now. Are certificates the only commonly used attributes without a
"native" encoding? I cant think of any others. So this is a deficiency
that should be rectified in my opinion.

> Removing the "MUST" request 

This is a natural consequence of defining a native encoding.
Clients now have a choice of transfer encoding to ask for. It just so
happens that in the case of certificates both native and ;binary
encodings happen to be the same (fortunately for signature checking :-)

>and use certificate attributes using

No, it wont necessitate any changes to clients. Once the LDAPv3 servers
make the change, this is backwards compatible and will work with any v3
client. The servers can accept any request, with or without ;binary, and
if they want can always return ;binary in their responses to cater for
both types of client.

> (server or client) to discover which attribute descriptions

This is not a certificate problem per se, this is a generic LDAP problem
isnt it?

> as for "*" and expect userCertificate;binary) 

agreed, if a server is to be consistent it probably would not put
;binary on any returned attributes, although I believe that todays
LDAPv3 servers will put ;binary on certificates because of the MUST
requirement. If they continued to do this in the future it would not
cause any problems with the clients, but would be inconsistent
behaviour.

>and most (if not

I think the change would not have been necessary if all existing
products had implemented the current spec. And thats bad!!

> OPTIONAL native encoding is to be used.

As a general rule I think that continuing to mandate one particular
tranfer encoding just for certificates is wrong and leaves us in hole in
the future if new transfer encodings are developed, such as XML (which
incidentally has now been standardised for certificates). So how would
you propose to cater for XML encodings in the future?

> functionality.   It seems redundant to me.

Clearly there is redundancy once a native encoding is specified and this
happens to be the same as the ;binary encoding, that is not in doubt.
But who is to say which one is the redundant one? It could be the use of
;binary. Maybe this transfer encoding is no longer needed by LDAPv3
(just being devil's advocate here :-)

David

-- 
*****************************************************************

David W. Chadwick, BSc PhD
Professor of Information Systems Security
IS Institute, University of Salford, Salford M5 4WT
Tel: +44 161 295 5351  Fax +44 161 745 8169
Mobile: +44 77 96 44 7184
Email: [email-masked]
Home Page:  http://www.salford.ac.uk/its024/chadwick.htm
Research Projects: http://sec.isi.salford.ac.uk
Understanding X.500:  http://www.salford.ac.uk/its024/X500.htm
X.500/LDAP Seminars: http://www.salford.ac.uk/its024/seminars.htm
Entrust key validation string: MLJ9-DU5T-HV8J
PGP Key ID is 0xBC238DE5

*****************************************************************

---

From: "Steven Legg" <[email-masked]>
Date: Thu, 4 Apr 2002 12:21:40 +1000

David,

David Chadwick wrote:
> Sent: Thursday, 4 April 2002 8:45
> To: Kurt D. Zeilenga
> Cc: Sharon Boeyen; 'Mark Wahl'; Mark C Smith; LDAP BIS; PKIX
> Subject: Re: LDAP Certificate transfer syntax

> "native" encoding? I cant think of any others.

I commonly use entryACI, prescriptiveACI and subentry ACI (ACI Item
syntax) and subtreeSpecification (Subtree Specification syntax).
Neither of the syntaxes for these attributes has a defined
LDAP-specific (i.e. native) encoding in RFC 2252 or RFC 2256.

> So this is a deficiency

Recent I-Ds have proposed human readable character string encodings
for both the ACI Item syntax and the Subtree Specification syntax,
by referencing the Generic String Encoding Rules, to correct the
deficiency in these syntaxes. This is in keeping with the statement
in RFC 2252 that "The encoding rules defined for a given attribute
syntax must produce octet strings. To the greatest extent possible,
encoded octet strings should be usable in their native encoded form
for display purposes".

Thus a "right" way to fix the Certificate syntax is to define
the LDAP-specific/native encoding to be the GSER encoding :-).

Had the ACI Item and Subtree Specification syntaxes previously been
defined such that their LDAP-specific encoding was the same as
their ;binary encoding it would not now be possible to define the
default format to be a human readable character string encoding.

> products had implemented the current spec. And thats bad!!

The fault is in the incorrectly implemented products. The obligation
to change is on them, not the correctly implemented products.

> you propose to cater for XML encodings in the future?

One way is to provide an update to the Certificate syntax to say that
values in the syntax MUST NOT be requested or returned without an
explicit transfer option. We could also say that now, in the current
draft.

> (just being devil's advocate here :-)

Only about a third of the syntaxes I support have defined LDAP-specific
encodings. I need a standard way to represent attribute values of the
other two thirds in LDAP and LDIF. The ;binary option serves that
purpose. Now if someone were to mandate that all additional LDAP syntaxes
must use GSER for the LDAP-specific encoding then I wouldn't need ;binary.

Regards,
Steven

---

From: David Chadwick <[email-masked]>
Date: Thu, 04 Apr 2002 14:09:26 +0100

Steven

thanks for your messages that have clearly stated the case that ;binary
is needed as a general transfer encoding, and that many attributes exist
without a native LDAP encoding being defined for them.

But this does raise the more general issue of how does a user who asks
for "all" attributes, deal with those which are returned, whose native
encoding he is unfamiliar with. Does the server assume the client knows
(in which case ;binary will only be used on those attributes with no
native encoding defined) or does not know (in which case ;binary will
need to be used on all attributes that are not defined in Internet
standard RFCs).

Seems like ASN.1 DAP had some advantages after all :-)

David

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 04 Apr 2002 09:22:58 -0800

>need to be used on all attributes that are not defined in Internet

This is being addressed, in general, in the LDAPbis protocol I-D
(draft-ietf-ldapbis-protocol-xx.txt).

I think we have consensus on the handling of options, including
transfer options, in general.  However, IIRC, Jim needs to revise
the text slightly to address a couple of cases (such as handling
of '*').

I think it very important certificate schema not be a 'special
case'.   I am concerned that adding a 'native' encoding to
certificate schema will necessity the need for a 'special case'.
In particular, in the handling of '*' as the general handling
is to return the native encoding (no transfer option) if one
is defined and supported.

>Seems like ASN.1 DAP had some advantages after all :-)

DAP, in many ways, is lighter than LDAP.  :-(

Kurt

---

From: Mark Wahl <[email-masked]>
Date: Thu, 04 Apr 2002 14:31:06 -0600

David Chadwick wrote:
> encoding he is unfamiliar with. 

An application which asks for all attributes will need to expect to see
returned in the search result entry some attributes whose syntax OID is not 
one known to it.

RFC 2256 section 4.3.1 states that clients which request that all 
attributes be returned from entries MUST be prepared to receive values
in binary (e.g. userCertificate;binary) and SHOULD NOT simply display
binary or unrecognized values to users.

What the application does with this is analogous to an email or web client 
that receives a Content-Type: application/x-foo-bar.  It might
 - ignore it, 
 - report an error,
 - save the bytes to the file,
 - attempt to 'guess' (e.g. if all the bytes are in a pattern that matches
   the UTF-8 rules, then maybe it is UTF-8).

> Does the server assume the client knows (in which case ;binary will only
> be used on those attributes with no native encoding defined) or does not 
> know (in which case ;binary will need to be used on all attributes that 
> are not defined in Internet  standard RFCs).

A server knows nothing about client's schema capabilities.  There is no 
operation for a client to 'upload' a syntax table to the server.  The
;binary is used when the client requests it, either on the Add or Modify
which introduces the attribute and value to the entry, or when the client
requests it in the search's attribute description list.

Mark Wahl
Sun Microsystems Inc.

---

From: David Chadwick <[email-masked]>
Date: Thu, 04 Apr 2002 23:02:08 +0100

"Kurt D. Zeilenga" wrote:

> case'.

Yes I agree with this. I think there really should be no special cases,
and that the core LDAPv3 specs should be able to cater for all
eventualities, such as native encoding exists, native encoding does not
exist, how to handle "any" in both cases etc.

>  I am concerned that adding a 'native' encoding to

The special case will be needed by the enhanced LDAPv3 server that
understands the certificate native encoding, so does not need to provide
;binary in the response, but might do so as to be backwards compatible
with clients that expect ;binary (ie. do not know the native encoding).
But this special case is not really so special anyway, because

a) is not mandatory so the server does not have to keep this backwards
compatibility if it does not want to and
b) this case is no different to a server that knows about a private
attribute "foobar" with a locally defined native encoding, when
returning this to a remote client from another domain. The server does
not have to provide ;binary encoding, but ought to because the remote
client wont know the native encoding. This is why I said that ;binary
can only be ignored for attributes whose native syntax is defined in an
RFC, and is internationally known, but not for locally defined
attributes.

David

---

From: David Chadwick <[email-masked]>
Date: Thu, 04 Apr 2002 23:55:42 +0100

> returned in the search result entry some attributes whose syntax OID is not

thats fine. But it does not quite go far enough. There are multiple
cases to consider by the server (irrespective of what the client knows
or does not know). The server might receive on an Add operation

i) an attribute whose LDAP syntax OID is known to the server and whose
native encoding is known e.g telephoneNumber
ii) an attribute whose LDAP syntax OID is known and whose native
encoding is not defined e.g. certificates and Steven Legg's multiple
examples of ACI etc (although it seems a bit odd to allocate an LDAP
syntax OID and not define what it means)
iii) an attribute whose syntax OID is not known and is sent to the
server as ;binary e.g. foobar;binary
iv) an attribute whose syntax OID is not known and is sent to the server
in a native encoding known to the client e.g foobar

I suspect that the server must reject case iv). I dont know what it will
do in case iii) when such an attribute is recieved. The server could
just store it as a binary BER blob (assuming the attribute type has been
configured in), or refuse to accept it. In fact case ii) and iii) dont
appear to be too dissimilar to me, since the LDAP syntax OID is not
passed in protocol, and in case ii) it is a pretty meaningless thing to
have as it is not defined? So a server could treat case ii) and iii) the
same. Can you clarify this for me Mark please.

The client submitting the attribute might well be different to the one
retrieving it, and have different amounts of schema knowledge. A dumb
client might not know any schema definitions!

So in general a client might expect to recieve

i) an attribute whose native encoding it knows, in native encoding e.g
telephoneNumber
ii) an attribute whose native encoding it knows, in ;binary encoding
e.g. telephoneNumber;binary
iii) an attribute whose native encoding it does not know, in native
encoding e.g. foobar
iv) an attribute whose native encoding it does not know, in binary
encoding e.g. foobar;binary

All of the above 8 cases (both receipt by the server and the client)
need to be described in the LDAPbis documents as they are general cases
independent of the certificate discussion. Dont you agree?

> requests it in the search's attribute description list.

What happens if a client Adds telephoneNumber;binary? This should be
readable in either native format or ;binary format, shouldn't it? One
could interpret what you said above that once submitted in the ;binary
form it is only ever readable in ;binary format. You did not imply this
did you?

Thanks

David

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 05 Apr 2002 12:35:23 -0800

I think adding a LDAP-specific "native" encoding to certificate
is a bad idea and will actually cause more problems than it
solves.

AFAIK, there are no interoperability problem between clients
and servers which adhere to the technical specification.  Things
are basically okay.

This I-D attempts to resolve interoperability problems between
non-conforming implementations and conforming implementations
by altering the specification to require conforming implementations
to accept and support the syntax and semantics of non-conforming
implementations.  However, there are a number of different
ways implementations have not conformed to the specification.

I have seen, in the wild, clients which:
	a) request "userCertificate" and expect the return of
	"userCertificate" using the (updated) LDAPv2 native
	string encoding.
	b) request "userCertificate" and expect the return of
	"userCertificate;binary" using binary transfer.
	c) request "*" and expect the return of
	"userCertificate;binary" using binary transfer.

This I-D caters to case a) in a manner which disallows servers
from supporting case b) and c).   I believe cases b) and c)
are actually for more common than case a).  I know of
implemenations which are liberal in accepting b) or c).  While
there may be implementations which are liberal in accepting
a), it should be noted that a) requires the server not to
be strict in what it sends.

I also note that b) and c) are generally the result of
the specification not be as clear as it should have been.
Where a) is the result of someone apply LDAPv2 syntax and
semantics to LDAPv3.

However, my objection to adding this I-D is that it that
client and server implementations of it won't interoperate
with existing server and client implementations which
conform to the existing specification.

I suggest we only make the minimal changes necessary to
resolve the issues which caused this schema to be removed
from the LDAP "core" technical specification.  It was
removed become of normative reference issues (e.g.,
2nd v 3rd v ... edition of X.500) and to align the
schema with that provided by X.509 (e.g., matching rules).

I note that it was NOT removed because of lack of multiple
independently developed implementations.  We need to
be careful to not to require changes to implementations
which have already demonstrated interoperability.

Kurt