Subject: [ldapext] Re: Comments to <draft-sermersheim-ldap-chaining>

From: "Jim Sermersheim" <[email-masked]>
Date: Sun, 20 Feb 2005 23:02:02 -0700

Thanks for the feedback,
 
I've updated the I-D and will submit. I think I addressed everything
except the question about criticality. For criticality, the instructions
in 4.1.11 of
http://www.ietf.org/internet-drafts/draft-ietf-ldapbis-protocol-30.txt.
Basically, if the server understands the control, the criticality is
ignored and the control is honored. If the control is not understood,
then criticality determines whether the control is ignored or causes the
operation to fail.
 
Jim

>>> Pierangelo Masarati <[email-masked]> 1/25/05 3:03:05 PM >>>
Jim,

I'm currently implementing the (now expired) version 02 of 
<draft-sermersheim-ldap-chaining> in OpenLDAP based on the "chain" 
overlay as first implemented by Howard Chu and subsequently extended by

myself. What follows is a collection of considerations that emerged 
from the implementation and that, perhaps owing to my lack of 
understanding of some details, I think require a clarification and
might 
be useful for the continuation of the work on the draft. The comments 
are quite unsorted.

1. Comments

1.1. Return code renaming
I suggest to change the name of "chainingRequired" into 
"requiresChaining" to avoid confusion and symbol clashes with the 
"requiresChaining" value in the "Behavior" enumeration.

1.2. Return code usage clarification
I think you should clarify when the special return codes related to the

control need be returned. For instance, if behavior is 
"chainingRequired" and the operation would return a referral that
cannot 
be chased, I'd return "cannotChain", but for any other "Behavior" case

returning a referral is acceptable, and returning a referral is not an

error: <draft-ietf-ldapbis-protocol> lists "referral" as a non-error 
result code in Appendix A.1, so I don't see an application for the
other 
error code you indicate ("chainingRequired"). I suggest you clarify 
your intentions.

1.3 OPTIONAL values and defaults
In the draft, both "Behavior" fields are marked as OPTIONAL. My point 
is: what's the reason for not providing any? For instance, the "chain"

overlay in OpenLDAP's slapd by default acts as if both "Behavior" were

"chainingPreferred". If the control is present, but no values are 
given, I see no point in using the control. Unless you mean that the 
default behavior of the DSA may differ if the control is present or
not; 
e.g. if no control is present, default to "referralsRequired", while if

the control is present, default to "chainingPreferred" (OpenLDAP's
slapd 
without and with the "chain" overlay), and in any case both defaults
are 
left to the implementor. If this is the case, I suggest it is
clarified.
[technical note: it is not clear to me if, in case none of the values
is 
provided, the crtls_value of the control structure should be empty or
null].

1.4 Operations.
It appears from the draft that the control applies to all operations, 
except "abandon, unbind, and StartTLS"; the second value applies only
to 
search continuations, and may apply to future operations. Should the 
presence of the second value in other operations among those already 
defined that do not require it be treated as an error? I think this 
should be clarified.

1.5 Behavior
1.5.1. "chainingPreferred" and "referralsPreferred" don't present any 
issue;
1.5.2. "chainingRequired" might be an issue if, in case it cannot be 
honored, no intermediate results should be returned at all (I'm talking

about searches). In fact, unless caching all intermediate results 
locally upon completion, it is very likely that a referral is met after

some entries/other intermediate results have already been returned. If

the referral cannot be chased, the operation should be terminated and 
"cannotChain" be returned. I think it should be clarified if this 
behavior is acceptable or not.
1.5.3 "referralsRequired" should never fail, because that's the 
regular behavior in case no chaining capabilities are available; I
don't 
see any issue that might prevent a DSA from returning a referral. Can 
you clarify why this option is ever supposed to fail?

1.6. Criticality
I think the behavior should be discussed in view of the criticality of

the control. Would it be correct to simply ignore the "Behavior" in 
case the control is not critical? It is my understanding that only the

"chainingRequired" behavior may suffer from the criticality field being

set when referrals cannot be chased; the other options should pose no 
issues. How should the behavior differ in this case based on the value

of the criticality flag?

2. Few notes on the implementation.

2.1. Approach
The "chain" overlay in OpenLDAP's slapd uses the LDAP backend to chase

referrals in case they're being returned by the underlying backend (or

at the frontend level, if used as a global overlay). Credits for both 
the overlay and the underlying LDAP backend essentially go to Howard
Chu.

2.2. Authorization and Security
As such, it can exploit the identity assertion capabilities of
back-ldap 
to proxyAuthz the identity of the client while chasing the referral; 
this provides features like SASL bind, authorization policies based on

pattern matching, grouping and so both at the local and at the remote 
site. The connection can be wrapped in a TLS layer (use ldaps:// 
referral URIs, or StartTLS over regular ldap:// URIs).

2.3. Relationship with other extensions
Interaction with the "manageDSAit" control as described in the draft, 
although not intended, seems to work as described in the draft. Of 
course it may need further testing.

2.4. Missing features and possible development
I plan to introduce:
2.4.1. authorization based on referral suffix (e.g. only chain 
requests rooted at <DN>)
2.4.2. authorization based on the referral URI (host/port portion)
2.4.3. authorization based on the protocol scheme (e.g. only ldaps://)
2.4.4. authorization based on some ssf
2.4.5. to allow chaining of multiple DSAs, back-ldap's identity 
assertion must be modified to allow authentication identity selection 
based on the DSA that is being chained (now a single identity can be 
specified, since back-ldap is intended to contact a single URI).
2.4.6. it is not clear yet how the control could interact with further

referral chasing operated at the client library's level by the LDAP 
backend (proxyAuthz and so).
2.4.7 there's yet no provisions to check for loops and so.

2.5. Specific numbers and #defines
I'm currently using OIDs and return codes under OpenLDAP's/private 
space; the macros and the numbers are(from OpenLDAP's HEAD ldap.h):

/* LDAP Chaining Behavior Control *//* work in progress */
/* <draft-sermersheim-ldap-chaining>;
* see also LDAP_REQUIRES_CHAINING, LDAP_CANNOT_CHAIN */
#ifdef LDAP_DEVEL
#define LDAP_CONTROL_X_CHAINING_BEHAVIOR "1.3.6.1.4.1.4203.666.11.3"

#define LDAP_CHAINING_PREFERRED 0
#define LDAP_CHAINING_REQUIRED 1
#define LDAP_REFERRALS_PREFERRED 2
#define LDAP_REFERRALS_REQUIRED 3
#endif

/* for the Chaining Behavior control (consecutive result codes
requested;
* see <draft-sermersheim-ldap-chaining> ) */
#ifdef LDAP_CONTROL_X_CHAINING_BEHAVIOR
#define LDAP_REQUIRES_CHAINING 0x4110
#define LDAP_CANNOT_CHAIN 0x4111
#endif

I hope the discussion above, and the quick overview on the current 
implementation and its todo list, may be of help in stabilizing and 
completing the draft.

Sincerely, p.

SysNet - via Dossi,8 27100 Pavia Tel: +390382573859 Fax: +390382476497

_______________________________________________
Ldapext mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ldapext

---

From: "Jim Sermersheim" <[email-masked]>
Date: Sun, 20 Feb 2005 23:26:02 -0700

I was just reviewing this again and I disagree with the current draft's
statement about interaction with ManageDsaIT. I believe ManageDsaIT has
the effect of disallowing chaining and disallowing referrals. It simply
causes the local distributed knowledge information to be updated and
interrogated.
 
I will change the draft to reflect this, but welcome further feedback.
 
Jim

Jim,

1. Comments

1.4 Operations.

1.5 Behavior

1.6. Criticality

of the criticality flag?

2.1. Approach

Sincerely, p.

---

From: Howard Chu <[email-masked]>
Date: Thu, 24 Feb 2005 19:45:17 -0800

Pierangelo Masarati wrote:
> understanding of some details, I think require a clarification and might 

I'm a bit skeptical about these features of the draft. X.511 only 
provides for "preferChaining" and "chainingProhibited." It strikes me 
that "chainingRequired" (and the implication that a request must fail if 
chaining is not available) is not useful. In the default case, where the 
control is not present, and the server does not support chaining, the 
server would just return a referral and the client would receive it and 
choose to follow it or drop it. Any client that goes to the effort of 
implementing this control will receive less information than it had 
without it. Seems like a lot of effort to go thru to be worse off than 
you were when you started. From another angle, the decision of whether 
to deal with referrals or not still resides with the client/App writer, 
and the server doesn't (and shouldn't have to) care.

I would eliminate the chainingRequired behavior, and leave the result 
code name alone (out of a foolish consistency, perhaps, but it matches 
the name used in X.511).

Further aligning with X.511, I would consider renaming 
"referralsRequired" to "chainingProhibited" since the two are 
equivalent. I personally don't see any use for "referralsPreferred" 
since (a) that's probably the server's default anyway and (b) it's only 
a preference, and the server will blithely ignore it if it's not the 
default. If you have a use case in mind I guess I'll let it slide, but 
surely any LDAP -> X.500 gateways would just delete this in any requests 
passed along. [By the way, I'm looking at a 2001 4th Edition X.500 
draft; if I'm out of date here please disregard...]

> "chainingRequired" and the operation would return a referral that cannot 
> result code in Appendix A.1, so I don't see an application for the other 
> default behavior of the DSA may differ if the control is present or not; 
> the control is present, default to "chainingPreferred" (OpenLDAP's slapd 
> without and with the "chain" overlay), and in any case both defaults are 
> left to the implementor.  If this is the case, I suggest it is clarified.
> [technical note: it is not clear to me if, in case none of the values is 
> except "abandon, unbind, and StartTLS"; the second value applies only to 
>  1.5.3 "referralsRequired" should never fail, because that's the regular 
> behavior in case no chaining capabilities are available; I don't see any 
> issue that might prevent a DSA from returning a referral.  Can you 
> clarify why this option is ever supposed to fail?
> 
> 1.6. Criticality
> the overlay and the underlying LDAP backend essentially go to Howard Chu.
> 
> As such, it can exploit the identity assertion capabilities of back-ldap 
>  2.4.1. authorization based on referral suffix (e.g. only chain requests 
> rooted at <DN>)
> /* for the Chaining Behavior control (consecutive result codes requested;

-- 
   -- Howard Chu
   Chief Architect, Symas Corp.       Director, Highland Sun
   http://www.symas.com               http://highlandsun.com/hyc
   Symas: Premier OpenSource Development and Support

---

From: "Jim Sermersheim" <[email-masked]>
Date: Thu, 24 Feb 2005 21:45:57 -0700

I have to admit that I disliked the names and limited usefulnes of the
controls as they are defined in X.511, and that may have influenced me
to simply define them in a symmetrical and (to me) more clear way for
LDAP.

I see usefulness in chainingRequired. For example, assume a client
wishes to perform a SSS or VLV search operation and knows that the data
in the result set may be distributed (and perhaps further assume some
DSAs support chaining these operations). If the client can specify
chainingRequired, it would know that the results it gets back are
already sorted and no extra work needs to be done. If the server cannot,
or will not chain, then the operation fails, and the client learns this
up front. In the absence of chainingRequired, the client may get back
many search results -- assuming they're sorted, only to find out in the
end that one or more search result references need to be followed to
complete the operation. I believe that chainingRequired may also be
useful for other multi-object operations which are served best (or can
only be completed) by the act of DSA chaining. Off the top of my head, I
can't think of a strong case for chainingRequired during name resolution
(otehr than the case I make below for referralsPreferred), but I see no
harm in allowing it's specification.

I also see usefulness in referralsPreferred. It allows one to possibly
override whatever the server's behavior is.  We have deployments where
the server's setting is chainingPreferred or chainingRequired. Clients
may prefer to receive referrals, yet want the operation to be chained
when policy prevents a referral from being returned.

Jim

>>> Howard Chu <[email-masked]> 02/24/05 8:45 PM >>>
might 

cannot 

other 

not; 
slapd 
are 
clarified.
is 
to 

>  1.5.3 "referralsRequired" should never fail, because that's the
regular 

Chu.
back-ldap 

>  2.4.1. authorization based on referral suffix (e.g. only chain
requests 
requested;

---

From: Howard Chu <[email-masked]>
Date: Thu, 24 Feb 2005 22:10:06 -0800

> to simply define them in a symmetrical and (to me) more clear way for

Fair enough, I can appreciate symmetry...

> complete the operation.

I believe you're seeing a false economy here. The point is, why did the 
client ask the question it asked - presumably, because it wanted some 
data. If your client sends a request with e.g. SSS, the server will 
proceed to gather results up, find that it can't obey the Chaining 
control, and return an error to the client. But as far as I can see, the 
client still wanted that data, else why did it ask in the first place. 
The client can just give up and say "sorry, I can't get what you want in 
my preferred fashion" but the more useful thing for it to do is to retry 
the request and make do with whatever results it gets. In that case, 
you've just doubled the amount of work the server has to do, because the 
client will re-issue the same request, with relaxed controls. Nothing 
changes the fact that extra work needs to be done by the client, (or it 
must just fail); the presence of the control doesn't help matters in any 
way.

Also, this SSS example isn't quite right - obviously a server needs to 
gather up the complete result set before it can begin returning sorted 
results to the client. As such, it will already know if references are 
present before sending any results to the client. It would be wrong for 
the server to begin sending entries back knowing that the result set 
contains unresolved references, so that search should automatically fail 
in the absence of chaining. Indeed, RFC2891 pretty much dictates that 
the request will fail, in section 2.1. In this regard, I would prefer 
that that spec be changed to require servers to send unresolved 
references first, ahead of regular entries, and let the client decide 
whether it wants to slog through or give up. Cutting off the information 
at the server prevents the DUA user from making an informed decision 
about what to do next. Giving the information to the client allows the 
possibility of a graceful recovery.

 > I believe that chainingRequired may also be

I believe the harm is in preventing useful information from reaching the 
client, thus making error recovery messier and more expensive than it 
needs to be.

> when policy prevents a referral from being returned.

To me, the absence of any chaining control already implies 
"referralsPreferred" but I'll have to think about it some more. It 
becomes useful if you assume that the client knows more about the 
referred server than the original DSA knows. Personally I believe this 
assumption is seldom true, but I guess I can't say "never" so I'll leave 
this alone for now.

---

From: "Jim Sermersheim" <[email-masked]>
Date: Fri, 25 Feb 2005 08:39:00 -0700

Ugh, now we're going to get into gritty implementation details of VLV
and SSS -- I shoulda used a different example.

Our (and I imagine other) implementations will service VLV and SSS
controls using an index if possible. In this case, entries may be
immediately returned as they are matched (as opposed to gathered,
sorted, then returned). In this scenario, (unless your proposed change
to SSS happens) a client may need to wade through many results *before*
it can even decide whether to begin presenting returned entries to a
user interface for exmple. This is because the server is free to return
referrals at any point (2.1 of rfc2891 does not excuse this).

In my example, the client did want data as you say, but only under
certain circumstances. When a number of DSAs hold copies of portions of
the tree, the client may need to contact a series of altServers in order
to locate one that can service the request in the manner that the client
wishes. Only after exhausting all DSAs might the client wish to fail
over to performing the distributed search and merge sort the results. 

I still don't see there being harm in, as you say, preventing useful
information from reaching the client. The client is aware of what it's
asking for, and has chosen to ask for a certain behavior. If the client
wishes for a different behavior, it should ask for different behavior.
There's a kind of unrelated analogy to the criticality of controls here.
 A client may wish to make a control critical to prevent the server from
performing the requested task (even though performing it might result in
useful information being returned) if it cannot service the request in
the exact way the client wants.

The final point is that there are implementations (two that I know of)
which allow all four of these behaviors to be set on a server-wide
basis, and the control can be used to override the server's behavior.
One implementation currently has its installation default as
chainingPreferred, the other has it as chainingRequired. The
implementation which defaults to chainingPreferred has been available
and widely used (most deployments are left in chainingPreferred mode)
for over ten years, so in my mind, there are a lot of people who assume
that in the absence of any control, their request is going to be
chained. Assumptions aside, it's the DSA's choice, and this control
allows as much flexibility as I could imagine is needed for today's and
tomorrow's need to override the server's default behavior.

Jim

>>> Howard Chu <[email-masked]> 2/24/05 11:10:06 PM >>>

control, and return an error to the client. But as far as I can see,
the 

The client can just give up and say "sorry, I can't get what you want
in 
my preferred fashion" but the more useful thing for it to do is to
retry 
the request and make do with whatever results it gets. In that case, 
you've just doubled the amount of work the server has to do, because
the 
client will re-issue the same request, with relaxed controls. Nothing 
changes the fact that extra work needs to be done by the client, (or it

must just fail); the presence of the control doesn't help matters in
any 
way.

the server to begin sending entries back knowing that the result set 
contains unresolved references, so that search should automatically
fail 
in the absence of chaining. Indeed, RFC2891 pretty much dictates that 
the request will fail, in section 2.1. In this regard, I would prefer 
that that spec be changed to require servers to send unresolved 
references first, ahead of regular entries, and let the client decide 
whether it wants to slog through or give up. Cutting off the
information 
at the server prevents the DUA user from making an informed decision 
about what to do next. Giving the information to the client allows the

> harm in allowing it's specification.

I believe the harm is in preventing useful information from reaching
the 
client, thus making error recovery messier and more expensive than it 
needs to be.

assumption is seldom true, but I guess I can't say "never" so I'll
leave 
this alone for now.