Subject: RE: Transactional issues with the LDAP Core Schema

From: Shai Herzog <[email-masked]>
Date: Sun, 13 Jun 1999 15:35:49 -0400

Dave and Bob,

Admiting to not being an LDAP/Directory guru, I'm wondering if we 
are doing the right thing by puting the burden of preventing half 
baked rules (groups) on the gui/tool. This opens up a lot of room 
for interpretation and attempt to close loopholes, which in turn 
lead to incompatible results.

Wouldn't a better approach would be to put a "semaphore lock" on
the root of the rule. Given the infrequent update, overhead is NOT
a problem, but it may solve virtually all the updating problems.

1. As long as the rule is being updated, it is locked, and PDPs
   should wait until the update is complete.

2. If multiple rules are updated as a transactional group, then
   all of them are locked, until they are all in, and then the
   lock is removed. This way, a keyword search would discover
   a few locked rules, and would cause the PDP to wait until
   they are cleared, and then re-issueing the keyword search
   to get all the rules.

What do you think?

Shai

At 01:36 PM 6/4/99, Durham, David wrote:
>Thanks Bob, this helps clear things up a bit with my second
>example. I see that by following a well-defined procedure 
>a policy editor tool can enter policy in a way that will 
>avoid many of the "half-baked" policy problems despite the lack 
>of transactions. This information should probably find its 
>way into the specification because policy editors that do 
>not follow the "correct" procedure will create problems.
>It would be even better if the LDAPized core schema could
>somehow force the correct procedure. 
>
>I still think the issue remains for policies that already 
>exist but are instead being edited/updated. Let me see if 
>I can conjure up an illustrative example for such a case...
>Say, policy "re-ordering" in the case rules are prioritied, 
>or redefining two groups already ANDed together within a 
>policy rule. Seems as if even more restrictions will have
>to be placed on the policy editor for cases as these where
>dependencies cross the boundaries of a single entry.
>
>Also, can you explain if there is a procedure for modifying 
>auxiliary attachments that act as labels (e.g. through
>the policyKeywords attribute) such that lack of transactional
>support is not a pitfall here?
>
>-Dave
>
>
>> -----Original Message-----
>> From:	[email-masked] [SMTP:[email-masked]]
>> Sent:	Friday, June 04, 1999 8:10 AM
>> To:	[email-masked]
>> Cc:	Durham, David; 'Raju Rajan'
>> Subject:	Transactional issues with the LDAP Core Schema
>> 
>> 
>> 
>> Dave,
>> 
>> You wrote the following in a response to Raju:
>> 
>> [[Dave]]  Getting the full LDAP object is not the problem, it
>>      is the solution. My complaint is that when you go beyond a single
>>      LDAP entry to express a policy, then you are effectively opening
>>      up LDAP's non-transactional Achilles heel. Keeping all ANDed
>>      conditions within a single entry will help prevent clients from
>>      retrieving incomplete (=incorrect) policies. Attaching conditions
>>      as separate sibling entries to a policy rule implies that a client
>>      can retrieve a portion of the DIT that a console is in the process
>>      of editing. Rules need to be built from the conditions and actions
>>      up, or entries need to contain all attributes applicable to a rule,
>>      so that PDPs do not get incorrect policies. Basically, the core
>>      schema should be designed to take advantage of atomic transactions
>>      within an entry where LDAP does have limited transactional support.
>>      The DNF form of complex policy rules violates this principle as
>>      do simple policy rules (and correct me if I am wrong) where
>>      conditions and actions are attached as sibling entries.
>> 
>>      I have a host of other transactional issues with using auxiliary
>>      attachments as labels which will have to be edited across DNs all
>>      the while PDPs (and other consoles) are potentially querying
>>      (and editing) the DIT!
>> 
>> While I understand the general point about LDAP's transactional
>> weakness and the resulting exposure arising from the use of multiple
>> entries to store related pieces of information, I'm having a hard time
>> finding the exposures in the specific case of the Core Policy Schema
>> classes.
>> 
>> For example, there's no chance for a client to get a "half-baked"
>> policy rule if the entries that make up the rule are added to the
>> directory (or, if they're already there, updated to say the right
>> thing) in the following order:
>> 
>> 1. Entries like Lee's addressRange object, that are pointed to by
>>    policy condition and/or policy action objects.
>> 2. All policyCondition and policyAction entries that the policyRule
>>    will point to.
>> 3. The policyRule entry itself.
>> 
>> It makes no difference whether the rule combines the conditions in DNF
>> or CNF, since by the time the rule is there to be retrieved, all the
>> conditions are already present in the directory.
>> 
>> Ordinarily this sequence of updates isn't something that the directory
>> administrator will need to worry about:  the Policy Tool should take
>> care of it automatically.  The standard should probably contain an
>> implementation note saying that a Policy Tool needs to write rule-
>> related information to the directory in this order, to avoid the
>> transactional pitfalls that Dave is worried about.
>> 
>> Another set of semantic relationships (besides those between a rule
>> and its conditions and actions) in the Core Schema are those between
>> a policy group and the policy rules or policy groups that it
>> aggregates.  Once again I don't see how a client can be misled if the
>> order of creation (or update) for these entries is:
>> 
>> 1. All the policyRules or policyGroups to be aggregated.
>> 2. The policyGroup instance that aggregates them.
>> 
>> The jurisdiction associations, realized by attaching the policy rule
>> or policy group pointers to something other than policyGroups, will
>> work in the same way:  first create the rules / groups being pointed
>> to, then create the pointers to them.  A client may come to the
>> directory before this process has been completed, in which case it
>> will fail to find any of its policy rules.  But this is the same
>> thing that would happen if it came to the directory before any of
>> the updates related to its policy rules had been done.  There is, once
>> again, no point in time when a client might retrieve some of the rules
>> related to its jurisdiction, while believing that it had retrieved all
>> of them.
>> 
>> The one case that does strike me as problematic is where a client
>> is retrieving all the policy rules whose priorities need to be
>> compared against each other to decide what to do in a particular set
>> of circumstances.  If the client gets to these rules via a policyGroup,
>> then there shouldn't be a problem.  The client will know that it's in
>> the middle of retrieving a set of rules, and so it knows that it must
>> not do a priority comparison until it finishes retrieving all the
>> rules.  (The preceding sentence becomes more complicated if we factor
>> in a PDP / PEP split, but I think the principle still applies.)
>> 
>> Where a client can't be sure it's gotten a complete picture is when
>> the policy rules whose priorities compete against each other are
>> identified only by a keyword.  In this case an unlucky client might
>> search a portion of the DIT for policy rules containing the keyword,
>> and get back only a partial set of rules, because it happened to do
>> the search while the rules were being added to the directory.
>> 
>> In this case, if the Policy Tool adds the rules to the directory in
>> order of decreasing priority, the likelihood of a mishap can be
>> reduced.  It isn't eliminated completely, though, because of the
>> following possible sequence of events:
>> 
>> 1. The client searches the portion of the DIT that will contain the
>>    highest priority rule, and it finds nothing.
>> 2. The Policy Tool adds the highest priority rule to the portion of
>>    the DIT that the client just searched.
>> 3. The Policy Tool adds a lower priority rule to a different portion
>>    of the DIT.
>> 4. The client searches this second portion of the DIT, and finds the
>>    lower priority rule.
>> 
>> At this point the client is set up to do the "wrong" thing (i.e., the
>> action(s) associated with the lower priority rule) if it's
>> presented with a stimulus that satisfies the conditions of both rules.
>> Notice, though, that this has nothing to do with splitting a single
>> policy rule into rule, condition, and action entries.  Exactly the
>> same thing could happen if each rule were represented by a single
>> entry.  I understand that Raju has raised some additional questions /
>> concerns related to prioritization of policy rules.  As we address
>> these concerns, we should also consider how to insure that a client
>> doesn't do a priority comparison when it has retrieved only a subset
>> of the rules that belong in the set to be compared.
>> 
>> Regards,
>> Bob
>> 
>> Bob Moore
>> IBM Networking Software
>> +1-919-254-4436
>> [email-masked]
>> 

__________________________________________________________
Shai Herzog                  Founder & CTO, IPHighway Inc.         
Cell: (408) 390-3045       http://www.iphighway.com/herzog           
Tel : (201) 585-0800                  Fax : (212) 656-1006

---

From: Randy Bush <[email-masked]>
Date: Sun, 13 Jun 1999 18:29:15 -0700 (PDT)

[ making trouble and very definitely not speaking as an iesg member ]

> a problem, but it may solve virtually all the updating problems.

if the lock is stored in ldap, then you have the problem of a holder
crashing or spacing out.  if it is not in ldap, then how can one share the
data across client workstations?  

we are about to learn all the lessons of opti/pessimistic concurrency,
rollback, two phase commit, ...  and there will be real fun if we want
multiple copies or otherwise distributed data.

"The first thing you should not do is treat the directory like a general
database."  -- _LDAP: Programming Directory-Enabled Applications with
Lightweight Directory Access Protocol_; T Howes, M Smith.

let me stir further trouble.  yes, ldap is the 'in thing' this year (well,
at least dns is not the victim any more:-).  but i hypothize that the vast
majority of users who will wanto to deploy the level of tools contemplated
here already have real relational database systems in-house.  and, not only
do those systems have transaction models, but they also won't require
embarrassing kludging to map cim's relational model onto a transaction-free
hierarchic dictionary.

randy

---

From: Michael Richardson <[email-masked]>
Date: Mon, 14 Jun 1999 11:39:40 -0400

>>>>> "Shai" == Shai Herzog <[email-masked]> writes:
    Shai> Admiting to not being an LDAP/Directory guru, I'm wondering if we 
    Shai> are doing the right thing by puting the burden of preventing half 
    Shai> baked rules (groups) on the gui/tool. This opens up a lot of room 
    Shai> for interpretation and attempt to close loopholes, which in turn 
    Shai> lead to incompatible results.

  Aren't we being rather LDAP specific?

  Most databases that I know of permit the concept of multiple record update.
Shouldn't an LDAP specific method be created within the LDAP group to deal
with this requirement and let's get on with defining how policies are
described, not how they are stored.

    Shai> What do you think?

  I think that you wouldn't want to wait for the LDAP lock to be released
due to network congestion in order to call 911.

   :!mcr!:            |  Solidum Systems Corporation, http://www.solidum.com
   Michael Richardson |For a better connected world,where data flows faster<tm>
 Personal: <A HREF="http://www.sandelman.ottawa.on.ca/People/Michael_Richardson/Bio.html">[email-masked]</A>. PGP key available.
 Corporate: <A HREF="mailto:[email-masked]">[email-masked]</A>.