Subject: Re: control combination was: Re: protocol-22 comments)

From: "Jim Sermersheim" <[email-masked]>
Date: Fri, 07 May 2004 11:39:49 -0600

If there are only two cases (defined and undefined), then we need to state what 'defined' means.

The whole point of this is to arrive at wording that promotes interoperability. So if one implementation twists 'defined' to mean one thing, and another twists it a different way, I think we failed.

If 'defined' means that it is documented in one or more of the control specifications, that's good * it means the behavior is well documented. If it means something else (like it's defined in the implementor's mind), it's not so good. 

Futhermore, if 'defined' means that it is documented in one or more of the control specifications, then I think Hallvard hasa point * we need to allow for non-error behavior for combinations of controls that are not defined, but happen to make sense and likely work together in some current implementations. Otherwise, someone will need to update those control specifications to define combination semantics in order to make the combination valid.

Jim

>>> "Kurt D. Zeilenga" <[email-masked]> 5/7/04 11:04:45 AM >>>
At 08:51 AM 5/7/2004, Jim Sermersheim wrote:
>So it seems there are three cases for a combination of controls:

I think there are only two cases relevant to protocol semantics:
  Are the semantics of the combination of controls defined or not?

The issue of where those semantics, when defined, are or should be
specified is orthonogal.

I think looking at it from the cases you outline may
be problematic, for instance:

>1) the combination is documented in a specification
  A specification can document that a combination of controls
  doesn't have a defined semantic.

>For #3, I think we are saying an error is returned.

I think we also need to state more clearly that the
combinations of undefined (or otherwise ambiguous) semantics
are not to be provided and, if provided, treated as a
protocol error.

>For #2, I think we need to state that behavior is undefined. This is
>because the lack of a specification can cause two implementations to act
>differently.

Here we're stating more what specifications of controls should
provide and how those specifications are to be interpreted if
they fail to provide it.  I think we need to separate this
issue from the above issue.

>If this is true, then we can use something like the following:
>
>"When a combination of controls is encountered whose semantics are not
>defined (or not known), the operation SHOULD fail with protocolError,
>otherwise the results are undefined."
>
>>>> "Kurt D. Zeilenga" <[email-masked]> 5/7/04 9:24:44 AM >>>
>There is a subtle difference in the suggested text from the
>current text that might be, well, a bit too subtle.
>
>The current text talks about avoiding combinations whose
>semantics are not SPECFIED while the suggested talks about
>avoiding combinations which are not DEFINED.  While defined
>and specified are synonyms of each other, the latter can
>be viewed as implying there is a specification (e.g., document)
>where the semantics are defined, the former only can be viewed
>only as implying the semantics are defined (without regard
>to where they are specified).
>
>In the example of Sort+SignedOp, I think it can be argued that
>the semantics are defined even though they are not explicitly
>specified.  The intent of the text is not to prohibit such
>combinations.
>
>The text is intended to disallow use of combinations where the
>semantics are undefined, such as a combination of PagedResults
>[RFC2696] and VLV [draft-ietf-ldapext-ldapv3-vlv-08.txt].
>
>If I reading everyone's comment well, I think we have consensus
>on what we want to say but just haven't found the words that say
>it well enough.  Specific suggestions welcomed.
>
>Kurt
>
>At 06:02 AM 5/7/2004, Hallvard B Furuseth wrote:
>>Kurt D. Zeilenga writes:
>>>At 08:55 PM 5/5/2004, Jim Sermersheim wrote:
>>>
>>>> "server MUST NOT make use of a combination of controls whose
>semantics
>>>> are not defined (or not known to it). When such a combination of
>>>> controls is encountered, it is RECOMMENDED that the server return
>>>> protocolError. Otherwise behavior is undefined.
>>> 
>>> I dislike the wording as it could be taken as allowing the server
>>> not to return an error (as opposed to suggesting which code to
>>> return in this case). 
>>
>>I thought that was the intent, which looked reasonable to me.
>>
>>> How about?
>>>   Controls are not to be combined (with other controls
>>>   of same or different types) unless the semantics of the
>>>   combination have been defined.  When an implementation
>>>   (client or server) encounters a combination of controls
>>>   whose semantics are not defined (or not known to that
>>>   implementation), the implementation MUST treat the
>>>   request as a protocol error.
>>
>>Sounds too strict to me.  Or at least, it can be interpreted too
>strict.
>>For example, does that forbid the combination of the rfc2649
>Operation
>>Signature control with e.g. the rfc2891 Server Side Sorting control?
>>Neither document explicitly states how the combination should work,
>but
>>rfc2649 does say how to apply that control to LDAPMessages in
>general,
>>and the combination seems to make sense to me.
>>
>>Also, I wonder about the combination of controls defined in different
>>"standards repositories" or whatever one should call it.  If all
>>controls were RFCs, one control spec could list all previously
>defined
>>controls.  However, in the previous discussions of controls, people
>were
>>referring to controls that I could not find RFCs for.
>>
>>Finally, one detail: If we stick to RFC controls, each control author
>>will need to check all previous RFC controls.  It might be easy to
>miss
>>one, and make a control combination with an obvious semantics
>invalid.
>>We could fix that with a requirement that all LDAP control RFCs must
>>have the phrase "LDAP control" in its title.  Or someone (IANA?) 
>could
>>maintain an official list of published controls.  However, it seems
>>simpler to me to allow controls that obviously do not interfere with
>>each other, to be combined even if their specs do not say so.
>>
>>> (Note: I generalized this for both clients and servers,
>>> and for requests which don't have responses.)
>>
>>-- 
>>Hallvard

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 07 May 2004 11:40:36 -0700

>If there are only two cases (defined and undefined), then we need to state what 'defined' means.

I think we can and should rely on the dictionary meaning of the word.

 From WordNet (r) 2.0 [wn]:
  defined
       adj 1: clearly characterized or delimited; "lost in a maze
       of words both defined and undefined"; "each child has clearly
       defined duties" [ant: {undefined}]

>The whole point of this is to arrive at wording that promotes interoperability.

And other qualities.

>So if one implementation twists 'defined' to mean one thing, and another twists it a different way, I think we failed.

There really is no way folks from twisting things, especially in
areas of extensions.

>If 'defined' means that it is documented in one or more of the control specifications, that's good * it means the behavior is well documented. If it means something else (like it's defined in the im…

but it might be "good enough".

Obviously, for there to be interoperability between to
implementations, there must be reasonable agreement of what
the semantics of the message are.  However, implementor X
don't need to agree with implementor Y to produce an
implementation that will interoperate with Z.  While
the base specification is intended to further
interoperability between independently developed
implementations feasible, it doesn't preclude creation
of extensions which rely dependent development for
interoperability.

---

From: John McMeeking <[email-masked]>
Date: Fri, 7 May 2004 13:47:28 -0500

Is this any better?  It still uses defined, but I'm hoping in a way that
the differences between defined, specified, and known that have been
discussed don't come into play.  Maybe some examples should be included.
English leaves a lot to be desired as a precise language.

<start of text>
Controls are not to be combined (with other controls of same or different
types) unless the semantics of the combination have been defined and are
supported by the implementation.  When an implementation (client or server)
encounters a combination of controls whose semantics are not supported by
that implementation, the implementation MUST treat the request as a
protocol error.
<end of text>

If the implementation supports the combination, honor the controls.  If the
implementation does not support the combination, return protocolError.

A server could implement both the page control and the sort control, but in
such a way that they cannot be used in conjunction.  The VLV + paged search
combination previously cited clearly makes no sense.  In both cases, the
implementation should return protocolError.

The desired behavior of sort + paged search is well defined by RFC.  Other
examples have been cited where the correct behavior seems clear but is not
explicitly defined (sort + signed op).  If the implementation supports the
combination, it should honor the request.

Examples have been cited of control combinations that clearly make no sense
(VLV + paged).  In these cases, the implementation should also return
protocolError.

John  McMeeking

"Jim Sermersheim" <[email-masked]> wrote on 05/07/2004 12:39:49 PM:

> The whole point of this is to arrive at wording that promotes
> interoperability. So if one implementation twists 'defined' to mean
> one thing, and another twists it a different way, I think we failed.
>
> If 'defined' means that it is documented in one or more of the
> control specifications, that's good * it means the behavior is well
> documented. If it means something else (like it's defined in the
> implementor's mind), it's not so good.
>
> Futhermore, if 'defined' means that it is documented in one or more
> of the control specifications, then I think Hallvard hasa point * we
> need to allow for non-error behavior for combinations of controls
> that are not defined, but happen to make sense and likely work
> together in some current implementations. Otherwise, someone will
> need to update those control specifications to define combination
> semantics in order to make the combination valid.
>
> Jim
>

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sun, 09 May 2004 10:40:26 -0700

>that implementation, the implementation MUST treat the request as a

I dislike use of "supported" here and, in particular,  I do not consider
it appropriate for an implementation to treat a well-defined combination
as a protocol error. 

>implementation does not support the combination, return protocolError.

unwillingToPerform would be more appropriate in this case.

>implementation should return protocolError.

I disagree.  In the former case, unwillingToPeform is more appropriate
as the semantics of the combination are well defined (as you note below).

---

From: Hallvard B Furuseth <[email-masked]>
Date: Sun, 9 May 2004 21:04:04 +0200

> I think we can and should rely on the dictionary meaning of the word.

Which shifts the question to: defined by what?  'defined by a control
spec' seems a quite reasonable interpretation in this context, but
that's too strict if we forbid everything else.  We could spell out
'Defined by a control spec or the implementation', which is permissive
enough - perhaps too permissive, but what does it mean?  'documented by
the implementation'?  I suggested to require that about something else
once, and it was shot down.

Maybe a better distinction is that the implementation should not give an
ambiguous control combination arbitrary semantics; it should either
return an error or give it deliberatly chosen semantics.  And perhaps
error should be RECOMMENDED.

-- 
Hallvard

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sun, 09 May 2004 18:12:32 -0700

At 12:04 PM 5/9/2004, Hallvard B Furuseth wrote:
>Which shifts the question to: defined by what?

Does it matter how it's defined?  It seems to me that all that
matters is that it is defined. 

>'defined by a control
>that's too strict if we forbid everything else.

Exactly.  Maybe it's implicitly agreed to by the implementors
of the client and server (maybe they are the same implementor).

>error should be RECOMMENDED.

One could argue that it would be impossible to implement
a combination of controls unless the semantics of that
control were defined.  That is, how does one implement
undefined behavior.  And by implementing it, it could be
considered defined.  So maybe we're just splitting hairs
here.

My view is that an implementation should not attach control
A to message X unless A is appropriate for X, that is:
X+A has well defined semantics.  And one should not attach
B to X+A unless B is appropriate for X+A, that is X+A+B has
well defined semantics.  Etc..

The processing of X+A+B is complicated by the fact that
A or B or both could be non-critical.  I suspect most
implementations act as follows on receipt of X+A+B:
  is X well formed? if not return protocolError.
  is A recognized and appropriate (for X)?
     If not and critical, return unavailableCriticalExtension
     If not and non-critical, ignore A
     Otherwise, is A well-formed?  If not return protocolError.
  is B recognized and appropriate (for X or X+A, depending on above)
     If not and critical, return unavailableCriticalExtension
     If not and non-critical, ignore B
     Otherwise, is B well-formed?  If not return protocolError.

Do implementations actually return protocolError for something
like search+A+B?  Or do they either ignore B if non-critical
or return unavailableCriticalExtension otherwise.

(Note that the above is note intended to be sequence order
dependent, but dependent on the order of examination (not
necessarily processing) by the implementation.

Kurt

---

From: Hallvard B Furuseth <[email-masked]>
Date: Mon, 10 May 2004 16:07:16 +0200

Kurt D. Zeilenga writes:
> Does it matter how it's defined?

If [Protocol] states what to do about '(un)defind' control combinations,
it should also clarify what '(un)defined' means, since there are several
equally reasonable but very different interpretations.

> It seems to me that all that matters is that it is defined. 

Then it's pointless for [Protocol] to specify what to do if a control
combination is undefined.  As you said below, an implementor could say
that by implementing it, it could be considered defined.  That reduces
anything [Protocol] says about it to mere word games, and the
implementor can simply ignore the paragraph about control combinations.

So if we are going to say something about control combinations, I'd
prefer other - or additional - criteria to defined/undefined.  As my
suggestion below:

>> return an error or give it deliberately chosen semantics.  And perhaps

> well defined semantics.  Etc..

Well, yes.  But now you are right back to 'defined' again.  So I think
'unless ... X+A+B either has unambiguous semantics, or how to best
resolve an ambiguity has been considered by the implementor' is a more
useful criterion.

>      Otherwise, is B well-formed?  If not return protocolError.

Good point.  However, we'd get more predictable results (for someone who
does not known the particular implementation) if the implementation
ignores all non-critical controls which are incompatible with some other
control in the message.  Maybe implementations MAY ignore selected
non-critical controls that conflict with other controls, and SHOULD if
so ignore _all_ non-critical controls that conflict with some other
controls?  OTOH, we are talking about control combinations that either
fail or are unpredictable in any case, so maybe not...

One other detail: I don't think it fits the definition of protocolError
in section A.2 to return that for a control combination which MAY be
treated either as defined or undefined:

        protocolError (2)
           Indicates the server received data which has incorrect
           structure.

If servers are to return protocolError for such control combinations,
that must be added to the list of exceptions to the basic definition of
protocolError in A.2.

I already dislike the length of that list, though.  Maybe
unwillingToPerform or unavailableCriticalExtension is better.

-- 
Hallvard

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Mon, 10 May 2004 11:47:45 -0700

In chewing about this more, I'm now thinking that text I suggested
is a bit problematic, mainly because it is at odds with current
implementations.  I am not aware of any implementation which treats
a combination of mutually exclusive controls as a protocol error.
As hinted at in my previous comment, I'm thinking that RFC 2251
actually handles this issue in its use of the term 'appropriate
for the operation'.

I think we need to look at ways of clarify the RFC 2251 text to
note that, in face of sequence of controls which are inappropriate
for the operation, the server is free to either treat the message as not
being well constructed (e.g, protocol error) OR to ignore any
number of the non-critical control so that the remaining controls
are recognized and appropriate for the operation OR, if to
arrive at a sequence of recognized and appropriate controls for
the operation would require ignoring a critical control, to
return unavailableCriticalExtension.

Kurt

Below are a few comments I wrote while chewing on this.  The
above is more reflective of where I am presently at on this issue.
I've attached them as there are a few related or tangential issues
I felt the need to comment on.

>equally reasonable but very different interpretations.

Maybe we need to move away from the term 'defined'.

RFC 2251, when describing message construct/deconstruction, used the
term: "appropriate for the operation".  In this thread, when we said
the semantics of the combination of the controls should be
defined, I think we mean the combination of the controls should
be appropriate for the operation.

Of course, this leads to the question "what's appropriate".
well, that's for the control's specification (whether it written,
or in other form) to say.

>implementor can simply ignore the paragraph about control combinations.

I think there will always be room for word games.

I think it reasonable for the protocol specification how implementations
are to respond when faced with constructs they do not recognize or they
do not otherwise think are appropriate.

This is completely orthogonal, in my mind, to how and where those
constructs are specified.

>Well, yes.  But now you are right back to 'defined' again.

Or back to 'appropriate'.  Note that problem is not just with
combinations, but also with controls when used individually.

What's 'defined'?  What's appropriate?... To this, I say, look
at the (definitive) specification of that construct.  However,
I cannot guarantee that there will a specification, let alone
a definitive specification.

>So I think 'unless ... X+A+B either has unambiguous semantics, or how to best
>control in the message.

I argue that more predictable results are not desirable here.  If a
set of controls are marked non-critical, then no real harm is by the
server ignoring any subset (including complete and empty sets) of the
controls.  I would argue instead that it more desirable to have the
implementation do "best effort" processing.

But note that I was attempting to describe what is the currently
implemented behavior.  My concern with "MUST treat as protocol error"
is that this isn't what current implementations behave.

>Maybe implementations MAY ignore selected
>non-critical controls that conflict with other controls,

RFC 2251 actually said:
   If the control is unrecognized or inappropriate but the criticality
   field is FALSE, the server MUST ignore the control.

If a non-critical control conflicts with some number of other controls,
the non-critical control may be viewed as inappropriate.  If so, it
MUST be ignored.

>and SHOULD if
>controls?

Besides that not being what implementations do today, I don't see your
reason for this imperative as enough justification for the SHOULD.

>OTOH, we are talking about control combinations that either

Note that I am not talking at all about the performance of the operation
as extended by the controls (e.g., which might fail or be unpredictable),
but whether the message is well constructed and its semantics are clear.

>that must be added to the list of exceptions to the basic definition of

I think, 'incorrect structure' is meant to broadly refer to
any invalid construct.

>unwillingToPerform or unavailableCriticalExtension is better.

Both of these, in my opinion, are inappropriate here.

The question of willingness to perform of an operation is generally
made after one de-constructs the message semantics.  While certainly
a server could be unwilling to de-construct (parse) the message,
this should not be confused with a de-construction error.

And, unavailableCriticalExtension is only applicable where the
unrecognized and inappropriate control is critical.