Subject: RE: Protocol: control specifications.

From: "Ramsay, Ron" <[email-masked]>
Date: Wed, 10 Mar 2004 12:21:36 +1100

I can see no point in discussions of how a server should handle incorrect criticality. Presumably, if the server knows the criticality is wrong then it knows enough to inplement it. Then, according to the rules, the criticality from the server's perspective is TRUE - controls that are implemented have to be performed. So, you would be asking a server to tell the client either
1) I did it this time, but don't send it to another server as you may get an incorrect result, or
2) I didn't do it. I could have but, na-na-na, you got the criticality wrong. Let that teach you a lesson!

I don't see either as acceptable server behaviour. I thought this had been settled. I thought I would see no more text on this.

Ron

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Kurt D. Zeilenga
Sent: Wednesday, 10 March 2004 09:43
To: Hallvard B Furuseth
Cc: [email-masked]
Subject: RE: Protocol: control specifications.

At 12:30 PM 3/9/2004, Hallvard B Furuseth wrote:
>Kurt D. Zeilenga writes:
>> Well, I think the question was more about what use scenarios
>> server-side enforcement of a particular criticality value would be
>> useful.  In the past, the use scenario discussed was so that client
>> developers would, if they tested against a server which implemented
>> such checks, receive a diagnostic (a protocolError) that would tell
>> them that they didn't provide the particular criticality value called
>> for.  The question was whether this wether or not there were other
>> (non-diagnostic) use cases.
>
>I cannot think of any.

noted.

>>At 01:11 PM 3/8/2004, Hallvard B Furuseth wrote:
>>>Kurt D. Zeilenga writes:
>
>>>- While I think control specs should be able to mandate the
>>>  criticality protocol field, they should not mandate that servers check
>>>  it.  That is a [Protocol] issue.  It would just add to the mess if one
>>>  could trust a server to catch wrong criticality for some controls, but
>>>  not for others.
>>
>
>> Problem here is that any mandated value is control-specific.  The
>> [protocol] discusses criticality handling which is not
>> control-specific.
>
>So if a server wishes to verify the criticality of controls in general,
>it must do it in the implementation of each control.

Well, I rather not get into implementation detail.  It is possible
that a server could implement their general control processing
in a manner which allowed them to verify criticality values
mandated by the control specification.  My point is that there
is a logic separation of control-neutral and control-specific
processing and any criticality verification is logically part of
the control-specific processing (as its specification is
control specific).

>I don't see a
>problem with that, as long as the server does it consistently.

Consistent with the control's specification!  The behavior
is control-specific...  one specification could say "incorrect
criticality is to be treated as a protocolError" while another
specification could say "incorrect criticality should be viewed
as constraintViolation" while another could say "if a incorrect
criticality is detected, it should be logged but otherwise
ignored".

I would hope most specifications would opt for the last (with
ability to disable logging) as a) it provides a useful diagnostic
and b) encourages servers to be liberal in what they accept.

I, for one, would likely ignore mandates to return errors where
I could instead process the operation unambigiously.

>Which is
>a quality of implementation issue, not a [protocol] or control spec
>issue.
>
>> While a control specification can certainly say
>> "servers implementing this control are to verify its critical field is
>> TRUE", (...), this is a control
>> specific semantic and not part of the control-neutral criticality
>> processing discussed in [Protocol].
>
>Yes.  Which is why I don't want control specs to say that.
>Because...
>
>> Whether or not such statements
>> are or aren't appropriate can be left to discussions regarding
>> particular control specifications which attempt to make them (as any
>> such verification is part of the control-specific processing).
>
>No.  If the control spec says that, it is altering the semantics of the
>criticality field.  Which [Protocol] says should not be done.

While [Protocol] says future specifications should not overload
criticality field, [Protocol] does not restrict the scope of
extension which a control might express.

>> Instead of saying the control specifications may require such a
>> verification (or verification of any number of other things), it would
>> like be better to clarify that the scope of any LDAP extension is only
>> limited by what can be supported in a truly optional manner.
>
>It's not clear to me what this suggestion means in practice.

Well, I would agree that some extension authors don't understand
that extensions should be truly optional.

>I _think_ it at least means to remove the text
>     (note: the semantics of the criticality field 
>     are defined above should not be altered by the control's 
>     specification)

This text is intended to highlight an important change in practice,
a change that resulted from significant discussion (namely in LDAPEXT)
of technical issues with existing extension specifications (some
"in progress", some published).  As the consensus was, and appears to
still be, that the overloading criticality is generally not appropriate,
it is appropriative for us to say so in this revision.  Otherwise
it is likely the practice would continue (especially given that not
all control specifications are subject to IETF review).

I note as well that this practice, aside from causing issues with
each overloading specification, also caused and continues to cause
confusion as to precise what are the criticality semantics.  Hence,
aside from clarifying what future specifications should avoid doing,
the note also reenforces that criticality semantics are just those
which were specified in RFC 2251.

>>>- I thought (and still suspect) rfc2251 said wrong criticality was an
>>>  error,
>> 
>> I do not believe your view is supported by the consensus of the WG
>
>It's been a while since the discussion, but at the moment I don't
>remember anyone but you and me saying what we think rfc2251 was meant to
>say.  Though I assume Jim agrees with you, since he adopted your changes
>so easily.

Consensus, especially when limited to just list posts, can be a
hard thing to judge.  I've tried to take all inputs (from list,
from personal conversations, from face-to-face meetings, etc..)
and weight them to come up with a belief of what the consensus
(or lack thereof) is.  I then generally try to state this belief
(as I have done here) as well as what actions I am considering
taking based upon this belief.  This is intended to allow
additional input (regarding either my view of consensus or the
proposed actions).

>> nor by existing implementations (many of which implement controls
>> whose specifications mandate clients provide particular criticality
>> values).
>
>Well, the question would be what most servers do;

Actually, the question is whether there are multiple independently
developed implementations.  2 servers would be enough... but I
not aware of any.

>> At present, I am inclined to either treat your request as a new feature
>> (and hence out-of-scope) or as a clarification to an un-implemented
>> interpretation (which we should avoid), or simply as a change which
>> is not supported by WG consensus.  However, to ensure the WG fully
>> considers all issues, it seemed appropriate to see if there were
>> other use scenarios.
>
>-- 
>Hallvard

---

From: Hallvard B Furuseth <[email-masked]>
Date: Wed, 10 Mar 2004 03:02:45 +0100

Ramsay, Ron writes:

> I can see no point in discussions of how a server should handle
> incorrect criticality.

Whether there is any point in discussing it anymore I don't know.

I do wish someone knew what some X.500 servers do about wrong
criticality.

> Presumably, if the server knows the criticality
> is wrong then it knows enough to inplement it.

Of course.

> Then, according to the rules, the criticality from the server's
> perspective is TRUE - controls that are implemented have to be
> performed.

The new rules, yes.

> So, you would be asking a server to tell the client either

s/asking/allowing/.

Yes.  You have a strange way of saying it, but since you do:

> I don't see either as acceptable server behaviour.

And I don't see why servers should be forbidden to catch this particular
client error.  There are plenty of client errors that servers may, but
need not, catch.  I think it's an important client error to catch, since
it can be so damaging.  As important as rejecting cleartext password
binds done without TLS.  After all, rejecting the bind doesn't protect
the password, that has already been sent in the clear.  But it
encourages the client to be fixed.

> I thought this had been settled. I thought I would see no more text on
> this.

I was arguing and arguing against the new rules when they were
suggested, but got little reply except a few misunderstandings.
OK, now I do get replies.

-- 
Hallvard

---

From: Mark Ennis <[email-masked]>
Date: Wed, 10 Mar 2004 13:38:18 +1100

Hallvard,

To answer your question about what X.500 servers do about wrong 
criticality, I can answer for our implementation of X.500. The answer is 
nothing. We use the criticalExtensions to determine whether any 
extensions we do not support are "critical" and if so, we generate an 
unavailableCriticalExtension error. Otherwise, we ignore the 
criticalExtensions except where critical extensions have been used to 
flag different behaviour (a misuse of critical extensions in my 
opinion), for example useAliasOnUpdate changes the behaviour of the 
dontDereferenceAliases service control option, rather than indicating 
the existence of a new flag for this purpose.

We treat the criticality flag in the LDAP controls in the same manner. 
If we support the control, we ignore the criticality flag.

- Mark.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Tue, 09 Mar 2004 19:22:06 -0800

>Whether there is any point in discussing it anymore I don't know.

It does seem we're beating a dead horse.

>I do wish someone knew what some X.500 servers do about wrong

I am pretty sure they behave in a manner consistent with most
LDAP servers.   That is, criticality is irrelevant to a server
making use of the extension information.  However, those
implementing DAP are welcomed to comment here.

But then, again, I don't think it matters all that much.  We
should be more concerned by what LDAP implementatoins are doing,
and how to revise the LDAP specification such that new
implementations are able to interoperate with existing as
well as other new implementations then attempting to re-engineer
LDAP to what we think was intended (or what we think it should
have been).

>client error.

They aren't forbidden per say, but prohibited from checking unless the
control specification states a specific requirement (against the guidance
against overloading criticality) for that they implement such a check.

Because that can lead to interoperability problems.  That is, while
you intend this to catch errors during development, it might have the
effect of producing non-conforming clients (because they follow a
server which doesn't provide the diagnostic, or provide s a bogus
diagnostic).

It also will lead to cases where an otherwise conforming client and
otherwise conforming server fail to interoperate simply because the
fail to agree to the proper value of criticality.  Such disagreements
should not be fatal to interoperability.

Also, having the server verify criticality limits the kinds of extensions
which can be made to LDAP.  That is, it makes it harder to extend
controls to support new operations, makes it harder to extend operations
extended by controls with new controls, etc..

In the same vein, consider issues regarding changes to the set of appropriate
messages which a control can be attached to.  Should a server verify that
a control whose specification said "MUST be attached only to a search request"
treat attachment of the control to a non-search request as a protocol error?
Shouldn't it be allowed for a new extended operation to say that this old
control can not be attached to the new operation?

What if the appropriate criticality for this new operation was different?
Would it be appropriate to new extended operation to say "old control MUST be
critical" even though the specification for that old control said it MUST
always be non-critical?

And what happens when a new security consideration arises that suggests
that a control, whose previous specification said be non-critical, should
not be critical in a some cases.  A sender verification requirement would
disallow simply changing the guidance provided to the client developer
(or user), but force the introduction of a replacement control.

That is, have criticality be "in the eye of the client" allows a great
deal of revision and extension flexibility.

>There are plenty of client errors that servers may, but
>it can be so damaging.

It can be damaging to unnecessarily catch errors that don't matter.
That is, the "be liberal in what you accept" principle is quite sound.

>As important as rejecting cleartext password binds done without TLS.

Bad example.  Use of cleartext passwords without TLS is not a protocol
error.

>After all, rejecting the bind doesn't protect the password, that has already been sent in the clear.  But it encourages the client to be fixed.

Some argue that such checks encourage complaints to be made to the
server developer not the client developer.  Some vendors don't think
its there place to police other vendors.

Kurt

---

From: Hallvard B Furuseth <[email-masked]>
Date: Thu, 11 Mar 2004 22:33:38 +0100

Kurt D. Zeilenga writes:

> what happens when a new security consideration arises that
> suggests that a control, whose previous specification said be
> non-critical, should not be critical in a some cases.  A sender
> verification requirement would disallow simply changing the guidance
> provided to the client developer (or user), but force the introduction
> of a replacement control.

Good point.  That applies to both client verification of user-supplied
criticality, which [Protocol] does allow, and server verification of
criticality.  It looks like X.500 knows what it is doing in only
allowing mandates of TRUE criticality.

I suggest we forbid control specs to mandate a request control to be
non-critical.

Leave response criticality as it is, since it is ignored, and since RFCs
2649 and 2891 already mandate response controls to be non-critical.

If there are existing control specs that mandate a request criticality
of FALSE, we could forbid both servers and clients to verify that,
but the real fix is to update the control specs.

-- 
Hallvard

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 11 Mar 2004 14:27:06 -0800

I don't think this is a good enough reason to mandate that of 
control specifications.   I am sure someone could come up with
a reasonable control extension which should never be marked
critical, for instance one specifically designed to test that
servers are properly ignoring non-critical controls.  I think
it better to leave whether such a mandate, if one is ever
proposed in a control specification, is appropriate or not
should be left to review/consideration of that specification
for publication (whether by the IETF or other party).

I think this kind of consideration (and I think it really just
something extension authors/reviewers need to consider) is best
left to a document detailing guidelines to authors of extensions.

Kurt

---

From: Hallvard B Furuseth <[email-masked]>
Date: Fri, 12 Mar 2004 00:09:48 +0100

Ron, you'll find an answer to your last message on the bottom.
Mark, thanks for your X.500 example.

I'll keep replying while the argument is going, but if we still disagree
after this message, and no X.500 (or LDAP?) servers we should allow for
turn up which do check criticality, I'll accept the case for disallowing
the criticality check as lost.

I still want language which explicitly allows control specs to mandate
criticality, or maybe just to mandate TRUE criticality, but I think
that's another subthread.

> LDAP servers.  (...)
> But then, again, I don't think it matters all that much.

Well, an LDAP gateway to an X.500 server which checked criticality would
be breaking the [Protocol] spec, unless each relevant control spec said
(against [Protocol]'s recommendation) that the server may return an
error for wrong criticality.  I agree there could be stronger arguments
against.  Anyway, the only X.500 example that have been posted so far
goes against me:-)

> We
> well as other new implementations (...)

A check would only break broken clients, so I'm not too concerned.

> against overloading criticality) for that they implement such a check.

Right, but I'm not talking about individual control specs that alter the
semantics to the criticality field.

I'm talking about control specs that simply mandate a criticality, and
servers that check mandated criticalities in general.
That and invalid values in search filters are the only errors I can
think of at the moment which servers will be forbidden to catch.

> server which doesn't provide the diagnostic, or provide s a bogus

That's the weirdest argument for anything I've heard for a long time.
Server bugs can happen in any area no matter what we specify.  Clients
that cater to such errors should take care to work with correct servers
as well.  If we follow your argument, LDAP should have had fewer and
less informative error codes, and no diagnosticMessage.  And e.g.
require clients to handle spaces after commas in DNs, in case some
server returns that.  That would be more interesting, since I still see
DNs with spaces.  I'm not even sure if that's a bug or not.

> should not be fatal to interoperability.

The same could be said of any bug or optional feature.  Spaces after
commas in DNs, integers prefixed with zeroes, not sending Bind after
StartTLS (even though other servers may invalidate the association).

(BTW, I assume either your client or server above would be conforming,
not just 'otherwise conforming'.  There are only two choices for
criticality, after all.)

> extended by controls with new controls, etc..

You mean it makes the criticality check harder to implement?  I don't
see that.  The criticality of most controls can be verified when the
control is parsed.  The exception is if the control spec - or the spec
of another extension which overrides the control spec - says that the
control is critical in circumstances which can not be determined just by
parsing that control.

Anyway, so what?  It's the implementor's choice how much error checking
to do.  Thorough error checking has always been tedious.

> In the same vein, consider issues regarding changes to the set of
> appropriate messages which a control can be attached to.

Now that you mention it, I note that a literal reading of rfc2251
forbids the server to return protocolError for the wrong criticality of
an inappropriate control.  (It must return unsupportedCriticalExtension
if it is marked critical, and ignore it otherwise.)  However, the same
goes for a control with malformed controlValue (inside a well-formed
OCTET STRING, that is), and I don't expect implementations take care to
ignore parse errors in inappropriate controls.

Except for that, this has nothing to do with criticality, but:

> Should a
> server verify that a control whose specification said "MUST be
> attached only to a search request" treat attachment of the control to
> a non-search request as a protocol error?

According to [Protocol] and rfc2251, attaching an inappropriate control
to an operation is not an error, it just means the control should not be
applied.  But if the control is marked critical as well, the server
returns an error.  (unsupportedCriticalExtension, not protocolError.)

I guess one could claim that your example doesn't actually state that
the control is only appropriate for Search, and therefore that the
server MAY return protocolError if it is attached to other operations.

> Shouldn't it be allowed for
> a new extended operation to say that this old control can not be
> attached to the new operation?

Sure.  So when a server which already implements the control, also
implements the new operation, it adds the new operation to the list
of operations appropriate for the control.  What of it?

> What if the appropriate criticality for this new operation was
> different?  Would it be appropriate to new extended operation to say
> "old control MUST be critical" even though the specification for that
> old control said it MUST always be non-critical?

I don't know.  I wouldn't write such an extension without asking
e.g. ldapext for advice.  But that's still another issue than verifying
criticality.  It is an example of making the criticality more cumbersome
to verify as discussed above, but in that regard it is no different from
if the control spec itself said that it is critical with one operation
and non-critical with another.

> deal of revision and extension flexibility.

Addressed in my recent message 'Mandated non-critical controls'.

>> There are plenty of client errors that servers may, but need not,
>> catch.  I think it's an important client error to catch, since it can
>> be so damaging.
> 

True, but as you may have noticed, my point is that this error does
matter.  If we are going to be liberal in _everything_ we accept, please
remove error returns from [Strprep] and [SASLPrep].  BTW, that's
something I want anyway, so I guess we both are on both sides of the
fence with regards to that principle.

> Bad example.  Use of cleartext passwords without TLS is not a protocol

I didn't mean to imply it was.  It is example of what Ron called
  "I didn't do it. I could have but, na-na-na, you got the (***)
  wrong. Let that teach you a lesson!"
which, as it happens, this example was a reply to.  And I think
both are for good reasons.

If you want an example of rejecting an error which one is able to handle
just because some other server might reject it (as Ron complained this
was not an example of), try a server which is stricter with the syntax
of attributes in update requests than with filter items, and refuses to
add an integer value with leading zeroes instead of stripping the
leading zeroes as it does in filters.  Or maybe rejecting DNs with
spaces, it's not clear to me if those are errors or not.

>> After all, rejecting the bind doesn't protect the password, that has
>> already been sent in the clear.  But it encourages the client to be
>> fixed.
> 

Vendors who think so are free to act accordingly.

Vendors can also implement optional checking in order to satisfy people
on both sides of that argument.  Except in the case of checking control
criticality.

For some types of applications, I really appreciate programs with an
option for really paranoid error checking, which I only turn on for bug
hunts and for development of other software.

-- 
Hallvard

---

From: Hallvard B Furuseth <[email-masked]>
Date: Fri, 12 Mar 2004 00:28:38 +0100

I wrote:

> which, as it happens, this example was a reply to.

Before someone complains: I just noticed that I read of 'wrong' as
something like 'bad, therefore unsupported here' while Ron probably
meant 'error'.  Except for that, the point is the same.

-- 
Hallvard