Subject: [ldapext] WG last call on the Virtual list View draft

From: Roland Hedberg <[email-masked]>
Date: Fri, 13 Sep 2002 07:57:36 +0200

Hi !

We (the wg chairs) feels that this draft is ready to progress to the
next level which is to regard it as done by the working group and
turn it over to the IESG for further processing on its way to become an RFC.

So we would like to hear from you within the working group if there
is anything in the draft that is questionable or wrong before we do that.

You have two (2) weeks to solicit your comments to myself, Mark Wahl,
the authors or the working group. 

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-ldapext-ldapv3-vlv-08.txt

-- Roland
_______________________________________________
Ldapext mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ldapext

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 10:58:24 -0700

I believe that VLV should be designed such that a client need
not request sorting of entries.

Just as one can page through unsorted results using Simple Page
control [RFC2696], one should be able to VLV through unsorted results.
All that is necessary is for the server to provide some repeatable
ordering of the entries involved. In many servers, the server maintains
all entries in this fashion. In other servers, the server may need
to maintain the order in some context specific way.  The contextID
feature allows for this. And, in servers which only care to support
VLV where a sort control has been provided, the server can return
an appropriate error (which has been defined in the I-D).

I suggest VLV only RECOMMEND implementation of RFC 2891 and
that VLV not require the sort control be provided.

With this change, the VLV operation would provide all the
functionality of the Simple Page control.  This would allow
the RFC 2696 (Informational) to be obsoleted in favor of
a Standard Track approach.   Otherwise, it will likely be
necessary to revise RFC 2696 for publication on the Standard
Track.

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 11:19:04 -0700

The document introduces an open-ended set of VLV-specific result
codes which differ only from existing codes in that they
"pertain specifically to the VLV operation".  As all existing
codes returned pertain specifically to the operation they are
returned with, it is unclear what the meaning of each new code
is.

>From the example, it seems the authors what servers to determine
whether the fault would have occurred if another operation
(same request less VLV control) had been issued and, if so,
use an existing code and if not use a new code.  This, IMO,
is not reasonable.  Generally, one should not determine the
result of one operation based upon the processing of some
other operation... especially one which never was submitted.

Consider timeLimitExceeded.  If timeLimit is exceeded, this
specification implies the server needs to determine whether it
would have been exceeded if the VLV control was not present.
Implementing this would likely be extremely difficult.

This document should new define error codes only where it
there are specific fault conditions which need to be called out.

For example, a code which indicates that greaterThanOrEqual
assertion value doesn't conform to the syntax of the
sort control matching rule should be defined.  (Note that
this is a condition which isn't presently called out but
should be.)

My review notes (which I will soon post) detail additional
concerns in this area.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 11:33:19 -0700

I would like to see the request control extended to allow
specification of a particular target entry by name.

It is often desirable to re-target a view about a particular
entry.  For example, when paging down, it is often
desirable to ensure that the last entry on the previous
page is the first entry on the following page.  Other
selection means are not suitable for this as they are
imprecise (in the face of concurrent DIT adds/deletes).

It is also desirable to center views around a particular
entry.  For example, one has a list displaying rankings
of players (of some game) and the user would like center the
list on a particular player (selected by some means
separate from the list).

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 11:35:31 -0700

I am surprised that negative offsets are not allowed, as allowed       
in many GUI list controls.  Possibly the designers intend that
reversing sorting be used instead.  I note, however, that reverse     
sorting may require the client to reverse the results to get the
desired behavior in the GUI.  Adding negative offsets to this control    
is likely straight forward.

Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 11:40:35 -0700

Attached are my review notes...

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Wed, 25 Sep 2002 13:48:52 -0700

> specification of a particular target entry by name.

Name ? Can you expand on this idea ?
VLV as specified is designed to allow
the display of entries before and after an
entry matching a particular value, in a sorted list.

It isn't clear to me what you want in addition to this capability.

---

From: "David Boreham" <[email-masked]>
Date: Wed, 25 Sep 2002 14:21:45 -0700

Comments on your notes:

>I believe this control should also be usable without a sorting
>control.  Just as one can page through unsorted results using
>[SPaged], one should be able to VLV through unsorted results.

This has been discussed several times in the past.
Can you give an example of where it would be useful
to use VLV on a randomly sorted result set ?

>This MUST should be dropped.  It should be acceptable for a VLV
>implementation not to support [SSS], just as a [SPaged] implementation
>need not support [SSS].  It should be possible to use VLV with other
>ordering controls instead of [SSS].

I've not been paying attention. Can you point me at the other
ordering controls please ? (when VLV was written , there were none).

>All that is necessary is is to say that implementations must provide
>unsorted results in some repeatable order.  That is, the order must
>be consistent for subsequent operations.  It should be sufficient
>to outline the ordering requirements and to state those requirements
>are not provided, the VLV control cannot be used.

Again, this has been discussed many times long ago.
The view taken was that 'repeatable order' is too vague a 
concept to be useful in the context of an interoperability standard.
Perhaps you could provide a use case where it would be
valuable ?

>I believe VLV should offer all the capabilities of [SPaged].  In
>fact, it's my opinion that VLV should obsolete [SPaged].

This was the original goal for VLV. However, due to the 
timeline for some vendors implementations, it didn't happen.
Your support is welcome.

>I note that there is no way to specify the target be a particular
>entry.  That is, by DN.  Whether or not select by DN is particularly
>useful is another matter.

Hmm....I remember thinking about this. Are there rules for
magnitude comparison of DN values ? If so then perhaps this
'just works (tm)'. 

>What if the client has two different virtual lists?  Should
>the contextID be associated with a particular virtual list?
>Also, is the client free to change any and all search/VLV
>parameters associated with a virtual list and, hence, a
>contextID?

Some background: the first VLV implementation was Netscape's.
Netscape's implementation was stateless, and had no context ID.
Later, another vendor with a different server architecture 
started to implement VLV. They discovered that their more stateful
design required that the client context cookie be added to the protocol.
The resulting wording in the RFC is designed to allow both designs
to interoperate.

So to answer: yes a contextID is associated with a particular
view onto a particular list (hence the term context rather than list
or client). It seems sensible to allow a client to submit whatever
it likes for the contextID (we're not the protocol police after all),
but if it doesn't play by the rules (submit the value previously 
supplied by the server), then performance and/or results could
be undefined. 

I won't comment on all the result code stuff: it's from after my 
work on the document, and it plain made my head spin !

Similarity to SPaged: discussion at previous WG meetings
persuaded me that SPaged and VLV are sufficiently semanticically
differnent that it was not appropriate to try to make VLV a superset
of SPaged functionality. For example: in SPaged, the server never
needs to 'go backwards' in the result set. Therefore the lack 
of a need to specify sort order is not a practical problem.
However, in VLV, we can go backwards, and the result set
could be spectacularly large. Hence there would be a need
to store sufficient state on the server side to ensure that regardless
of the clients's partying back and forth on the result set, we never
get lost as to what we've returned before and what comes 
next. It is not at all clear to me that this is a) tractable and b) 
advisable. This is one of the reasons why sort order is required
in VLV. 

I note further than some implementations will have a 'natural order'.
e.g. UMich-derived implemenations will typically return results in
entryID order. So it is tempting to say 'heck, no sort control, we 
just return entries in entryID order'. I believe that this thinking is
too much driven by experience with that particular implementation.
Further, if there is such a thing as 'the natural sort order', I'd rather
see that specified with some kind of ordering control, rather than
implied by the absence of any ordering control. To me, no ordering
specified means totally random and not repeatable order, in the worst case.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 14:31:29 -0700

>Name ? Can you expand on this idea ?

distinguished name.

>entry matching a particular value, in a sorted list.

A particular value selects only the first entry which
has a greater than or equal value.  There could be
numerous entries which are all equal to the particular
value.  The client many want to select a particular target
entry.

>It isn't clear to me what you want in addition to this capability.

        VirtualListViewRequest
        ...
          CHOICE {
                ...
                name    [2] distinguishedName
          }
        }

(as indicated in my review notes, I believe the CHOICE
should be given an identifier)

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Wed, 25 Sep 2002 14:51:17 -0700

> distinguished name.

Ah !

>           CHOICE {

So you're asking for the ability to specify DN in place of
an attribute name, in the target specification ?

Seems reasonable. Is there a reason why 'dn' can't
be submitted as an attribute name ? (I suspect there is,
I don't know enough about X.500 to undersand why 
dn is different to other attribute names).

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 14:59:18 -0700

>to use VLV on a randomly sorted result set ?

Paging of results.  Presently there is no standard track
mechanism to page through results.

>ordering controls please ? (when VLV was written , there were none).

My point is, there will be other ordering controls defined
eventually.  Whether they are defined because RFC 2891 doesn't
fulfill a particular ordering requirement or because RFC 2891
is flawed or whatever, is not terrible relevant.  VLV should not
care how the ordering is provided, just that ordering is provided.
RFC 2891 should be viewed only as one way to provide ordering.

>concept to be useful in the context of an interoperability standard.

First, I don't think 'repeatable order' is too vague.

Second, the ordering provided by RFC 2891 is equally vague.

For example, consider a set of entries who all have as
the attribute "givenName" the value "joe" which have been
sorted, per RFC 2891, by this attribute.  RFC 2891 does
not detail what order they are to be returned in.

VLV, however, assumes that the order is repeatable.
This is all that is necessary for VLV to work.

>Perhaps you could provide a use case where it would be

Paging of results.  Determining the number of entries
to be returned without requiring them to be returned.

>magnitude comparison of DN values ? If so then perhaps this

There is no ordering rule for DNs.  But there doesn't need
to be for a DN to be used to select a target entry from
an ordered set.

>supplied by the server), then performance and/or results could

The wording in the document implies the last contextID returned
is to be returned, not the last contextID for a particular
virtual list view should be returned.  Likely some simple
rewording would resolving this issue.

>specified means totally random and not repeatable order, in the worst case.

I note that worse case under RFC 2891 is the same as no ordering
control.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 25 Sep 2002 15:04:28 -0700

>an attribute name, in the target specification ?

No, I'm asking to specify DN in place of byoffset or
greaterThanOrEqual.

>dn is different to other attribute names).

A number of reasons, but most importantly:

The DN is to specify the target entry in an ordered list,
not how to order the list (which what the attribute
name is used for).

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Thu, 26 Sep 2002 12:28:53 -0700

> No, I'm asking to specify DN in place of byoffset or

I can't argue that this is a useless or insane thing to want.
However, I don't see a compelling use case which would
warrant changing the protocol at this stage in the game.

Suggest that this belongs in a VLV2 protocol.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 26 Sep 2002 13:51:10 -0700

At 12:28 PM 2002-09-26, David Boreham wrote:
>warrant changing the protocol

The use case is precise "scroll up" and "scroll down" actions.

Without this, a "scroll up" may result in entries below
the current view being returned.  Without this, a "scroll down"
may result in entries above the current view being returned.

Note only do I believe this feature should be added, that the
document needs to be rewritten to to use precise target
entry selection (by DN) for scrolling.

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Thu, 26 Sep 2002 14:05:57 -0700

> may result in entries above the current view being returned.

This was discussed at length a couple of years ago.
The solution proposed, agreed, documented and implemented
was the cookie mechanism.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 26 Sep 2002 14:32:31 -0700

>was the cookie mechanism.

If cookies were mandatory to implement, always provided by
the servers, always returned by the client, and the
specification stated that the server must ensure continuity
between scroll up/down events, the cookie mechanism might
work.

However, even with this, VLV still wouldn't provide a 
precise mechanism to select a particular target entry.

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Thu, 26 Sep 2002 14:51:49 -0700

> between scroll up/down events, the cookie mechanism might

Cookies are optional because they're a performance optimization
required only by a subset of implementations. 

I'm not clear what you're asking for by 'continuity'.
Are you concerned about missing entries from the list in the general
case, or where the list is modified concurrently with the scrolling ?

> precise mechanism to select a particular target entry.

Correct. Isn't this orthogonal ? Can we seprate the two issues ?

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 26 Sep 2002 15:42:15 -0700

>required only by a subset of implementations. 

Then don't sell it as a solution to list continuity issues.
In the below discussions I will assume no cookie was
provided by the server.

>I'm not clear what you're asking for by 'continuity'.

Discontinuity would be:

User is provided a view into a list showing "Apples, Washington"
to "Oranges, Florida".  The user requests the list be scrolled
down expecting the new view to show "Oranges, Florida" to whatever.
(The application is designed to show overlap of one list
entry between views and the list is sorted first by "fruit"
and then by "state").

However, the new list shows "Pinapples, Hawaii" to
"Tangerines, California".

The new view is not continuous with the prior view.  There
is a discontinuity.

Selection by DN is useful not only to provide continuity
but to detect and, where detected, resolve address discontinuity
issues.

For instance, selection by DN would have allowed the client
select the entry listed as "Tangerines, California" as the
target.  Or, the client requesting additional overlapping
entries, could actually detection changes to the target
entry which affect its ordering and then choose whether
to continue using it, or to select another target entry.

>Correct. Isn't this orthogonal ?

No.  Because selection by DN is a mechanism for precisely
selecting a particular target entry.  No other mechanism
for precisely selecting a particular target entry has been
suggest.

>Can we seprate the two issues ?

The issue is there is no precise mechanism.
Selection by DN is a precise mechanism.

I don't see any reason to separate the issue from a possible
solution.

---

From: "David Boreham" <[email-masked]>
Date: Thu, 26 Sep 2002 16:00:12 -0700

> The new view is not continuous with the prior view.  There

Ah yes. We had this discussion a couple of years ago too.

>case, or where the list is modified concurrently with the scrolling ?

It would help to know which case you are concerned about.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 26 Sep 2002 16:47:31 -0700

>Ah yes. We had this discussion a couple of years ago too.

I don't dispute that the problem was previously discussed.

>>case, or where the list is modified concurrently with the scrolling ?

I am concerned primarily by the latter.

If we look back at the scenario I described:

  However, the new list shows "Pinapples, Hawaii" to "Tangerines,
  California".

The obvious problem is that the server doesn't know what
actions the user has taken, it only knows what the parameters
the client provided to it.  The current mechanism requires the
server to guess whether the new offset / content count
indicates a new (independent) positioning by their ratio
(where "jumping" to new view is expected) or they indicate a
scroll up/down action.

As long as the server has to guess, there will be problems.

An appropriate solution needs to avoid discontinuities,
allow for discontinuities to be detected, and provide
a means to address discontinuities.

Additionally, the solution should not require complex
state management.  The solution should be "stateless".

I think an scrolling approach based upon select by DN
is not only an appropriate solution (as described above),
but also doesn't involve significant state management.
In fact, from a protocol perspective, the DN approach
is "stateless".

Kurt