Subject: Re: KITTEN: IETF 75 - 76

From: Michael B Allen <[email-masked]>
Date: Tue, 1 Sep 2009 13:12:02 -0400

On Tue, 18 Aug 2009 00:13:28 +0200
Leif Johansson <[email-masked]> wrote:

> > >    2. listing/iterating credentials
> > >    3. exporting/importing credentials
> >
> > I think Leif has expressed interest in (3) before.
> 
> That was exporting unfinished contexts actually - slightly different.
> 
> >
> > >    4. error message reporting
> > >    5. asynchronous calls
> >
> > The last 2 are generally required for use by modern applications.
> 
> hmm, is asynchronous calls related to exportable contexts perhaps...?

Exactly they are. To implement async (properly) you would need to be able to serialize a context at any step.

Mike

-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 1 Sep 2009 12:31:11 -0500

> able to serialize a context at any step.

Internally, maybe, but as far as the application goes: no.

To me async means a pair of functions called, say,
gss_init_sec_context_async() and gss_accept_sec_context_async() that do
not block, but may return an indication that the call did not complete,
and which provide some way for event notification so that the
application can call them again when they are ready to be called again.

Here's one illustration based on event notification:

	major = gss_init_sec_context_async_event(&minor,
	    cred, &ctx, target, mech, req_flags,
	    GSS_C_INDEFINITE, cb, &input_token, &output_token,
	    &ret_flags, NULL, &event_notification);

	if (major == GSS_S_CONTINUE_NEEDED &&
	    output_token.length == 0) {
		/*
		 * hook event notification and contextual args
		 * to gss_init_sec_context_async() into our event loop
		 * and return.  when the event fires we'll get here
		 * again and call gss_init_sec_context_async() to make
		 * progress.
		 */

		queue_up(&event_notification, cred, &ctx, target, mech,
		    ...);
	}

Here's another based on background threads and a completion callback
instead of an event notification:

	major = gss_init_sec_context_async_cb(&minor,
	    cred, &ctx, target, mech, req_flags,
	    GSS_C_INDEFINITE, cb, &input_token, &output_token,
	    &ret_flags, NULL, cb_func, &cb_data);

	if (major == GSS_S_CONTINUE_NEEDED &&
	    output_token.length == 0) {
		/*
		 * Our callback will get called when
		 * gss_init_sec_context_async_cb() completes its work.
		 * The callback will arrange to get us called again.
		 */
		return (...);
	}

No context serialization needed.

The problem with the event notification variant is that event
notification is very OS-specific, and even library specific (think
kqueues, event ports, poll(), select(), /dev/poll, libevent, ...).  On
Unix and Unix-like systems we can probably make the event_notification
be just an array of file descriptors.

The problem with the callback variant is that it implies a threaded
process model (which I don't mind, but which won't be feasible for some
implementors).

Nico
--

---

From: Shawn M Emery <[email-masked]>
Date: Tue, 01 Sep 2009 11:53:32 -0600

Folks,

We've had great discussion on the list about the new work items.  So now 
we need volunteers for editors of each of the items.  They would need to 
filter this thread and capture the design in corresponding ID(s).  If 
they have further questions/clarifications then please post this to the 
list as there has been feed back from multiple sources.

If you would like to be an editor then please e-mail the list, so that 
we can coordinate assignments.  We would prefer to have at least one 
editor to progress this WG.

Thanks,

Shawn.
--
Shawn M Emery wrote:
> Folks,
>
> As our AD (Tim) had mentioned at the last session, we should gage 
> interest in the working group going forward by polling the members to 
> see what items they would be willing to work on between now and IETF 
> 76 (starting November 8).  Tim would like one or two new drafts to be 
> created as an indicator.
>
> Love had given a set of new work items that would be of interest, as 
> follows:
>
>    1. initialization/new credentials
> ...along with Alexey's recent request for policy/encryption strength.
>
> We are looking for authors and editors for any of these new items or 
> something else that you would like to see developed within KITTEN.
>

-- 
Shawn.

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 01 Sep 2009 11:13:03 -0700

> process model (which I don't mind, but which won't be feasible for  

It does not require threads, the caller of the cb enabled ISC will  
have to call the event loop, that in turn will call the callback call.

It does require though that you select one event process system (or  
threads).

Love

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 1 Sep 2009 13:52:10 -0500

> It does require though that you select one event process system (or  

Yes, you're right, the callback variant implies one of: threaded process
model or async I/O event loop model.  We should at least specify that
the callback may be called in a different thread in the case of threaded
process models.

Nico
--

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 01 Sep 2009 14:27:04 -0700

> 	    &ret_flags, NULL, cb_func, &cb_data);

The callback should take all output variables, for example:

OM_uint32
gss_acquire_cred_ex_f(gss_status_id_t /* status */,
		      const gss_name_t /* desired_name */,
		      OM_uint32 /* flags */,
		      OM_uint32 /* time_req */,
		      const gss_OID /*desired_mech */,
		      gss_cred_usage_t /* cred_usage */,
		      gss_auth_identity_t /* identity */,
		      void * /* ctx */,
		      void (* /* complete */)(void *, OM_uint32, gss_status_id_t,  
gss_cred_id_t, gss_OID_set, OM_uint32));

OM_uint32
gss_iter_creds_f(gss_status_id_t /* status */,
		 OM_uint32 /* flags */,
		 gss_OID /* mech */,
		 void * /* userctx */,
		 void (* /*cred_iter */ )(void *, gss_OID, gss_cred_id_t));

Love

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 1 Sep 2009 16:38:53 -0500

> The callback should take all output variables, for example:

As I imagined it the callback would only be to indicate that GISCACB()
must get called again to complete its work.  (And it might well need
fail to complete at that point.

Think of krb5_gisc: it may need to do multiple DNS lookups, and multiple
TGS exchanges; each time that some I/O is ready that it needs it could
call the callback, the app then calls back into GISCACB(), krb5_gisc
makes some progress, and if it'd block then it returns continue-needed
without an output token.  Lather, rinse, repeat.

I like it better that way.  If we add a new version of GISC with new
outputs we'll need not only a new GISC_async, but also a new callback
function prototype.  Having the callback lead to the application calling
GISCACB() again seems reasonable and simple enough to me.  I'm not
wedded to this, but it does seem simpler.

The callback in the credential iterator can be the way you propose.
That's because it doesn't need every output of gss_inquire_cred*(), just
a cred handle and mechanism OID, and it can find the rest from that.

Nico
--

---

From: Michael B Allen <[email-masked]>
Date: Tue, 1 Sep 2009 18:13:07 -0400

On Tue, 01 Sep 2009 14:27:04 -0700

> gss_cred_id_t, gss_OID_set, OM_uint32));

Holy moses that's complicated. Do you really need callbacks? Why not just have a flag like GSS_C_NOWAIT that indicates the call should return immediately even if the overall operation has not completed yet? If the status indicates more processing is required (EAGAIN for C binding for example) it means that you need to call it again later. I use this technique in my own stuff and the implementation turns out to be surprisingly clean.

Also note that in an HTTP server if you cannot serialize the context while juggle concurrent authentications you may have to commit to using resources unnecessarily (such as processes on UNIX). That was my primary thought when I first posted to this thread. It's not clear to me that your model would help that scenario and that scenario is fairly important IMO. Would callbacks work with a stateless transport like HTTP?

Mike

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 01 Sep 2009 15:18:13 -0700

1 sep 2009 kl. 15:13 skrev Michael B Allen:

>  it means that you need to call it again later

When is later unless you get a callback ?

So far it have not been complicated, note that gss_acquire_cred_ex()  
does more then gss_acquire_cred().

Love

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 1 Sep 2009 17:20:59 -0500

> When is later unless you get a callback ?

I agree.  Callbacks are easy.  My construction (for ISC) is simpler: no
need for all the ISC output args in the callback.

Love's credential iterator callback is trivial.

Callbacks are very common in API design.

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 01 Sep 2009 15:39:33 -0700

> need for all the ISC output args in the callback.

So what is input token for the second call ? Can input token be freed  
between first and second call ?

Why do I have to pass all those argument that might (or might not) do  
something.

Love

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 1 Sep 2009 17:54:02 -0500

On Tue, Sep 01, 2009 at 03:39:33PM -0700, Love Hörnquist Åstrand wrote:
> >I agree.  Callbacks are easy.  My construction (for ISC) is simpler:
> >no need for all the ISC output args in the callback.
> 

It should be the same as in the previous call.  But that can be relaxed
-- we could say that the mech must keep an internal copy if it will
continue needing it after the first call.

> Why do I have to pass all those argument that might (or might not) do  

If it's hard to track them you can always place them in the callback
function's argument -- a void * that would be a pointer to a struct with
all the context that the callback needs, including those GISC args if
you're not storing them elsewhere.

---

From: Michael B Allen <[email-masked]>
Date: Tue, 1 Sep 2009 21:40:59 -0400

On Tue, 01 Sep 2009 15:18:13 -0700

> When is later unless you get a callback ?

When you call the function with the flag GSS_C_NOWAIT, the implementation transmits the network request [1] and then checks to see if data can be read (such as by using select(2)). If data is not available, the function simply returns a status of EAGAIN. Now the caller is free to perform other work (and thus it is async). But the caller now knows that the function needs to be called again to complete the operation. So later, it calls the function again. If the network response has been received, it returns success. Otherwise, if GSS_C_NOWAIT was supplied, it again returns EAGAIN. If GSS_C_NOWAIT was not supplied, it waits for the operation to complete (e.g. for a network response to be received) just like the non-async call. In fact async and non-async uses the same function - the GSS_C_NOWAIT flag alone controls async behavior.

Mike

[1] A network request is only one example, the behavior is the same for any blocking operation.

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 1 Sep 2009 22:23:54 -0700

> you're not storing them elsewhere.

I find this more complicated then just returning the values to the  
callback function.

In fact, I find it more complicated to have my callback function have  
to save values so it can call the same function is just was called  
from just to call the called function again. It just doesn't make sense.

Love

---

From: "Stefan (metze) Metzmacher" <[email-masked]>
Date: Wed, 02 Sep 2009 09:46:33 +0200

> When you call the function with the flag GSS_C_NOWAIT, the implementation transmits the network request [1] and then checks to see if data can be read (such as by using select(2)). If data is not a…

That is true for system calls like send() and recv(), where the caller
is supposed to call select. But for gssapi the caller would have to poll
like this:

while (1) {

	ret = gss_call(&minor, ...);
	if (ret != 0 && minor == EAGAIN) {
		continue;
	}
	...
	break;
}

Or the application needs to try every few milli seconds...

What about a generic callback infrastructure:

typedef void *gss_async_cookie_t;

OM_uint32 gss_init_async_cookie(
	void (*callback)(gss_async_cookie_t *cookie,
                         void *callback_private),
	void *callback_private,
	gss_async_cookie_t **cookie
	);

OM_uint32 gss_release_async_cookie(gss_async_cookie_t **cookie);

#define GSS_S_ASYNC 12345678

and then either one function that autodetects if it should use
the input parameters based on its state in the async cookie.

OM_uint32 GSSAPI_LIB_FUNCTION gss_init_sec_context_async
           (OM_uint32 * /*minor_status*/,
            const gss_cred_id_t /*initiator_cred_handle*/,
            gss_ctx_id_t * /*context_handle*/,
            const gss_name_t /*target_name*/,
            const gss_OID /*mech_type*/,
            OM_uint32 /*req_flags*/,
            OM_uint32 /*time_req*/,
            const gss_channel_bindings_t /*input_chan_bindings*/,
            const gss_buffer_t /*input_token*/,
            gss_OID * /*actual_mech_type*/,
            gss_buffer_t /*output_token*/,
            OM_uint32 * /*ret_flags*/,
            OM_uint32 * /*time_rec*/,
            gss_async_cookie_t * /* cookie */
           );

or two functions.

OM_uint32 GSSAPI_LIB_FUNCTION gss_init_sec_context_input
           (OM_uint32 * /*minor_status*/,
            const gss_cred_id_t /*initiator_cred_handle*/,
            gss_ctx_id_t * /*context_handle*/,
            const gss_name_t /*target_name*/,
            const gss_OID /*mech_type*/,
            OM_uint32 /*req_flags*/,
            OM_uint32 /*time_req*/,
            const gss_channel_bindings_t /*input_chan_bindings*/,
            const gss_buffer_t /*input_token*/,
            gss_async_cookie_t * /* cookie */
           );

OM_uint32 GSSAPI_LIB_FUNCTION gss_init_sec_context_output
           (OM_uint32 * /*minor_status*/,
            gss_OID * /*actual_mech_type*/,
            gss_buffer_t /*output_token*/,
            OM_uint32 * /*ret_flags*/,
            OM_uint32 * /*time_rec*/,
            gss_async_cookie_t * /* cookie */
           );

If the gss_init_sec_context implementation has finished the work
it notifies the callback that it's done. But to get the actual results
the caller needs to call the gss_init_sec_context_async() function again
(or the gss_init_sec_context_output() function) and pass the same async
cookie.

I think something like this would make the api at least useful for Samba.

metze

---

From: Michael B Allen <[email-masked]>
Date: Wed, 2 Sep 2009 03:10:04 -0400

On Tue, 1 Sep 2009 22:21:57 -0700
Love Hörnquist Åstrand <[email-masked]> wrote:

> 
> 1 sep 2009 kl. 18:40 skrev Michael B Allen:
> 
> > But the caller now knows that the function needs to be called again  
> > to complete the operation.
> 
> so consumer got to poll the library, that is no got from the  
> performance view.

Actually now that I think about it I only use the NOWAIT flag technique for client oriented routines. For example, if you wanted to perform 10 HTTP requests in parallel, you could use the NOWAIT flag to initiate the requests and then not use NOWAIT when collecting the responses.

If someone wanted to use async for an event loop (a single thread / process handling an arbitrary number of contexts concurrently) then the NOWAIT flag technique by itself would indeed require polling and would be inefficient.

However, if the caller had access to the underlying polling routine (like through a "gss_process_events" function that called select or whatever), then the caller could know if the status of *any* operation had changed and thus polling would not be required. The benefit of doing this would be that a single thread / process could be used to do everything.

Mike

---

From: Love Hörnquist Åstrand <[email-masked]>
Date: Tue, 1 Sep 2009 22:21:57 -0700

Love

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 2 Sep 2009 10:32:41 -0500

On Wed, Sep 02, 2009 at 09:46:33AM +0200, Stefan (metze) Metzmacher wrote:
> Michael B Allen schrieb:
> > When you call the function with the flag GSS_C_NOWAIT, the
> > implementation transmits the network request [1] and then checks to
> > see if data can be read (such as by using select(2)). If data is not
> > available, the function simply returns a status of EAGAIN. Now the
> > caller is free to perform other work (and thus it is async). But the
> > caller now knows that the function needs to be called again to
> > complete the operation. So later, it calls the function again. If
> > the network response has been received, it returns success.
> > Otherwise, if GSS_C_NOWAIT was supplied, it again returns EAGAIN. If
> > GSS_C_NOWAIT was not supplied, it waits for the operation to
> > complete (e.g. for a network response to be received) just like the
> > non-async call. In fact async and non-async uses the same function -
> > the GSS_C_NOWAIT flag alone controls async behavior.
> 

Absolutely no polling.

Async I/O is not rocket science.  We can quibble over details, but we've
only got so many workable models.  And we have to be mindful that not
one model will result in fully portable application code.

Here are some reasonable models:

 - Completion callback (or, rather, a callback indicating that there's
   work to do, not necessarily completion)

   This implies either a threaded process model or an async I/O process
   model with one common event loop provided by the OS.

   This model is the simplest for apps to use, and it will be portable
   to Linux, *BSD, Solaris, Windows.  But not necessarily to embedded
   devices, real-time OSes, etc...

 - Return handles to I/O resources that the app can feed into its event
   loop.

   This is much more OS-specific.  On Unix/Unix-like OSes this means
   returning an array of file descriptors.  On Windows it means
   returning and array of file handles.  This too may not be terribly
   portable.  And it requires that the application plug the returned
   resources into its event loop, and take them out, on every call to
   these GSS functions.

I much prefer the callback model.  It will yield the most portable and
simplest code, even if it does not yield universally portable code.

> (or the gss_init_sec_context_output() function) and pass the same async

The partially established context argument is suitable for use as a
"cookie", therefore we don't need an extra cookie.

Callers of GISC and GASC today already have to ensure they're using all
the same input arguments (except for input tokens, of course), on every
call related to one security context establishement.  Therefore it seems
to me that it's not much to ask that async callers do the same.  That
means we can keep the callback prototype simple: it should take one, at
most two, arguments: a void * private data argument, and possibly the
"cookie" that you mention, which in this case would be the partially
established security context.

> I think something like this would make the api at least useful for Samba.

Anything that will actually work will, indeed, be useful to Samba.

Note that Samba can implement such a thing _now_ by using a worker
thread.

---

From: Michael B Allen <[email-masked]>
Date: Wed, 2 Sep 2009 17:20:15 -0400

On Wed, 2 Sep 2009 10:32:41 -0500

> > while (1) {
        gss_process_events(&minor, ...);
> > 
> > 	ret = gss_call(&minor, GSS_C_NOWAIT, ...);

The gss_process_events function would power the mechanism's event loop and wait for the status of an operation to change. Basically it would just call an implementation specific event loop without the 'loop'.

So there is no callback, it is not polling and it would be portable to pretty much any system because it does not require multiple threads / processes. You could juggle many GSS contexts with a single thread / process without breaking a sweat.

Note that I'm not necessarily advocating this technique. Currently I actually do something quite different for event loop stuff which seems to be the use-case we're talking about. There might be some horrendous flaw with it but I can't think of one at the moment.

Mike

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 2 Sep 2009 16:26:52 -0500

>         gss_process_events(&minor, ...);

The problem with this model is that it has the GSS-API impose its own
event loop on the application.  Imagine if every library did that!  An
application that uses multiple such libraries, where some such libraries
use other such libraries, would be very difficult to get to work at all,
particularly without threading or sub-processes.

I don't understand your aversion to callbacks.  Callbacks are
exceedingly common in C APIs nowadays, and that's a good thing, IMO.

Nico
--

---

From: Michael B Allen <[email-masked]>
Date: Wed, 2 Sep 2009 21:12:00 -0400

On Wed, 2 Sep 2009 16:26:52 -0500

> exceedingly common in C APIs nowadays, and that's a good thing, IMO.

I think you're exaggerating, if not misrepresenting, the detractions of the NOWAIT flag technique. But it is no matter. For a variety of reasons I will not pursue the issue further.

However, before I go back to lurking, I have to declare that callbacks are NOT simple. They can be quite complicated. Just from looking at function signatures posted in this thread I personally have no idea how these callbacks would actually work. Are they called by another thread, by a signal, through another call or by another caller? Is locking required? What data is safe to modify in the callback? They seem C oriented - do the semantics change if a different language is used like Java? Hopefully the answers to these questions will be clear if these callbacks do end up in a GSSAPI implementations.

Mike

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 03 Sep 2009 10:38:49 -0400

>> > 	if (ret != 0 && minor == EAGAIN) {

That's not another model.  That's equivalent to using the synchronous form 
of the call, if gss_process_events() blocks until the mech is ready to be 
called, or to polling, if it eventually times out.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 3 Sep 2009 10:26:38 -0500

On Wed, Sep 02, 2009 at 09:12:00PM -0400, Michael B Allen wrote:
> I think you're exaggerating, if not misrepresenting, the detractions
> of the NOWAIT flag technique. But it is no matter. For a variety of
> reasons I will not pursue the issue further.

I have worked on applications that use over half a dozen layered
components, at least two of which need to do blocking I/O operations,
plus the application itself being async I/O event driven.

Putting together such an application with your async I/O model would
result in an enormous amount of complication.

I'd much rather have the library give me either file descriptors to
select on, or a callback.

> However, before I go back to lurking, I have to declare that callbacks
> are NOT simple. They can be quite complicated. Just from looking at
> function signatures posted in this thread I personally have no idea
> how these callbacks would actually work. Are they called by another
> thread, by a signal, through another call or by another caller? Is
> locking required? What data is safe to modify in the callback? They
> seem C oriented - do the semantics change if a different language is
> used like Java? Hopefully the answers to these questions will be clear
> if these callbacks do end up in a GSSAPI implementations.

Callbacks are simple (and the ones I proposed certainly are).  One
should specify the calling context, as you point out.  To be most
portable we should say that callback functions must be async-signal-
safe, which means that callback functions should limit themselves to:
signalling condition variables, writing to a pipe, or otherwise posting
an event to another event loop.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 3 Sep 2009 10:31:16 -0500

On Thu, Sep 03, 2009 at 10:45:01AM -0400, Jeffrey Hutzelman wrote:
> Really, I think we have three options here:
> 
> 1) Assume POSIX threads(*).
> 2) Assume POSIX threads(*), but allow the existence of an implementation
>   dependent means of indicating use of some other threading model.

(2) is feasible.  You simply provide completion callbacks and say that
they must be async-signal-safe.

> 3) Recognize that there is no sane way to provide an portable async API
>   without either a portable threading model or a portable event model,
>   and give up.  Applications which want to call the GSS-API in an
>   asynchronous manner can simulate it by running the GSS-API call in
>   its own thread (under a platform-specific thread model).

4) Assume file descriptors and one or more of poll(), select(),
   /dev/poll, kevent, event ports, ...

Since (3) works today I'm very tempted to do nothing.

> (*) This means "you can't have this API unless you have threads".
> Unfortunately, it also means we're in the unenviable position of pushing
> an API which requires the use of threads but is not itself MT-safe.

The GSS-API is not inherently MT-unsafe.  Its MT-safety is not defined
-- _that_ is a problem.  (Should one lock around per-msg token functions
called with the same security context handle?)

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 3 Sep 2009 11:05:23 -0500

On Thu, Sep 03, 2009 at 05:43:11PM +0200, Volker Lendecke wrote:
> Sorry to step in here, but there's tons of example for
> portable async APIs. One that I just adapted Samba to is the
> API used in the avahi libs. You hand the avahi libs a set of
> callback functions like "listen on this fd for read" and so
> on, and you are called back by avahi when something
> interesting happens.

Yes, that too is possible: abstract the application's event loop and
export it to libraries.  That is complicated and only sightly more
portable than just exporting file descriptor lists from the library to
the app.

> This is definitely doable without the requirement for
> threads. It may be a complex API, but async programming
> never has been easy.

Indeed.

I'm not sure we'll ever reach consensus on a single async GSS API.  We
might have to do more than one, or none at all.

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 03 Sep 2009 10:45:01 -0400

--On Wednesday, September 02, 2009 09:12:00 PM -0400 Michael B Allen 
<[email-masked]> wrote:

> I think you're exaggerating, if not misrepresenting, the detractions of
> the NOWAIT flag technique.

No, he's not.  Nonblocking API's are OK, but there has to be a way to tell 
when it's time to try the call again, and neither periodic polling nor 
busy-wait is acceptable.  The GSS-API MUST NOT force such wasteful and 
performance-killing techniques on applications.

-- Jeff

---

From: Michael B Allen <[email-masked]>
Date: Thu, 3 Sep 2009 11:17:49 -0400

On Thu, 03 Sep 2009 10:38:49 -0400
> called, or to polling, if it eventually times out.

But there only needs to be one caller of gss_process_events for any number of NOWAIT calls. So you can build event loops with it. I'm not sure I understand the callback model being discussed but it seems to be an event loop powered by a separate thread firing callbacks in response to changes in status of calls. If that is in fact the case, you could build the callback model you're talking about using only the NOWAIT flag model I'm talking about.

Mike

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 3 Sep 2009 14:37:22 -0500

On Thu, Sep 03, 2009 at 11:17:49AM -0400, Michael B Allen wrote:
> But there only needs to be one caller of gss_process_events for any
> number of NOWAIT calls. So you can build event loops with it. I'm not
> sure I understand the callback model being discussed but it seems to
> be an event loop powered by a separate thread firing callbacks in
> response to changes in status of calls. If that is in fact the case,
> you could build the callback model you're talking about using only the
> NOWAIT flag model I'm talking about.

It may be reasonable to have libraries use an externally provided event
API (an alternative offered by Volker Lendecke).  (A callback model with
async-signal-safe callback functions will effectively do just that.)

It is not reasonable to have libraries provide an event loop for
applications to use.  The reason is as I've explained before.

Try putting together an application (and libraries) that uses:

 - an LDAP library, which uses a SASL library, which, through a plug-in,
   uses a GSS-API library, and through the GSS-API there's the Kerberos
   V5 GSS-API mechanism, which in turn might use a pluggable crypto
   lubrary, ...

 - also uses the GSS-API/SASL/crypto directly and independently of the
   LDAP library

Now, just for fun, add PKCROSS into the picture, deep in the bowels of
the krb5 mech.

If each library wants to have its own event loop then each one had
better provide a way for the caller to use that event loop for other
purposes, else there's no way to construct such an application and
libraries without resorting to threads.  Now say that each library does
provide a way for the caller to use the library's event loop, then each
layer in the above stack will have to do just that.

Once you've built that monstrosity there'll only be one thing left to
do: debug it!  ;)

That kind of complexity is a non-starter.  It's much easier to just wrap
a thread around any synchronous and slow operation that I want made
asynchronous.

Still, if you want to write an Internet-Draft outlining your proposal,
we'll review it.

Nico
--

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 03 Sep 2009 16:55:54 -0400

--On Thursday, September 03, 2009 11:17:49 AM -0400 Michael B Allen 
<[email-masked]> wrote:

>> That's not another model.  That's equivalent to using the synchronous
>> form  of the call, if gss_process_events() blocks until the mech is
>> ready to be  called, or to polling, if it eventually times out.
>
> number of NOWAIT calls. So you can build event loops with it.

No, you really can't.  Sure, if your goal is to establish six contexts 
simultaneously and then go on to something else, you can do that.  But if 
your goal is to establish one context while still allowing PalmOS to 
interact with the user, this doesn't cut it.  In fact, that's not entirely 
fair, because in that model you can't have threads _or_ asynchronous 
callbacks -- the library's only choice would be to send the application an 
event when its ready for another call.  But the point still holds -- if you 
give me an interface that blocks until it's ready, then I can't run my own 
event loop, and if you give me an interface that I have to poll, then I'm 
polling, not running an event loop.

A proper event loop blocks until there is an event (_any_ event), then does 
some work.  Polling is not an event loop.  Polling wastes cycles and can 
result in serious increases in power use besides.  Not to mention all of 
the points Nico has made with respect to the complexities of an API trying 
to impose an event model on an application with several intermediate 
libraries in between.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Carnegie Mellon University - Pittsburgh, PA

---

From: Volker Lendecke <[email-masked]>
Date: Thu, 3 Sep 2009 17:43:11 +0200

Volker

---

From: Volker Lendecke <[email-masked]>
Date: Thu, 3 Sep 2009 23:18:34 +0200

On Thu, Sep 03, 2009 at 04:55:54PM -0400, Jeffrey Hutzelman wrote:

> A proper event loop blocks until there is an event (_any_ event), then 
> does some work.  Polling is not an event loop.  Polling wastes cycles and 
> can result in serious increases in power use besides.  Not to mention all 
> of the points Nico has made with respect to the complexities of an API 
> trying to impose an event model on an application with several 
> intermediate libraries in between.

That's why I pointed at the avahi libs. Samba internally has
its own version of an event loop, and it would be outright
impossible to adapt that to an event loop from an external
lib. Adapting to those avahi libs was really easy for me,
given that I have done quite a bit of async programming
inside Samba before. There are other samples out there, but
many of those (I've looked at some async DNS libs) assume
select(2) as the blocking call. This won't work in modern
systems anymore that use epoll other more efficient
mechanisms.

I'm not saying that avahi is the last word in this, but it
worked pretty well for me.

Internally the libs would use the async calls only, the sync
wrappers would then provide a stub event loop around the
async calls. That's at least what we do in the native Samba
async code.

Volker