headers
Michael B Allen | 1 Sep 2009 19:12

Re: KITTEN: IETF 75 - 76

On Tue, 18 Aug 2009 00:13:28 +0200
Leif Johansson <leifj <at> mnt.se> wrote:

> > >    2. listing/iterating credentials
> > >    3. exporting/importing credentials
> >
> > I think Leif has expressed interest in (3) before.
> 
> That was exporting unfinished contexts actually - slightly different.
> 
> >
> > >    4. error message reporting
> > >    5. asynchronous calls
> >
> > The last 2 are generally required for use by modern applications.
> 
> hmm, is asynchronous calls related to exportable contexts perhaps...?

Exactly they are. To implement async (properly) you would need to be able to serialize a context at any step.

Mike

--

-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/
Permalink | Reply |
headers
Nicolas Williams | 1 Sep 2009 19:31
Picon

Re: KITTEN: IETF 75 - 76

On Tue, Sep 01, 2009 at 01:12:02PM -0400, Michael B Allen wrote:
> On Tue, 18 Aug 2009 00:13:28 +0200
> Leif Johansson <leifj <at> mnt.se> wrote:
> 
> > > >    2. listing/iterating credentials
> > > >    3. exporting/importing credentials
> > >
> > > I think Leif has expressed interest in (3) before.
> > 
> > That was exporting unfinished contexts actually - slightly different.
> > 
> > >
> > > >    4. error message reporting
> > > >    5. asynchronous calls
> > >
> > > The last 2 are generally required for use by modern applications.
> > 
> > hmm, is asynchronous calls related to exportable contexts perhaps...?
> 
> Exactly they are. To implement async (properly) you would need to be
> able to serialize a context at any step.

Internally, maybe, but as far as the application goes: no.

To me async means a pair of functions called, say,
gss_init_sec_context_async() and gss_accept_sec_context_async() that do
not block, but may return an indication that the call did not complete,
and which provide some way for event notification so that the
application can call them again when they are ready to be called again.
(Continue reading)

Permalink | Reply |
headers
Love Hörnquist Åstrand | 1 Sep 2009 23:27
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 10:31 skrev Nicolas Williams:

> Here's another based on background threads and a completion callback
> instead of an event notification:
>
> 	major = gss_init_sec_context_async_cb(&minor,
> 	    cred, &ctx, target, mech, req_flags,
> 	    GSS_C_INDEFINITE, cb, &input_token, &output_token,
> 	    &ret_flags, NULL, cb_func, &cb_data);

The callback should take all output variables, for example:

OM_uint32
gss_acquire_cred_ex_f(gss_status_id_t /* status */,
		      const gss_name_t /* desired_name */,
		      OM_uint32 /* flags */,
		      OM_uint32 /* time_req */,
		      const gss_OID /*desired_mech */,
		      gss_cred_usage_t /* cred_usage */,
		      gss_auth_identity_t /* identity */,
		      void * /* ctx */,
		      void (* /* complete */)(void *, OM_uint32, gss_status_id_t,  
gss_cred_id_t, gss_OID_set, OM_uint32));

OM_uint32
gss_iter_creds_f(gss_status_id_t /* status */,
		 OM_uint32 /* flags */,
		 gss_OID /* mech */,
		 void * /* userctx */,
(Continue reading)

Permalink | Reply |
headers
Michael B Allen | 2 Sep 2009 00:13

Re: KITTEN: IETF 75 - 76

On Tue, 01 Sep 2009 14:27:04 -0700
Love Hörnquist Åstrand <lha <at> kth.se> wrote:

> 
> 1 sep 2009 kl. 10:31 skrev Nicolas Williams:
> 
> > Here's another based on background threads and a completion callback
> > instead of an event notification:
> >
> > 	major = gss_init_sec_context_async_cb(&minor,
> > 	    cred, &ctx, target, mech, req_flags,
> > 	    GSS_C_INDEFINITE, cb, &input_token, &output_token,
> > 	    &ret_flags, NULL, cb_func, &cb_data);
> 
> The callback should take all output variables, for example:
> 
> OM_uint32
> gss_acquire_cred_ex_f(gss_status_id_t /* status */,
> 		      const gss_name_t /* desired_name */,
> 		      OM_uint32 /* flags */,
> 		      OM_uint32 /* time_req */,
> 		      const gss_OID /*desired_mech */,
> 		      gss_cred_usage_t /* cred_usage */,
> 		      gss_auth_identity_t /* identity */,
> 		      void * /* ctx */,
> 		      void (* /* complete */)(void *, OM_uint32, gss_status_id_t,  
> gss_cred_id_t, gss_OID_set, OM_uint32));

Holy moses that's complicated. Do you really need callbacks? Why not just have a flag like GSS_C_NOWAIT
that indicates the call should return immediately even if the overall operation has not completed yet? If
(Continue reading)

Permalink | Reply |
headers
Love Hörnquist Åstrand | 2 Sep 2009 00:18
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 15:13 skrev Michael B Allen:

>  it means that you need to call it again later

When is later unless you get a callback ?

So far it have not been complicated, note that gss_acquire_cred_ex()  
does more then gss_acquire_cred().

Love
Permalink | Reply |
headers
Michael B Allen | 2 Sep 2009 03:40

Re: KITTEN: IETF 75 - 76

On Tue, 01 Sep 2009 15:18:13 -0700
Love Hörnquist Åstrand <lha <at> kth.se> wrote:

> 
> 1 sep 2009 kl. 15:13 skrev Michael B Allen:
> 
> >  it means that you need to call it again later
> 
> When is later unless you get a callback ?

When you call the function with the flag GSS_C_NOWAIT, the implementation transmits the network request
[1] and then checks to see if data can be read (such as by using select(2)). If data is not available, the
function simply returns a status of EAGAIN. Now the caller is free to perform other work (and thus it is
async). But the caller now knows that the function needs to be called again to complete the operation. So
later, it calls the function again. If the network response has been received, it returns success.
Otherwise, if GSS_C_NOWAIT was supplied, it again returns EAGAIN. If GSS_C_NOWAIT was not supplied, it
waits for the operation to complete (e.g. for a network response to be received) just like the non-async
call. In fact async and non-async uses the same function - the GSS_C_NOWAIT flag alone controls async behavior.

Mike

[1] A network request is only one example, the behavior is the same for any blocking operation.

--

-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/
Permalink | Reply |
headers
Love Hörnquist Åstrand | 2 Sep 2009 07:21
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 18:40 skrev Michael B Allen:

> But the caller now knows that the function needs to be called again  
> to complete the operation.

so consumer got to poll the library, that is no got from the  
performance view.

Love
Permalink | Reply |
headers
Michael B Allen | 2 Sep 2009 09:10

Re: KITTEN: IETF 75 - 76

On Tue, 1 Sep 2009 22:21:57 -0700
Love Hörnquist Åstrand <lha <at> kth.se> wrote:

> 
> 1 sep 2009 kl. 18:40 skrev Michael B Allen:
> 
> > But the caller now knows that the function needs to be called again  
> > to complete the operation.
> 
> so consumer got to poll the library, that is no got from the  
> performance view.

Actually now that I think about it I only use the NOWAIT flag technique for client oriented routines. For
example, if you wanted to perform 10 HTTP requests in parallel, you could use the NOWAIT flag to initiate
the requests and then not use NOWAIT when collecting the responses.

If someone wanted to use async for an event loop (a single thread / process handling an arbitrary number of
contexts concurrently) then the NOWAIT flag technique by itself would indeed require polling and would
be inefficient.

However, if the caller had access to the underlying polling routine (like through a "gss_process_events"
function that called select or whatever), then the caller could know if the status of *any* operation had
changed and thus polling would not be required. The benefit of doing this would be that a single thread /
process could be used to do everything.

Mike

--

-- 
Michael B Allen
Java Active Directory Integration
(Continue reading)

Permalink | Reply |
headers
Stefan (metze) Metzmacher | 2 Sep 2009 09:46
Picon
Favicon

Re: KITTEN: IETF 75 - 76

Michael B Allen schrieb:
> On Tue, 01 Sep 2009 15:18:13 -0700
> Love Hörnquist Åstrand <lha <at> kth.se> wrote:
> 
>> 1 sep 2009 kl. 15:13 skrev Michael B Allen:
>>
>>>  it means that you need to call it again later
>> When is later unless you get a callback ?
> 
> When you call the function with the flag GSS_C_NOWAIT, the implementation transmits the network request
[1] and then checks to see if data can be read (such as by using select(2)). If data is not available, the
function simply returns a status of EAGAIN. Now the caller is free to perform other work (and thus it is
async). But the caller now knows that the function needs to be called again to complete the operation. So
later, it calls the function again. If the network response has been received, it returns success.
Otherwise, if GSS_C_NOWAIT was supplied, it again returns EAGAIN. If GSS_C_NOWAIT was not supplied, it
waits for the operation to complete (e.g. for a network response to be received) just like the non-async
call. In fact async and non-async uses the same function - the GSS_C_NOWAIT flag alone controls async behavior.

That is true for system calls like send() and recv(), where the caller
is supposed to call select. But for gssapi the caller would have to poll
like this:

while (1) {

	ret = gss_call(&minor, ...);
	if (ret != 0 && minor == EAGAIN) {
		continue;
	}
	...
	break;
(Continue reading)

Permalink | Reply |
headers
Nicolas Williams | 2 Sep 2009 17:32
Picon

Re: KITTEN: IETF 75 - 76

On Wed, Sep 02, 2009 at 09:46:33AM +0200, Stefan (metze) Metzmacher wrote:
> Michael B Allen schrieb:
> > When you call the function with the flag GSS_C_NOWAIT, the
> > implementation transmits the network request [1] and then checks to
> > see if data can be read (such as by using select(2)). If data is not
> > available, the function simply returns a status of EAGAIN. Now the
> > caller is free to perform other work (and thus it is async). But the
> > caller now knows that the function needs to be called again to
> > complete the operation. So later, it calls the function again. If
> > the network response has been received, it returns success.
> > Otherwise, if GSS_C_NOWAIT was supplied, it again returns EAGAIN. If
> > GSS_C_NOWAIT was not supplied, it waits for the operation to
> > complete (e.g. for a network response to be received) just like the
> > non-async call. In fact async and non-async uses the same function -
> > the GSS_C_NOWAIT flag alone controls async behavior.
> 
> That is true for system calls like send() and recv(), where the caller
> is supposed to call select. But for gssapi the caller would have to poll
> like this:
> 
> while (1) {
> 
> 	ret = gss_call(&minor, ...);
> 	if (ret != 0 && minor == EAGAIN) {
> 		continue;
> 	}
> 	...
> 	break;
> }
>
(Continue reading)

Permalink | Reply |
headers
Michael B Allen | 2 Sep 2009 23:20

Re: KITTEN: IETF 75 - 76

On Wed, 2 Sep 2009 10:32:41 -0500
Nicolas Williams <Nicolas.Williams <at> sun.com> wrote:
> Absolutely no polling.
> 
> Async I/O is not rocket science.  We can quibble over details, but we've
> only got so many workable models.  And we have to be mindful that not
> one model will result in fully portable application code.
> 
> Here are some reasonable models:
> 
>  - Completion callback (or, rather, a callback indicating that there's
>    work to do, not necessarily completion)
> 
>    This implies either a threaded process model or an async I/O process
>    model with one common event loop provided by the OS.
> 
>    This model is the simplest for apps to use, and it will be portable
>    to Linux, *BSD, Solaris, Windows.  But not necessarily to embedded
>    devices, real-time OSes, etc...
> 
>  - Return handles to I/O resources that the app can feed into its event
>    loop.
> 
>    This is much more OS-specific.  On Unix/Unix-like OSes this means
>    returning an array of file descriptors.  On Windows it means
>    returning and array of file handles.  This too may not be terribly
>    portable.  And it requires that the application plug the returned
>    resources into its event loop, and take them out, on every call to
>    these GSS functions.
(Continue reading)

Permalink | Reply |
headers
Nicolas Williams | 2 Sep 2009 23:26
Picon

Re: KITTEN: IETF 75 - 76

On Wed, Sep 02, 2009 at 05:20:15PM -0400, Michael B Allen wrote:
> There is another model:
> 
> > > while (1) {
>         gss_process_events(&minor, ...);
> > > ...

The problem with this model is that it has the GSS-API impose its own
event loop on the application.  Imagine if every library did that!  An
application that uses multiple such libraries, where some such libraries
use other such libraries, would be very difficult to get to work at all,
particularly without threading or sub-processes.

I don't understand your aversion to callbacks.  Callbacks are
exceedingly common in C APIs nowadays, and that's a good thing, IMO.

Nico
--

--
Permalink | Reply |
headers
Michael B Allen | 3 Sep 2009 03:12

Re: KITTEN: IETF 75 - 76

On Wed, 2 Sep 2009 16:26:52 -0500
Nicolas Williams <Nicolas.Williams <at> sun.com> wrote:

> On Wed, Sep 02, 2009 at 05:20:15PM -0400, Michael B Allen wrote:
> > There is another model:
> > 
> > > > while (1) {
> >         gss_process_events(&minor, ...);
> > > > ...
> 
> The problem with this model is that it has the GSS-API impose its own
> event loop on the application.  Imagine if every library did that!  An
> application that uses multiple such libraries, where some such libraries
> use other such libraries, would be very difficult to get to work at all,
> particularly without threading or sub-processes.
> 
> I don't understand your aversion to callbacks.  Callbacks are
> exceedingly common in C APIs nowadays, and that's a good thing, IMO.

I think you're exaggerating, if not misrepresenting, the detractions of the NOWAIT flag technique. But it
is no matter. For a variety of reasons I will not pursue the issue further.

However, before I go back to lurking, I have to declare that callbacks are NOT simple. They can be quite
complicated. Just from looking at function signatures posted in this thread I personally have no idea how
these callbacks would actually work. Are they called by another thread, by a signal, through another call
or by another caller? Is locking required? What data is safe to modify in the callback? They seem C oriented
- do the semantics change if a different language is used like Java? Hopefully the answers to these
questions will be clear if these callbacks do end up in a GSSAPI implementations.

Mike
(Continue reading)

Permalink | Reply |
headers
Nicolas Williams | 3 Sep 2009 17:26
Picon

Re: KITTEN: IETF 75 - 76

On Wed, Sep 02, 2009 at 09:12:00PM -0400, Michael B Allen wrote:
> On Wed, 2 Sep 2009 16:26:52 -0500
> Nicolas Williams <Nicolas.Williams <at> sun.com> wrote:
> > The problem with this model is that it has the GSS-API impose its own
> > event loop on the application.  Imagine if every library did that!  An
> > application that uses multiple such libraries, where some such libraries
> > use other such libraries, would be very difficult to get to work at all,
> > particularly without threading or sub-processes.
> > 
> > I don't understand your aversion to callbacks.  Callbacks are
> > exceedingly common in C APIs nowadays, and that's a good thing, IMO.
> 
> I think you're exaggerating, if not misrepresenting, the detractions
> of the NOWAIT flag technique. But it is no matter. For a variety of
> reasons I will not pursue the issue further.

I have worked on applications that use over half a dozen layered
components, at least two of which need to do blocking I/O operations,
plus the application itself being async I/O event driven.

Putting together such an application with your async I/O model would
result in an enormous amount of complication.

I'd much rather have the library give me either file descriptors to
select on, or a callback.

> However, before I go back to lurking, I have to declare that callbacks
> are NOT simple. They can be quite complicated. Just from looking at
> function signatures posted in this thread I personally have no idea
> how these callbacks would actually work. Are they called by another
(Continue reading)

Permalink | Reply |
headers
Nicolas Williams | 2 Sep 2009 00:20
Picon

Re: KITTEN: IETF 75 - 76

On Tue, Sep 01, 2009 at 03:18:13PM -0700, Love Hörnquist Åstrand wrote:
> 
> 1 sep 2009 kl. 15:13 skrev Michael B Allen:
> 
> > it means that you need to call it again later
> 
> When is later unless you get a callback ?

I agree.  Callbacks are easy.  My construction (for ISC) is simpler: no
need for all the ISC output args in the callback.

Love's credential iterator callback is trivial.

Callbacks are very common in API design.
Permalink | Reply |
headers
Love Hörnquist Åstrand | 2 Sep 2009 00:39
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 15:20 skrev Nicolas Williams:

> On Tue, Sep 01, 2009 at 03:18:13PM -0700, Love Hörnquist Åstrand  
> wrote:
>>
>> 1 sep 2009 kl. 15:13 skrev Michael B Allen:
>>
>>> it means that you need to call it again later
>>
>> When is later unless you get a callback ?
>
> I agree.  Callbacks are easy.  My construction (for ISC) is simpler:  
> no
> need for all the ISC output args in the callback.

So what is input token for the second call ? Can input token be freed  
between first and second call ?

Why do I have to pass all those argument that might (or might not) do  
something.

Love
Permalink | Reply |
headers
Nicolas Williams | 2 Sep 2009 00:54
Picon

Re: KITTEN: IETF 75 - 76

On Tue, Sep 01, 2009 at 03:39:33PM -0700, Love Hörnquist Åstrand wrote:
> 1 sep 2009 kl. 15:20 skrev Nicolas Williams:
> >I agree.  Callbacks are easy.  My construction (for ISC) is simpler:
> >no need for all the ISC output args in the callback.
> 
> So what is input token for the second call ? Can input token be freed  
> between first and second call ?

It should be the same as in the previous call.  But that can be relaxed
-- we could say that the mech must keep an internal copy if it will
continue needing it after the first call.

> Why do I have to pass all those argument that might (or might not) do  
> something.

If it's hard to track them you can always place them in the callback
function's argument -- a void * that would be a pointer to a struct with
all the context that the callback needs, including those GISC args if
you're not storing them elsewhere.
Permalink | Reply |
headers
Love Hörnquist Åstrand | 2 Sep 2009 07:23
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 15:54 skrev Nicolas Williams:

> On Tue, Sep 01, 2009 at 03:39:33PM -0700, Love Hörnquist Åstrand  
> wrote:
>> 1 sep 2009 kl. 15:20 skrev Nicolas Williams:
>>> I agree.  Callbacks are easy.  My construction (for ISC) is simpler:
>>> no need for all the ISC output args in the callback.
>>
>> So what is input token for the second call ? Can input token be freed
>> between first and second call ?
>
> It should be the same as in the previous call.  But that can be  
> relaxed
> -- we could say that the mech must keep an internal copy if it will
> continue needing it after the first call.
>
>> Why do I have to pass all those argument that might (or might not) do
>> something.
>
> If it's hard to track them you can always place them in the callback
> function's argument -- a void * that would be a pointer to a struct  
> with
> all the context that the callback needs, including those GISC args if
> you're not storing them elsewhere.

I find this more complicated then just returning the values to the  
callback function.

In fact, I find it more complicated to have my callback function have
(Continue reading)

Permalink | Reply |
headers
Nicolas Williams | 1 Sep 2009 23:38
Picon

Re: KITTEN: IETF 75 - 76

On Tue, Sep 01, 2009 at 02:27:04PM -0700, Love Hörnquist Åstrand wrote:
> >Here's another based on background threads and a completion callback
> >instead of an event notification:
> >
> >	major = gss_init_sec_context_async_cb(&minor,
> >	    cred, &ctx, target, mech, req_flags,
> >	    GSS_C_INDEFINITE, cb, &input_token, &output_token,
> >	    &ret_flags, NULL, cb_func, &cb_data);
> 
> The callback should take all output variables, for example:

As I imagined it the callback would only be to indicate that GISCACB()
must get called again to complete its work.  (And it might well need
fail to complete at that point.

Think of krb5_gisc: it may need to do multiple DNS lookups, and multiple
TGS exchanges; each time that some I/O is ready that it needs it could
call the callback, the app then calls back into GISCACB(), krb5_gisc
makes some progress, and if it'd block then it returns continue-needed
without an output token.  Lather, rinse, repeat.

I like it better that way.  If we add a new version of GISC with new
outputs we'll need not only a new GISC_async, but also a new callback
function prototype.  Having the callback lead to the application calling
GISCACB() again seems reasonable and simple enough to me.  I'm not
wedded to this, but it does seem simpler.

The callback in the credential iterator can be the way you propose.
That's because it doesn't need every output of gss_inquire_cred*(), just
a cred handle and mechanism OID, and it can find the rest from that.
(Continue reading)

Permalink | Reply |
headers
Love Hörnquist Åstrand | 1 Sep 2009 20:13
Picon
Picon
Favicon

Re: KITTEN: IETF 75 - 76


1 sep 2009 kl. 10:31 skrev Nicolas Williams:

> The problem with the callback variant is that it implies a threaded
> process model (which I don't mind, but which won't be feasible for  
> some
> implementors).

It does not require threads, the caller of the cb enabled ISC will  
have to call the event loop, that in turn will call the callback call.

It does require though that you select one event process system (or  
threads).

Love
Permalink | Reply |
headers
Nicolas Williams | 1 Sep 2009 20:52
Picon

Re: KITTEN: IETF 75 - 76

On Tue, Sep 01, 2009 at 11:13:03AM -0700, Love Hörnquist Åstrand wrote:
> 1 sep 2009 kl. 10:31 skrev Nicolas Williams:
> >The problem with the callback variant is that it implies a threaded
> >process model (which I don't mind, but which won't be feasible for  
> >some
> >implementors).
> 
> It does not require threads, the caller of the cb enabled ISC will  
> have to call the event loop, that in turn will call the callback call.
> 
> It does require though that you select one event process system (or  
> threads).

Yes, you're right, the callback variant implies one of: threaded process
model or async I/O event loop model.  We should at least specify that
the callback may be called in a different thread in the case of threaded
process models.

Nico
--
Permalink | Reply |

Gmane