Soren Dreijer | 9 Oct 18:48 2013
Picon

Director in Ruby hangs Ruby VM

Hi folks!

I'm using directors in my Ruby SWIG project to allow C++ code to call back in to Ruby. I have an interface defined like so:
%feature("director") IHostEventHandler;

In Ruby, I instantiate a class that derives from IHostEventHandler and then pass that to my C++ code via a SWIGified function.

For testing purposes, when the IHostEventHandler is registered, I create a C++ thread which calls one of the methods on the IHostEventHandler interface every 2 seconds.

The call correctly comes in to Ruby. However, once the call returns, the Ruby VM appears to be in a funky state and I'm no longer able to execute other commands on the console. Interestingly, the C++ code is still able to call in to Ruby via the interface; it's just like everything else in the Ruby VM has stopped working.

Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for me to execute the callback on a non-Ruby thread?

Any help greatly appreciated,
Soren
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Swig-user mailing list
Swig-user <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-user
Klaus Kaempf | 9 Oct 20:16 2013
Picon

Re: Director in Ruby hangs Ruby VM

* Soren Dreijer <dreijer+list <at> echobit.net> [Oct 09. 2013 18:50]:
> 
> The call correctly comes in to Ruby. However, once the call returns, the
> Ruby VM appears to be in a funky state and I'm no longer able to execute
> other commands on the console. Interestingly, the C++ code is still able to
> call in to Ruby via the interface; it's just like everything else in the
> Ruby VM has stopped working.

Which Ruby version are you using ?

> 
> Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
> me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.

For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.

See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0

Hth,

Klaus
--

-- 
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
Maxfeldstraße 5, 90409 Nürnberg, Germany

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Swig-user mailing list
Swig-user <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-user
Soren Dreijer | 17 Oct 17:30 2013
Picon

Re: Director in Ruby hangs Ruby VM

I'm wondering if that isn't the issue here. I took a quick look through the generated SWIG code and I don't see any Ruby threads being used for the callback.

---
(It looks like I accidentally replied directly to Klaus rather than the list earlier, so here's a quick catch up:)

I'm using Ruby 1.9.3p448.

I'm already using rb_thread_blocking_region() when making calls from Ruby into native code for long operations. However,I'm not sure why rb_thread_blocking_region() is relevant in this particular case. I'm calling from native code *into* Ruby, not the other way around. That is, I don't have a GIL to release at all.

It includes a test.rb script, which creates an instance of the SWIG Hang class and then sets a director via set_event_handler().

set_event_handler() creates a native thread (using C++11) that calls back in to Ruby every 2 seconds. As soon as the first callback has been executed, however, the Ruby VM hangs and takes no further user input. Interestingly, SWIG is still able to continue executing callbacks; it's just that all other Ruby execution has stopped (I can't even Ctrl+C the process)




On Wed, Oct 9, 2013 at 11:16 AM, Klaus Kaempf <kkaempf <at> suse.de> wrote:
* Soren Dreijer <dreijer+list <at> echobit.net> [Oct 09. 2013 18:50]:
>
> The call correctly comes in to Ruby. However, once the call returns, the
> Ruby VM appears to be in a funky state and I'm no longer able to execute
> other commands on the console. Interestingly, the C++ code is still able to
> call in to Ruby via the interface; it's just like everything else in the
> Ruby VM has stopped working.

Which Ruby version are you using ?

>
> Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
> me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.

For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.

See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0

Hth,

Klaus
--
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
Maxfeldstraße 5, 90409 Nürnberg, Germany

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Swig-user mailing list
Swig-user <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-user
Soren Dreijer | 17 Oct 18:44 2013
Picon

Re: Director in Ruby hangs Ruby VM

Alright, turns out it *was* because SWIG isn't calling back in to Ruby on a native Ruby thread. Below, I've rewritten the original test code to instead spawn a Ruby thread and then perform the SWIG callback on that thread. That seems to fix the issue.

I've submitted a bug report: https://sourceforge.net/p/swig/bugs/1344/

Working code:

static VALUE DoSleep(void* pData)
{
    sleep(2);
    return 0;
}

static void CancelSleep(void* pData)
{   
}

static VALUE SomeRubyThread(void* pData)
{
    IEventHandler* pEventHandler = reinterpret_cast<IEventHandler*>(pData);

    while (true)
    {
        rb_thread_blocking_region(&DoSleep, nullptr, &CancelSleep, nullptr);

        pEventHandler->authenticate("moo", "moo");
    }

    return Qnil;
}

void Hang::set_event_handler(IEventHandler* pEventHandler)
{
    rb_thread_create(&SomeRubyThread, reinterpret_cast<void*>(pEventHandler));
}


On Thu, Oct 17, 2013 at 8:30 AM, Soren Dreijer <dreijer+list <at> echobit.net> wrote:
I'm wondering if that isn't the issue here. I took a quick look through the generated SWIG code and I don't see any Ruby threads being used for the callback.

---
(It looks like I accidentally replied directly to Klaus rather than the list earlier, so here's a quick catch up:)

I'm using Ruby 1.9.3p448.

I'm already using rb_thread_blocking_region() when making calls from Ruby into native code for long operations. However,I'm not sure why rb_thread_blocking_region() is relevant in this particular case. I'm calling from native code *into* Ruby, not the other way around. That is, I don't have a GIL to release at all.

It includes a test.rb script, which creates an instance of the SWIG Hang class and then sets a director via set_event_handler().

set_event_handler() creates a native thread (using C++11) that calls back in to Ruby every 2 seconds. As soon as the first callback has been executed, however, the Ruby VM hangs and takes no further user input. Interestingly, SWIG is still able to continue executing callbacks; it's just that all other Ruby execution has stopped (I can't even Ctrl+C the process)




On Wed, Oct 9, 2013 at 11:16 AM, Klaus Kaempf <kkaempf <at> suse.de> wrote:
* Soren Dreijer <dreijer+list <at> echobit.net> [Oct 09. 2013 18:50]:
>
> The call correctly comes in to Ruby. However, once the call returns, the
> Ruby VM appears to be in a funky state and I'm no longer able to execute
> other commands on the console. Interestingly, the C++ code is still able to
> call in to Ruby via the interface; it's just like everything else in the
> Ruby VM has stopped working.

Which Ruby version are you using ?

>
> Am I misunderstanding how directors work in Ruby? Is it perhaps wrong for
> me to execute the callback on a non-Ruby thread?

Internal thread handling changed between Ruby 1.8 and 1.9. Ruby 2.0
changed the interface once more.

For Ruby 1.9 'rb_thread_blocking_region' is the magic function to call.
See e.g.
http://www.spacevatican.org/2012/7/5/whos-afraid-of-the-big-bad-lock/
for more information.

See https://bugs.ruby-lang.org/issues/8660 for Ruby 2.0

Hth,

Klaus
--
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
Maxfeldstraße 5, 90409 Nürnberg, Germany


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Swig-user mailing list
Swig-user <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/swig-user

Gmane