Thiago Padilha | 19 Nov 15:59 2013
Picon

Need help with very simple C++ FFI call

Hi

I'm trying write a ffi binding to the v8 javascript engine (C++
library). Theres one already at https://github.com/sol/v8 but I want
to do it again as an exercise to improve by haskell skills.

I chose this library be because it will force me to deal with impure
functions using monads. Since its not a trivial library, it will also
improve my knowledge of writing FFI bindings and cabal packages in
case I want to wrap more C++ libraries in the future.

My first step is to port the 'hello world' example in v8 embedders
guide(https://developers.google.com/v8/get_started) as a haskell
binding. So I have downloaded/compiled v8 and created the following
directory structure:

src/V8.hs
cbits/haskell-v8.cpp
# v8 libraries
deps/lib/libv8_base.a
deps/lib/libv8_snapshot.a
# v8 headers
deps/include/

Here's the contents of the haskell-v8.cpp file:
http://pastebin.com/RfYCCxFQ (basically a copy of the v8 embedders
example function, except that it returns a char pointer).

and here's the haskell module: http://pastebin.com/fnXVbEeU

(Continue reading)

Yuras Shumovich | 19 Nov 17:06 2013
Picon

Re: Need help with very simple C++ FFI call


Hello,

IIRC, strlen doesn't count the terminating '\0', so you'll need to
malloc(strlen(*ascii) + 1).

Not sure why python works, maybe just different memory layout hides the
bug.

Thanks,
Yuras

On Tue, 2013-11-19 at 12:59 -0200, Thiago Padilha wrote:
> Hi
> 
> I'm trying write a ffi binding to the v8 javascript engine (C++
> library). Theres one already at https://github.com/sol/v8 but I want
> to do it again as an exercise to improve by haskell skills.
> 
> I chose this library be because it will force me to deal with impure
> functions using monads. Since its not a trivial library, it will also
> improve my knowledge of writing FFI bindings and cabal packages in
> case I want to wrap more C++ libraries in the future.
> 
> My first step is to port the 'hello world' example in v8 embedders
> guide(https://developers.google.com/v8/get_started) as a haskell
> binding. So I have downloaded/compiled v8 and created the following
> directory structure:
> 
> src/V8.hs
(Continue reading)

Thiago Padilha | 19 Nov 17:19 2013
Picon

Re: Need help with very simple C++ FFI call

Thanks for pointing the strlen issue, but as I stated before, the
segmentation fault occurs even before the copying the resulting
string. To illustrate the error better, heres a very trimmed(20 lines
long) version of the c++ file that still causes the segmentation
fault: http://pastebin.com/cJvkLQPg . Commenting line 14 will fix the
segfault.

On Tue, Nov 19, 2013 at 1:06 PM, Yuras Shumovich <shumovichy <at> gmail.com> wrote:
>
> Hello,
>
> IIRC, strlen doesn't count the terminating '\0', so you'll need to
> malloc(strlen(*ascii) + 1).
>
> Not sure why python works, maybe just different memory layout hides the
> bug.
>
> Thanks,
> Yuras
>
> On Tue, 2013-11-19 at 12:59 -0200, Thiago Padilha wrote:
>> Hi
>>
>> I'm trying write a ffi binding to the v8 javascript engine (C++
>> library). Theres one already at https://github.com/sol/v8 but I want
>> to do it again as an exercise to improve by haskell skills.
>>
>> I chose this library be because it will force me to deal with impure
>> functions using monads. Since its not a trivial library, it will also
>> improve my knowledge of writing FFI bindings and cabal packages in
(Continue reading)

Yuras Shumovich | 19 Nov 17:50 2013
Picon

Re: Need help with very simple C++ FFI call


Ok, then I don't see anything obviously wrong.

I was embeding v8 into haskell project few months ago, and almost the
same code works for me.

I don't have v8 installed right now, so I can't try your code at the
moment, but I see only one not trivial difference with my code. I'm
creating isolate manually, and you are using the default one. Check that
Isolate::GetCurrent() returns anything. I can imaging that your code is
running in unbound green thread, or it is bound to OS thread other then
the main one. v8 use thread local storage, that can be the issue.

On Tue, 2013-11-19 at 14:19 -0200, Thiago Padilha wrote:
> Thanks for pointing the strlen issue, but as I stated before, the
> segmentation fault occurs even before the copying the resulting
> string. To illustrate the error better, heres a very trimmed(20 lines
> long) version of the c++ file that still causes the segmentation
> fault: http://pastebin.com/cJvkLQPg . Commenting line 14 will fix the
> segfault.
> 
> On Tue, Nov 19, 2013 at 1:06 PM, Yuras Shumovich <shumovichy <at> gmail.com> wrote:
> >
> > Hello,
> >
> > IIRC, strlen doesn't count the terminating '\0', so you'll need to
> > malloc(strlen(*ascii) + 1).
> >
> > Not sure why python works, maybe just different memory layout hides the
> > bug.
(Continue reading)

Thiago Padilha | 19 Nov 18:13 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 1:50 PM, Yuras Shumovich <shumovichy <at> gmail.com> wrote:
>
> Ok, then I don't see anything obviously wrong.
>
> I was embeding v8 into haskell project few months ago, and almost the
> same code works for me.
>
> I don't have v8 installed right now, so I can't try your code at the
> moment, but I see only one not trivial difference with my code. I'm
> creating isolate manually, and you are using the default one. Check that
> Isolate::GetCurrent() returns anything. I can imaging that your code is
> running in unbound green thread, or it is bound to OS thread other then
> the main one. v8 use thread local storage, that can be the issue.

Yuras, you are correct now :)

I've replaced the first line by these:

     Isolate* isolate = Isolate::New();
    // Sets this isolate as the entered one for the current thread
    isolate->Enter();

And the example worked as expected. Thanks for this very useful tip.

I still dont understand why ghci is not using the default thread to
run the binding, but from now on I will be more alert when binding to
libraries that uses threads.

Theres something else I also dont understood about FFI: When trying to
debug the problem I've written many printf statements  but none of
(Continue reading)

Carter Schonwald | 19 Nov 18:15 2013
Picon

Re: Need help with very simple C++ FFI call

There's a flag you can pass to ghci for that thread local issue. I forget what it is, but it should be in the manual. 

On Tuesday, November 19, 2013, Thiago Padilha wrote:

On Tue, Nov 19, 2013 at 1:50 PM, Yuras Shumovich <shumovichy <at> gmail.com> wrote:
>
> Ok, then I don't see anything obviously wrong.
>
> I was embeding v8 into haskell project few months ago, and almost the
> same code works for me.
>
> I don't have v8 installed right now, so I can't try your code at the
> moment, but I see only one not trivial difference with my code. I'm
> creating isolate manually, and you are using the default one. Check that
> Isolate::GetCurrent() returns anything. I can imaging that your code is
> running in unbound green thread, or it is bound to OS thread other then
> the main one. v8 use thread local storage, that can be the issue.

Yuras, you are correct now :)

I've replaced the first line by these:

     Isolate* isolate = Isolate::New();
    // Sets this isolate as the entered one for the current thread
    isolate->Enter();

And the example worked as expected. Thanks for this very useful tip.

I still dont understand why ghci is not using the default thread to
run the binding, but from now on I will be more alert when binding to
libraries that uses threads.

Theres something else I also dont understood about FFI: When trying to
debug the problem I've written many printf statements  but none of
them showed on the screen, but now that the example is working I've
notice that all printed text appears once I exit ghci. Why is that
happening?
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Brandon Allbery | 19 Nov 18:17 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 12:13 PM, Thiago Padilha <tpadilha84 <at> gmail.com> wrote:
I still dont understand why ghci is not using the default thread to
run the binding, but from now on I will be more alert when binding to
libraries that uses threads.

-fno-default-sandbox ? (this changes in upcoming ghci, I believe)
 
Theres something else I also dont understood about FFI: When trying to
debug the problem I've written many printf statements  but none of
them showed on the screen, but now that the example is working I've
notice that all printed text appears once I exit ghci. Why is that
happening?

Because C/C++ stdio buffers by default, as does Haskell's --- but they are independent buffers, so the state of the Haskell buffering mode is not relevant to C/C++ or vice versa and flushing in one does not affect the other. Evidently the C one is only getting flushed in an atexit() hook. (You may want to look at setvbuf().)

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Sven Panne | 19 Nov 20:02 2013
Picon

Re: Need help with very simple C++ FFI call

2013/11/19 Yuras Shumovich <shumovichy <at> gmail.com>:
> [...] but I see only one not trivial difference with my code. I'm
> creating isolate manually, and you are using the default one. Check that
> Isolate::GetCurrent() returns anything. I can imaging that your code is
> running in unbound green thread, or it is bound to OS thread other then
> the main one. v8 use thread local storage, that can be the issue.

Yes, v8 uses TLS to store the current Isolate (= VM instance) for
historical reasons, and if you want to use the default Isolate
(created even before main() is entered) things get tricky if you are
not using it from the main thread. Furthermore, if you use an Isolate
from various threads (sequentially, re-entry is not allowed) you have
to use v8's Locker to tell v8 what's happening. Isolate::Scope has to
be used to tell v8 when you are switching between several Isolates in
a single thread.

This is all sub-optimal (<- huge understatement) and we regret the
introduction of TLS in v8 every single day, it is a constant source of
bugs, confusion and complexity. Having said that, we are currently in
the process of removing TLS and make the Isolate very explicit in v8's
API. But this is easier said than done, given all the various complex
embedders v8 has (Chrome, node.js, ...). In a bright future, v8 uses
no TLS, all Isolates are explicit, there is no default Isolate and no
"current Isolate", and Isolate::Scope and Locker are merged. Until
then things are a bit tricky, but you can make your life easier by
building a debug version of v8, which has tons of assertions, making
stuff like a NULL Isolate blindingly obvious.

Choosing a combination of 2 multi-threaded runtimes for learning
purposes has a slightly masochistic touch... ;-)
Thiago Padilha | 19 Nov 21:22 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 4:02 PM, Sven Panne <svenpanne <at> gmail.com> wrote:
> Choosing a combination of 2 multi-threaded runtimes for learning
> purposes has a slightly masochistic touch... ;-)

Its as they say: "No pain, no gain".

Anyway, thank you all for the help
Brandon Allbery | 19 Nov 17:19 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 11:06 AM, Yuras Shumovich <shumovichy <at> gmail.com> wrote:
IIRC, strlen doesn't count the terminating '\0', so you'll need to
malloc(strlen(*ascii) + 1).

Not sure why python works, maybe just different memory layout hides the
bug.

Also note that different memory allocators have different behaviors; depending on block size rounding, string length and arena accounting overhead, the allocated block may be larger than requested. So the malloc() that Python provides may possibly be returning something "accidentally" large enough to handle the NUL anyway. (I don't know if Python actually overrides/replaces the default malloc(); some other interpreters do.)

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Thiago Padilha | 19 Nov 17:31 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 1:19 PM, Brandon Allbery <allbery.b <at> gmail.com> wrote:

> Also note that different memory allocators have different behaviors;
> depending on block size rounding, string length and arena accounting
> overhead, the allocated block may be larger than requested. So the malloc()
> that Python provides may possibly be returning something "accidentally"
> large enough to handle the NUL anyway. (I don't know if Python actually
> overrides/replaces the default malloc(); some other interpreters do.)

Line 14(which causes the segfault) comment says the HandleScope is
being allocated on the stack:

    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);

I dont know any C++. Is it possible that this line of code is
allocating something on the heap? If so could ghci have overriden
memory allocation routines in a way that would mess that allocation?
Brandon Allbery | 19 Nov 17:43 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 11:31 AM, Thiago Padilha <tpadilha84 <at> gmail.com> wrote:
On Tue, Nov 19, 2013 at 1:19 PM, Brandon Allbery <allbery.b <at> gmail.com> wrote:
> Also note that different memory allocators have different behaviors;

Line 14(which causes the segfault) comment says the HandleScope is
being allocated on the stack:

    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);

I dont know any C++. Is it possible that this line of code is
allocating something on the heap? If so could ghci have overriden
memory allocation routines in a way that would mess that allocation?

I don't think ghc/ghci overrides malloc() like that. And it is possible that the constructor, while allocating a stack object, also allocates some heap-based memory; but that is a detail of the C++ code of the constructor. That said, I believe g++ uses malloc/free behind the scenes for new/delete to avoid such conflicts (this is common because it is often necessary to mix C and C++ code, and it's just bad to have multiple memory allocators working in the same heap).

(That said, the fact that it uses malloc/free behind the scenes does *not* guarantee that you can substitute malloc() for new/new[] or free() for delete/delete[]. But in this case, no C++ allocations should be escaping the C++ code, so that also should not be relevant.)

It might be a good idea to run the test program under valgrind and see what it has to say about the problem.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Carter Schonwald | 19 Nov 17:21 2013
Picon

Re: Need help with very simple C++ FFI call

If you're wanting to call c++ code from ghci, you may need to use GHC 7.8 (current head), at least if you've hit any linking problems with using ghci in 7.6 or older ghc's.  Just an advanced warning. :-)


Happy hacking. 

On Tuesday, November 19, 2013, Thiago Padilha wrote:
Hi

I'm trying write a ffi binding to the v8 javascript engine (C++
library). Theres one already at https://github.com/sol/v8 but I want
to do it again as an exercise to improve by haskell skills.

I chose this library be because it will force me to deal with impure
functions using monads. Since its not a trivial library, it will also
improve my knowledge of writing FFI bindings and cabal packages in
case I want to wrap more C++ libraries in the future.

My first step is to port the 'hello world' example in v8 embedders
guide(https://developers.google.com/v8/get_started) as a haskell
binding. So I have downloaded/compiled v8 and created the following
directory structure:

src/V8.hs
cbits/haskell-v8.cpp
# v8 libraries
deps/lib/libv8_base.a
deps/lib/libv8_snapshot.a
# v8 headers
deps/include/

Here's the contents of the haskell-v8.cpp file:
http://pastebin.com/RfYCCxFQ (basically a copy of the v8 embedders
example function, except that it returns a char pointer).

and here's the haskell module: http://pastebin.com/fnXVbEeU

I compiled a shared library exposing the c "hello" function with the
following command:

    $ g++ \
      -shared -o libhaskellv8.so \
      -I./deps/include \
      -L./deps/lib \
      ./cbits/haskell-v8.cpp \
      -lv8_base \
      -lv8_snapshot \
      -lpthread \
      -lrt

And loaded into ghci with the following:

    $ ghci -L. -lhaskellv8 src/V8.hs

The problem is that I get a segmentation fault when I call the 'hello'
function from ghci. To ensure the library was compiled correctly, I
successfully called it from python:

    $ python -c \
      'from ctypes import CDLL, c_char_p; lv8=CDLL("libhaskellv8.so");
lv8.hello.restype = c_char_p; print lv8.hello()'
    Hello, World!

I have selectively uncommented lines from the c++ file, and the
segmentation faults starts happening when I uncomment line 14
(HandleScope handle_scope(isolate)).

What am I missing here?
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Thiago Padilha | 19 Nov 17:36 2013
Picon

Re: Need help with very simple C++ FFI call

On Tue, Nov 19, 2013 at 1:21 PM, Carter Schonwald
<carter.schonwald <at> gmail.com> wrote:
> If you're wanting to call c++ code from ghci, you may need to use GHC 7.8
> (current head), at least if you've hit any linking problems with using ghci
> in 7.6 or older ghc's.  Just an advanced warning. :-)
>
> Happy hacking.

I'm using GHC 7.6 but the shared library was compiled and linked using
gcc. Would this GHC incompatibility cause troubles when loading
already-linked shared libraries?

Gmane