Questions, help, and general non-development discussion related to Cython. ()

Gabriel Jacobo | 24 Jul 2012 22:05

BunnyMark benchmark

Hey guys, I've been doing some "sort of real world" benchmarking experiments with my game engine, which embeds the CPython VM plus Cython generated code, going from a pure Python implementation to a Cython+STL alternative, and I'm getting upwards of 35x speed improvements. I've done a write up about it (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)... I just wanted to say Cython rocks!

Also, I'm really baffled (though I shouldn't be, everyone says so) at how slow the CPython VM is. I've been looking at other embedding alternatives, but it seems there's nothing else available (PyPy doesn't seem to be ready for this task yet). Any suggestions? Is it worth it? I've read a message from Stefan in the Python mailing list where he says JIT compilers won't bring much speed improvement in real world applications...despite this I would like to try and see what I get out of it.

--
Gabriel.

headers

Stefan Behnel | 25 Jul 2012 07:52

Re: BunnyMark benchmark

Gabriel Jacobo, 24.07.2012 22:05:
> Hey guys, I've been doing some "sort of real world" benchmarking
> experiments with my game engine, which embeds the CPython VM plus Cython
> generated code, going from a pure Python implementation to a Cython+STL
> alternative, and I'm getting upwards of 35x speed improvements. I've done a
> write up about it
> (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)...
> I just wanted to say Cython rocks!

Thanks for sharing.

> Also, I'm really baffled (though I shouldn't be, everyone says so) at how
> slow the CPython VM is.

It's not - it just depends on what you do with it.

> I've read a message
> from Stefan in the Python mailing list where he says JIT compilers won't
> bring much speed improvement in real world applications...

I never said that. They certainly do, as long as they optimise the code
well (which is orthogonal to them being JIT compilers). And PyPy contains
lots of smart optimisations.

However, even for optimising JIT compilers, it depends on what you do with
them. They will usually win the competition with an interpreter, but they
may or may not make your code as fast as you need it, and they won't give
you much control for the cases where it's not.

> despite this I would like to try and see what I get out of it.

As long as it doesn't work at all, it won't be easy to try. Also note that
Cython is way faster in CPython than in PyPy.

Stefan

headers

Jerome Kieffer | 25 Jul 2012 08:29

Re: BunnyMark benchmark

On Tue, 24 Jul 2012 17:05:34 -0300
Gabriel Jacobo <gabomdq <at> gmail.com> wrote:

> Hey guys, I've been doing some "sort of real world" benchmarking
> experiments with my game engine, which embeds the CPython VM plus Cython
> generated code, going from a pure Python implementation to a Cython+STL
> alternative, and I'm getting upwards of 35x speed improvements. I've done a
> write up about it
> (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)...

Hi Gabriel,

I am surprised by the OpenMP consideration ... I think I managed to
have Cython OpenMP code running under windows when compiled with
mingw32 and providing the additional pthread DLL ? I have to admit I
did not play much with Visual Studio & OpenMP there.

Cheers,

--

-- 
Jerome Kieffer <google <at> terre-adelie.org>

headers

Gabriel Jacobo | 25 Jul 2012 14:28

Re: BunnyMark benchmark

2012/7/25 Jerome Kieffer <google <at> terre-adelie.org>

On Tue, 24 Jul 2012 17:05:34 -0300
Gabriel Jacobo <gabomdq <at> gmail.com> wrote:

> Hey guys, I've been doing some "sort of real world" benchmarking
> experiments with my game engine, which embeds the CPython VM plus Cython
> generated code, going from a pure Python implementation to a Cython+STL
> alternative, and I'm getting upwards of 35x speed improvements. I've done a
> write up about it
> (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)...

Hi Gabriel,

I am surprised by the OpenMP consideration ... I think I managed to
have Cython OpenMP code running under windows when compiled with
mingw32 and providing the additional pthread DLL ? I have to admit I
did not play much with Visual Studio & OpenMP there.

Cheers,

--
Jerome Kieffer <google <at> terre-adelie.org>

The mingw compilers that come with Ubuntu don't include the libgomp library "out of the box", I suppose it can be made to work compiling it manually, but in the end I don't feel it's a good choice for me as this is a GNU library licensed under the GPL, so distributing it on the iPhone (where no shared linking is possible) implies that you'd have to GPL license your own code, that's why I'm thinking about trying ZeroMQ, which thankfully Cython should make an easy task.

Gabriel.

headers

Jérôme Kieffer | 25 Jul 2012 19:37

Re: BunnyMark benchmark

On Wed, 25 Jul 2012 09:28:12 -0300
Gabriel Jacobo <gabomdq <at> gmail.com> wrote:

> The mingw compilers that come with Ubuntu don't include the libgomp library
> "out of the box",

Please correct me if I am wrong but mingw is supposed to be a port of gcc to windows, no ?

Are you cross compiling linux -> windows ?

I had to ship the "libpthread.dll" under windows to have multithreading (hence openMP) working on windows
but honnestly I did not care that much as my code was anyway GPL (I am lucky, my company enforces that :)

> I suppose it can be made to work compiling it manually,
> but in the end I don't feel it's a good choice for me as this is a GNU
> library licensed under the GPL, so distributing it on the iPhone (where no
> shared linking is possible) implies that you'd have to GPL license your own
> code, that's why I'm thinking about trying ZeroMQ, which thankfully Cython
> should make an easy task.

I never played with such toys :)

Cheers,

--

-- 
Jérôme Kieffer <google <at> terre-adelie.org>

headers

Robert Bradshaw | 26 Jul 2012 06:48

Re: BunnyMark benchmark

On Tue, Jul 24, 2012 at 1:05 PM, Gabriel Jacobo <gabomdq <at> gmail.com> wrote:
> Hey guys, I've been doing some "sort of real world" benchmarking experiments
> with my game engine, which embeds the CPython VM plus Cython generated code,
> going from a pure Python implementation to a Cython+STL alternative, and I'm
> getting upwards of 35x speed improvements. I've done a write up about it
> (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)... I just wanted
> to say Cython rocks!

Glad you're enjoying it! Mind if we pull a quote from that article for
our web page (or you could write one).

- Robert

headers

Gabriel Jacobo | 26 Jul 2012 15:08

Re: BunnyMark benchmark

2012/7/26 Robert Bradshaw <robertwb <at> gmail.com>

On Tue, Jul 24, 2012 at 1:05 PM, Gabriel Jacobo <gabomdq <at> gmail.com> wrote:
> Hey guys, I've been doing some "sort of real world" benchmarking experiments
> with my game engine, which embeds the CPython VM plus Cython generated code,
> going from a pure Python implementation to a Cython+STL alternative, and I'm
> getting upwards of 35x speed improvements. I've done a write up about it
> (http://mdqinc.com/blog/2012/07/how-to-make-a-chicken-fly/)... I just wanted
> to say Cython rocks!

Glad you're enjoying it! Mind if we pull a quote from that article for
our web page (or you could write one).

- Robert

Sure, feel free to do so. If you can't find something suitable to quote let me know and I'll write something up. Thanks again.

--
Gabriel.