headers
Samuele Kaplun | 9 Aug 14:32
X-Face
Picon
Picon

Segmentation Fault with -O0

Dear Pyrex & Cython lists,
I've found a strange behaviour of my code developed with Pyrex (and Cython) 
which I can't solve.
I wrote and integer bit set extension to Python with Pyrex. It has the 
possibility to be searialized into a string and deserialized from that 
string.
I pass a correctly serialized representation to the constructor of my code (in 
the rhs variable):
[...]
cdef class intbitset:
[...]
    cdef IntBitSet *bitset
[...]
    def __new__(intbitset self, rhs=0, int minsize=-1):
[...]
        cdef size_t size
        cdef void *buf
[...]
                tmp = zlib.decompress(rhs)
                PyObject_AsReadBuffer(tmp, &buf, &size)
                self.bitset = intBitSetCreateFromBuffer(buf, size)
[...]
as you can see, I decompress the rhs string into the tmp Python object, I then 
retrieve a pointer to tmp and then I retrieve a pointer and the size of the 
pointed area, in order to pass it to my C constructor.

When I compile everything without optimization (-O0) it always crashes after 
exiting from __new__.
I tried to selectively commenting-out the code and I found that the 
incriminated row is:
(Continue reading)

Permalink | Reply |
headers
Samuele Kaplun | 9 Aug 17:47
X-Face
Picon
Picon

Re: [Cython-dev] Segmentation Fault with -O0

To the lists:

In data giovedì 09 agosto 2007, Samuele Kaplun scribacchiaste dette parole:
> Dear Pyrex & Cython lists,
> I've found a strange behaviour of my code developed with Pyrex (and Cython)
> which I can't solve.
> I wrote and integer bit set extension to Python with Pyrex. It has the
> possibility to be searialized into a string and deserialized from that
> string.
> I pass a correctly serialized representation to the constructor of my code
> (in the rhs variable):
> [...]
> cdef class intbitset:
> [...]
>     cdef IntBitSet *bitset
> [...]
>     def __new__(intbitset self, rhs=0, int minsize=-1):
> [...]
>         cdef size_t size
>         cdef void *buf
> [...]
>                 tmp = zlib.decompress(rhs)
>                 PyObject_AsReadBuffer(tmp, &buf, &size)
>                 self.bitset = intBitSetCreateFromBuffer(buf, size)
> [...]
> as you can see, I decompress the rhs string into the tmp Python object, I
> then retrieve a pointer to tmp and then I retrieve a pointer and the size
> of the pointed area, in order to pass it to my C constructor.
>
> When I compile everything without optimization (-O0) it always crashes
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 9 Aug 18:48
Picon
Favicon

Re: Segmentation Fault with -O0


Samuele Kaplun wrote:
>> When I compile everything without optimization (-O0) it always crashes
>> after exiting from __new__.
>> I tried to selectively commenting-out the code and I found that the
>> incriminated row is:
>> 		PyObject_AsReadBuffer(tmp, &buf, &size)
>> But I really don't know why.
>> The most interesting thing, is that it never segfaults when compiled with
>> -02 or -03.
>> If checked this with latest Cython & Pyrex and I obtain the same behaviour.
>>
> Some additional information: I run the tests with gcc-3.4.6, 4.1.2 and 4.2.0, 
> with the same behaviour (crash with -O0, -O, no crash with -O2, -O3), on an 
> amd64 machine and Python 2.5. It never crash on gcc-3.x on 32bits.

How did you declare the function? Did you use Py_ssize_t for the size parameter?

Stefan
Permalink | Reply |
headers
Samuele Kaplun | 9 Aug 19:06
X-Face
Picon
Picon

Re: [Cython-dev] Segmentation Fault with -O0

In data giovedì 09 agosto 2007, Stefan Behnel scribacchiaste dette parole:
> Samuele Kaplun wrote:
> How did you declare the function? Did you use Py_ssize_t for the size
> parameter?
>
> Stefan

Bingo! There was the bug!! 
Thanks a lot, I owe you a beer or a pizza! :-)
Best regards,
	Samuele

--

-- 
.O.
..O
OOO
Permalink | Reply |
headers
Samuele Kaplun | 9 Aug 18:58
X-Face
Picon
Picon

Re: Segmentation Fault with -O0

In data giovedì 09 agosto 2007, Stefan Behnel scribacchiaste dette parole:
> Samuele Kaplun wrote:
> >> When I compile everything without optimization (-O0) it always crashes
> >> after exiting from __new__.
> >> I tried to selectively commenting-out the code and I found that the
> >> incriminated row is:
> >> 		PyObject_AsReadBuffer(tmp, &buf, &size)
> >> But I really don't know why.
> >> The most interesting thing, is that it never segfaults when compiled
> >> with -02 or -03.
> >> If checked this with latest Cython & Pyrex and I obtain the same
> >> behaviour.
> >
> > Some additional information: I run the tests with gcc-3.4.6, 4.1.2 and
> > 4.2.0, with the same behaviour (crash with -O0, -O, no crash with -O2,
> > -O3), on an amd64 machine and Python 2.5. It never crash on gcc-3.x on
> > 32bits.
>
> How did you declare the function? Did you use Py_ssize_t for the size
> parameter?
>
> Stefan

Dear Stefan
I used "int" taking it from the Python doc on 
http://docs.python.org/api/abstract-buffer.html
I will test with Py_ssize_t.
Best regards,
	Samuele
(Continue reading)

Permalink | Reply |
headers
Samuele Kaplun | 9 Aug 19:00
X-Face
Picon
Picon

Re: [Cython-dev] Segmentation Fault with -O0

In data giovedì 09 agosto 2007, Samuele Kaplun scribacchiaste dette parole:
> In data giovedì 09 agosto 2007, Stefan Behnel scribacchiaste dette parole:
> > Samuele Kaplun wrote:
> > >> When I compile everything without optimization (-O0) it always crashes
> > >> after exiting from __new__.
> > >> I tried to selectively commenting-out the code and I found that the
> > >> incriminated row is:
> > >> 		PyObject_AsReadBuffer(tmp, &buf, &size)
> > >> But I really don't know why.
> > >> The most interesting thing, is that it never segfaults when compiled
> > >> with -02 or -03.
> > >> If checked this with latest Cython & Pyrex and I obtain the same
> > >> behaviour.
> > >
> > > Some additional information: I run the tests with gcc-3.4.6, 4.1.2 and
> > > 4.2.0, with the same behaviour (crash with -O0, -O, no crash with -O2,
> > > -O3), on an amd64 machine and Python 2.5. It never crash on gcc-3.x on
> > > 32bits.
> >
> > How did you declare the function? Did you use Py_ssize_t for the size
> > parameter?
> >
> > Stefan
>
> Dear Stefan
> I used "int" taking it from the Python doc on
> http://docs.python.org/api/abstract-buffer.html
> I will test with Py_ssize_t.
> Best regards,
> 	Samuele
(Continue reading)

Permalink | Reply |
headers
Samuele Kaplun | 15 Aug 17:30
X-Face
Picon
Picon

Re: Segmentation Fault with -O0

Dear Stefan

In data giovedì 09 agosto 2007, Samuele Kaplun scribacchiaste dette parole:
> Herm, readingly carefully I see Py_ssize_t!
> Let's see, then.
> Best regards,
> 	Samuele

I've just discovered that I was right in saying that in the docs I red int. In 
fact the buffer interface use int in Python 2.3, 2.4, and Py_ssize_t in 
Python 2.5
I wonder if there's a macro or a recipe to make everything happy with Pyrex. 
Otherwise do you think that it's possible to build a .pyx that is aware of 
the Python version?
Best regards,
	Samuele

--

-- 
.O.
..O
OOO
Permalink | Reply |
headers
Stefan Behnel | 15 Aug 17:48
Picon
Favicon

Re: Segmentation Fault with -O0


Samuele Kaplun wrote:
> In data giovedì 09 agosto 2007, Samuele Kaplun scribacchiaste dette parole:
>> Herm, readingly carefully I see Py_ssize_t!
>> Let's see, then.
>> Best regards,
>> 	Samuele
> 
> I've just discovered that I was right in saying that in the docs I red int. In 
> fact the buffer interface use int in Python 2.3, 2.4, and Py_ssize_t in 
> Python 2.5
> I wonder if there's a macro or a recipe to make everything happy with Pyrex. 
> Otherwise do you think that it's possible to build a .pyx that is aware of 
> the Python version?

Well, have you tried compiling it under Python 2.4? It should work out-of-the-box.

Stefan
Permalink | Reply |
headers
Samuele Kaplun | 15 Aug 18:21
X-Face
Picon
Picon

Re: Segmentation Fault with -O0

In data mercoledì 15 agosto 2007, Stefan Behnel scribacchiaste dette parole:
> Samuele Kaplun wrote:
>
> Well, have you tried compiling it under Python 2.4? It should work
> out-of-the-box.
>
> Stefan

If I declare with:
ctypedef int Py_ssize_t in the global space, then it's ok, but I'm not sure 
this makes Python/Pyrex/Gcc happy.
Otherwise if import Py_ssize_t with:

cdef extern from "Python.h":
    ctypedef int Py_ssize_t

Then this works only on Python 2.5.

I found a PEP about Py_ssize_t stating that it was introduced after 2.5.0
So for now I've added a macro in a personal .h header with

#include <Python.h>
#if PY_VERSION_HEX < 0x02050000
typedef int Py_ssize_t;
#endif

and then I import Py_ssize_t from my header.
Do you think is a proper solution or even the first run should be perfectly 
fine?
Best regards,
(Continue reading)

Permalink | Reply |
headers
Robert Bradshaw | 15 Aug 18:54
Favicon

Re: Segmentation Fault with -O0

This is precisely what cython does--at the top of the generated c  
file it puts

#if PY_VERSION_HEX < 0x02050000
   typedef int Py_ssize_t;
   #define PY_SSIZE_T_MAX INT_MAX
   #define PY_SSIZE_T_MIN INT_MIN
   #define PyInt_FromSsize_t(z) PyInt_FromLong(z)
   #define PyInt_AsSsize_t(o)   PyInt_AsLong(o)
#endif

to be able to handle pre- and post- Python 2.5 without having to  
worry about it.

On Aug 15, 2007, at 9:21 AM, Samuele Kaplun wrote:

> In data mercoledì 15 agosto 2007, Stefan Behnel scribacchiaste  
> dette parole:
>> Samuele Kaplun wrote:
>>
>> Well, have you tried compiling it under Python 2.4? It should work
>> out-of-the-box.
>>
>> Stefan
>
> If I declare with:
> ctypedef int Py_ssize_t in the global space, then it's ok, but I'm  
> not sure
> this makes Python/Pyrex/Gcc happy.
> Otherwise if import Py_ssize_t with:
(Continue reading)

Permalink | Reply |

Gmane