Questions, help, and general non-development discussion related to Cython. ()

Felix Schlesinger | 2 Mar 2010 05:47

Re: cdef struct for nested ndarray

On Mar 1, 5:15 pm, Robert Bradshaw <rober... <at> math.washington.edu>
wrote:
> On Mar 1, 2010, at 12:43 PM, Felix Schlesinger wrote:

> > I am working with large numpy ndarrays in cython. These are 'record
> > arrays' like
> > np.array([], dtype= [('foo', 'i1'),('bar','S1'),('foobars',('i2',5)),
> > ('barfoos',('i2',5))  ])
>
> > What is the right way to declare this type as a struct in cython for
> > static typing and speed-up?
>
> > cdef packed struct arr:
> >    np.int8_t foo
> >    np.character_t bar
> >    np.ndarray[int16_t] foobars
> >    np.ndarray[int16_t] barfoos

> This also won't work because you're trying to use a struct--there's  
> not really a good way to do refcounting of struct members so not even  
> objects are allowed. Use a cdef class instead.

But the cdef class could not be used as an array type, right?

When I try I get
cdef class foobar:
	cdef np.ndarray i
	cdef np.ndarray j

def foo(np.ndarray[foobar]):

(Continue reading)

headers

Dag Sverre Seljebotn | 2 Mar 2010 08:21

Re: Re: cdef struct for nested ndarray

Felix Schlesinger wrote:
> On Mar 1, 5:15 pm, Robert Bradshaw <rober... <at> math.washington.edu>
> wrote:
>   
>> On Mar 1, 2010, at 12:43 PM, Felix Schlesinger wrote:
>>     
>
>   
>>> I am working with large numpy ndarrays in cython. These are 'record
>>> arrays' like
>>> np.array([], dtype= [('foo', 'i1'),('bar','S1'),('foobars',('i2',5)),
>>> ('barfoos',('i2',5))  ])
>>>       
>>> What is the right way to declare this type as a struct in cython for
>>> static typing and speed-up?
>>>       
>>> cdef packed struct arr:
>>>    np.int8_t foo
>>>    np.character_t bar
>>>    np.ndarray[int16_t] foobars
>>>    np.ndarray[int16_t] barfoos
>>>       
>
>   
>> This also won't work because you're trying to use a struct--there's  
>> not really a good way to do refcounting of struct members so not even  
>> objects are allowed. Use a cdef class instead.
>>     
>
> But the cdef class could not be used as an array type, right?

(Continue reading)

headers

Felix Schlesinger | 2 Mar 2010 16:55

Re: cdef struct for nested ndarray

> cdef packed struct MyDtype:
>     cdef int a
>     cdef float nested[10][20]

The 'cdef's inside the struct are typos, right? At least in cython
0.12 they are a syntax error.

> This *might* work now if you do
>
> cdef np.ndarray[MyDtype, cast=True] arr = numpy.zeros(....)

def foo(np.ndarray arr):
	cdef np.ndarray[MyDtype,cast=True] nestedarr = arr
	return 0

leads to an assertion error at Cython/Compiler/Buffer.py line 630 (see
end of message).

> Nested arrays contained in NumPy arrays are very different from the
> NumPy arrays themselves, as they are fixed-size at compilation time and
> don't act as their own object but just describe layout in memory.

But shouldn't their fixed size and lack of 'ownership' make it easier
to map them to C arrays in cython (if they are in C memory layout in
numpy)?

thanks
  Felix

Traceback (most recent call last):

(Continue reading)