General discussion about the Cython source code generator for Python extensions

Stefan Behnel | 1 Jun 2008 14:10

Re: [Cython] about let users have more control on garbage collection

Hi,

Lisandro Dalcin wrote:
> cdef class Mat: # <- proxy class
>    cdef PetscMat # <- PETSc object, a pointer to a refcounted struct.
> 
> I need to store arbitrary Python objects inside the PETSc object, I'm
> actually storing a dictionary. This dictionary have to survive until
> the PETSc object is deallocated, but that deallocation do not
> necesarily occurs when the instance of my proxy class is deallocated,
> because the PETSc object can still 'live' (that is, a referece being
> owned) inside other PETSc objects.

Can't you work with singleton proxies? That's what we do in lxml. There's
never more than one Python Element proxy for an XML node struct. We use a
factory function to build a proxy for a struct and keep a back pointer from
the struct to the class, so that we can reuse an already existing proxy. All
the Python state is kept in attributes of the proxy object, and the struct can
be freed when the proxy is garbage collected by Python.

Stefan

headers

Lisandro Dalcin | 2 Jun 2008 16:58

Re: [Cython] about let users have more control on garbage collection

On 6/1/08, Stefan Behnel <stefan_ml@...> wrote:
> Hi,
> Can't you work with singleton proxies? That's what we do in lxml. There's
>  never more than one Python Element proxy for an XML node struct.

I'll take a look, but I believe such approach, even if possible, would
complicate a lot my implementation.

Stefan, iff it is no much work for you, could you point me a link to
the actual lxml code implementing all this machinery?

--

-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

headers

Stefan Behnel | 2 Jun 2008 21:13

Re: [Cython] about let users have more control on garbage collection

Hi,

Lisandro Dalcin wrote:
> On 6/1/08, Stefan Behnel <stefan_ml@...> wrote:
>> Hi,
>> Can't you work with singleton proxies? That's what we do in lxml. There's
>>  never more than one Python Element proxy for an XML node struct.
> 
> I'll take a look, but I believe such approach, even if possible, would
> complicate a lot my implementation.

Not sure. A factory function for creating proxies is a good idea in general,
and once that's in place, you can do all sorts of weird things in there.

> Stefan, iff it is no much work for you, could you point me a link to
> the actual lxml code implementing all this machinery?

Ah, asking about the deep magic, right? ;)

Here's the _elementFactory() function:

http://codespeak.net/svn/lxml/trunk/src/lxml/lxml.etree.pyx

and a bit more of the proxy machinery is in here:

http://codespeak.net/svn/lxml/trunk/src/lxml/proxy.pxi

But that truly is black magic...

Stefan

(Continue reading)