headers
P.J. Eby | 15 Jul 2011 20:41
Gravatar

Re: recursion limit

At 07:36 PM 7/15/2011 +0200, nicky van foreest wrote:
>Hi,
>
>I ran into a RuntimeError: maximum recursion depth exceeded in cmp
>with Trellis in this (simple) piece of code.

I ran the code and it does not produce an error for me; it just runs.

>I think I can repair this by resetting the recursion depth. However, I
>suppose this will not work if I run a simulation with a 10e6 jobs. Why
>actually does trellis run into this problem?

Since I'm not able to reproduce the problem, I couldn't 
say.  However, if you are getting recursion in a comparison, it's 
possible that it's because you are using a __cmp__ or other 
comparison method in your code that's not in the sample you sent me, 
wherein that comparison relies on trellis properties.  (OTOH, if that 
were the case, then changing the recursion depth wouldn't fix it.)

>  It makes me a bit
>suspicious about the scaleability of trellis, or should I not worry
>about this?

I don't think so.  The recalculation algorithm is only recursive if 
you request calculation of a value that has not previously been 
calculated.  That is, if you set up a chain of a million items, and 
only then request some info about the last one, *and* the values it 
depends on are lazy (that is, they don't get automatically computed 
just by initializing their owner object), then yes, it will have to 
recurse to one million depth (times a constant).
(Continue reading)

Permalink | Reply |
headers
P.J. Eby | 16 Jul 2011 00:04
Gravatar

Re: recursion limit

At 09:02 PM 7/15/2011 +0200, nicky van foreest wrote:
>For me this works too, but if you change the range from 30 to 3000,
>say, I suspect you will get an error. In fact, at my machine I already
>get an error when the range is 100.

Ok, confirmed at 100, and switching the 'compute' attrs to 'maintain' 
attrs makes it go away, even up to 10000.

>Ok. I get this. So the trick is to prevent this, like you propose with
>using trellis.maintain.

Yes.  Lazy computation is for things that you may or may not need to 
compute.  If you know you are going to need it from the start, then 
you should use maintain.

>I have some more theoretical questions about trellis. If you don't
>have time, please skip them.  First, if I were to start something like
>trellis, I think I would use some topological sort on a graph. I
>cannot find this in the trellis code.

Cell objects have a level attribute that's used to indicate their 
distance from a dependency graph root; is that what you're asking about?

>  Second, I would like to have a
>program/parser that figures out by itself the dependency graph of the
>attributes and functions. Now I have to make explicit the
>trellis.attrs and the trellis.compute.attrs/trellis.maintain.attrs,
>which I would prefer not to do by hand, but let the computer figure it
>out. Why is this is not implemented in trellis? (I must admit that I
>don't quite know how to do this; it is just a question out of
(Continue reading)

Permalink | Reply |
headers
nicky van foreest | 16 Jul 2011 22:43
Picon

Re: recursion limit

HI Philip,

Thanks for your explanations.

> Ok, confirmed at 100, and switching the 'compute' attrs to 'maintain' attrs
> makes it go away, even up to 10000.

Here it also works at 10000 too, but it becomes noticebly slow.
However, this is as expected, given the amount of work.

> Cell objects have a level attribute that's used to indicate their distance
> from a dependency graph root; is that what you're asking about?

Yes.

>
>
>>  Second, I would like to have a
>> program/parser that figures out by itself the dependency graph of the
>> attributes and functions. Now I have to make explicit the
>> trellis.attrs and the trellis.compute.attrs/trellis.maintain.attrs,
>> which I would prefer not to do by hand, but let the computer figure it
>> out. Why is this is not implemented in trellis? (I must admit that I
>> don't quite know how to do this; it is just a question out of
>> interest.)
>
> I don't understand what feature you're asking for here.  The distinction
> between compute, maintain, and attrs is part of the design of your program,
> not something that a parser can figure out.  By stating that something is a
> maintained value, it means that it may have side-effects and also that it
(Continue reading)

Permalink | Reply |

Gmane