headers
Stefan Behnel | 10 Jul 2012 09:24
Picon
Favicon

[Cython] failing memory view tests in Python 2.4/5

Hi,

I've fixed some of the errors that current block the tests. The remaining
failures in the memory view tests exist in both Py2.4 and 2.5, which hints
at a problem with the buffer support in numpy.pxd.

https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py25-ext/504/testReport/

The memslice test failure is annoying. It looks like slice() fails to
properly deep-copy in Py2.[45]. I don't know if we can do anything about
this, seems more like a CPython bug. Deep copying is used to split function
definitions for fused types and I guess slice objects are used in
combination with the memory slices. So both don't currently work together
in Py2.[45].

I'd like to release a 0.17 alpha soon, so please get these sorted out in
one way or another ASAP. I'm also fine with declaring these features
unusable in older Python releases if we can't fix them, but it would
obviously be better to get them working.

Stefan
Permalink | Reply |
headers
mark florisson | 10 Jul 2012 11:20
Picon
Gravatar

Re: [Cython] failing memory view tests in Python 2.4/5

On 10 July 2012 08:24, Stefan Behnel <stefan_ml@...> wrote:
> Hi,
>
> I've fixed some of the errors that current block the tests. The remaining
> failures in the memory view tests exist in both Py2.4 and 2.5, which hints
> at a problem with the buffer support in numpy.pxd.
>
> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py25-ext/504/testReport/
>
> The memslice test failure is annoying. It looks like slice() fails to
> properly deep-copy in Py2.[45]. I don't know if we can do anything about
> this, seems more like a CPython bug. Deep copying is used to split function
> definitions for fused types and I guess slice objects are used in
> combination with the memory slices. So both don't currently work together
> in Py2.[45].

I'm pretty sure I fixed the slice copying problem through a
__deepcopy__ overload a long time ago elsewhere. I don't think these
tests should be excluded from the test run yet. Perhaps we should run
the 2x tests from py27 down to 2.4. Tests in newer python's are less
likely to fail.

Do you want _refactor_indexnode merged for this release?

> I'd like to release a 0.17 alpha soon, so please get these sorted out in
> one way or another ASAP. I'm also fine with declaring these features
> unusable in older Python releases if we can't fix them, but it would
> obviously be better to get them working.

I'll try to find some time this week.
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 17 Jul 2012 10:58
Picon
Favicon

Re: [Cython] failing memory view tests in Python 2.4/5

mark florisson, 10.07.2012 11:20:
> On 10 July 2012 08:24, Stefan Behnel wrote:
>> I've fixed some of the errors that current block the tests. The remaining
>> failures in the memory view tests exist in both Py2.4 and 2.5, which hints
>> at a problem with the buffer support in numpy.pxd.
>>
>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py25-ext/504/testReport/
>>
>> The memslice test failure is annoying. It looks like slice() fails to
>> properly deep-copy in Py2.[45]. I don't know if we can do anything about
>> this, seems more like a CPython bug. Deep copying is used to split function
>> definitions for fused types and I guess slice objects are used in
>> combination with the memory slices. So both don't currently work together
>> in Py2.[45].
> 
> I'm pretty sure I fixed the slice copying problem through a
> __deepcopy__ overload a long time ago elsewhere. I don't think these
> tests should be excluded from the test run yet.

Ok.

> Perhaps we should run
> the 2x tests from py27 down to 2.4. Tests in newer python's are less
> likely to fail.

Test failures are bad in general, though. Just because you hear about them
a couple of minutes later doesn't mean that they are less important.

>> I'd like to release a 0.17 alpha soon, so please get these sorted out in
>> one way or another ASAP. I'm also fine with declaring these features
(Continue reading)

Permalink | Reply |
headers
mark florisson | 17 Jul 2012 12:15
Picon
Gravatar

Re: [Cython] failing memory view tests in Python 2.4/5

On 17 July 2012 09:58, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 10.07.2012 11:20:
>> On 10 July 2012 08:24, Stefan Behnel wrote:
>>> I've fixed some of the errors that current block the tests. The remaining
>>> failures in the memory view tests exist in both Py2.4 and 2.5, which hints
>>> at a problem with the buffer support in numpy.pxd.
>>>
>>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/BACKEND=c,PYVERSION=py25-ext/504/testReport/
>>>
>>> The memslice test failure is annoying. It looks like slice() fails to
>>> properly deep-copy in Py2.[45]. I don't know if we can do anything about
>>> this, seems more like a CPython bug. Deep copying is used to split function
>>> definitions for fused types and I guess slice objects are used in
>>> combination with the memory slices. So both don't currently work together
>>> in Py2.[45].
>>
>> I'm pretty sure I fixed the slice copying problem through a
>> __deepcopy__ overload a long time ago elsewhere. I don't think these
>> tests should be excluded from the test run yet.
>
> Ok.
>
>
>> Perhaps we should run
>> the 2x tests from py27 down to 2.4. Tests in newer python's are less
>> likely to fail.
>
> Test failures are bad in general, though. Just because you hear about them
> a couple of minutes later doesn't mean that they are less important.
>
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 17 Jul 2012 13:12
Picon
Favicon

Re: [Cython] failing memory view tests in Python 2.4/5

mark florisson, 17.07.2012 12:15:
> On 17 July 2012 09:58, Stefan Behnel wrote:
>> mark florisson, 10.07.2012 11:20:
>>> Perhaps we should run
>>> the 2x tests from py27 down to 2.4. Tests in newer python's are less
>>> likely to fail.
>>
>> Test failures are bad in general, though. Just because you hear about them
>> a couple of minutes later doesn't mean that they are less important.
> 
> If tests fail in a lower version and not a higher version it often
> means an easy to fix version incompatibility

And also the other way round, but that's usually more related to Py3 bugs.

> (although sometimes,
> especially with buffers and it's special casing, this is really not
> true). But the point is you know whether something is broken in
> general, or it's a version related thing, and that information in
> itself is useful.

We currently run the tests for Py2.[47] and Py3.[23] in parallel before all
others. I think that should be enough to handle that case. However, if one
of them fails, the tests against the remaining Python versions will not be
triggered. I think it might be more helpful to always have all of them run,
so that it's clearer which of the release series are affected (just 2.4? or
2.4 and 2.5, as in this case?)

Stefan
(Continue reading)

Permalink | Reply |
headers
mark florisson | 17 Jul 2012 13:18
Picon
Gravatar

Re: [Cython] failing memory view tests in Python 2.4/5

On 17 July 2012 12:12, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 17.07.2012 12:15:
>> On 17 July 2012 09:58, Stefan Behnel wrote:
>>> mark florisson, 10.07.2012 11:20:
>>>> Perhaps we should run
>>>> the 2x tests from py27 down to 2.4. Tests in newer python's are less
>>>> likely to fail.
>>>
>>> Test failures are bad in general, though. Just because you hear about them
>>> a couple of minutes later doesn't mean that they are less important.
>>
>> If tests fail in a lower version and not a higher version it often
>> means an easy to fix version incompatibility
>
> And also the other way round, but that's usually more related to Py3 bugs.
>
>
>> (although sometimes,
>> especially with buffers and it's special casing, this is really not
>> true). But the point is you know whether something is broken in
>> general, or it's a version related thing, and that information in
>> itself is useful.
>
> We currently run the tests for Py2.[47] and Py3.[23] in parallel before all
> others. I think that should be enough to handle that case. However, if one
> of them fails, the tests against the remaining Python versions will not be
> triggered. I think it might be more helpful to always have all of them run,
> so that it's clearer which of the release series are affected (just 2.4? or
> 2.4 and 2.5, as in this case?)
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 17 Jul 2012 11:02
Picon
Favicon

[Cython] merging _refactor_indexnode branch (was: failing memory view tests in Python 2.4/5)

mark florisson, 10.07.2012 11:20:
> Do you want _refactor_indexnode merged for this release?

I was ok with it when I first skipped over it. It would be good to give it
another pair of eyeballs, but apart from that, yes, I think it should go
in. I'll merge it as soon as Jenkins is happy with the test suite again
(and after re-enabling the broken tests, which are related after all), so
that we can see if everything is ok after the merge.

Stefan
Permalink | Reply |
headers
mark florisson | 17 Jul 2012 12:20
Picon
Gravatar

Re: [Cython] merging _refactor_indexnode branch (was: failing memory view tests in Python 2.4/5)

On 17 July 2012 10:02, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 10.07.2012 11:20:
>> Do you want _refactor_indexnode merged for this release?
>
> I was ok with it when I first skipped over it. It would be good to give it
> another pair of eyeballs, but apart from that, yes, I think it should go
> in. I'll merge it as soon as Jenkins is happy with the test suite again
> (and after re-enabling the broken tests, which are related after all), so
> that we can see if everything is ok after the merge.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel@...
> http://mail.python.org/mailman/listinfo/cython-devel

Hm, some things changed here. I rebased my current working branch
(_array_expressions) on that branch, and then I moved the buffer and
memoryview tests into their own directories in master, and I think I
rebased my devel branch along with _refactor_indexnode on master. So
I'll have to renew that pull request using the final rebase. I'll then
rebase the rest of my working branch on master.
Permalink | Reply |
headers
Stefan Behnel | 17 Jul 2012 12:52
Picon
Favicon

Re: [Cython] merging _refactor_indexnode branch

mark florisson, 17.07.2012 12:20:
> On 17 July 2012 10:02, Stefan Behnel wrote:
>> mark florisson, 10.07.2012 11:20:
>>> Do you want _refactor_indexnode merged for this release?
>>
>> I was ok with it when I first skipped over it. It would be good to give it
>> another pair of eyeballs, but apart from that, yes, I think it should go
>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>> (and after re-enabling the broken tests, which are related after all), so
>> that we can see if everything is ok after the merge.
>
> Hm, some things changed here. I rebased my current working branch
> (_array_expressions) on that branch, and then I moved the buffer and
> memoryview tests into their own directories in master, and I think I
> rebased my devel branch along with _refactor_indexnode on master. So
> I'll have to renew that pull request using the final rebase. I'll then
> rebase the rest of my working branch on master.

Ok, just take care to not overwrite it so that we keep the original comments.

Stefan
Permalink | Reply |
headers
Stefan Behnel | 23 Jul 2012 19:54
Picon
Favicon

Re: [Cython] merging _refactor_indexnode branch

mark florisson, 17.07.2012 12:20:
> On 17 July 2012 10:02, Stefan Behnel wrote:
>> mark florisson, 10.07.2012 11:20:
>>> Do you want _refactor_indexnode merged for this release?
>>
>> I was ok with it when I first skipped over it. It would be good to give it
>> another pair of eyeballs, but apart from that, yes, I think it should go
>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>> (and after re-enabling the broken tests, which are related after all), so
>> that we can see if everything is ok after the merge.
> 
> Hm, some things changed here. I rebased my current working branch
> (_array_expressions) on that branch, and then I moved the buffer and
> memoryview tests into their own directories in master, and I think I
> rebased my devel branch along with _refactor_indexnode on master. So
> I'll have to renew that pull request using the final rebase. I'll then
> rebase the rest of my working branch on master.

Would you have an estimate when this will be done? It would be good if this
could be merged for the alpha, to avoid too much code churn after the first
user tests.

Stefan
Permalink | Reply |
headers
mark florisson | 23 Jul 2012 20:07
Picon
Gravatar

Re: [Cython] merging _refactor_indexnode branch

On 23 July 2012 18:54, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 17.07.2012 12:20:
>> On 17 July 2012 10:02, Stefan Behnel wrote:
>>> mark florisson, 10.07.2012 11:20:
>>>> Do you want _refactor_indexnode merged for this release?
>>>
>>> I was ok with it when I first skipped over it. It would be good to give it
>>> another pair of eyeballs, but apart from that, yes, I think it should go
>>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>>> (and after re-enabling the broken tests, which are related after all), so
>>> that we can see if everything is ok after the merge.
>>
>> Hm, some things changed here. I rebased my current working branch
>> (_array_expressions) on that branch, and then I moved the buffer and
>> memoryview tests into their own directories in master, and I think I
>> rebased my devel branch along with _refactor_indexnode on master. So
>> I'll have to renew that pull request using the final rebase. I'll then
>> rebase the rest of my working branch on master.
>
> Would you have an estimate when this will be done? It would be good if this
> could be merged for the alpha, to avoid too much code churn after the first
> user tests.

Sorry, I forgot to do this. Actually if you want I can do it now, but
since it doesn't really bring any features it might be better to have
it wait until next release if it happens to break things. Your call,
I'll make a new PR.

> Stefan
>
(Continue reading)

Permalink | Reply |
headers
mark florisson | 23 Jul 2012 20:38
Picon
Gravatar

Re: [Cython] merging _refactor_indexnode branch

On 23 July 2012 19:07, mark florisson <markflorisson88@...> wrote:
> On 23 July 2012 18:54, Stefan Behnel <stefan_ml@...> wrote:
>> mark florisson, 17.07.2012 12:20:
>>> On 17 July 2012 10:02, Stefan Behnel wrote:
>>>> mark florisson, 10.07.2012 11:20:
>>>>> Do you want _refactor_indexnode merged for this release?
>>>>
>>>> I was ok with it when I first skipped over it. It would be good to give it
>>>> another pair of eyeballs, but apart from that, yes, I think it should go
>>>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>>>> (and after re-enabling the broken tests, which are related after all), so
>>>> that we can see if everything is ok after the merge.
>>>
>>> Hm, some things changed here. I rebased my current working branch
>>> (_array_expressions) on that branch, and then I moved the buffer and
>>> memoryview tests into their own directories in master, and I think I
>>> rebased my devel branch along with _refactor_indexnode on master. So
>>> I'll have to renew that pull request using the final rebase. I'll then
>>> rebase the rest of my working branch on master.
>>
>> Would you have an estimate when this will be done? It would be good if this
>> could be merged for the alpha, to avoid too much code churn after the first
>> user tests.
>
> Sorry, I forgot to do this. Actually if you want I can do it now, but
> since it doesn't really bring any features it might be better to have
> it wait until next release if it happens to break things. Your call,
> I'll make a new PR.

Ok, it's here: https://github.com/cython/cython/pull/137
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 23 Jul 2012 20:46
Picon
Favicon

Re: [Cython] merging _refactor_indexnode branch

mark florisson, 23.07.2012 20:38:
> On 23 July 2012 19:07, mark florisson wrote:
>> On 23 July 2012 18:54, Stefan Behnel wrote:
>>> mark florisson, 17.07.2012 12:20:
>>>> I'll have to renew that pull request using the final rebase. I'll then
>>>> rebase the rest of my working branch on master.
>>>
>>> Would you have an estimate when this will be done? It would be good if this
>>> could be merged for the alpha, to avoid too much code churn after the first
>>> user tests.
>>
>> Sorry, I forgot to do this. Actually if you want I can do it now, but
>> since it doesn't really bring any features it might be better to have
>> it wait until next release if it happens to break things. Your call,
>> I'll make a new PR.
> 
> Ok, it's here: https://github.com/cython/cython/pull/137

Cool, I'll give it a try.

> Are we going for a release branch again, or a devel branch? Last time
> many people assumed the master branch was the release branch.

I think we can leave things in the master for now and recreate the release
branch after the release. People can work in their local branches if they
have anything for 0.18 already.

Stefan
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 23 Jul 2012 20:55
Picon
Favicon

Re: [Cython] merging _refactor_indexnode branch

mark florisson, 23.07.2012 20:07:
> On 23 July 2012 18:54, Stefan Behnel wrote:
>> mark florisson, 17.07.2012 12:20:
>>> On 17 July 2012 10:02, Stefan Behnel wrote:
>>>> mark florisson, 10.07.2012 11:20:
>>>>> Do you want _refactor_indexnode merged for this release?
>>>>
>>>> I was ok with it when I first skipped over it. It would be good to give it
>>>> another pair of eyeballs, but apart from that, yes, I think it should go
>>>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>>>> (and after re-enabling the broken tests, which are related after all), so
>>>> that we can see if everything is ok after the merge.
>>>
>>> Hm, some things changed here. I rebased my current working branch
>>> (_array_expressions) on that branch, and then I moved the buffer and
>>> memoryview tests into their own directories in master, and I think I
>>> rebased my devel branch along with _refactor_indexnode on master. So
>>> I'll have to renew that pull request using the final rebase. I'll then
>>> rebase the rest of my working branch on master.
>>
>> Would you have an estimate when this will be done? It would be good if this
>> could be merged for the alpha, to avoid too much code churn after the first
>> user tests.
> 
> Sorry, I forgot to do this. Actually if you want I can do it now, but
> since it doesn't really bring any features it might be better to have
> it wait until next release if it happens to break things. Your call,
> I'll make a new PR.

Hmm, revisiting the amount of code that this changes now, I would prefer
(Continue reading)

Permalink | Reply |
headers
mark florisson | 23 Jul 2012 21:02
Picon
Gravatar

Re: [Cython] merging _refactor_indexnode branch

On 23 July 2012 19:55, Stefan Behnel <stefan_ml@...> wrote:
> mark florisson, 23.07.2012 20:07:
>> On 23 July 2012 18:54, Stefan Behnel wrote:
>>> mark florisson, 17.07.2012 12:20:
>>>> On 17 July 2012 10:02, Stefan Behnel wrote:
>>>>> mark florisson, 10.07.2012 11:20:
>>>>>> Do you want _refactor_indexnode merged for this release?
>>>>>
>>>>> I was ok with it when I first skipped over it. It would be good to give it
>>>>> another pair of eyeballs, but apart from that, yes, I think it should go
>>>>> in. I'll merge it as soon as Jenkins is happy with the test suite again
>>>>> (and after re-enabling the broken tests, which are related after all), so
>>>>> that we can see if everything is ok after the merge.
>>>>
>>>> Hm, some things changed here. I rebased my current working branch
>>>> (_array_expressions) on that branch, and then I moved the buffer and
>>>> memoryview tests into their own directories in master, and I think I
>>>> rebased my devel branch along with _refactor_indexnode on master. So
>>>> I'll have to renew that pull request using the final rebase. I'll then
>>>> rebase the rest of my working branch on master.
>>>
>>> Would you have an estimate when this will be done? It would be good if this
>>> could be merged for the alpha, to avoid too much code churn after the first
>>> user tests.
>>
>> Sorry, I forgot to do this. Actually if you want I can do it now, but
>> since it doesn't really bring any features it might be better to have
>> it wait until next release if it happens to break things. Your call,
>> I'll make a new PR.
>
(Continue reading)

Permalink | Reply |
headers
Stefan Behnel | 23 Jul 2012 21:08
Picon
Favicon

Re: [Cython] merging _refactor_indexnode branch

mark florisson, 23.07.2012 21:02:
> Was this comment "Note that some
> aspects of the memory views feature may not currently work in Python
> 2.4.x. This is a known bug." in the release notes about the copying of
> Python slice objects?

Yes, about the failing tests. I removed it now.

Stefan
Permalink | Reply |

Gmane