Dmitrey | 12 Dec 16:09 2009
Picon

Suppressing of numpy __mul__, __div__ etc

hello all,
I have class oofun (in a package FuncDesigner, that is already used by lots of people) where some operations on numpy arrays (along with Python lists and numbers) are overloaded, such as __mul__, __add__, __pow__, __div__ etc.
There is a problem with numpy arrays: if I use a*f where a is array and f is oofun, it returns array with elements a[i]*f. Would it omit calling array.__mul__ and call only oofun.__rmul__, like it is done in numpy.matrix and Python lists/numbers, all would work ok as expected; but now it makes FuncDesigner code to work much slower or unexpectedly or doesn't work at all instead.
So, does anyone mind if I'll commit some changes to numpy __mul__, __div__ etc?
I intend to implement the following walkaround:
Now the code looks like this for array:

    def __mul__(self, i):
        return asarray(multiply(self, i))

and like this for numpy/matrixlib/defmatrix.py:

    def __mul__(self, other):
        if isinstance(other,(N.ndarray, list, tuple)) :
            return N.dot(self, asmatrix(other))
        if isscalar(other) or not hasattr(other, '__rmul__') :
            return N.dot(self, other)
        return NotImplemented

and I want to add an empty class named "CNumpyLeftOperatorOverloaded" to numpy, and if someone defines his class as a child of the one, __mul__ and others will not invoke __div__ etc, calling otherClass.__rdiv__ etc:

    def __mul__(self, i):
        return asarray(multiply(self, i)) if not isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
and declare my class as a child of the one.

As far as I understood, the changes should be added to numpy/core/defcherarray.py
So, does anyone mind me to implement it?
D.

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 12 Dec 21:56 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



2009/12/12 Dmitrey <tmp50 <at> ukr.net>
hello all,
I have class oofun (in a package FuncDesigner, that is already used by lots of people) where some operations on numpy arrays (along with Python lists and numbers) are overloaded, such as __mul__, __add__, __pow__, __div__ etc.
There is a problem with numpy arrays: if I use a*f where a is array and f is oofun, it returns array with elements a[i]*f. Would it omit calling array.__mul__ and call only oofun.__rmul__, like it is done in numpy.matrix and Python lists/numbers, all would work ok as expected; but now it makes FuncDesigner code to work much slower or unexpectedly or doesn't work at all instead.
So, does anyone mind if I'll commit some changes to numpy __mul__, __div__ etc?
I intend to implement the following walkaround:
Now the code looks like this for array:

    def __mul__(self, i):
        return asarray(multiply(self, i))

and like this for numpy/matrixlib/defmatrix.py:

    def __mul__(self, other):
        if isinstance(other,(N.ndarray, list, tuple)) :
            return N.dot(self, asmatrix(other))
        if isscalar(other) or not hasattr(other, '__rmul__') :
            return N.dot(self, other)
        return NotImplemented

and I want to add an empty class named "CNumpyLeftOperatorOverloaded" to numpy, and if someone defines his class as a child of the one, __mul__ and others will not invoke __div__ etc, calling otherClass.__rdiv__ etc:

    def __mul__(self, i):
        return asarray(multiply(self, i)) if not isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
and declare my class as a child of the one.

As far as I understood, the changes should be added to numpy/core/defcherarray.py
So, does anyone mind me to implement it?
D.


Sounds like you are exporting an array interface or subclassing ndarray. If the latter, you might be able to manipulate the value of __array_priority__. I haven't experimented with these things myself.

As to the proposed solutions, they are the start of a slippery slope of trying to identify all objects to which they should apply. I don't think we want to go there.

Chuck
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Robert Kern | 12 Dec 22:16 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On Sat, Dec 12, 2009 at 14:56, Charles R Harris
<charlesr.harris <at> gmail.com> wrote:
>
> 2009/12/12 Dmitrey <tmp50 <at> ukr.net>
>>
>> hello all,
>> I have class oofun (in a package FuncDesigner, that is already used by
>> lots of people) where some operations on numpy arrays (along with Python
>> lists and numbers) are overloaded, such as __mul__, __add__, __pow__,
>> __div__ etc.
>> There is a problem with numpy arrays: if I use a*f where a is array and f
>> is oofun, it returns array with elements a[i]*f. Would it omit calling
>> array.__mul__ and call only oofun.__rmul__, like it is done in numpy.matrix
>> and Python lists/numbers, all would work ok as expected; but now it makes
>> FuncDesigner code to work much slower or unexpectedly or doesn't work at all
>> instead.
>> So, does anyone mind if I'll commit some changes to numpy __mul__, __div__
>> etc?
>> I intend to implement the following walkaround:
>> Now the code looks like this for array:
>>
>>     def __mul__(self, i):
>>         return asarray(multiply(self, i))
>>
>> and like this for numpy/matrixlib/defmatrix.py:
>>
>>     def __mul__(self, other):
>>         if isinstance(other,(N.ndarray, list, tuple)) :
>>             return N.dot(self, asmatrix(other))
>>         if isscalar(other) or not hasattr(other, '__rmul__') :
>>             return N.dot(self, other)
>>         return NotImplemented
>>
>> and I want to add an empty class named "CNumpyLeftOperatorOverloaded" to
>> numpy, and if someone defines his class as a child of the one, __mul__ and
>> others will not invoke __div__ etc, calling otherClass.__rdiv__ etc:
>>
>>     def __mul__(self, i):
>>         return asarray(multiply(self, i)) if not
>> isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
>> and declare my class as a child of the one.
>>
>> As far as I understood, the changes should be added to
>> numpy/core/defcherarray.py
>> So, does anyone mind me to implement it?
>> D.
>>
>
> Sounds like you are exporting an array interface or subclassing ndarray. If
> the latter, you might be able to manipulate the value of __array_priority__.
> I haven't experimented with these things myself.

I don't think he's subclassing ndarray, but does have a class that
shouldn't be interpreted by ndarray as a scalar. In any case,
__array_priority__ doesn't matter; it just controls which type the
output of a multi-input ufunc.

> As to the proposed solutions, they are the start of a slippery slope of
> trying to identify all objects to which they should apply. I don't think we
> want to go there.

I think what he is asking for is an empty mixin class which other
folks could subclass to mark their classes. It would say "Hey,
ndarray! Let my __mul__, __rmul__, etc., take priority over yours,
regardless of which of us comes first in the expression." Otherwise,
ndarray will gladly consume pretty much any object on the other side
of the operator because it will treat it as an object scalar.

We could also define a standard attribute that could mark such classes
instead of requiring a mixin subclass.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 12 Dec 22:52 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



On Sat, Dec 12, 2009 at 2:16 PM, Robert Kern <robert.kern <at> gmail.com> wrote:

<snip>
 
I think what he is asking for is an empty mixin class which other
folks could subclass to mark their classes. It would say "Hey,
ndarray! Let my __mul__, __rmul__, etc., take priority over yours,
regardless of which of us comes first in the expression." Otherwise,
ndarray will gladly consume pretty much any object on the other side
of the operator because it will treat it as an object scalar.

We could also define a standard attribute that could mark 
such classes
instead of requiring a mixin subclass.


Ah, I completely misunderstood. Nevermind...

I think such a mixin base class would be useful.

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
josef.pktd | 12 Dec 23:06 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On Sat, Dec 12, 2009 at 4:52 PM, Charles R Harris
<charlesr.harris <at> gmail.com> wrote:
>
>
> On Sat, Dec 12, 2009 at 2:16 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
>
> <snip>
>
>>
>> I think what he is asking for is an empty mixin class which other
>> folks could subclass to mark their classes. It would say "Hey,
>> ndarray! Let my __mul__, __rmul__, etc., take priority over yours,
>> regardless of which of us comes first in the expression." Otherwise,
>> ndarray will gladly consume pretty much any object on the other side
>> of the operator because it will treat it as an object scalar.
>>
>> We could also define a standard attribute that could mark
>>
>> such classes
>> instead of requiring a mixin subclass.
>>
>
> Ah, I completely misunderstood. Nevermind...
>
> I think such a mixin base class would be useful.

In a similar direction, I was recently thinking whether it would be
possible to get an attribute or some indication for classes that
implement a similar interface as numpy.arrays, i.e. same list of
methods. This would make it easier to write functions for any kind of
ducks.

Instead of converting with np.array or np.asarray, we could check
whether the instances in the function arguments implement the required
interface, and if yes just use the methods without converting to
arrays.
I would rule out matrices in this because it changes the meaning of
the multiplication completely.

I don't know whether this could be handled by a mixin class. ?

Josef

> Chuck
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
Dmitrey | 13 Dec 09:38 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On Sat, Dec 12, 2009 at 4:52 PM, Robert Kern wrote:
> We could also define a standard attribute that could mark such classes instead of requiring a mixin subclass.

I had thought about the idea, but I suspect it will work slower:

    def __mul__(self, i):
        return asarray(multiply(self, i)) if not getattr(i, '_numpyLeftOperatorsOverloaded', False) else i.__rmul__(self)
vs
    def __mul__(self, i):
        return asarray(multiply(self, i)) if not isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)

But it doesn't matter for me essentially which way to chose; so which one do you agree to be implemented?

On Sat, Dec 12, 2009 at 4:52 PM, Charles R Harris
<charlesr.harris <at> gmail.com> wrote:
>Instead of converting with np.array or np.asarray, we could check whether the instances in the function arguments implement the required interface, and if yes just use the methods without converting to arrays.

Do you mean something like
    def __mul__(self, i):
        return asarray(multiply(self, i)) if not hasattr(i, '__rmul__', False) else i.__rmul__(self)
?
I guess this is wrong idea, because sometimes user will want to get result that is masked array of i instances. BTW, lots of data types have its own __rmul__ (Python lists, numbers, even numpy arrays and matrices), so I guess it will not work as expected (as it is done now) for them.


_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Alan G Isaac | 13 Dec 15:07 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On 12/13/2009 3:38 AM, Dmitrey wrote:
>      def __mul__(self, i):
>          return asarray(multiply(self, i)) if not getattr(i,
> '_numpyLeftOperatorsOverloaded', False) else i.__rmul__(self)
> vs
>      def __mul__(self, i):
>          return asarray(multiply(self, i)) if not
> isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
>

I am not at all speaking against this, but I am wondering
what exactly your object is getting out of overriding multiplication.

Also, for clarification, the proposal is not just to override
multiplication, but *all* arithmetic operations. Right?
(The examples have all been multiplication.)

Alan Isaac
Charles R Harris | 13 Dec 17:11 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



On Sun, Dec 13, 2009 at 7:07 AM, Alan G Isaac <aisaac <at> american.edu> wrote:
On 12/13/2009 3:38 AM, Dmitrey wrote:
>      def __mul__(self, i):
>          return asarray(multiply(self, i)) if not getattr(i,
> '_numpyLeftOperatorsOverloaded', False) else i.__rmul__(self)
> vs
>      def __mul__(self, i):
>          return asarray(multiply(self, i)) if not
> isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
>


I am not at all speaking against this, but I am wondering
what exactly your object is getting out of overriding multiplication.

Also, for clarification, the proposal is not just to override
multiplication, but *all* arithmetic operations. Right?
(The examples have all been multiplication.)


Yes, I think we would want to override all of them, but in ndarray itself. For instance, what happens now is:

In [1]: from numpy.polynomial import Polynomial as poly

In [2]: p = poly([0,1])

In [3]: ones(2) * p
Out[3]: array([poly([ 0.  1.], [-1.  1.]), poly([ 0.  1.], [-1.  1.])], dtype=object)

In [4]: p * ones(2)
Out[4]: Polynomial([ 0.,  1.,  1.], [-1.,  1.])

This is because the * ufunc treats the Polynomial class as a scalar, promotes the ones(2) to an object array, and does the multiplication. What a mixin class would do is exempt the polynomial from being treated as a scalar and force python to call its __rmul__ method instead.

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dag Sverre Seljebotn | 13 Dec 23:05 2009
Picon
Picon

Re: Suppressing of numpy __mul__, __div__ etc

Robert Kern wrote:
> On Sat, Dec 12, 2009 at 14:56, Charles R Harris
> <charlesr.harris <at> gmail.com> wrote:
>>
>> 2009/12/12 Dmitrey <tmp50 <at> ukr.net>
>>>
>>> hello all,
>>> I have class oofun (in a package FuncDesigner, that is already used by
>>> lots of people) where some operations on numpy arrays (along with
>>> Python
>>> lists and numbers) are overloaded, such as __mul__, __add__, __pow__,
>>> __div__ etc.
>>> There is a problem with numpy arrays: if I use a*f where a is array and
>>> f
>>> is oofun, it returns array with elements a[i]*f. Would it omit calling
>>> array.__mul__ and call only oofun.__rmul__, like it is done in
>>> numpy.matrix
>>> and Python lists/numbers, all would work ok as expected; but now it
>>> makes
>>> FuncDesigner code to work much slower or unexpectedly or doesn't work
>>> at all
>>> instead.
>>> So, does anyone mind if I'll commit some changes to numpy __mul__,
>>> __div__
>>> etc?
>>> I intend to implement the following walkaround:
>>> Now the code looks like this for array:
>>>
>>>     def __mul__(self, i):
>>>         return asarray(multiply(self, i))
>>>
>>> and like this for numpy/matrixlib/defmatrix.py:
>>>
>>>     def __mul__(self, other):
>>>         if isinstance(other,(N.ndarray, list, tuple)) :
>>>             return N.dot(self, asmatrix(other))
>>>         if isscalar(other) or not hasattr(other, '__rmul__') :
>>>             return N.dot(self, other)
>>>         return NotImplemented
>>>
>>> and I want to add an empty class named "CNumpyLeftOperatorOverloaded"
>>> to
>>> numpy, and if someone defines his class as a child of the one, __mul__
>>> and
>>> others will not invoke __div__ etc, calling otherClass.__rdiv__ etc:
>>>
>>>     def __mul__(self, i):
>>>         return asarray(multiply(self, i)) if not
>>> isinstance(i,CNumpyLeftOperatorOverloaded) else i.__rmul__(self)
>>> and declare my class as a child of the one.
>>>
>>> As far as I understood, the changes should be added to
>>> numpy/core/defcherarray.py
>>> So, does anyone mind me to implement it?
>>> D.
>>>
>>
>> Sounds like you are exporting an array interface or subclassing ndarray.
>> If
>> the latter, you might be able to manipulate the value of
>> __array_priority__.
>> I haven't experimented with these things myself.
>
> I don't think he's subclassing ndarray, but does have a class that
> shouldn't be interpreted by ndarray as a scalar. In any case,
> __array_priority__ doesn't matter; it just controls which type the
> output of a multi-input ufunc.
>
>> As to the proposed solutions, they are the start of a slippery slope of
>> trying to identify all objects to which they should apply. I don't think
>> we
>> want to go there.
>
> I think what he is asking for is an empty mixin class which other
> folks could subclass to mark their classes. It would say "Hey,
> ndarray! Let my __mul__, __rmul__, etc., take priority over yours,
> regardless of which of us comes first in the expression." Otherwise,
> ndarray will gladly consume pretty much any object on the other side
> of the operator because it will treat it as an object scalar.
>
> We could also define a standard attribute that could mark such classes
> instead of requiring a mixin subclass.

FWIW, I'd like to submit a patch to Sage for making NumPy arrays and Sage
matrices work better together which would require this functionality. +1
for an attribute and not a mixin, as that would allow Cython classes as
well.

Something like "__do_not_treat_as_scalar__ = True". The idea is simply to
stop ndarray from handling the operation, and let other classes implement
specific support for arithmetic with NumPy arrays. Which is not a slippery
slope at all.

Dag Sverre
Dmitrey | 14 Dec 12:25 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

I don't think the field name '__do_not_treat_as_scalar__ ' is a good choice. At first, it will be unclear what it is used for, at 2nd, some users / classes, that certainly are not scalars with overloaded __rdiv__ etc sometimes will prefer numpy __div__ method to be used.
As for me, I would prefer something like '__use_self_operators = True' or '__use_numpy_operations = False' or '__involve_self_overloaded_methods__ = True'.
I have to release FuncDesigner tomorrow (to keep my quarterly schedule) so please inform me ASAP which name do you finally choose.
Thank you in advance, D.

Dag Sverre wrote:
FWIW, I'd like to submit a patch to Sage for making NumPy arrays and Sage
matrices work better together which would require this functionality. +1
for an attribute and not a mixin, as that would allow Cython classes as
well.

Something like "__do_not_treat_as_scalar__ = True". The idea is simply to
stop ndarray from handling the operation, and let other classes implement
specific support for arithmetic with NumPy arrays. Which is not a slippery
slope at all.

Dag Sverre

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 14 Dec 19:06 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



2009/12/14 Dmitrey <tmp50 <at> ukr.net>
I don't think the field name '__do_not_treat_as_scalar__ ' is a good choice. At first, it will be unclear what it is used for, at 2nd, some users / classes, that certainly are not scalars with overloaded __rdiv__ etc sometimes will prefer numpy __div__ method to be used.
As for me, I would prefer something like '__use_self_operators = True' or '__use_numpy_operations = False' or '__involve_self_overloaded_methods__ = True'.
I have to release FuncDesigner tomorrow (to keep my quarterly schedule) so please inform me ASAP which name do you finally choose.
Thank you in advance, D.


I think it should have numpy or ndarray in the name somewhere to indicate that it is numpy specific. Hmm <looks in thesaurus>, maybe,

__supercede_ndarray__,
__disallow_ndarray__,
__deny_ndarray__,
__reject_ndarray__,
__refuse_ndarray__,
__exclude_ndarray__,
__reject_ndarray__, ...

My preference among those would be __deny_ndarray__.

Chuck
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 14 Dec 19:16 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



--- Исходное сообщение ---
От кого: Charles R Harris <charlesr.harris <at> gmail.com>
Кому: SciPy Developers List <scipy-dev <at> scipy.org>
Дата: 14 декабря, 20:06:46
Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey <tmp50 <at> ukr.net>

> I don't think the field name '__do_not_treat_as_scalar__ ' is a good
> choice. At first, it will be unclear what it is used for, at 2nd, some users
> / classes, that certainly are not scalars with overloaded __rdiv__ etc
> sometimes will prefer numpy __div__ method to be used.
> As for me, I would prefer something like '__use_self_operators = True' or
> '__use_numpy_operations = False' or '__involve_self_overloaded_methods__ =
> True'.
> I have to release FuncDesigner tomorrow (to keep my quarterly schedule) so
> please inform me ASAP which name do you finally choose.
> Thank you in advance, D.
>
>
I think it should have numpy or ndarray in the name somewhere to indicate
that it is numpy specific. Hmm <looks in thesaurus>, maybe,

__supercede_ndarray__,
__disallow_ndarray__,
__deny_ndarray__,
__reject_ndarray__,
__refuse_ndarray__,
__exclude_ndarray__,
__reject_ndarray__, ...

My preference among those would be __deny_ndarray__.

But isn't the issue present with numpy matrices or scipy.sparse matrices as well?
So I guess instead of ndarray another word should be used.
D.

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 14 Dec 19:26 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



2009/12/14 Dmitrey <tmp50 <at> ukr.net>


--- Исходное сообщение ---
От кого: Charles R Harris <charlesr.harris <at> gmail.com>
Кому: SciPy Developers List <scipy-dev <at> scipy.org>
Дата: 14 декабря, 20:06:46
Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey <tmp50 <at> ukr.net>

> I don't think the field name '__do_not_treat_as_scalar__ ' is a good
> choice. At first, it will be unclear what it is used for, at 2nd, some users
> / classes, that certainly are not scalars with overloaded __rdiv__ etc
> sometimes will prefer numpy __div__ method to be used.
> As for me, I would prefer something like '__use_self_operators = True' or
> '__use_numpy_operations = False' or '__involve_self_overloaded_methods__ =
> True'.
> I have to release FuncDesigner tomorrow (to keep my quarterly schedule) so
> please inform me ASAP which name do you finally choose.
> Thank you in advance, D.
>
>
I think it should have numpy or ndarray in the name somewhere to indicate
that it is numpy specific. Hmm <looks in thesaurus>, maybe,

__supercede_ndarray__,
__disallow_ndarray__,
__deny_ndarray__,
__reject_ndarray__,
__refuse_ndarray__,
__exclude_ndarray__,
__reject_ndarray__, ...

My preference among those would be __deny_ndarray__.

But isn't the issue present with numpy matrices or scipy.sparse matrices as well?
So I guess instead of ndarray another word should be used.

__has_precedence__
__is_prior__

 Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 14 Dec 19:40 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



--- Исходное сообщение ---
От кого: Charles R Harris <charlesr.harris <at> gmail.com>
Кому: SciPy Developers List <scipy-dev <at> scipy.org>
Дата: 14 декабря, 20:26:12
Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey <tmp50 <at> ukr.net>

>
>
> --- Исходное сообщение ---
> От кого: Charles R Harris <charlesr.harris <at> gmail.com>
> Кому: SciPy Developers List <scipy-dev <at> scipy.org>
> Дата: 14 декабря, 20:06:46
> Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>
> 2009/12/14 Dmitrey <tmp50 <at> ukr.net>
>
> > I don't think the field name '__do_not_treat_as_scalar__ ' is a good
> > choice. At first, it will be unclear what it is used for, at 2nd, some
> users
> > / classes, that certainly are not scalars with overloaded __rdiv__ etc
> > sometimes will prefer numpy __div__ method to be used.
> > As for me, I would prefer something like '__use_self_operators = True' or
>
> > '__use_numpy_operations = False' or '__involve_self_overloaded_methods__
> =
> > True'.
> > I have to release FuncDesigner tomorrow (to keep my quarterly schedule)
> so
> > please inform me ASAP which name do you finally choose.
> > Thank you in advance, D.
> >
> >
> I think it should have numpy or ndarray in the name somewhere to indicate
> that it is numpy specific. Hmm <looks in thesaurus>, maybe,
>
> __supercede_ndarray__,
> __disallow_ndarray__,
> __deny_ndarray__,
> __reject_ndarray__,
> __refuse_ndarray__,
> __exclude_ndarray__,
> __reject_ndarray__, ...
>
> My preference among those would be __deny_ndarray__.
>
>
> But isn't the issue present with numpy matrices or scipy.sparse matrices as
> well?
> So I guess instead of ndarray another word should be used.
>

__has_precedence__
__is_prior__

Chuck

I guess some numpy developers should choose the final name in numpy IRC channel and inform the list (ASAP) about their collective (and hence final) decision. I'm not skilled in English quite enough, but so short names seems too uninformative to me, they will be not used too often so I guess more informative should be preferred.
D.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 14 Dec 21:01 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



2009/12/14 Dmitrey <tmp50 <at> ukr.net>


--- Исходное сообщение ---
От кого: Charles R Harris <charlesr.harris <at> gmail.com>
Кому: SciPy Developers List <scipy-dev <at> scipy.org>
Дата: 14 декабря, 20:26:12
Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey <tmp50 <at> ukr.net>

>
>
> --- Исходное сообщение ---
> От кого: Charles R Harris <charlesr.harris <at> gmail.com>
> Кому: SciPy Developers List <scipy-dev <at> scipy.org>
> Дата: 14 декабря, 20:06:46
> Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>
> 2009/12/14 Dmitrey <tmp50 <at> ukr.net>
>
> > I don't think the field name '__do_not_treat_as_scalar__ ' is a good
> > choice. At first, it will be unclear what it is used for, at 2nd, some
> users
> > / classes, that certainly are not scalars with overloaded __rdiv__ etc
> > sometimes will prefer numpy __div__ method to be used.
> > As for me, I would prefer something like '__use_self_operators = True' or
>
> > '__use_numpy_operations = False' or '__involve_self_overloaded_methods__
> =
> > True'.
> > I have to release FuncDesigner tomorrow (to keep my quarterly schedule)
> so
> > please inform me ASAP which name do you finally choose.
> > Thank you in advance, D.
> >
> >
> I think it should have numpy or ndarray in the name somewhere to indicate
> that it is numpy specific. Hmm <looks in thesaurus>, maybe,
>
> __supercede_ndarray__,
> __disallow_ndarray__,
> __deny_ndarray__,
> __reject_ndarray__,
> __refuse_ndarray__,
> __exclude_ndarray__,
> __reject_ndarray__, ...
>
> My preference among those would be __deny_ndarray__.
>
>
> But isn't the issue present with numpy matrices or scipy.sparse matrices as
> well?
> So I guess instead of ndarray another word should be used.
>

__has_precedence__
__is_prior__

Chuck

I guess some numpy developers should choose the final name in numpy IRC channel and inform the list (ASAP) about their collective (and hence final) decision. I'm not skilled in English quite enough, but so short names seems too uninformative to me, they will be not used too often so I guess more informative should be preferred.

I don't think we should rush this. I'm waiting for more people to weigh in or offer different solutions.

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 14 Dec 22:03 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

It seems I was wrong - the issue doesn't matter for matrices, so we could use something like __exclude_ndarray_operations__ . If there will be no collective opinion of numpy developers in term of several hours, please don't mind me to commit the changes, it would be very important to have quarterly release of FuncDesigner being free of the bug.
Regards, D.

P.S. scipy.org server is down for now.


--- Исходное сообщение ---
От кого: Charles R Harris <charlesr.harris <at> gmail.com>
Кому: SciPy Developers List <scipy-dev <at> scipy.org>
Дата: Dec 14, 2009 22:01:53
Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey

>
>
> --- Исходное сообщение ---
> От кого: Charles R Harris
> Кому: SciPy Developers List
> Дата: 14 декабря, 20:26:12
> Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>
> 2009/12/14 Dmitrey
>
> >
> >
> > --- Исходное сообщение ---
> > От кого: Charles R Harris
> > Кому: SciPy Developers List
> > Дата: 14 декабря, 20:06:46
> > Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
> >
> > 2009/12/14 Dmitrey
> >
> > > I don't think the field name '__do_not_treat_as_scalar__ ' is a good
> > > choice. At first, it will be unclear what it is used for, at 2nd, some
> > users
> > > / classes, that certainly are not scalars with overloaded __rdiv__ etc
> > > sometimes will prefer numpy __div__ method to be used.
> > > As for me, I would prefer something like '__use_self_operators = True'
> or
> >
> > > '__use_numpy_operations = False' or
> '__involve_self_overloaded_methods__
> > =
> > > True'.
> > > I have to release FuncDesigner tomorrow (to keep my quarterly schedule)
>
> > so
> > > please inform me ASAP which name do you finally choose.
> > > Thank you in advance, D.
> > >
> > >
> > I think it should have numpy or ndarray in the name somewhere to indicate
>
> > that it is numpy specific. Hmm , maybe,
> >
> > __supercede_ndarray__,
> > __disallow_ndarray__,
> > __deny_ndarray__,
> > __reject_ndarray__,
> > __refuse_ndarray__,
> > __exclude_ndarray__,
> > __reject_ndarray__, ...
> >
> > My preference among those would be __deny_ndarray__.
> >
> >
> > But isn't the issue present with numpy matrices or scipy.sparse matrices
> as
> > well?
> > So I guess instead of ndarray another word should be used.
> >
>
> __has_precedence__
> __is_prior__
>
> Chuck
>
>
> I guess some numpy developers should choose the final name in numpy IRC
> channel and inform the list (ASAP) about their collective (and hence final)
> decision. I'm not skilled in English quite enough, but so short names seems
> too uninformative to me, they will be not used too often so I guess more
> informative should be preferred.
>

I don't think we should rush this. I'm waiting for more people to weigh in
or offer different solutions.

Chuck





_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Robert Kern | 14 Dec 22:24 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

2009/12/14 Dmitrey <tmp50 <at> ukr.net>:
> It seems I was wrong - the issue doesn't matter for matrices, so we could
> use something like __exclude_ndarray_operations__ . If there will be no
> collective opinion of numpy developers in term of several hours, please
> don't mind me to commit the changes, it would be very important to have
> quarterly release of FuncDesigner being free of the bug.

Commit what changes to which repository? We won't be modifying
anything in numpy or scipy in that time frame, sorry.

> P.S. scipy.org server is down for now.

It's up again.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
Dmitrey | 15 Dec 12:53 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

От кого: Robert Kern <robert.kern <at> gmail.com>

Commit what changes to which repository? We won't be modifying
anything in numpy or scipy in that time frame, sorry.

I meant commit those several lines of code into numpy repository, then I would propose FuncDesigner users to install  numpy version from latest svn snapshot.
Ok, now I'll create a ticket and inform FD users they should wait for next numpy release to get rid of the bug.
D.

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dag Sverre Seljebotn | 14 Dec 22:34 2009
Picon
Picon

Re: Suppressing of numpy __mul__, __div__ etc

Dmitrey wrote:
> It seems I was wrong - the issue doesn't matter for matrices, so we 
> could use something like __exclude_ndarray_operations__ . If there will 
> be no collective opinion of numpy developers in term of several hours, 
> please don't mind me to commit the changes, it would be very important 
> to have quarterly release of FuncDesigner being free of the bug.
> Regards, D.

Please, calm down. It's not unlikely that there will pass another 
quarter before the release of the next version of NumPy anyway; 1.4.0 is 
in release candidate mode. And won't your users use old versions of 
NumPy for some time to come anyway?

--

-- 
Dag Sverre
Dag Sverre Seljebotn | 16 Dec 09:12 2009
Picon
Picon

Re: Suppressing of numpy __mul__, __div__ etc

Charles R Harris wrote:
>
>
> 2009/12/14 Dmitrey <tmp50 <at> ukr.net <mailto:tmp50 <at> ukr.net>>
>
>
>
>     --- Исходное сообщение ---
>     От кого: Charles R Harris <charlesr.harris <at> gmail.com
>     <mailto:charlesr.harris <at> gmail.com>>
>     Кому: SciPy Developers List <scipy-dev <at> scipy.org
>     <mailto:scipy-dev <at> scipy.org>>
>     Дата: 14 декабря, 20:26:12
>     Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>
>         > --- Исходное сообщение ---
>         > От кого: Charles R Harris <charlesr.harris <at> gmail.com
>         <mailto:charlesr.harris <at> gmail.com>>
>         > Кому: SciPy Developers List <scipy-dev <at> scipy.org
>         <mailto:scipy-dev <at> scipy.org>>
>         > Дата: 14 декабря, 20:06:46
>         > Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>
>         > I think it should have numpy or ndarray in the name
>         somewhere to indicate
>         > that it is numpy specific. Hmm <looks in thesaurus>, maybe,
>         >
>         > __supercede_ndarray__,
>         > __disallow_ndarray__,
>         > __deny_ndarray__,
>         > __reject_ndarray__,
>         > __refuse_ndarray__,
>         > __exclude_ndarray__,
>         > __reject_ndarray__, ...
>         >
>         > My preference among those would be __deny_ndarray__.
>         >
>         >
>         > But isn't the issue present with numpy matrices or
>         scipy.sparse matrices as
>         > well?
>         > So I guess instead of ndarray another word should be used.
>         >
>
>         __has_precedence__
>         __is_prior__
>
>         Chuck
>
>     N
>     I guess some numpy developers should choose the final name in
>     numpy IRC channel and inform the list (ASAP) about their
>     collective (and hence final) decision. I'm not skilled in English
>     quite enough, but so short names seems too uninformative to me,
>     they will be not used too often so I guess more informative should
>     be preferred.
>
>
> I don't think we should rush this. I'm waiting for more people to 
> weigh in or offer different solutions.

Well, a different solution would be to have a standard with "operand 
precedence", i.e. an integer which could be compared between objects, 
and the highest one wins and gets to decide how the arithmetic operation 
is carried out.

This could be used to simplify NumPy itself. I.e. assign e.g. 
__operand_precedence__ = 0 for ndarray, 100 for matrix, and use it 
internally in NumPy to decide who carries out the operation.

Between other libraries it gets messy to try to figure out what range of 
values to use though. One would really want to be able to make 
statements about a partially ordered set ("I take priority over ndarray; 
a Frobnicator takes priority over me..."), but it's probably way too 
time consuming and complicated.

I like the "precedence" word though. So my suggestions are
  __operand_precedence__ # boolean or integer?
  __numpy_operand_precedence__ # boolean or integer?
  __precedence_over_ndarray__ # bool

Dag Sverre
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Sebastian Walter | 16 Dec 23:53 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

I have also implemented/wrapped various automatic differentiation
tools in python
(http://github.com/b45ch1/algopy , http://github.com/b45ch1/pyadolc,
http://github.com/b45ch1/pycppad if someone is interested)
and I have also come across some numpy glitches and inconsistencies.

However, this particular problem I have not encountered. I'm not sure
I understand the rationale why an expression like numpy.array([1,2] *
oofun(1) should  call the oofun.__rmul__ operator.
Hence, I'm a little sceptical about this enhancement.

What is wrong with the following implementation? It works perfectly fine...

--------------- start code snippet -----------------

class oofun:
    def __init__(self,x):
        self.x = x

    def __mul__(self, rhs):
        print 'called __mul__'
        if isinstance(rhs, oofun):
            return oofun(self.x * rhs.x)
        else:
            return rhs * self

    def __rmul__(self, lhs):
        print 'called __rmul__'
        return oofun(self.x * lhs)

    def __str__(self):
        return str(self.x)+'a'

    def __repr__(self):
        return str(self)

------------- end code snippet ----------------

--------- output ----------
basti <at> shlp:~/Desktop$ python live_demo.py
called __mul__
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]
------------- end output --------------

regards,
Sebastian

2009/12/16 Dag Sverre Seljebotn <dagss <at> student.matnat.uio.no>:
> Charles R Harris wrote:
>>
>>
>> 2009/12/14 Dmitrey <tmp50 <at> ukr.net <mailto:tmp50 <at> ukr.net>>
>>
>>
>>
>>     --- Исходное сообщение ---
>>     От кого: Charles R Harris <charlesr.harris <at> gmail.com
>>     <mailto:charlesr.harris <at> gmail.com>>
>>     Кому: SciPy Developers List <scipy-dev <at> scipy.org
>>     <mailto:scipy-dev <at> scipy.org>>
>>     Дата: 14 декабря, 20:26:12
>>     Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>>
>>         > --- Исходное сообщение ---
>>         > От кого: Charles R Harris <charlesr.harris <at> gmail.com
>>         <mailto:charlesr.harris <at> gmail.com>>
>>         > Кому: SciPy Developers List <scipy-dev <at> scipy.org
>>         <mailto:scipy-dev <at> scipy.org>>
>>         > Дата: 14 декабря, 20:06:46
>>         > Тема: Re: [SciPy-dev] Suppressing of numpy __mul__, __div__ etc
>>
>>         > I think it should have numpy or ndarray in the name
>>         somewhere to indicate
>>         > that it is numpy specific. Hmm <looks in thesaurus>, maybe,
>>         >
>>         > __supercede_ndarray__,
>>         > __disallow_ndarray__,
>>         > __deny_ndarray__,
>>         > __reject_ndarray__,
>>         > __refuse_ndarray__,
>>         > __exclude_ndarray__,
>>         > __reject_ndarray__, ...
>>         >
>>         > My preference among those would be __deny_ndarray__.
>>         >
>>         >
>>         > But isn't the issue present with numpy matrices or
>>         scipy.sparse matrices as
>>         > well?
>>         > So I guess instead of ndarray another word should be used.
>>         >
>>
>>         __has_precedence__
>>         __is_prior__
>>
>>         Chuck
>>
>>     N
>>     I guess some numpy developers should choose the final name in
>>     numpy IRC channel and inform the list (ASAP) about their
>>     collective (and hence final) decision. I'm not skilled in English
>>     quite enough, but so short names seems too uninformative to me,
>>     they will be not used too often so I guess more informative should
>>     be preferred.
>>
>>
>> I don't think we should rush this. I'm waiting for more people to
>> weigh in or offer different solutions.
>
> Well, a different solution would be to have a standard with "operand
> precedence", i.e. an integer which could be compared between objects,
> and the highest one wins and gets to decide how the arithmetic operation
> is carried out.
>
> This could be used to simplify NumPy itself. I.e. assign e.g.
> __operand_precedence__ = 0 for ndarray, 100 for matrix, and use it
> internally in NumPy to decide who carries out the operation.
>
> Between other libraries it gets messy to try to figure out what range of
> values to use though. One would really want to be able to make
> statements about a partially ordered set ("I take priority over ndarray;
> a Frobnicator takes priority over me..."), but it's probably way too
> time consuming and complicated.
>
> I like the "precedence" word though. So my suggestions are
>  __operand_precedence__ # boolean or integer?
>  __numpy_operand_precedence__ # boolean or integer?
>  __precedence_over_ndarray__ # bool
>
>
> Dag Sverre
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 17 Dec 04:18 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



2009/12/16 Sebastian Walter <sebastian.walter <at> gmail.com>
I have also implemented/wrapped various automatic differentiation
tools in python
(http://github.com/b45ch1/algopy , http://github.com/b45ch1/pyadolc,
http://github.com/b45ch1/pycppad if someone is interested)
and I have also come across some numpy glitches and inconsistencies.

However, this particular problem I have not encountered. I'm not sure
I understand the rationale why an expression like numpy.array([1,2] *
oofun(1) should  call the oofun.__rmul__ operator.
Hence, I'm a little sceptical about this enhancement.

What is wrong with the following implementation? It works perfectly fine...

--------------- start code snippet -----------------

class oofun:
   def __init__(self,x):
       self.x = x

   def __mul__(self, rhs):
       print 'called __mul__'
       if isinstance(rhs, oofun):
           return oofun(self.x * rhs.x)
       else:
           return rhs * self

   def __rmul__(self, lhs):
       print 'called __rmul__'
       return oofun(self.x * lhs)

   def __str__(self):
       return str(self.x)+'a'

   def __repr__(self):
       return str(self)

------------- end code snippet ----------------

--------- output ----------
basti <at> shlp:~/Desktop$ python live_demo.py
called __mul__
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]
------------- end output --------------



That makes the behavior consistent. But suppose as a convenience one wants to implement left/right multiplication by python integers but doesn't want to allow multiplication by arrays? Or wants multiplication by arrays array-wise, not element-wise?

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 17 Dec 08:38 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

От кого: Sebastian Walter <sebastian.walter <at> gmail.com>

I have also implemented/wrapped various automatic differentiation
tools in python
(http://github.com/b45ch1/algopy , http://github.com/b45ch1/pyadolc,
http://github.com/b45ch1/pycppad if someone is interested)
and I have also come across some numpy glitches and inconsistencies.

However, this particular problem I have not encountered. I'm not sure
I understand the rationale why an expression like numpy.array([1,2] *
oofun(1) should call the oofun.__rmul__ operator.
Hence, I'm a little sceptical about this enhancement.

What is wrong with the following implementation? It works perfectly fine...

--------------- start code snippet -----------------

class oofun:
def __init__(self,x):
self.x = x

def __mul__(self, rhs):
print 'called __mul__'
if isinstance(rhs, oofun):
return oofun(self.x * rhs.x)
else:
return rhs * self

def __rmul__(self, lhs):
print 'called __rmul__'
return oofun(self.x * lhs)

def __str__(self):
return str(self.x)+'a'

def __repr__(self):
return str(self)

------------- end code snippet ----------------

--------- output ----------
basti <at> shlp:~/Desktop$ python live_demo.py
called __mul__
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]
called __rmul__
called __rmul__
called __rmul__
[2.0a 2.0a 2.0a]

But I don't want to get ndarray of N oofuncs, I just want to get single oofunc. At first, evaluating of each oofunc is rather costly, so for N >> 1 I get serious slow down; at second, sometimes my code doesn't work at all - for example, when I create a constraint c = oof1 < oof2 (or lots of other samples). I expect result of a*oof, a/oof etc to be of type oofun while it doesn't (it yields ndarray of oofuns).


_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dag Sverre Seljebotn | 17 Dec 10:45 2009
Picon
Picon

Re: Suppressing of numpy __mul__, __div__ etc

Sebastian Walter wrote:
> I have also implemented/wrapped various automatic differentiation
> tools in python
> (http://github.com/b45ch1/algopy , http://github.com/b45ch1/pyadolc,
> http://github.com/b45ch1/pycppad if someone is interested)
> and I have also come across some numpy glitches and inconsistencies.
>
> However, this particular problem I have not encountered. I'm not sure
> I understand the rationale why an expression like numpy.array([1,2] *
> oofun(1) should  call the oofun.__rmul__ operator.
> Hence, I'm a little sceptical about this enhancement.
>
> What is wrong with the following implementation? It works perfectly fine...
>   
Well, if you feel that way, then simply don't use this feature. This 
just leads to more flexibility for writers of libraries which depend on 
NumPy.

The suggestion isn't really inconsistent with Python. Even if it is true 
that left-mul takes precedence in Python, it is NOT common that an 
object handles all arithmetic operations whether they make sense or not. So

2 * MyClassWhichKnowsAboutPythonIntegers()

works, because the "2" doesn't try to do anything and rmul is called! 
NumPy is special because rmul never gets called with NumPy arrays, which 
can be quite inconvenient at times.

Dag Sverre
Sebastian Walter | 17 Dec 14:12 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

1) From an abstract point of view, oofun should be a data type.
And the container of choice would be a numpy.array of dtype ofun, not
of dtype object.

so array([1,2,3],dtype=float) * oofun(2) should return
array([2,4,6],dtype=returntype(oofun,float))

I.e. there should be a way to implement new data types.
Maybe add something like

class oofun:
    dtype = True

then ndarray.__mul__ would call appropriate interface functions
provided by the oofun class.
I think that's what josef suggested in an earlier post.

2) I understand that the current behaviour is not nice, e.g. for array
* matrix the matrix.__rmul__ operator should be called.
In fact, I run into the same problem a lot.

I think the only special case of interest are objects that are
themselves containers. E.g. matrix is a container, lil_matrix is a
container, etc.

ndarray knows how to treat some containers, e.g. lists.
The question is, what should ndarray do when it encounters an object
that is a container it doesn't know how to handle.

Wouldn't the following make sense:

When ndarray.__mul__(self, other) is called, and other is a container,
it should simply check if it knows how to handle that container.
If ndarray doesn't know, it should call the other.__rmul__ and hope
that `other` knows what to do.

E.g.

class matrix:
    iscontainer = True
    ....

class ndarray:

    def __mul__(self, other):
       if other.iscontainer:
           if not know_what_to_do_with(other):
               other.__rmul__(self)

      else:
          do what is done now: treat other as dtype=object

then array * matrix
would call the ndarray.__mul__ operator
it then realizes that matrix is a container but ndarray doesnt know
what to do with it. It would therefore call matrix.__rmul__

If other is not a container it is treated as object and we would get
the current behaviour. This would allow to stay backward compatible.

On Thu, Dec 17, 2009 at 10:45 AM, Dag Sverre Seljebotn
<dagss <at> student.matnat.uio.no> wrote:
> Sebastian Walter wrote:
>> I have also implemented/wrapped various automatic differentiation
>> tools in python
>> (http://github.com/b45ch1/algopy , http://github.com/b45ch1/pyadolc,
>> http://github.com/b45ch1/pycppad if someone is interested)
>> and I have also come across some numpy glitches and inconsistencies.
>>
>> However, this particular problem I have not encountered. I'm not sure
>> I understand the rationale why an expression like numpy.array([1,2] *
>> oofun(1) should  call the oofun.__rmul__ operator.
>> Hence, I'm a little sceptical about this enhancement.
>>
>> What is wrong with the following implementation? It works perfectly fine...
>>
> Well, if you feel that way, then simply don't use this feature. This
> just leads to more flexibility for writers of libraries which depend on
> NumPy.
>
> The suggestion isn't really inconsistent with Python. Even if it is true
> that left-mul takes precedence in Python, it is NOT common that an
> object handles all arithmetic operations whether they make sense or not. So
>
> 2 * MyClassWhichKnowsAboutPythonIntegers()
>
> works, because the "2" doesn't try to do anything and rmul is called!
> NumPy is special because rmul never gets called with NumPy arrays, which
> can be quite inconvenient at times.
>
> Dag Sverre
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
Dmitrey | 17 Dec 14:28 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

От кого: Sebastian Walter <sebastian.walter <at> gmail.com>

1) From an abstract point of view, oofun should be a data type.
And the container of choice would be a numpy.array of dtype ofun, not
of dtype object.

so array([1,2,3],dtype=float) * oofun(2) should return
array([2,4,6],dtype=returntype(oofun,float))

What do you mean under oofun(2) ?!
oofun is
array([1,2,3],dtype=float) * oofun_instance should return result of type oofun and nothing else.
Same to other classes - SAGE, polinomials, etc.

When ndarray.__mul__(self, other) is called, and other is a container,
it should simply check if it knows how to handle that container.
If ndarray doesn't know, it should call the other.__rmul__ and hope
that `other` knows what to do.

E.g.

class matrix:
iscontainer = True
....

class ndarray:

def __mul__(self, other):
if other.iscontainer:
if not know_what_to_do_with(other):
other.__rmul__(self)

else:
do what is done now: treat other as dtype=object

then array * matrix
would call the ndarray.__mul__ operator
it then realizes that matrix is a container but ndarray doesnt know
what to do with it. It would therefore call matrix.__rmul__
 

If other is not a container it is treated as object and we would get
the current behaviour. This would allow to stay backward compatible.

As for me, I dislike the idea - it's too complicated, and some containers sometimes could want to take operations priority, using their __rmul__  etc instead of ndarray __mul__ etc.

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Sebastian Walter | 17 Dec 16:22 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

2009/12/17 Dmitrey <tmp50 <at> ukr.net>:
> От кого: Sebastian Walter <sebastian.walter <at> gmail.com>
>
> 1) From an abstract point of view, oofun should be a data type.
> And the container of choice would be a numpy.array of dtype ofun, not
> of dtype object.
>
> so array([1,2,3],dtype=float) * oofun(2) should return
> array([2,4,6],dtype=returntype(oofun,float))
>
> What do you mean under oofun(2) ?!
i don't know the intrinsic implementation of your AD tool. Maybe I'm
wrong, but in automatic differentiation,
a computational graph of an algorithm is

> oofun is
> array([1,2,3],dtype=float) * oofun_instance should return result of type
> oofun and nothing else.
> Same to other classes - SAGE, polinomials, etc.

could you elaborate on why someone would
want to do an   `array * polynomial`  operation and not expect it to
be an array of polynomial as result?

>
> When ndarray.__mul__(self, other) is called, and other is a container,
> it should simply check if it knows how to handle that container.
> If ndarray doesn't know, it should call the other.__rmul__ and hope
> that `other` knows what to do.
>
> E.g.
>
> class matrix:
> iscontainer = True
> ....
>
> class ndarray:
>
> def __mul__(self, other):
> if other.iscontainer:
> if not know_what_to_do_with(other):
> other.__rmul__(self)
>
> else:
> do what is done now: treat other as dtype=object
>
> then array * matrix
> would call the ndarray.__mul__ operator
> it then realizes that matrix is a container but ndarray doesnt know
> what to do with it. It would therefore call matrix.__rmul__
>
>
>
> If other is not a container it is treated as object and we would get
> the current behaviour. This would allow to stay backward compatible.
>
> As for me, I dislike the idea - it's too complicated, and some containers
> sometimes could want to take operations priority, using their __rmul__  etc
> instead of ndarray __mul__ etc.

you would simply add

class oofun:
    iscontainer = True

to your oofun implementation and you would get exactly what you want....
I don't think this in any way complicated. It is basically what you
have asked for in your original post.

>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 17 Dec 16:37 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

От кого: Sebastian Walter <sebastian.walter <at> gmail.com>
could you elaborate on why someone would
want to do an `array * polynomial` operation and not expect it to
be an array of polynomial as result?

this has been already mentioned in the thread
http://permalink.gmane.org/gmane.comp.python.scientific.devel/12445
In [1]: from numpy.polynomial import Polynomial as poly

In [2]: p = poly([0,1])

In [3]: ones(2) * p
Out[3]: array([poly([ 0.  1.], [-1.  1.]), poly([ 0.  1.], [-1.  1.])], dtype=object)

In [4]: p * ones(2)
Out[4]: Polynomial([ 0.,  1.,  1.], [-1.,  1.])

you would simply add

class oofun:
iscontainer = True

to your oofun implementation and you would get exactly what you want....
I guess "iscontainer" is a bad choice for the field, but if numpy developers decide to use this one, let it be, it doesn't matter for me sufficiently.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Sebastian Walter | 17 Dec 17:13 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

2009/12/17 Dmitrey <tmp50 <at> ukr.net>:
> От кого: Sebastian Walter <sebastian.walter <at> gmail.com>
>
> could you elaborate on why someone would
> want to do an `array * polynomial` operation and not expect it to
> be an array of polynomial as result?
>
> this has been already mentioned in the thread
> http://permalink.gmane.org/gmane.comp.python.scientific.devel/12445
> In [1]: from numpy.polynomial import Polynomial as poly
>
> In [2]: p = poly([0,1])
>
> In [3]: ones(2) * p
> Out[3]: array([poly([ 0.  1.], [-1.  1.]), poly([ 0.  1.], [-1.  1.])],
> dtype=object)
>
> In [4]: p * ones(2)
> Out[4]: Polynomial([ 0.,  1.,  1.], [-1.,  1.])

let me rephrase then. I don't understand why p * ones(2) should give
Polynomial([ 0.,  1.,  1.], [-1.,  1.]).
A polynomial over the reals is a data type with a ring structure and
should therefore behave "similarly" to floats IMHO.

>
> you would simply add
>
> class oofun:
> iscontainer = True
>
> to your oofun implementation and you would get exactly what you want....
>
> I guess "iscontainer" is a bad choice for the field, but if numpy developers
> decide to use this one, let it be, it doesn't matter for me sufficiently.

I'm just making suggestions ;). I'm as much numpy dev as you are...
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Dmitrey | 17 Dec 18:07 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

От кого: Sebastian Walter <sebastian.walter <at> gmail.com>


let me rephrase then. I don't understand why p * ones(2) should give
Polynomial([ 0., 1., 1.], [-1., 1.]). A polynomial over the reals is a data type with a ring structure and
should therefore behave "similarly" to floats IMHO.

Since I'm not a numpy developer, I cannot give you irrefutable answer, but I guess it is much more useful for numpy users that are mostly engineering programmers, not researchers of a data type with a ring structure.

Also, this is not only up to polynomials - as it has been mentioned, this issue is important for stacking with SAGE data types, oofuns etc, where users certainly want to get same type instead of an ndarray.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
James Bergstra | 17 Dec 22:27 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

I develop another symbolic-over-numpy package called theano, and
somehow we avoid this problem.

In [1]: import theano

In [2]: import numpy

In [3]: numpy.ones(4) * theano.tensor.dmatrix()
Out[3]: Elemwise{mul,no_inplace}.0

In [4]: theano.tensor.dmatrix() * theano.tensor.dmatrix()
Out[4]: Elemwise{mul,no_inplace}.0

In [5]: theano.tensor.dmatrix() * numpy.ones(4)
Out[5]: Elemwise{mul,no_inplace}.0

The dmatrix() function returns an instance of the TensorVariable class
defined in this file:
http://trac-hg.assembla.com/theano/browser/theano/tensor/basic.py#L901

I think the only thing we added  for numpy was __array_priority__ =
1000, which has already been suggested here.  I'm confused by why this
thread goes on.

James

2009/12/17 Dmitrey <tmp50 <at> ukr.net>:
> От кого: Sebastian Walter <sebastian.walter <at> gmail.com>
>
>
> let me rephrase then. I don't understand why p * ones(2) should give
> Polynomial([ 0., 1., 1.], [-1., 1.]).
>
> A polynomial over the reals is a data type with a ring structure and
>
> should therefore behave "similarly" to floats IMHO.
>
>
>
> Since I'm not a numpy developer, I cannot give you irrefutable answer, but I
> guess it is much more useful for numpy users that are mostly engineering
> programmers, not researchers of a data type with a ring structure.
>
> Also, this is not only up to polynomials - as it has been mentioned, this
> issue is important for stacking with SAGE data types, oofuns etc, where
> users certainly want to get same type instead of an ndarray.
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev <at> scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>

--

-- 
http://www-etud.iro.umontreal.ca/~bergstrj
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 18 Dec 00:35 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



On Thu, Dec 17, 2009 at 2:27 PM, James Bergstra <james.bergstra <at> gmail.com> wrote:
I develop another symbolic-over-numpy package called theano, and
somehow we avoid this problem.

In [1]: import theano

In [2]: import numpy

In [3]: numpy.ones(4) * theano.tensor.dmatrix()
Out[3]: Elemwise{mul,no_inplace}.0

In [4]: theano.tensor.dmatrix() * theano.tensor.dmatrix()
Out[4]: Elemwise{mul,no_inplace}.0

In [5]: theano.tensor.dmatrix() * numpy.ones(4)
Out[5]: Elemwise{mul,no_inplace}.0


The dmatrix() function returns an instance of the TensorVariable class
defined in this file:
http://trac-hg.assembla.com/theano/browser/theano/tensor/basic.py#L901

I think the only thing we added  for numpy was __array_priority__ =
1000, which has already been suggested here.  I'm confused by why this
thread goes on.


Hmm,

That does seem to work. I wonder if it is intended or just fortuitous, the documentation says:

The __array_priority__ attribute

__array_priority__
This attribute allows simple but flexible determination of which sub- type should be considered “primary” when an operation involving two or more sub-types arises. In operations where different sub-types are being used, the sub-type with the largest __array_priority__ attribute will determine the sub-type of the output(s). If two sub- types have the same __array_priority__ then the sub-type of the first argument determines
the output. The default __array_priority__ attribute returns a value of 0.0 for the base ndarray type and 1.0 for a sub-type. This attribute can also be defined by objects that are not sub-types of the ndarray and can be used to determine which __array_wrap__ method should be called for the return output.

Which doesn't seem directly applicable. Perhaps the documentation is wrong, the last sentence is a bit confusing.

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 18 Dec 01:54 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



On Thu, Dec 17, 2009 at 4:35 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:


On Thu, Dec 17, 2009 at 2:27 PM, James Bergstra <james.bergstra <at> gmail.com> wrote:
I develop another symbolic-over-numpy package called theano, and
somehow we avoid this problem.

In [1]: import theano

In [2]: import numpy

In [3]: numpy.ones(4) * theano.tensor.dmatrix()
Out[3]: Elemwise{mul,no_inplace}.0

In [4]: theano.tensor.dmatrix() * theano.tensor.dmatrix()
Out[4]: Elemwise{mul,no_inplace}.0

In [5]: theano.tensor.dmatrix() * numpy.ones(4)
Out[5]: Elemwise{mul,no_inplace}.0


The dmatrix() function returns an instance of the TensorVariable class
defined in this file:
http://trac-hg.assembla.com/theano/browser/theano/tensor/basic.py#L901

I think the only thing we added  for numpy was __array_priority__ =
1000, which has already been suggested here.  I'm confused by why this
thread goes on.


Hmm,

That does seem to work. I wonder if it is intended or just fortuitous, the documentation says:

The __array_priority__ attribute

__array_priority__
This attribute allows simple but flexible determination of which sub- type should be considered “primary” when an operation involving two or more sub-types arises. In operations where different sub-types are being used, the sub-type with the largest __array_priority__ attribute will determine the sub-type of the output(s). If two sub- types have the same __array_priority__ then the sub-type of the first argument determines
the output. The default __array_priority__ attribute returns a value of 0.0 for the base ndarray type and 1.0 for a sub-type. This attribute can also be defined by objects that are not sub-types of the ndarray and can be used to determine which __array_wrap__ method should be called for the return output.

Which doesn't seem directly applicable. Perhaps the documentation is wrong, the last sentence is a bit confusing.


OK, looks intended:

    /*
     * FAIL with NotImplemented if the other object has
     * the __r<op>__ method and has __array_priority__ as
     * an attribute (signalling it can handle ndarray's)
     * and is not already an ndarray or a subtype of the same type.
     */

This is in ufunc_object.c. However, it doesn't works for general ufuncs, i.e.,  np.multiply(a,b) isn't the same as "a * b"

Chuck
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Charles R Harris | 18 Dec 17:00 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc



On Thu, Dec 17, 2009 at 5:54 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:


On Thu, Dec 17, 2009 at 4:35 PM, Charles R Harris <charlesr.harris <at> gmail.com> wrote:


On Thu, Dec 17, 2009 at 2:27 PM, James Bergstra <james.bergstra <at> gmail.com> wrote:
I develop another symbolic-over-numpy package called theano, and
somehow we avoid this problem.

In [1]: import theano

In [2]: import numpy

In [3]: numpy.ones(4) * theano.tensor.dmatrix()
Out[3]: Elemwise{mul,no_inplace}.0

In [4]: theano.tensor.dmatrix() * theano.tensor.dmatrix()
Out[4]: Elemwise{mul,no_inplace}.0

In [5]: theano.tensor.dmatrix() * numpy.ones(4)
Out[5]: Elemwise{mul,no_inplace}.0


The dmatrix() function returns an instance of the TensorVariable class
defined in this file:
http://trac-hg.assembla.com/theano/browser/theano/tensor/basic.py#L901

I think the only thing we added  for numpy was __array_priority__ =
1000, which has already been suggested here.  I'm confused by why this
thread goes on.


Hmm,

That does seem to work. I wonder if it is intended or just fortuitous, the documentation says:

The __array_priority__ attribute

__array_priority__
This attribute allows simple but flexible determination of which sub- type should be considered “primary” when an operation involving two or more sub-types arises. In operations where different sub-types are being used, the sub-type with the largest __array_priority__ attribute will determine the sub-type of the output(s). If two sub- types have the same __array_priority__ then the sub-type of the first argument determines
the output. The default __array_priority__ attribute returns a value of 0.0 for the base ndarray type and 1.0 for a sub-type. This attribute can also be defined by objects that are not sub-types of the ndarray and can be used to determine which __array_wrap__ method should be called for the return output.

Which doesn't seem directly applicable. Perhaps the documentation is wrong, the last sentence is a bit confusing.


OK, looks intended:

    /*
     * FAIL with NotImplemented if the other object has
     * the __r<op>__ method and has __array_priority__ as
     * an attribute (signalling it can handle ndarray's)
     * and is not already an ndarray or a subtype of the same type.
     */

This is in ufunc_object.c. However, it doesn't works for general ufuncs, i.e.,  np.multiply(a,b) isn't the same as "a * b"


What makes this confusing is that it is in the wrong place. This sort of behaviour should be enforced in multiarray, not in the ufunc object.

Chuck

_______________________________________________
SciPy-Dev mailing list
SciPy-Dev <at> scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
Alan G Isaac | 17 Dec 15:21 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On 12/17/2009 4:45 AM, Dag Sverre Seljebotn wrote:
> NumPy is special because rmul never gets called with NumPy arrays, which
> can be quite inconvenient at times.

Is this not the real problem?
After all, np.multiply could be used when the need arises,
removing any need to override the standard Python precedence
for *.

I know this view will be unpopular, and someone will surely
argue that changing it would make the interaction between matrices
and arrays too obscure. But in the long run, it may prove worth
asking whether this convenience feature is actually inconvenient.

Alan Isaac
Nathan Bell | 13 Dec 23:19 2009
Picon

Re: Suppressing of numpy __mul__, __div__ etc

On Sat, Dec 12, 2009 at 4:16 PM, Robert Kern <robert.kern <at> gmail.com> wrote:
>
> I think what he is asking for is an empty mixin class which other
> folks could subclass to mark their classes. It would say "Hey,
> ndarray! Let my __mul__, __rmul__, etc., take priority over yours,
> regardless of which of us comes first in the expression." Otherwise,
> ndarray will gladly consume pretty much any object on the other side
> of the operator because it will treat it as an object scalar.
>
> We could also define a standard attribute that could mark such classes
> instead of requiring a mixin subclass.
>

We could use this functionality in scipy.sparse too.  In particular,
it would be nice if asarray(some_sparse_matrix) just worked so we
could toss (presumably small) sparse matrices into functions expecting
ndarrays.

Like Dmitrey, we need to invoke sparse.__rmul__(dense) when
encountering dense * sparse.

--

-- 
Nathan Bell wnbell <at> gmail.com
http://www.wnbell.com/

Gmane