headers
cool-RR | 29 May 2010 18:40
Gravatar

Re: Pickling unbound methods on Python 3

On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray-2P5OLIWfD11Wk0Htik3J/w@public.gmane.org> wrote:
On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
> One person told me that given an unbound method in Python 3.x, it's
> *impossible* to tell to which class it belongs. Is it true?

I believe that that is true.  In Python3 there is no such thing as an
unbound method as a distinct object type.  There are functions, and
there are bound methods.  See the second sentence in this section:

http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods

I see. Where would be a good place to discuss this decision? I would want 3.2 to allow pickling of unbound methods.

 
You'll probably have to explain more about the problem you are
trying to solve in order for us to help you find a solution.

--
R. David Murray                                      www.bitdance.com

I found a hacky workaround: When I have a method `my_method` that I want to pickle, I write `my_method = my_class.my_method` in the method's module.

So no need to burden this list with the specifics of my problem.

Now I only worry about the future: I want 3.2 to enable pickling of unbound methods so I could get rid of this workaround in the future.

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
M.-A. Lemburg | 29 May 2010 18:56
Favicon

Re: Pickling unbound methods on Python 3

cool-RR wrote:
> On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray@...>wrote:
> 
>> On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
>>> One person told me that given an unbound method in Python 3.x, it's
>>> *impossible* to tell to which class it belongs. Is it true?
>>
>> I believe that that is true.  In Python3 there is no such thing as an
>> unbound method as a distinct object type.  There are functions, and
>> there are bound methods.  See the second sentence in this section:
>>
>>
>> http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods
> 
> 
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.

Unbound methods don't exist in Python3. You only have functions and
(bound) methods.

You can see that if you try to call an unbound method with a
non-instance first arg:

>>> class X:
...  def test(self): return 42
...
>>> X.test(X())
42
>>> X.test(3)
42

Doing the same in Python2 gives an error:

>>> X.test(X())
42
>>> X.test(3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unbound method test() must be called with X instance as first argument (got int instance
instead)

--

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 29 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK                50 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
Permalink | Reply |
headers
cool-RR | 29 May 2010 19:59
Gravatar

Re: Pickling unbound methods on Python 3

On Sat, May 29, 2010 at 6:56 PM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+AlfA@public.gmane.org> wrote:
cool-RR wrote:
> On Sat, May 29, 2010 at 5:46 PM, R. David Murray <rdmurray-2P5OLIWfD11Wk0Htik3J/w@public.gmane.org>wrote:
>
>> On Fri, 28 May 2010 21:55:28 +0200, cool-RR wrote:
>>> One person told me that given an unbound method in Python 3.x, it's
>>> *impossible* to tell to which class it belongs. Is it true?
>>
>> I believe that that is true.  In Python3 there is no such thing as an
>> unbound method as a distinct object type.  There are functions, and
>> there are bound methods.  See the second sentence in this section:
>>
>>
>> http://docs.python.org/release/3.0.1/whatsnew/3.0.html#operators-and-special-methods
>
>
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.
 
Unbound methods don't exist in Python3. You only have functions and
(bound) methods.

I know that unbound methods are of the function type in Python 3. I'm calling them unbound methods because they are methods of classes which are not bound to an instance. I'm aware they have no special stance, but I still refer to them as unbound methods.

You can see that if you try to call an unbound method with a
non-instance first arg:

>>> class X:
...  def test(self): return 42
...
>>> X.test(X())
42
>>> X.test(3)
42

Doing the same in Python2 gives an error:

>>> X.test(X())
42
>>> X.test(3)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: unbound method test() must be called with X instance as first argument (got int instance
instead)

Thanks for the interesting example, Marc-Andre. 

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 29 May 2010 20:43
Picon
Gravatar

Re: Pickling unbound methods on Python 3

> I know that unbound methods are of the function type in Python 3. I'm
> calling them unbound methods because they are methods of classes which
> are not bound to an instance. I'm aware they have no special stance, but
> I still refer to them as unbound methods.

Neglecting reality doesn't help your cause, though. You'll need to 
understand *why* pickling fails, and then see whether there might be
a solution.

A work-around is to put all methods you want to pickle into module-scope
of your module

class Foo:
   def bar(self):
     pass

bar = Foo.bar

Now you can pickle Foo.bar.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 29 May 2010 22:52
Gravatar

Re: Pickling unbound methods on Python 3

Hello Martin, thanks for joining the discussion.

On Sat, May 29, 2010 at 8:43 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
I know that unbound methods are of the function type in Python 3. I'm
calling them unbound methods because they are methods of classes which
are not bound to an instance. I'm aware they have no special stance, but
I still refer to them as unbound methods.

Neglecting reality doesn't help your cause, though. You'll need to understand *why* pickling fails, and then see whether there might be
a solution.

I think I understand why pickling fails. An unbound method is not differentiated in any way from a function. The `save_global` function tries to look for it in the main module namespace, instead of in the class.

 
A work-around is to put all methods you want to pickle into module-scope
of your module

class Foo:
 def bar(self):
   pass

bar = Foo.bar

Now you can pickle Foo.bar.

Yes, I've already done this, but I'd prefer a less hackish solution. 

 
Regards,
Martin

I can think of a few solutions:

1. Make `save_global` look in the classes inside the module as well.

2. Allow me to specify a reducer for FunctionType. This is currently not working, see here: http://stackoverflow.com/questions/2932742/python-using-copyreg-to-define-reducers-for-types-that-already-have-reducers


Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 30 May 2010 09:19
Picon
Gravatar

Re: Pickling unbound methods on Python 3

> I can think of a few solutions:
>
> 1. Make `save_global` look in the classes inside the module as well.

How exactly would you do this? I don't think this is implementable,
in a reasonable way.

> 2. Allow me to specify a reducer for FunctionType.

How would the reducer work?

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 30 May 2010 12:00
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
I can think of a few solutions:

1. Make `save_global` look in the classes inside the module as well.

How exactly would you do this? I don't think this is implementable,
in a reasonable way.

I don't understand what the problem is with just looking in the classes defined in the module:

"""
def find_containing_object_of_function(function):
    function_name = function.__name__
    module = sys.modules[function.__module__]
    objects_to_check = [module]
    while objects_to_check:
        object_to_check = objects_to_check.pop()
        if getattr(object_to_check, function_name, None) is function:
            return object_to_check
        try:
            sub_objects = vars(object_to_check)
        except TypeError: # The object doesn't have a vars/__dict__
            continue
        for sub_object in sub_objects.values():
            if isinstance(sub_object, type):
                objects_to_check.append(sub_object)
        
    raise LookupError('''Couldn't find the %s function for pickling/deepcopying \
it. I tried looking around in the %s module, but it either wasn't there or \
was in a non-obvious place.''' % (function, module))
    
    
def reduce_function(function):
    containing_object = find_containing_object_of_function(function)
    return (getattr, (containing_object, function.__name__))
"""

This is not bulletproof, of course. But it'll work for most cases.


2. Allow me to specify a reducer for FunctionType.

How would the reducer work?


As above.

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Lennart Regebro | 30 May 2010 12:50
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr@...> wrote:
> I don't understand what the problem is with just looking in the classes
> defined in the module:

How do you handle name conflicts?

--

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
Permalink | Reply |
headers
cool-RR | 30 May 2010 12:52
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-4Sw9Ecu7DfZBDgjK7y7TUQ@public.gmane.org> wrote:
> I don't understand what the problem is with just looking in the classes
> defined in the module:

How do you handle name conflicts?

Can you specify which kind of naming conflict you mean?


Ram.


_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Lennart Regebro | 30 May 2010 12:58
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 12:52, cool-RR <cool-rr@...> wrote:
> On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro@...> wrote:
>>
>> On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr@...> wrote:
>> > I don't understand what the problem is with just looking in the classes
>> > defined in the module:
>>
>> How do you handle name conflicts?
>
> Can you specify which kind of naming conflict you mean?

Two methods with the same name but in different classes would get a
name class. They would be unpickled as the same method, leading to a
lot of really hairy debugging. Especially if they take the same
parameters, which seems likely in your case. You would simply unpickle
and call the wrong method, with no errors, just unexpected behavior.

Or in other words: Your solution is way more hackish and brittle than
the one Martin suggested, which is simple, neat and Pythonic.

--

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
Permalink | Reply |
headers
cool-RR | 30 May 2010 13:02
Gravatar
Subject: Re: Pickling unbound methods on Python 3

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 12:58 PM, Lennart Regebro <regebro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
On Sun, May 30, 2010 at 12:52, cool-RR <cool-rr-4Sw9Ecu7DfZBDgjK7y7TUQ@public.gmane.org> wrote:
> On Sun, May 30, 2010 at 12:50 PM, Lennart Regebro <regebro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>
>> On Sun, May 30, 2010 at 12:00, cool-RR <cool-rr-4Sw9Ecu7DfZBDgjK7y7TUQ@public.gmane.org> wrote:
>> > I don't understand what the problem is with just looking in the classes
>> > defined in the module:
>>
>> How do you handle name conflicts?
>
> Can you specify which kind of naming conflict you mean?

Two methods with the same name but in different classes would get a
name class. They would be unpickled as the same method, leading to a
lot of really hairy debugging. Especially if they take the same
parameters, which seems likely in your case. You would simply unpickle
and call the wrong method, with no errors, just unexpected behavior.

I don't understand how that would happen. The code that I gave confirms that the function found inside the class `is` the function that is being pickled, so if it finds a different function with the same name, it will ignore it. Or maybe I'm misunderstanding something?

 
Or in other words: Your solution is way more hackish and brittle than
the one Martin suggested, which is simple, neat and Pythonic.

I disagree, Lennart.


Ram. 
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Lennart Regebro | 30 May 2010 13:33
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr@...> wrote:
> I don't understand how that would happen. The code that I gave confirms that
> the function found inside the class `is` the function that is being pickled,
> so if it finds a different function with the same name, it will ignore it.

How can it do that? How does it know that it should ignore it?

But you are right, Martins solution wasn't as neat as I thought first,
you basically need to do it the other way around, ie define the
methods as functions, and then set them on the classes. Probably
wrapper methods is neater and less hacky.

--

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
Permalink | Reply |
headers
cool-RR | 30 May 2010 13:35
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 1:33 PM, Lennart Regebro <regebro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr-4Sw9Ecu7DfZBDgjK7y7TUQ@public.gmane.org> wrote:
> I don't understand how that would happen. The code that I gave confirms that
> the function found inside the class `is` the function that is being pickled,
> so if it finds a different function with the same name, it will ignore it.

How can it do that? How does it know that it should ignore it?

I'm not completely sure we're talking about the same thing here. Did you read the code and try to follow the problematic case?

Ram. 
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Lennart Regebro | 30 May 2010 13:39
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 13:35, cool-RR <cool-rr@...> wrote:
> On Sun, May 30, 2010 at 1:33 PM, Lennart Regebro <regebro@...> wrote:
>>
>> On Sun, May 30, 2010 at 13:02, cool-RR <cool-rr@...> wrote:
>> > I don't understand how that would happen. The code that I gave confirms
>> > that
>> > the function found inside the class `is` the function that is being
>> > pickled,
>> > so if it finds a different function with the same name, it will ignore
>> > it.
>>
>> How can it do that? How does it know that it should ignore it?
>
> I'm not completely sure we're talking about the same thing here. Did you
> read the code and try to follow the problematic case?

You are right, I misread.
Permalink | Reply |
headers
Martin v. Löwis | 30 May 2010 23:34
Picon
Gravatar

Re: Pickling unbound methods on Python 3

Am 30.05.2010 12:00, schrieb cool-RR:
> On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <martin <at> v.loewis.de
> <mailto:martin <at> v.loewis.de>> wrote:
>
>         I can think of a few solutions:
>
>         1. Make `save_global` look in the classes inside the module as well.
>
>
>     How exactly would you do this? I don't think this is implementable,
>     in a reasonable way.
>
>
> I don't understand what the problem is with just looking in the classes
> defined in the module:

I wouldn't call this a reasonable implementation. It is fairly expensive.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 30 May 2010 23:43
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 11:34 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
Am 30.05.2010 12:00, schrieb cool-RR:
On Sun, May 30, 2010 at 9:19 AM, "Martin v. Löwis" <martin <at> v.loewis.de
<mailto:martin <at> v.loewis.de>> wrote:

       I can think of a few solutions:

       1. Make `save_global` look in the classes inside the module as well.


   How exactly would you do this? I don't think this is implementable,
   in a reasonable way.


I don't understand what the problem is with just looking in the classes
defined in the module:

I wouldn't call this a reasonable implementation. It is fairly expensive.

Regards,
Martin

What if we'll try the expensive stuff after the old way (shallow module search) fails and is about to raise an exception?

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 30 May 2010 23:48
Picon
Gravatar

Re: Pickling unbound methods on Python 3

> What if we'll try the expensive stuff after the old way (shallow module
> search) fails and is about to raise an exception?

Feel free to submit a patch. However, if this needs to be fixed at all 
(and I'm not convinced it does), then I'd rather see a proper fix, i.e. 
one where you find the class from the function object directly, or which
has an even more general way of identifying global objects.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 31 May 2010 00:57
Gravatar

Re: Pickling unbound methods on Python 3

On Sun, May 30, 2010 at 11:48 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
What if we'll try the expensive stuff after the old way (shallow module
search) fails and is about to raise an exception?

Feel free to submit a patch. However, if this needs to be fixed at all (and I'm not convinced it does), then I'd rather see a proper fix, i.e. one where you find the class from the function object directly, or which
has an even more general way of identifying global objects.

Regards,
Martin

I agree, and that was what I wanted from the beginning. But several people have told me that it's *impossible* to find the class that an unbound method was defined on. So I came up with this workaround instead.

I'll consider making a patch. Thanks for your help Martin.

By the way, when this discussion continues / when I make the patch, should I post on python-dev instead of here?

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
M.-A. Lemburg | 31 May 2010 09:31
Favicon

Re: Pickling unbound methods on Python 3

cool-RR wrote:
> On Sun, May 30, 2010 at 11:48 PM, "Martin v. Löwis" <martin <at> v.loewis.de>wrote:
> 
>> What if we'll try the expensive stuff after the old way (shallow module
>>> search) fails and is about to raise an exception?
>>>
>>
>> Feel free to submit a patch. However, if this needs to be fixed at all (and
>> I'm not convinced it does), then I'd rather see a proper fix, i.e. one where
>> you find the class from the function object directly, or which
>> has an even more general way of identifying global objects.
>>
>> Regards,
>> Martin
>>
> 
> I agree, and that was what I wanted from the beginning. But several people
> have told me that it's *impossible* to find the class that an unbound method
> was defined on. So I came up with this workaround instead.
> 
> I'll consider making a patch. Thanks for your help Martin.
> 
> By the way, when this discussion continues / when I make the patch, should I
> post on python-dev instead of here?

python-dev would be more appropriate.

I suppose the easiest way to add the information about the defining
class is to have the class constructor add a function attribute
to the (unbound) method functions. This should be the name of
the defining class to avoid circular references.

--

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 31 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK                48 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
Permalink | Reply |
headers
cool-RR | 31 May 2010 12:08
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal <at> egenix.com> wrote:
I suppose the easiest way to add the information about the defining
class is to have the class constructor add a function attribute
to the (unbound) method functions. This should be the name of
the defining class to avoid circular references.

--
Marc-Andre Lemburg

As a workaround that'll work, yeah. But we were discussing whether we can make a real solution. (Unless you were proposing that Python itself will do this and not just my program; In that case I'm totally with you, but it seems that the Python community isn't.) 

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
M.-A. Lemburg | 31 May 2010 12:18
Favicon

Re: Pickling unbound methods on Python 3

cool-RR wrote:
> On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal@...> wrote:
>>
>> I suppose the easiest way to add the information about the defining
>> class is to have the class constructor add a function attribute
>> to the (unbound) method functions. This should be the name of
>> the defining class to avoid circular references.
>>
>> --
>> Marc-Andre Lemburg
>>
> 
> As a workaround that'll work, yeah. But we were discussing whether we can
> make a real solution. (Unless you were proposing that Python itself will do
> this and not just my program; In that case I'm totally with you, but it
> seems that the Python community isn't.)

I was thinking of having this in Python itself. The compiler has
all the information available, so why not add some of this useful
meta-information to the created objects ?!

This is not the same as creating a new bound method type - we'd
only add some extra information for function objects that happen
to be defined as methods.

--

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 31 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK                48 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
Permalink | Reply |
headers
cool-RR | 31 May 2010 12:20
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 12:18 PM, M.-A. Lemburg <mal <at> egenix.com> wrote:
cool-RR wrote:
> On Mon, May 31, 2010 at 9:31 AM, M.-A. Lemburg <mal-SVD0I98eSHvQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> I suppose the easiest way to add the information about the defining
>> class is to have the class constructor add a function attribute
>> to the (unbound) method functions. This should be the name of
>> the defining class to avoid circular references.
>>
>> --
>> Marc-Andre Lemburg
>>
>
> As a workaround that'll work, yeah. But we were discussing whether we can
> make a real solution. (Unless you were proposing that Python itself will do
> this and not just my program; In that case I'm totally with you, but it
> seems that the Python community isn't.)

I was thinking of having this in Python itself. The compiler has
all the information available, so why not add some of this useful
meta-information to the created objects ?!

This is not the same as creating a new bound method type - we'd
only add some extra information for function objects that happen
to be defined as methods.

--
Marc-Andre Lemburg

I totally agree with you. If you start talking on this in python-dev, `cc` me.

Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 31 May 2010 22:07
Picon
Gravatar

Re: Pickling unbound methods on Python 3

>     I suppose the easiest way to add the information about the defining
>     class is to have the class constructor add a function attribute
>     to the (unbound) method functions. This should be the name of
>     the defining class to avoid circular references.
>
> As a workaround that'll work, yeah. But we were discussing whether we
> can make a real solution. (Unless you were proposing that Python itself
> will do this and not just my program; In that case I'm totally with you,
> but it seems that the Python community isn't.)

What makes you think so? I (personally) have objected to your hackish 
way of scanning the all classes in the module. Adding a function 
attribute is fine with me.

I only wonder if it would be useful to generalize this beyond functions.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 31 May 2010 22:12
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 10:07 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
   I suppose the easiest way to add the information about the defining
   class is to have the class constructor add a function attribute
   to the (unbound) method functions. This should be the name of
   the defining class to avoid circular references.

As a workaround that'll work, yeah. But we were discussing whether we
can make a real solution. (Unless you were proposing that Python itself
will do this and not just my program; In that case I'm totally with you,
but it seems that the Python community isn't.)

What makes you think so? I (personally) have objected to your hackish way of scanning the all classes in the module. Adding a function attribute is fine with me.

Great! I prefer adding an attribute to the unbound methods as well. Would you like to raise this on python-dev or would you like me to? (I haven't been active there and I don't know which issues are in consensus and which aren't.) In any case `cc` me if you talk about it.
 
I only wonder if it would be useful to generalize this beyond functions.

Sounds good.

 
Ram.

_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 31 May 2010 22:19
Picon
Gravatar

Re: Pickling unbound methods on Python 3

> Great! I prefer adding an attribute to the unbound methods as well.
> Would you like to raise this on python-dev or would you like me to?

I'm very skeptical that "raising" it will have any effect. Submit a 
patch instead.

> (I haven't been active there and I don't know which issues are
> in consensus and which aren't.)

Consensus is irrelevant in absence of a patch.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 31 May 2010 22:25
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
Great! I prefer adding an attribute to the unbound methods as well.
Would you like to raise this on python-dev or would you like me to?

I'm very skeptical that "raising" it will have any effect. Submit a patch instead.

I don't understand Martin. Unbound methods had an `.im_class` attribute in Python 2.x, which is what we're talking about, and for Python 3.x that attribute was purposefully removed. I am assuming it was a deliberate decision by the python-dev community and that they had good reasons for it. Am I not supposed to ask them about this before I put it back?


Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Brett Cannon | 31 May 2010 23:10
Favicon
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 13:25, cool-RR <cool-rr <at> cool-rr.com> wrote:
> On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <martin <at> v.loewis.de>
> wrote:
>>>
>>> Great! I prefer adding an attribute to the unbound methods as well.
>>> Would you like to raise this on python-dev or would you like me to?
>>
>> I'm very skeptical that "raising" it will have any effect. Submit a patch
>> instead.
>
> I don't understand Martin. Unbound methods had an `.im_class` attribute in
> Python 2.x, which is what we're talking about, and for Python 3.x that
> attribute was purposefully removed. I am assuming it was a deliberate
> decision by the python-dev community and that they had good reasons for it.
> Am I not supposed to ask them about this before I put it back?

What Martin is saying is that you will need a solid proposal that
includes a patch in order to convince python-dev to change something.
Making changes to the function type are not taken lightly and a clear
definition of what you want to happen needs to exist, and code is the
clearest definition you can have. Then it can be discussed on
python-dev to try to convince us to accept the patch.
_______________________________________________
Python-porting mailing list
Python-porting <at> python.org
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Martin v. Löwis | 31 May 2010 23:26
Picon
Gravatar

Re: Pickling unbound methods on Python 3

Am 31.05.2010 22:25, schrieb cool-RR:
> On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <martin <at> v.loewis.de
> <mailto:martin <at> v.loewis.de>> wrote:
>
>         Great! I prefer adding an attribute to the unbound methods as well.
>         Would you like to raise this on python-dev or would you like me to?
>
>
>     I'm very skeptical that "raising" it will have any effect. Submit a
>     patch instead.
>
>
> I don't understand Martin. Unbound methods had an `.im_class` attribute
> in Python 2.x, which is what we're talking about, and for Python 3.x
> that attribute was purposefully removed. I am assuming it was a
> deliberate decision by the python-dev community and that they had good
> reasons for it. Am I not supposed to ask them about this before I put it
> back?

Sure, you can ask. However, instead of asking, please study the code: 
the subversion log is available to the general public. Find out (for 
yourself) what specific revision made the change, and you can save other 
contributors the time of doing the research for you.

You are mistaken that the the attribute was removed. It was not removed. 
Instead, the object returned from the attribute lookup was changed, and 
the "new" type just happened to have no im_class attribute.
It didn't have that attribute in 2.x, either.

Regards,
Martin
Permalink | Reply |
headers
cool-RR | 1 Jun 2010 13:13
Gravatar

Re: Pickling unbound methods on Python 3

On Mon, May 31, 2010 at 11:26 PM, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
Am 31.05.2010 22:25, schrieb cool-RR:
On Mon, May 31, 2010 at 10:19 PM, "Martin v. Löwis" <martin <at> v.loewis.de
<mailto:martin <at> v.loewis.de>> wrote:

       Great! I prefer adding an attribute to the unbound methods as well.
       Would you like to raise this on python-dev or would you like me to?


   I'm very skeptical that "raising" it will have any effect. Submit a
   patch instead.


I don't understand Martin. Unbound methods had an `.im_class` attribute
in Python 2.x, which is what we're talking about, and for Python 3.x
that attribute was purposefully removed. I am assuming it was a
deliberate decision by the python-dev community and that they had good
reasons for it. Am I not supposed to ask them about this before I put it
back?

Sure, you can ask. However, instead of asking, please study the code: the subversion log is available to the general public. Find out (for yourself) what specific revision made the change, and you can save other contributors the time of doing the research for you.

You are mistaken that the the attribute was removed. It was not removed. Instead, the object returned from the attribute lookup was changed, and the "new" type just happened to have no im_class attribute.
It didn't have that attribute in 2.x, either.

Regards,
Martin

Thanks for the clarifications, Martin and Brett.

At this time I'm both under-qualified and don't have time to work on a patch. (I'm under-qualified because I don't code C and I'm not familiar with Python's innards.)

Thanks for your help.


Ram.
_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |
headers
Jesus Cea | 2 Jun 2010 14:39
Picon
Favicon

Re: Pickling unbound methods on Python 3


On 01/06/10 13:13, cool-RR wrote:
> Thanks for the clarifications, Martin and Brett.
> 
> At this time I'm both under-qualified and don't have time to work on a
> patch. (I'm under-qualified because I don't code C and I'm not familiar
> with Python's innards.)

Although having a patch is important, I think that first we need a
discussion about this subject in python-dev. Then opening an issue in
the bugtracker. The patch doesn't need to be written by you, and
python-dev must decide first it is a good idea (for somebody other to
develop! :) to avoid wasting the developer time.

--

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@... - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea@...         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
Permalink | Reply |
headers
Jesus Cea | 31 May 2010 23:23
Picon
Favicon

Re: Pickling unbound methods on Python 3


On 31/05/10 22:07, "Martin v. Löwis" wrote:
> What makes you think so? I (personally) have objected to your
> hackish way of scanning the all classes in the module. Adding a
> function attribute is fine with me.
> 
> I only wonder if it would be useful to generalize this beyond
> functions.

Could be done thru a metaclass, trivially.

Adding an attribute automatically in python 3.2/3.3 would be some I
would support. It really needs a brainstorm in python-dev, though.

--

-- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea@... - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea@...         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
Permalink | Reply |
headers
Lennart Regebro | 30 May 2010 00:13
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sat, May 29, 2010 at 20:43, "Martin v. Löwis" <martin <at> v.loewis.de> wrote:
> bar = Foo.bar
>
> Now you can pickle Foo.bar.

Ah yes, of course. So simple and neat.

--

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
Permalink | Reply |
headers
Lennart Regebro | 29 May 2010 19:41
Picon
Gravatar

Re: Pickling unbound methods on Python 3

On Sat, May 29, 2010 at 18:40, cool-RR <cool-rr@...> wrote:
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.

That would be python-dev@...

However, it has already been discussed:

  http://mail.python.org/pipermail/python-dev/2005-January/050625.html
  http://mail.python.org/pipermail/python-dev/2007-November/075279.html

It turns out, even the pickling argument was discussed.

  http://mail.python.org/pipermail/python-dev/2005-January/051143.html

Essentially, the argument is that it's easier to use a function
instead of writing a pickler for a method in the first case, so
support for pickling unbound methods isn't a high priority as there
are easier and less magic ways to solve the use case.

I know that's not what you want to hear, but that's likely to be the
answer you'll get.

--

-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64
Permalink | Reply |
headers
cool-RR | 29 May 2010 20:03
Gravatar

Re: Pickling unbound methods on Python 3

On Sat, May 29, 2010 at 7:41 PM, Lennart Regebro <regebro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
On Sat, May 29, 2010 at 18:40, cool-RR <cool-rr-4Sw9Ecu7DfZBDgjK7y7TUQ@public.gmane.org> wrote:
> I see. Where would be a good place to discuss this decision? I would want
> 3.2 to allow pickling of unbound methods.

That would be python-dev <at> python.org

However, it has already been discussed:

 http://mail.python.org/pipermail/python-dev/2005-January/050625.html
 http://mail.python.org/pipermail/python-dev/2007-November/075279.html

It turns out, even the pickling argument was discussed.

 http://mail.python.org/pipermail/python-dev/2005-January/051143.html

Essentially, the argument is that it's easier to use a function
instead of writing a pickler for a method in the first case, so
support for pickling unbound methods isn't a high priority as there
are easier and less magic ways to solve the use case.


I know that's not what you want to hear, but that's likely to be the
answer you'll get.

--
Lennart Regebro

Thanks for the info Lennart.

Ram. 

_______________________________________________
Python-porting mailing list
Python-porting@...
http://mail.python.org/mailman/listinfo/python-porting
Permalink | Reply |

Gmane