John Reid | 18 Feb 12:47 2013
Picon

Differences creating tuples and collections.namedtuples

Hi,

I was hoping namedtuples could be used as replacements for tuples in all instances. There seem to be some
differences between how tuples and namedtuples are created. For example with a tuple I can do:

a=tuple([1,2,3])

with namedtuples I get a TypeError:

from collections import namedtuple
B=namedtuple('B', 'x y z')
b=B([1,2,3])

TypeError: __new__() takes exactly 4 arguments (2 given)
> <ipython-input-23-d1da2ef851fb>(3)<module>()
      1 from collections import namedtuple
      2 B=namedtuple('B', 'x y z')
----> 3 b=B([1,2,3])

I'm seeing this problem because of the following code in IPython:

def canSequence(obj):
    if isinstance(obj, (list, tuple)):
        t = type(obj)
        return t([can(i) for i in obj])
    else:
        return obj

where obj is a namedtuple and t([can(i) for i in obj]) fails with the TypeError. See
http://article.gmane.org/gmane.comp.python.ipython.user/10270 for more info.
(Continue reading)

Oscar Benjamin | 18 Feb 13:03 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
> Hi,
>
> I was hoping namedtuples could be used as replacements for tuples in all instances.

namedtuples are not really intended to serves as tuples anywhere. They
are intended to provide lightweight, immutable, hashable objects with
*named* (rather than numbered) values.

> There seem to be some differences between how tuples and namedtuples are created. For example with a tuple
I can do:
>
> a=tuple([1,2,3])
>
> with namedtuples I get a TypeError:
>
> from collections import namedtuple
> B=namedtuple('B', 'x y z')
> b=B([1,2,3])

For a namedtuple you need to unpack the arguments

b = B(*[1, 2, 3])

or

b = B(1, 2, 3)

>
> TypeError: __new__() takes exactly 4 arguments (2 given)
(Continue reading)

Oscar Benjamin | 18 Feb 13:05 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18 February 2013 12:03, Oscar Benjamin <oscar.j.benjamin <at> gmail.com> wrote:
> On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
>> I'm seeing this problem because of the following code in IPython:
>>
>> def canSequence(obj):
>>     if isinstance(obj, (list, tuple)):
>>         t = type(obj)
>>         return t([can(i) for i in obj])
>>     else:
>>         return obj
>
> What is the point of the code above? If obj is a list or a tuple you
> create a new list or tuple with the same data and then return it
> otherwise you just return the object. What about:

Sorry, I didn't read this properly. I see that you want apply can() to
all the elements. What is the reason for wanting to preserve the type
of the sequence?

Oscar
John Reid | 18 Feb 14:49 2013
Picon

Re: Differences creating tuples and collections.namedtuples


On 18/02/13 12:05, Oscar Benjamin wrote:
> On 18 February 2013 12:03, Oscar Benjamin <oscar.j.benjamin <at> gmail.com> wrote:
>> On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
>>> I'm seeing this problem because of the following code in IPython:
>>>
>>> def canSequence(obj):
>>>     if isinstance(obj, (list, tuple)):
>>>         t = type(obj)
>>>         return t([can(i) for i in obj])
>>>     else:
>>>         return obj
>> What is the point of the code above? If obj is a list or a tuple you
>> create a new list or tuple with the same data and then return it
>> otherwise you just return the object. What about:
> Sorry, I didn't read this properly. I see that you want apply can() to
> all the elements. What is the reason for wanting to preserve the type
> of the sequence?
>
>
Well like I said it is not me that wants to do this. It is part of the
code in IPython for sending messages between clients and engines.

John Reid | 18 Feb 14:51 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18/02/13 12:03, Oscar Benjamin wrote:
> On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
>> Hi,
>>
>> I was hoping namedtuples could be used as replacements for tuples in all instances.
> namedtuples are not really intended to serves as tuples anywhere. They
> are intended to provide lightweight, immutable, hashable objects with
> *named* (rather than numbered) values.
If they are not supposed to be tuples then calling them namedtuples and
inheriting from tuple seems a little odd.

Oscar Benjamin | 18 Feb 15:12 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18 February 2013 13:51, John Reid <johnbaronreid <at> gmail.com> wrote:
> On 18/02/13 12:03, Oscar Benjamin wrote:
>> On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
>>> Hi,
>>>
>>> I was hoping namedtuples could be used as replacements for tuples in all instances.
>> namedtuples are not really intended to serves as tuples anywhere. They
>> are intended to provide lightweight, immutable, hashable objects with
>> *named* (rather than numbered) values.
>
> If they are not supposed to be tuples then calling them namedtuples and
> inheriting from tuple seems a little odd.

You can use namedtuple instances in places that expect tuples.
Inheriting from tuples enables them to be all the things I said:
lightweight, immutable and hashable. The type object itself has a
different interface for the constructor, though.

Oscar
John Reid | 18 Feb 15:18 2013
Picon

Re: Differences creating tuples and collections.namedtuples


On 18/02/13 14:12, Oscar Benjamin wrote:
> On 18 February 2013 13:51, John Reid <johnbaronreid <at> gmail.com> wrote:
>> On 18/02/13 12:03, Oscar Benjamin wrote:
>>> On 18 February 2013 11:47, John Reid <johnbaronreid <at> gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I was hoping namedtuples could be used as replacements for tuples in all instances.
>>> namedtuples are not really intended to serves as tuples anywhere. They
>>> are intended to provide lightweight, immutable, hashable objects with
>>> *named* (rather than numbered) values.
>> If they are not supposed to be tuples then calling them namedtuples and
>> inheriting from tuple seems a little odd.
> You can use namedtuple instances in places that expect tuples.
> Inheriting from tuples enables them to be all the things I said:
> lightweight, immutable and hashable. The type object itself has a
> different interface for the constructor, though.
>
>
Then I can't use them in every place I use tuples. For example IPython
relies upon the type's interface for the constructor as part of a
serialization mechanism.

I wonder why they have a different interface. It seems to restrict their
usability. No doubt there were other factors involved in the design of
the interface.

John.

(Continue reading)

Dave Angel | 18 Feb 13:11 2013

Re: Differences creating tuples and collections.namedtuples

On 02/18/2013 06:47 AM, John Reid wrote:
> Hi,
>
> I was hoping namedtuples could be used as replacements for tuples in all instances. There seem to be some
differences between how tuples and namedtuples are created. For example with a tuple I can do:
>
> a=tuple([1,2,3])
>
> with namedtuples I get a TypeError:
>
> from collections import namedtuple
> B=namedtuple('B', 'x y z')
> b=B([1,2,3])

You are passing a single list to the constructor, but you specified that 
the namedtuple was to have 3 items.  So you need two more.

>
> TypeError: __new__() takes exactly 4 arguments (2 given)
>> <ipython-input-23-d1da2ef851fb>(3)<module>()
>        1 from collections import namedtuple
>        2 B=namedtuple('B', 'x y z')
> ----> 3 b=B([1,2,3])
>
> I'm seeing this problem because of the following code in IPython:
>
> def canSequence(obj):
>      if isinstance(obj, (list, tuple)):
>          t = type(obj)
>          return t([can(i) for i in obj])
(Continue reading)

John Reid | 18 Feb 15:09 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18/02/13 12:11, Dave Angel wrote:
> On 02/18/2013 06:47 AM, John Reid wrote:
>> Hi,
>>
>> I was hoping namedtuples could be used as replacements for tuples in
>> all instances. There seem to be some differences between how tuples
>> and namedtuples are created. For example with a tuple I can do:
>>
>> a=tuple([1,2,3])
>>
>> with namedtuples I get a TypeError:
>>
>> from collections import namedtuple
>> B=namedtuple('B', 'x y z')
>> b=B([1,2,3])
> 
> You are passing a single list to the constructor, but you specified that
> the namedtuple was to have 3 items.  So you need two more.

I'm aware how to construct the namedtuple and the tuple. My point was
that they use different syntaxes for the same operation and this seems
to break ipython. I was wondering if this is a necessary design feature
or perhaps just an oversight on the part of the namedtuple author or
ipython developers.
Oscar Benjamin | 18 Feb 15:15 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18 February 2013 14:09, John Reid <j.reid <at> mail.cryst.bbk.ac.uk> wrote:
> On 18/02/13 12:11, Dave Angel wrote:
>> On 02/18/2013 06:47 AM, John Reid wrote:
>>> Hi,
>>>
>>> I was hoping namedtuples could be used as replacements for tuples in
>>> all instances. There seem to be some differences between how tuples
>>> and namedtuples are created. For example with a tuple I can do:
>>>
>>> a=tuple([1,2,3])
>>>
>>> with namedtuples I get a TypeError:
>>>
>>> from collections import namedtuple
>>> B=namedtuple('B', 'x y z')
>>> b=B([1,2,3])
>>
>> You are passing a single list to the constructor, but you specified that
>> the namedtuple was to have 3 items.  So you need two more.
>
> I'm aware how to construct the namedtuple and the tuple. My point was
> that they use different syntaxes for the same operation and this seems
> to break ipython. I was wondering if this is a necessary design feature
> or perhaps just an oversight on the part of the namedtuple author or
> ipython developers.

I would say that depending on isinstance(obj, tuple) was the error. I
can't offer a suggestion as you haven't clarified what the purpose of
this code in canSequence() is or what constraints it is expected to
satisfy.
(Continue reading)

John Reid | 18 Feb 15:23 2013
Picon

Re: Differences creating tuples and collections.namedtuples


On 18/02/13 14:15, Oscar Benjamin wrote:
> On 18 February 2013 14:09, John Reid <j.reid <at> mail.cryst.bbk.ac.uk> wrote:
>> On 18/02/13 12:11, Dave Angel wrote:
>>> On 02/18/2013 06:47 AM, John Reid wrote:
>>>> Hi,
>>>>
>>>> I was hoping namedtuples could be used as replacements for tuples in
>>>> all instances. There seem to be some differences between how tuples
>>>> and namedtuples are created. For example with a tuple I can do:
>>>>
>>>> a=tuple([1,2,3])
>>>>
>>>> with namedtuples I get a TypeError:
>>>>
>>>> from collections import namedtuple
>>>> B=namedtuple('B', 'x y z')
>>>> b=B([1,2,3])
>>>
>>> You are passing a single list to the constructor, but you specified that
>>> the namedtuple was to have 3 items.  So you need two more.
>>
>> I'm aware how to construct the namedtuple and the tuple. My point was
>> that they use different syntaxes for the same operation and this seems
>> to break ipython. I was wondering if this is a necessary design feature
>> or perhaps just an oversight on the part of the namedtuple author or
>> ipython developers.
> 
> I would say that depending on isinstance(obj, tuple) was the error. I
> can't offer a suggestion as you haven't clarified what the purpose of
(Continue reading)

Oscar Benjamin | 18 Feb 15:53 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18 February 2013 14:23, John Reid <j.reid <at> mail.cryst.bbk.ac.uk> wrote:
[snip]
> That said it would be nice to know the rationale for
> namedtuple.__new__ to have a different signature to tuple.__new__. I'm
> guessing namedtuple._make has a similar interface to tuple.__new__. Does
> anyone know what the rationale was for this design?

Because namedtuples use names for the arguments in the constructor:

>>> from collections import namedtuple
>>> Point = namedtuple('Point', 'x y')
>>> p1 = Point(x=2, y=3)
>>> p1
Point(x=2, y=3)
>>> p1.x
2

Oscar
John Reid | 18 Feb 16:07 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 18/02/13 14:53, Oscar Benjamin wrote:
> On 18 February 2013 14:23, John Reid <j.reid <at> mail.cryst.bbk.ac.uk> wrote:
> [snip]
>> That said it would be nice to know the rationale for
>> namedtuple.__new__ to have a different signature to tuple.__new__. I'm
>> guessing namedtuple._make has a similar interface to tuple.__new__. Does
>> anyone know what the rationale was for this design?
> 
> Because namedtuples use names for the arguments in the constructor:
> 
>>>> from collections import namedtuple
>>>> Point = namedtuple('Point', 'x y')
>>>> p1 = Point(x=2, y=3)
>>>> p1
> Point(x=2, y=3)
>>>> p1.x
> 2
> 
That's a good point. I haven't used __new__ much before but wouldn't
something like this __new__() cater for both uses? (Example taken from
namedtuple docs
http://docs.python.org/2/library/collections.html#collections.namedtuple).

>>> Point = namedtuple('Point', ['x', 'y'], verbose=True)
class Point(tuple):
    'Point(x, y)'

    __slots__ = ()

    _fields = ('x', 'y')
(Continue reading)

Terry Reedy | 18 Feb 22:28 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On 2/18/2013 6:47 AM, John Reid wrote:

> I was hoping namedtuples could be used as replacements for tuples
 >  in all instances.

This is a mistake in the following two senses. First, tuple is a class 
with instances while namedtuple is a class factory that produces 
classes. (One could think of namedtuple as a metaclass, but it was not 
implemented that way.) Second, a tuple instance can have any length and 
different instances can have different lengths. On the other hand, all 
instances of a particular namedtuple class have a fixed length. This 
affects their initialization. So does the fact that Oscar mentioned, 
that fields can be initialized by name.

 > There seem to be some differences between how tuples and namedtuples
 > are created. For example with a tuple I can do:
>
> a=tuple([1,2,3])

But no sensible person would ever do that, since it creates an 
unnecessary list and is equivalent to

a = 1,2,3

The actual api is tuple(iterable). I presume you know that, but it gets 
to the question you ask about 'why the difference?'. The only reason to 
use an explicit tuple() call is when you already have an iterable, 
possibly of unknown length, rather than the individual field objects. In 
the latter case, one should use a display.

(Continue reading)

alex23 | 19 Feb 02:47 2013
Picon

Re: Differences creating tuples and collections.namedtuples

On Feb 18, 9:47 pm, John Reid <johnbaronr... <at> gmail.com> wrote:
> See http://article.gmane.org/gmane.comp.python.ipython.user/10270 for more info.

One quick workaround would be to use a tuple where required and then
coerce it back to Result when needed as such:

def sleep(secs):
    import os, time, parallel_helper
    start = time.time()
    time.sleep(secs)
    return tuple(parallel_helper.Result(os.getpid(), time.time() -
start))

rc = parallel.Client()
v = rc.load_balanced_view()
async_result = v.map_async(sleep, range(3, 0, -1), ordered=False)
for ar in async_result:
    print parallel_helper.Result(*ar)

You can of course skip the creation of Result in sleep and only turn
it into one in the display loop, but it all depends on additional
requirements (and adds some clarity to what is happening, I think).
--

-- 
http://mail.python.org/mailman/listinfo/python-list

John Reid | 19 Feb 10:36 2013
Picon

Re: Differences creating tuples and collections.namedtuples


On 19/02/13 01:47, alex23 wrote:
> On Feb 18, 9:47 pm, John Reid <johnbaronr... <at> gmail.com> wrote:
>> See http://article.gmane.org/gmane.comp.python.ipython.user/10270 for more info.
> 
> One quick workaround would be to use a tuple where required and then
> coerce it back to Result when needed as such:
> 
> def sleep(secs):
>     import os, time, parallel_helper
>     start = time.time()
>     time.sleep(secs)
>     return tuple(parallel_helper.Result(os.getpid(), time.time() -
> start))
> 
> rc = parallel.Client()
> v = rc.load_balanced_view()
> async_result = v.map_async(sleep, range(3, 0, -1), ordered=False)
> for ar in async_result:
>     print parallel_helper.Result(*ar)
> 
> You can of course skip the creation of Result in sleep and only turn
> it into one in the display loop, but it all depends on additional
> requirements (and adds some clarity to what is happening, I think).
> 

Thanks all I really need is a quick work around but it is always nice to
discuss these things. Also this class decorator seems to do the job for
ipython although it does change the construction syntax a little and is
probablty overkill. No doubt the readers of this list can improve it
(Continue reading)


Gmane