headers
pataphor | 4 Jun 2009 15:37
Picon

Making the case for repeat

This probably has a snowballs change in hell of ending up in builtins or
even some in some module, but such things should not prevent one to
try and present the arguments for what one thinks is right. Else one
would end up with consequentialism and that way lies madness and
hyperreality.

So here is my proposed suggestion for a once and for all reconciliation
of various functions in itertools that can not stand on their own and
keep a straight face. Because of backwards compatibility issues we
cannot remove them but we can boldly jump forward and include the right
repeat in the builtin namespace, which I think would be the best thing.
Alternatively -- the second best solution -- would be to give this
function its own namespace where it can supersede the old incongruencies
in itertools. Combiniter or combinator?

P.

from itertools import count
from functools import partial

def repeat(iterable, cycles = None, times = 1):
    L = []
    for x in iterable:
        for i in xrange(times):
            yield x
        L.append(x)
    counter = count if cycles is None else partial(xrange,cycles-1)
    for _ in counter():
        for x in L:
            for i in xrange(times):
(Continue reading)

Permalink | Reply |
headers
Raymond Hettinger | 8 Jun 2009 01:46
Favicon

Re: Making the case for repeat

[pataphor]
> So here is my proposed suggestion for a once and for all reconciliation
> of various functions in itertools that can not stand on their own and
> keep a straight face.

Interesting phraseology ;-)  Enticing and yet fallacious in its
presumption of known and accepted usability problems.  FWIW, when I
designed the module, I started by researching constructs that had
proven success in functional languages and then adapted them to the
needs of Python applications.  That being said, I'm always open to
hearing new ideas.

After reading this thread a couple times, I have a few thoughts
to offer.

1. The Pythonic Way(tm) is to avoid combining too much functionality
in a single function, preferring to split when possible.  That is why
ifilter() and ifilterfalse() are separate functions.

(FWIW, the principle is considered pythonic because it was articulated
by Guido and has been widely applied throughout the language.)

There is a natural inclination to do the opposite.  We factor code
to eliminate redundancy, but that is not always a good idea with
an API.  The goal for code factoring is to minimize redundancy.
The goal for API design is having simple parts that are easily
learned and can be readily combined (i.e. the notion of an
iterator algebra).

It is not progress to mush the parts together in a single function
(Continue reading)

Permalink | Reply |
headers
Boris Borcic | 12 Jun 2009 17:12
Picon

Re: Making the case for repeat

Raymond Hettinger wrote:
> There is a natural inclination to do the opposite.  We factor code
> to eliminate redundancy, but that is not always a good idea with
> an API.  The goal for code factoring is to minimize redundancy.
> The goal for API design is having simple parts that are easily
> learned and can be readily combined (i.e. the notion of an
> iterator algebra).

This reminds me of an early programming experience that left me with a 
fascination. At a time where code had to fit in a couple dozens kilobytes, I 
once had to make significant room in what was already very tight and terse code. 
Code factoring *did* provide the room in the end, but the fascinating part came 
before.

There was strictly no redundancy apparent at first, and finding a usable one 
involved contemplating code execution paths for hours until some degree of 
similarity was apparent between two code path families. And then, the 
fascinating part, was to progressively mutate both *away* from minimality of 
code, in order to enhance the similarity until it could be factored out.

I was impressed; in various ways. First; that the effort could be characterized 
quite mechanically and in a sense stupidly as finding a shortest equivalent 
program, while the subjective feeling was that the task exerted perceptive 
intelligence to the utmost. Second; by the notion that a global constraint of 
code minimization could map more locally to a constraint that drew code to 
expand. Third; that the process resulted in bottom-up construction of what's 
usually constructed top-down, mimicking the willful design of the latter case, 
eg. an API extension, as we might call it nowadays.

Cheers, BB
(Continue reading)

Permalink | Reply |
headers
Steven D'Aprano | 13 Jun 2009 14:15
Picon

Re: Making the case for repeat

Boris Borcic wrote:

> This reminds me of an early programming experience that left me with a
> fascination. At a time where code had to fit in a couple dozens kilobytes,
> I once had to make significant room in what was already very tight and
> terse code. Code factoring *did* provide the room in the end, but the
> fascinating part came before.
> 
> There was strictly no redundancy apparent at first, and finding a usable
> one involved contemplating code execution paths for hours until some
> degree of similarity was apparent between two code path families. And
> then, the fascinating part, was to progressively mutate both *away* from
> minimality of code, in order to enhance the similarity until it could be
> factored out.
> 
> I was impressed; in various ways. First; that the effort could be
> characterized quite mechanically and in a sense stupidly as finding a
> shortest equivalent program, while the subjective feeling was that the
> task exerted perceptive intelligence to the utmost. Second; by the notion
> that a global constraint of code minimization could map more locally to a
> constraint that drew code to expand. Third; that the process resulted in
> bottom-up construction of what's usually constructed top-down, mimicking
> the willful design of the latter case, eg. an API extension, as we might
> call it nowadays.

This is much the same that happens in maximisation problems: the value gets
trapped in a local maximum, and the only way to reach a greater global
maximum is to go downhill for a while.

I believe that hill-climbing algorithms allow some downhill movement for
(Continue reading)

Permalink | Reply |
headers
K4NTICO | 13 Jun 2009 15:48
Picon

Re: Making the case for repeat

On 13 juin, 14:15, Steven D'Aprano
<st... <at> REMOVETHIS.cybersource.com.au> wrote:
> Boris Borcic wrote:
> > This reminds me of an early programming experience that left me with a
> > fascination. At a time where code had to fit in a couple dozens kilobytes,
> > I once had to make significant room in what was already very tight and
> > terse code. Code factoring *did* provide the room in the end, but the
> > fascinating part came before.
>
> > There was strictly no redundancy apparent at first, and finding a usable
> > one involved contemplating code execution paths for hours until some
> > degree of similarity was apparent between two code path families. And
> > then, the fascinating part, was to progressively mutate both *away* from
> > minimality of code, in order to enhance the similarity until it could be
> > factored out.
>
> > I was impressed; in various ways. First; that the effort could be
> > characterized quite mechanically and in a sense stupidly as finding a
> > shortest equivalent program, while the subjective feeling was that the
> > task exerted perceptive intelligence to the utmost. Second; by the notion
> > that a global constraint of code minimization could map more locally to a
> > constraint that drew code to expand. Third; that the process resulted in
> > bottom-up construction of what's usually constructed top-down, mimicking
> > the willful design of the latter case, eg. an API extension, as we might
> > call it nowadays.
>
> This is much the same that happens in maximisation problems: the value gets
> trapped in a local maximum, and the only way to reach a greater global
> maximum is to go downhill for a while.
>
(Continue reading)

Permalink | Reply |
headers
Steven D'Aprano | 8 Jun 2009 10:09
Picon

Re: Making the case for repeat

On Sun, 07 Jun 2009 16:46:23 -0700, Raymond Hettinger wrote:

> We factor code
> to eliminate redundancy, but that is not always a good idea with an API.
>  The goal for code factoring is to minimize redundancy. The goal for API
> design is having simple parts that are easily learned and can be readily
> combined (i.e. the notion of an iterator algebra).

Wonderfully said! That has articulated something which I only recently 
came to appreciate, but couldn't put into words.

-- 
Steven
--

-- 
http://mail.python.org/mailman/listinfo/python-list
Permalink | Reply |
headers
Ben Finney | 8 Jun 2009 10:30
Picon

Re: Making the case for repeat

Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:

> On Sun, 07 Jun 2009 16:46:23 -0700, Raymond Hettinger wrote:
> 
> > We factor code to eliminate redundancy, but that is not always a
> > good idea with an API. The goal for code factoring is to minimize
> > redundancy. The goal for API design is having simple parts that are
> > easily learned and can be readily combined (i.e. the notion of an
> > iterator algebra).
> 
> Wonderfully said! That has articulated something which I only recently 
> came to appreciate, but couldn't put into words.

As originally defined by Martin Fowler, re-factoring always means the
external behaviour is unchanged <URL:http://refactoring.com/>.

So, there's no such thing as a re-factoring that changes the API.
Anything that changes an external attribute of the code is a different
kind of transformation, not a re-factoring.

-- 
 \          “Better not take a dog on the space shuttle, because if he |
  `\   sticks his head out when you're coming home his face might burn |
_o__)                                                up.” —Jack Handey |
Ben Finney
--

-- 
http://mail.python.org/mailman/listinfo/python-list
Permalink | Reply |
headers
Steven D'Aprano | 8 Jun 2009 11:44
Picon

Re: Making the case for repeat

On Mon, 08 Jun 2009 18:30:37 +1000, Ben Finney wrote:

> Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:
> 
>> On Sun, 07 Jun 2009 16:46:23 -0700, Raymond Hettinger wrote:
>> 
>> > We factor code to eliminate redundancy, but that is not always a good
>> > idea with an API. The goal for code factoring is to minimize
>> > redundancy. The goal for API design is having simple parts that are
>> > easily learned and can be readily combined (i.e. the notion of an
>> > iterator algebra).
>> 
>> Wonderfully said! That has articulated something which I only recently
>> came to appreciate, but couldn't put into words.
> 
> As originally defined by Martin Fowler, re-factoring always means the
> external behaviour is unchanged <URL:http://refactoring.com/>.
> 
> So, there's no such thing as a re-factoring that changes the API.
> Anything that changes an external attribute of the code is a different
> kind of transformation, not a re-factoring.

Possibly a *factoring*, without the "re-", just like Raymond said.

Also, keep in mind that when creating a new API, you have no existing API 
to re-factor.

-- 
Steven
--

--
(Continue reading)

Permalink | Reply |
headers
jkn | 8 Jun 2009 11:56
Picon

Re: Making the case for repeat


> Possibly a *factoring*, without the "re-", just like Raymond said.
>
> Also, keep in mind that when creating a new API, you have no existing API
> to re-factor.

Exactly.

I think this has come up before, but I can't remember the answers; any
suggestions for pointer to examples of very well-designed APIs, and
maybe some background as to how the design was achieved?

    Thanks
    jon N
--

-- 
http://mail.python.org/mailman/listinfo/python-list
Permalink | Reply |
headers
jkn | 8 Jun 2009 10:39
Picon

Re: Making the case for repeat

On Jun 8, 9:30 am, Ben Finney <ben+pyt... <at> benfinney.id.au> wrote:

[...]
>
> As originally defined by Martin Fowler, re-factoring always means the
> external behaviour is unchanged <URL:http://refactoring.com/>.
>
> So, there's no such thing as a re-factoring that changes the API.
> Anything that changes an external attribute of the code is a different
> kind of transformation, not a re-factoring.

... and Steven was not calling the two things by the same name. He,
and Raymond, were distinguishing between refactoring, and API design.
That was their point, I think.

    Jon N
--

-- 
http://mail.python.org/mailman/listinfo/python-list
Permalink | Reply |
headers
Gabriel Genellina | 5 Jun 2009 03:44
Picon
Favicon
Gravatar

Re: Making the case for repeat

En Thu, 04 Jun 2009 10:37:45 -0300, pataphor <pataphor <at> gmail.com> escribió:

> So here is my proposed suggestion for a once and for all reconciliation
> of various functions in itertools that can not stand on their own and
> keep a straight face. Because of backwards compatibility issues we
> cannot remove them but we can boldly jump forward and include the right
> repeat in the builtin namespace, which I think would be the best thing.
> Alternatively -- the second best solution -- would be to give this
> function its own namespace where it can supersede the old incongruencies
> in itertools. Combiniter or combinator?

Ok, you're proposing a "bidimensional" repeat. I prefer to keep things
simple, and I'd implement it in two steps. First, something similar to
your repeat_each function in another post:

py> thing = ['1','2','3','4']
py> chain.from_iterable(repeat(elem, 3) for elem in thing)
<itertools.chain object at 0x00BECB90>
py> list(_)
['1', '1', '1', '2', '2', '2', '3', '3', '3', '4', '4', '4']

Note that this doesn't require any additional storage. Second step would
be to build a bidimensional repeat:

py> one = chain.from_iterable(repeat(elem, 3) for elem in thing)
py> two = chain.from_iterable(tee(one, 2))
py> list(two)
['1', '1', '1', '2', '2', '2', '3', '3', '3', '4', '4', '4', '1', '1',
'1', '2',
   '2', '2', '3', '3', '3', '4', '4', '4']
(Continue reading)

Permalink | Reply |
headers
Steven D'Aprano | 4 Jun 2009 15:53
Picon

Re: Making the case for repeat

On Thu, 04 Jun 2009 13:37:45 +0000, pataphor wrote:

> This probably has a snowballs change in hell of ending up in builtins or
> even some in some module, but such things should not prevent one to try
> and present the arguments for what one thinks is right. Else one would
> end up with consequentialism and that way lies madness and hyperreality.

It would be cruel of me to say "Too late", so I shall merely ask, what on 
earth are you going on about?

> So here is my proposed suggestion for a once and for all reconciliation
> of various functions in itertools that can not stand on their own and
> keep a straight face. Because of backwards compatibility issues we
> cannot remove them but we can boldly jump forward and include the right
> repeat in the builtin namespace, which I think would be the best thing.

What is "the right repeat"? What's wrong with the old one? If you're 
going to make a proposal, you have to actually *make the proposal* and 
not just say "Here, have some code, now put it in the builtins because 
the rest of itertools is teh suxor!!!". That rarely goes down well.

(I don't know if that's exactly what you're trying to say, but it seems 
that way to me.)

I've run your test code, and I don't know what I'm supposed to be 
impressed by.

-- 
Steven
--

--
(Continue reading)

Permalink | Reply |
headers
pataphor | 4 Jun 2009 16:34
Picon

Re: Making the case for repeat

Steven D'Aprano wrote:

> I've run your test code, and I don't know what I'm supposed to be 
> impressed by.

Thank you for trying out the code. That you're unimpressed actually is a
huge encouragement because code should just run the way people expect,
without unnecessary surprises.

P.
--

-- 
http://mail.python.org/mailman/listinfo/python-list
Permalink | Reply |

Gmane