Manlio Perillo | 13 Jan 23:51
Favicon

walking a directory tree efficiently

Hi.

During a tentative (quite unsuccessfull) to convert a simple Python 
script that prints on stdout a directory and all its subdirectory [1] in 
a good Haskell (mostly to start to do real practice with the language), 
I came across this blog post:
http://blog.moertel.com/articles/2007/03/28/directory-tree-printing-in-haskell-part-three-lazy-i-o

Since recently I read about alternatives to lazy IO (like iteratee), I'm 
curious to know if a flexible, efficient and safe alternative exists, 
for the task of display a directory tree.

[1] http://paste.pocoo.org/show/99523/

Thanks  Manlio perillo
Don Stewart | 14 Jan 00:27
Gravatar

Re: walking a directory tree efficiently

manlio_perillo:
> Hi.
> 
> During a tentative (quite unsuccessfull) to convert a simple Python 
> script that prints on stdout a directory and all its subdirectory [1] in 
> a good Haskell (mostly to start to do real practice with the language), 
> I came across this blog post:
> http://blog.moertel.com/articles/2007/03/28/directory-tree-printing-in-haskell-part-three-lazy-i-o
> 
> 
> Since recently I read about alternatives to lazy IO (like iteratee), I'm 
> curious to know if a flexible, efficient and safe alternative exists, 
> for the task of display a directory tree.
> 
> 
> [1] http://paste.pocoo.org/show/99523/
> 

If you can do it with strict IO in Python, do the same thing in Haskell
with System.IO.Strict. It should be mechanical to translate Python
programs directly into naive IO-based Haskell using strict IO. Boring,
but mechanical.

There's no iteratee/fold-based IO system yet.

-- Don
Thomas Hartman | 14 Jan 02:49

Re: walking a directory tree efficiently

> There's no iteratee/fold-based IO system yet.

What about

http://sites.google.com/site/haskell/notes/lazy-io-considered-harmful-way-to-go-left-fold-enumerator
?

It's not on hackage, but at least it's public domain.

Oleg, of course.

2009/1/13 Don Stewart <dons <at> galois.com>:
> manlio_perillo:
>> Hi.
>>
>> During a tentative (quite unsuccessfull) to convert a simple Python
>> script that prints on stdout a directory and all its subdirectory [1] in
>> a good Haskell (mostly to start to do real practice with the language),
>> I came across this blog post:
>> http://blog.moertel.com/articles/2007/03/28/directory-tree-printing-in-haskell-part-three-lazy-i-o
>>
>>
>> Since recently I read about alternatives to lazy IO (like iteratee), I'm
>> curious to know if a flexible, efficient and safe alternative exists,
>> for the task of display a directory tree.
>>
>>
>> [1] http://paste.pocoo.org/show/99523/
>>
>
(Continue reading)

Manlio Perillo | 14 Jan 15:49
Favicon

Re: walking a directory tree efficiently

Don Stewart ha scritto:
> manlio_perillo:
>> Hi.
>>
>> During a tentative (quite unsuccessfull) to convert a simple Python 
>> script that prints on stdout a directory and all its subdirectory [1] in 
>> a good Haskell (mostly to start to do real practice with the language), 
>> I came across this blog post:
>> http://blog.moertel.com/articles/2007/03/28/directory-tree-printing-in-haskell-part-three-lazy-i-o
>>
>>
>> Since recently I read about alternatives to lazy IO (like iteratee), I'm 
>> curious to know if a flexible, efficient and safe alternative exists, 
>> for the task of display a directory tree.
>>
>>
>> [1] http://paste.pocoo.org/show/99523/
>>
> 
> If you can do it with strict IO in Python, do the same thing in Haskell
> with System.IO.Strict. 
> It should be mechanical to translate Python
> programs directly into naive IO-based Haskell using strict IO. Boring,
> but mechanical.
> 

But that's not the purpose of what I'm doing ;).

I'm trying to practice with Haskell, by converting small Python scripts 
I have written.
(Continue reading)

Paolo Losi | 15 Jan 01:04

Re: walking a directory tree efficiently

Hi Manlio

Manlio Perillo wrote:
> By the way, I have managed to have a working program:
> http://hpaste.org/13919

I've made some some minor refinements according
to my own tastes :-)
http://hpaste.org/13919/diff?old=0&new=2

Please note that in both cases IO exceptions are not handled.

> I would like to receive some advices:
> 1) I have avoided the do notation, using functions like liftM.
>    Is this a good practice?

Avoinding the do notation is not good in itself.
If it improves readability, is ok.
But IMHO should not be used as a "smell" for bad coding style.

>    Is this as efficient as using do notation?

Note that the do notation is desugared by the compiler as one of
the first steps.

do print "foo"
    print "foo"

is exactly equivalent to

(Continue reading)

Luke Palmer | 15 Jan 01:52

Re: Re: walking a directory tree efficiently

On Wed, Jan 14, 2009 at 5:04 PM, Paolo Losi <paolo <at> hypersonic.it> wrote:


2) I have written some support functions: mapM' and filterM'
  Are they well written and generic?

mapM' is generic and already implemented: fmap
(Note that a Monad is also a Functor)

Except for when it isn't, which is really annoying because it ought to be.  I just pretend it is, and then grit my teeth and kill a baby when I get bitten.

And mapM' is not fmap, but fmap.fmap (or liftM.map if you prefer).
 


filterM' is specific enough not to deserve any package
filterM' = fmap . map

Uh, I think you're looking at the wrong one.

filterM' :: (a -> m Bool) -> m [a] -> m [a]

I would write this as:

filterM' p = filterM p . return

And then probably just go inline it.
 
that I like more...


  Are they already available in some package?
  Can you suggest better names?
3) I find
  (,) node `liftM` walkTree' path
  not very readable.
  Is it possible to express it in a more (not too much) verbose way?

I find it readable... It's ok IMO.

A bit ugly, perhaps. With the notation from Control.Applicative:

(,) node <$> walkTree' path

I would really like to be able to do a proper-looking tuple section though;

(node,) <$> walkTree' path

Luke

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
wren ng thornton | 18 Jan 08:37

Re: walking a directory tree efficiently

Manlio Perillo wrote:
> By the way, I have managed to have a working program:
> http://hpaste.org/13919
> 
> I would like to receive some advices:
> 1) I have avoided the do notation,

As Paolo Losi says, there's nothing wrong with do-notation. You should 
use whichever style makes your code the easiest to understand.

In order to increase your understanding it's a helpful exercise to write 
monadic code in applicative style instead of do-notation, just as it can 
be helpful to use points-free style instead of lambda abstractions. But 
these are just exercises to help you become more familiar with doing the 
transformations in your mind. At the end of the day, you should use 
whichever of these styles makes your code easiest to read. Sometimes 
that even means switching between them and using different styles in 
different places. There's no performance difference between them because 
the compiler does various transformations to normalize things before 
doing real work.

> using functions like liftM.
>    Is this a good practice?
>    Is this as efficient as using do notation?

I prefer using (liftM f m) instead of (m >>= return . f) which is more 
verbose and obfuscates the purity of f. Though personally liftM doesn't 
strike me as an infix operator and so I think it's clearer to use it in 
prefix form. The name doesn't support infixation as well as `elem`, 
`asTypeOf`, `isPrefixOf`, etc.

If all of your Monads define instances for Functor as well ---and they 
should!--- then you can use (<$>) from Control.Applicative as a good 
infix name. The functions liftM and fmap are equal. The Monad class 
should have Functor as a superclass since all monads are functors, but 
brokenly it doesn't. The only difference between using liftM and using 
fmap is which dictionaries get passed around for polymorphic functions 
(assuming they're not all compiled away). Sometimes there might be a 
small difference in clarity too, since not everyone realizes all monads 
are functors.

> 2) I have written some support functions: mapM' and filterM'
>    Are they well written and generic?
>    Are they already available in some package?
>    Can you suggest better names?

Your mapM' can be written as (fmap . fmap) which is more generic. 
Assuming your Monads are Functors, of course.

Your filterM' can be rewritten as ((=<<) . filterM) or as the somewhat 
more verbose (\f m -> m >>= filterM f)

Since these functions are so short and are used only once, I would 
suggest just writing the definitions inline instead of breaking them out 
and giving them names. Giving them names makes them seem more important 
than they are.

> 3) I find
>    (,) node `liftM` walkTree' path
>    not very readable.
>    Is it possible to express it in a more (not too much) verbose way?

liftM (\x -> (node,x)) (walkTree' path)

Don't be afraid of verbosity if you think it makes the code clearer to read.

--

-- 
Live well,
~wren

Re: Re: walking a directory tree efficiently


Hi,
 what about avoid the use of the unfold over the tree and construct it
directly (e.g. see http://hpaste.org/13919#a3)? I wonder if there is (an
easy) possibility to construct the tree lazily so that output start
immediately for large trees.

best,
Massimiliano Gubinelli
--

-- 
View this message in context: http://www.nabble.com/walking-a-directory-tree-efficiently-tp21446337p21475846.html
Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com.
Paolo Losi | 19 Jan 09:51

Re: walking a directory tree efficiently

Massimiliano Gubinelli wrote:
> Hi,
>  what about avoid the use of the unfold over the tree and construct it
> directly (e.g. see http://hpaste.org/13919#a3)? 

Nice solution!

> I wonder if there is (an
> easy) possibility to construct the tree lazily so that output start
> immediately for large trees.

I think the modular approach would be that of using
a fold-left enumerator that produces the list of paths
and navigation operations by traversing the dir hierarchy "Depth First":

data DirTraversalInfo = Path String
                       | DirUp
		      | DirDown

I guess you know:
http://okmij.org/ftp/papers/LL3-collections-talk.pdf

> best,
> Massimiliano Gubinelli

Ciao
Paolo

Gmane