oleg | 11 May 07:32 2013

Stream processing


I'm a bit curious
  > * be reliable in the presence of async exceptions (solved by conduit,
  >   pipes-safe),
  >
  > * hold on to resources only as long as necessary (solved by conduit
  >   and to some degree by pipes-safe),

Are you aware of
        http://okmij.org/ftp/Streams.html#regions
which describes both resource deallocation and async signals. Could
you tell what you think is deficient in that code?

  > * ideally also allow upstream communication (solved by pipes and to
  >   some degree by conduit).

Are you aware (of, admittedly) old message whose title was specifically
  ``Sending messages up-and-down the iteratee-enumerator chain''
http://www.haskell.org/pipermail/haskell-cafe/2011-May/091870.html
(there were several messages in that thread). Here is the code for
those messages
        http://okmij.org/ftp/Haskell/Iteratee/UpDown.hs
Ertugrul Söylemez | 11 May 13:24 2013
Picon

Re: Stream processing

oleg <at> okmij.org wrote:

> I'm a bit curious
> > * be reliable in the presence of async exceptions (solved by
> > conduit, pipes-safe),
> >
> > * hold on to resources only as long as necessary (solved by conduit
> > and to some degree by pipes-safe),
>
> Are you aware of
>         http://okmij.org/ftp/Streams.html#regions
> which describes both resource deallocation and async signals. Could
> you tell what you think is deficient in that code?

Conceptually all of them could solve that in some way.  After all they
all have an underlying monad, which could implement that.  In fact
that's what conduit does:  it uses ResourceT.

> > * ideally also allow upstream communication (solved by pipes and to
> > some degree by conduit).
>
> Are you aware (of, admittedly) old message whose title was
> specifically ``Sending messages up-and-down the iteratee-enumerator
> chain''
> http://www.haskell.org/pipermail/haskell-cafe/2011-May/091870.html
> (there were several messages in that thread). Here is the code for
> those messages http://okmij.org/ftp/Haskell/Iteratee/UpDown.hs

Same here.  Conceptually you could solve it in all stream processing
abstractions by small adjustments.
(Continue reading)

Conrad Parker | 13 May 06:11 2013

Re: Stream processing

On 11 May 2013 19:24, Ertugrul Söylemez <es <at> ertes.de> wrote:
>
> However, my real question hasn't been answered so far.  Is my
> formulation of the stream processing problem accurate/complete?
>

Yes, you've summarized the commonly asked questions well. Perhaps you
could make a wiki page which lists out these questions, and provide
links to each of the answers from Oleg's site or elsewhere.

cheers,

Conrad.

Gmane