Phil Scott | 9 May 19:57 2013

Stream instance for Parsec + conduits

Hi all.

I would like to have a Parsec Stream instance for Data.Text streams in yesod's ConduitM. So far, I have this:

The idea is that because Yesod's conduits will be chunking values in Data.Text, I should have a wrapper
StreamSource to cache chunked values.

For some reason, the test parser fails, saying:

[Left "Blah" (line 1, column 1):
unexpected "g"
expecting "h" or "g"]

Any ideas?


Haskell-Cafe mailing list
Haskell-Cafe <at>
Roman Cheplyaka | 28 May 21:20 2013

Re: Stream instance for Parsec + conduits

Hi Phil,

Sorry for the late answer — somehow I missed your email when it was
originally posted.

Parsec wasn't designed for incremental input, and adding it without
modifying the internals would be tricky.

The problem with your code is this: at the branching point, Parsec
remembers the current state of the stream — i.e. "".

Then it runs the first branch, and, after that has
failed, the second branch — both with the same state of the stream, "".

The first branch, when being run, grabs the "g" from conduit.
The second branch, since it was passed "", also asks conduit for the
input. Alas, conduit has already given away its only chunk of data. It
has no idea that this is a different branch which is unaware of the
previous events.

One way around this is to have a wrapper around the conduit monad — a
simple StateT should suffice. It must cache the whole input. The stream
itself (i.e. your StreamSource) can be simply an Int which denotes the
position in the stream. Your monad then can supply the data requested by
its position, without losing data when switching branches.

A drawback of this is that you don't know how long to hold that data,
and thus have a memory leak. Attoparsec has it, too, by the way, but
it is not related to being incremental. It is possible to alter Parsec
so that you have both incremental input support and the ability to
(Continue reading)