headers
Sebastiaan Visser | 27 Jul 2011 11:46
Picon

Upgrade to Happstack 6 - request bodies

Hello all,

We have some problems upgrading one of our server applications to Happstack 6, lots of things related to
request bodies and request parameters have changed. Unfortunately, the new situation doesn't seem that
straightforward to me.

In the previous version of Happstack we were able to access the request body directly using the `rqBody`
function. In the new situation we have to first decode the body using `decodeBody` and than access it by
reading from an MVar. The utility functions accessing this variables ensure the body is only request
once, all consecutive calls will error.

We're pretty sure we're accessing the body only once and after decoding it, but we still get the following
error: "askRqEnv failed because the request body has not been decoded yet. Try using 'decodeBody'." This
happens when accessing the cookie using 'readCookieValue'. Isn't strange that accessing the cookie
value uses the request body at all?

So the pattern we use is:

>   do decodeBody (defaultBodyPolicy "/tmp/" (10 * 1024 * 1024) (10 * 1024 * 1024) (1024 * 1024))
>      liftIO (print 1)
>      c <- lookCookieValue "tid"
>      liftIO (print 2)
>      b <- takeRequestBody
>      ...

And the result is:

>   1
>   askRqEnv failed because the request body has not been decoded yet. Try using 'decodeBody'.
(this happens using happstack-server-6.0.3, in 6.1.6 the same error string is sent to the client)
(Continue reading)

Permalink | Reply |
headers
Jeremy Shaw | 27 Jul 2011 18:21
Gravatar

Re: Upgrade to Happstack 6 - request bodies

Hello,

I think you have uncovered a bug. But first let's clear up how things
are supposed to work.

In previous versions of Happstack, the body was always decoded if the
content-type was multipart/form-data or
application/x-www-form-urlencoded. At the same time you also could
access the entire raw request body.

From an easy-of-use point of view, that was wonderful. But, it also
means that if the entire request body was likely to be forced into
RAM. If a Request contained file uploads, then the entirely file
contents would be forced into RAM. That is clearly not acceptable.

If the request body is just a field in the Request like:

 data Request = Request { body :: Lazy.ByteString, ... }

then there is no way to avoid forcing the whole request body into RAM.
Even if you use Lazy.writeFile to write the file to disk, the Request
itself will still have a reference to the body and so none of the data
can be garbage collected until after the Response is sent and the
whole Request gets garbage collected.

So, instead we need to introduce an MVar which allows us to detach the
body contents from the Request type so that it can be garbage
collected separately. So that is why we have:

data Request = Request { rqBody :: MVar RqBody, ... }
(Continue reading)

Permalink | Reply |
headers
Jeremy Shaw | 27 Jul 2011 20:29
Gravatar

Re: Upgrade to Happstack 6 - request bodies

Hello,

I have uploaded happstack-server 6.2.2 which fixes the issue you reported.

You should now be able to use lookCookieValue for a POST/PUT request
with out having to call decodeBody.

The only time you need to call decodeBody now is if you are calling a
function that actually looks at the form-data. For example:

  lookText "somefield"

That will look at both the query string and form-data in the request
body, so it does require decodeBody to be called. But only if you did
a PUT/POST request that could actually contain form data. For example,
if it is a GET request, then only the query_string will be checked,
since there could not be any form data in the body.

If the request body *could* contain form-data, but you have not
decoded it yet, but still want to get the query string value you can
do:

 queryString $ lookText "somefield"

That will only examine the query string and does not require you to
call decodeBody.

So, I believe the behavior is now sensible, and what was originally intended:

 1. you can get the raw request body by simply calling takeRequestBody
(Continue reading)

Permalink | Reply |
headers
Sebastiaan Visser | 28 Jul 2011 12:51
Picon

Re: Upgrade to Happstack 6 - request bodies

Jeremy,

Thanks for the very quick response and fix! 

I'm trying the new code now and at first sight things seem fine. I'll let you know when we encounter more problems.

Thanks again!

Sebastiaan

On Jul 27, 2011, at 8:29 PM, Jeremy Shaw wrote:
> Hello,
> 
> I have uploaded happstack-server 6.2.2 which fixes the issue you reported.
> 
> You should now be able to use lookCookieValue for a POST/PUT request
> with out having to call decodeBody.
> 
> The only time you need to call decodeBody now is if you are calling a
> function that actually looks at the form-data. For example:
> 
>  lookText "somefield"
> 
> That will look at both the query string and form-data in the request
> body, so it does require decodeBody to be called. But only if you did
> a PUT/POST request that could actually contain form data. For example,
> if it is a GET request, then only the query_string will be checked,`
> since there could not be any form data in the body.
> 
> If the request body *could* contain form-data, but you have not
(Continue reading)

Permalink | Reply |

Gmane