Corey O'Connor | 21 Oct 22:34 2013
Picon

a tiny HTTP server.. that doesn't work

I attempted to write a tiny HTTP server using only Haskell Platform packages:


Which works... Unless the threaded runtime is used.

When compiled using the threaded runtime and run with +RTS -N this server fails to reply correctly to ~3% of requests. The expectation is that the reply will be exactly the request body. However, 3% of the time the HTTP request fails to be parsed due to an "ErrorClosed"

The server then executes:


which responds to the client. The client receives the response correctly most of the time.

My current hypothesis is that some aspect of lazy IO is not playing nice. The connection is being closed before the request can be completely parsed.

Is this correct? What am I missing?

I know there are other HTTP server packages that could be used. However, the exercise was to build a HTTP server using only Haskell Platform packages.

Below is the main body of the code:

main = withSocketsDo $ do
    http_socket <- listenOn $ PortNumber 9090
    dispatch_on_accept http_socket $ either handle_failed_request handle_valid_request
    sClose http_socket

handle_failed_request failure = return $ Response (4,0,0) "Bad Request" [mkHeader HdrConnection "close"]
                                                  (encodeUtf8 $ pack $ show failure)

handle_valid_request request = do
    let request_body = rqBody request
    return $ Response (2,0,0) "OK" [mkHeader HdrConnection "close"]
                      (encodeUtf8 $ pack $ show request_body)

dispatch_on_accept http_socket handler = forever $ accept http_socket >>= forkIO . httpHandler . fst
    where
        httpHandler client_socket = bracket (socketConnection "client" 0 client_socket)
                                            Network.HTTP.close
                                            client_interact
        client_interact :: HandleStream BS.ByteString -> IO ()
        client_interact byte_stream = receiveHTTP byte_stream >>= handler >>= respondHTTP byte_stream

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Jeremy Shaw | 21 Oct 22:57 2013

Re: a tiny HTTP server.. that doesn't work

my hypothesis is that you are getting 'ErrorClosed' when calling
'sClose' because the client side has already beat you to the punch.
But that is just a hypothesis.

- jeremy

On Mon, Oct 21, 2013 at 3:34 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
> I attempted to write a tiny HTTP server using only Haskell Platform
> packages:
>
> * https://github.com/coreyoconnor/tiny-http-hp
>
> Which works... Unless the threaded runtime is used.
>
> When compiled using the threaded runtime and run with +RTS -N this server
> fails to reply correctly to ~3% of requests. The expectation is that the
> reply will be exactly the request body. However, 3% of the time the HTTP
> request fails to be parsed due to an "ErrorClosed"
>
> The server then executes:
>
> * https://github.com/coreyoconnor/tiny-http-hp/blob/master/TinyHttp.hs#L23
>
> which responds to the client. The client receives the response correctly
> most of the time.
>
> My current hypothesis is that some aspect of lazy IO is not playing nice.
> The connection is being closed before the request can be completely parsed.
>
> Is this correct? What am I missing?
>
> I know there are other HTTP server packages that could be used. However, the
> exercise was to build a HTTP server using only Haskell Platform packages.
>
> Below is the main body of the code:
>
>
> main = withSocketsDo $ do
>
>     http_socket <- listenOn $ PortNumber 9090
>
>     dispatch_on_accept http_socket $ either handle_failed_request
> handle_valid_request
>
>     sClose http_socket
>
>
>
> handle_failed_request failure = return $ Response (4,0,0) "Bad Request"
> [mkHeader HdrConnection "close"]
>
>                                                   (encodeUtf8 $ pack $ show
> failure)
>
>
> handle_valid_request request = do
>
>     let request_body = rqBody request
>
>     return $ Response (2,0,0) "OK" [mkHeader HdrConnection "close"]
>
>                       (encodeUtf8 $ pack $ show request_body)
>
>
> dispatch_on_accept http_socket handler = forever $ accept http_socket >>=
> forkIO . httpHandler . fst
>
>     where
>         httpHandler client_socket = bracket (socketConnection "client" 0
> client_socket)
>
>                                             Network.HTTP.close
>
>                                             client_interact
>         client_interact :: HandleStream BS.ByteString -> IO ()
>
>         client_interact byte_stream = receiveHTTP byte_stream >>= handler
>>>= respondHTTP byte_stream
>
>
> -Corey O'Connor
> coreyoconnor <at> gmail.com
> http://corebotllc.com/
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
Corey O'Connor | 21 Oct 23:15 2013
Picon

Re: a tiny HTTP server.. that doesn't work

Thanks for the input!

The sClose on line 21 is actually never called: the "dispatch_on_request" loops forever.
The sClose implied by Network.HTTP.close on line 34 is called, but (in theory) after the "ErrorClosed" error is produced. This error comes from the "receiveHTTP" on line 37. Which should occur before the Network.HTTP.close.

The reasoning of "client side has already closed the connection" makes sense, but I would imagine the client (JMeter in this case) would wait for a response before closing the connection. JMeter does receive a reply in the failure cases. Which is the reply formed on line 23.

Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

Cheers,
Corey




On Mon, Oct 21, 2013 at 1:57 PM, Jeremy Shaw <jeremy <at> n-heptane.com> wrote:
my hypothesis is that you are getting 'ErrorClosed' when calling
'sClose' because the client side has already beat you to the punch.
But that is just a hypothesis.

- jeremy

On Mon, Oct 21, 2013 at 3:34 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
> I attempted to write a tiny HTTP server using only Haskell Platform
> packages:
>
> * https://github.com/coreyoconnor/tiny-http-hp
>
> Which works... Unless the threaded runtime is used.
>
> When compiled using the threaded runtime and run with +RTS -N this server
> fails to reply correctly to ~3% of requests. The expectation is that the
> reply will be exactly the request body. However, 3% of the time the HTTP
> request fails to be parsed due to an "ErrorClosed"
>
> The server then executes:
>
> * https://github.com/coreyoconnor/tiny-http-hp/blob/master/TinyHttp.hs#L23
>
> which responds to the client. The client receives the response correctly
> most of the time.
>
> My current hypothesis is that some aspect of lazy IO is not playing nice.
> The connection is being closed before the request can be completely parsed.
>
> Is this correct? What am I missing?
>
> I know there are other HTTP server packages that could be used. However, the
> exercise was to build a HTTP server using only Haskell Platform packages.
>
> Below is the main body of the code:
>
>
> main = withSocketsDo $ do
>
>     http_socket <- listenOn $ PortNumber 9090
>
>     dispatch_on_accept http_socket $ either handle_failed_request
> handle_valid_request
>
>     sClose http_socket
>
>
>
> handle_failed_request failure = return $ Response (4,0,0) "Bad Request"
> [mkHeader HdrConnection "close"]
>
>                                                   (encodeUtf8 $ pack $ show
> failure)
>
>
> handle_valid_request request = do
>
>     let request_body = rqBody request
>
>     return $ Response (2,0,0) "OK" [mkHeader HdrConnection "close"]
>
>                       (encodeUtf8 $ pack $ show request_body)
>
>
> dispatch_on_accept http_socket handler = forever $ accept http_socket >>=
> forkIO . httpHandler . fst
>
>     where
>         httpHandler client_socket = bracket (socketConnection "client" 0
> client_socket)
>
>                                             Network.HTTP.close
>
>                                             client_interact
>         client_interact :: HandleStream BS.ByteString -> IO ()
>
>         client_interact byte_stream = receiveHTTP byte_stream >>= handler
>>>= respondHTTP byte_stream
>
>
> -Corey O'Connor
> coreyoconnor <at> gmail.com
> http://corebotllc.com/
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Joey Adams | 22 Oct 04:28 2013
Picon

Re: a tiny HTTP server.. that doesn't work

On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection.  This seems consistent with your results:

 * With the close call: connection is closed right after client sends a second request, so client whines.

 * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Corey O'Connor | 23 Oct 01:54 2013
Picon

Re: a tiny HTTP server.. that doesn't work

On Mon, Oct 21, 2013 at 7:28 PM, Joey Adams <joeyadams3.14159 <at> gmail.com> wrote:
On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection.  This seems consistent with your results:

 * With the close call: connection is closed right after client sends a second request, so client whines.
 * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.

​Thanks!​ I'll try your test and see if anything changes.

Though I thought the code accounted for this: A connection close header is added to each response. Which I thought would cause the client to close the connection.​​ I could be totally wrong tho. There could be some additional aspect of HTTP connections I do not understand.

Properly supporting persistent connections should be pretty easy. There is another Network.HTTP server package that does this: http://hackage.haskell.org/package/http-server-1/docs/Network-HTTP-Server.html

So I'll give that a shot and report back.

Thanks!
Corey

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Corey O'Connor | 6 Nov 03:10 2013
Picon

Re: a tiny HTTP server.. that doesn't work

I've updated the code to handle persistent connections. As best I can figure anyways..
Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

The test is done via JMeter. See the perf_test.jmx file. I've confirmed JMeter is include "connection: close" headers in all requests. Which means the persistent connection code is not even being exercised. From instrumenting the code I can confirm this is the case: All requests include "connection: close" and the client never performs a second request using the same stream.

An additional data point: The same test when run with 1 capability (EG: +RTS -N1) does *not* fail. There are no connection close errors. Only when running with multiple capabilities does the test fail.

Cheers,
Corey




On Tue, Oct 22, 2013 at 4:54 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
On Mon, Oct 21, 2013 at 7:28 PM, Joey Adams <joeyadams3.14159 <at> gmail.com> wrote:
On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection.  This seems consistent with your results:

 * With the close call: connection is closed right after client sends a second request, so client whines.
 * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.

​Thanks!​ I'll try your test and see if anything changes.

Though I thought the code accounted for this: A connection close header is added to each response. Which I thought would cause the client to close the connection.​​ I could be totally wrong tho. There could be some additional aspect of HTTP connections I do not understand.

Properly supporting persistent connections should be pretty easy. There is another Network.HTTP server package that does this: http://hackage.haskell.org/package/http-server-1/docs/Network-HTTP-Server.html

So I'll give that a shot and report back.

Thanks!
Corey


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Corey O'Connor | 6 Nov 20:55 2013
Picon

Re: a tiny HTTP server.. that doesn't work

My current guess: This is a bug in how the socket is set up or one of the underlying libraries.

Using the stream debug hooks indicates the failure is due to "readLine" returning an empty line when parsing the request header. I cannot tell if an empty line is returned due to an actual 0 byte return from read/recv or some processing of the read data. Still, this would indicate the HTTP library considers the request to be malformed.

The question is then: Are the requests from JMeter malformed?

A wireshark capture was performed of the JMeter test traffic. This wireshark data indicated 5000 HTTP requests were made. Which is what JMeter produced. All 5000 of these requests were byte identical. This capture did not affect the test results: A similar number of requests failed.

I tried to instrument the read/recv calls using dtrace. This indicated read, and not recv, was used when reading from teh socket. I was unable to determine anything else tho. Probably a better idea to instrument HTTP library a bit more.

Cheers,
Corey





On Tue, Nov 5, 2013 at 6:10 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
I've updated the code to handle persistent connections. As best I can figure anyways..
Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

The test is done via JMeter. See the perf_test.jmx file. I've confirmed JMeter is include "connection: close" headers in all requests. Which means the persistent connection code is not even being exercised. From instrumenting the code I can confirm this is the case: All requests include "connection: close" and the client never performs a second request using the same stream.

An additional data point: The same test when run with 1 capability (EG: +RTS -N1) does *not* fail. There are no connection close errors. Only when running with multiple capabilities does the test fail.

Cheers,
Corey

On Tue, Oct 22, 2013 at 4:54 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
On Mon, Oct 21, 2013 at 7:28 PM, Joey Adams <joeyadams3.14159 <at> gmail.com> wrote:
On Mon, Oct 21, 2013 at 5:15 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
Curiously: If the Network.HTTP.close is removed the server does not fail, but some requests takes several seconds to process. I presume the ordering of effects is then correct, but without the explicit close the close occurs only a GC time.

The client may be trying to reuse the connection; see http://en.wikipedia.org/wiki/HTTP_persistent_connection.  This seems consistent with your results:

 * With the close call: connection is closed right after client sends a second request, so client whines.
 * Without the close call: client waits around for a response, then gives up and establishes another connection.

To confirm, add another receiveHTTP call client_interact and see if it returns another Request.

​Thanks!​ I'll try your test and see if anything changes.

Though I thought the code accounted for this: A connection close header is added to each response. Which I thought would cause the client to close the connection.​​ I could be totally wrong tho. There could be some additional aspect of HTTP connections I do not understand.

Properly supporting persistent connections should be pretty easy. There is another Network.HTTP server package that does this: http://hackage.haskell.org/package/http-server-1/docs/Network-HTTP-Server.html

So I'll give that a shot and report back.

Thanks!
Corey



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Joey Adams | 7 Nov 01:37 2013
Picon

Re: a tiny HTTP server.. that doesn't work

On Tue, Nov 5, 2013 at 9:10 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
I've updated the code to handle persistent connections. As best I can figure anyways..
Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

What OS are you testing on? 
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Corey O'Connor | 8 Nov 23:28 2013
Picon

Re: a tiny HTTP server.. that doesn't work

This is on Mac OS 10.8 and 10.7.
I'll be testing NixOS (Linux) this weekend. Was having troubles getting jmeter running...

Cheers,
Corey



On Wed, Nov 6, 2013 at 4:37 PM, Joey Adams <joeyadams3.14159 <at> gmail.com> wrote:
On Tue, Nov 5, 2013 at 9:10 PM, Corey O'Connor <coreyoconnor <at> gmail.com> wrote:
I've updated the code to handle persistent connections. As best I can figure anyways..
Unfortunately this does not resolve the issue. The server still reports the connection being closed before the full response is read.

What OS are you testing on? 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane