Isak Hansen | 9 Jul 19:40 2014
Picon

Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

Let's just imagine there aren't enough poker sites out there already and that it makes sense to build another one. Basically, there will be players connecting to one or more tables, most likely talking some kind of JSON based protocol with the server(s).

#1 - What's a good way to set up and maintain compressed and encrypted connections from the clients to my cluster? I've been wanting to try out websockets, think that'll do?

#2 - I'm new to Haskell and would appreciate thoughts on how to store and process game state. Imagine a plain application managing 1000 tables of poker, responding to player input (that magically appears) and timer events (e.g. folding players that fail to take action). What data structures and libraries should I be looking at here? Thoughts on concurrency and how I organize program flow?

#3 - I'm thinking of ZMQ to wire components together. Would I be better off basing my cluster on Cloud Haskell or some other library for distributed work?


Would love some quick pointers and feedback here, thanks in advance,
Isak

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Andrew Gibiansky | 9 Jul 20:01 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

I don't have any comments on the specifics of your application, but I've used zeromq4-haskell [0] for IHaskell and it's a solid library that works well. There's also zeromq-haskell (instead of zeromq4-haskell) if you want older ZeroMQ. I was using it for local communication, not networked though, so not sure if that changes anything. It is fairly low-level, though, so I imagine Cloud Haskell can give you a higher level of abstraction when dealing with networking stuff.

[0] https://hackage.haskell.org/package/zeromq4-haskell


On Wed, Jul 9, 2014 at 10:40 AM, Isak Hansen <isak.hansen <at> gmail.com> wrote:
Let's just imagine there aren't enough poker sites out there already and that it makes sense to build another one. Basically, there will be players connecting to one or more tables, most likely talking some kind of JSON based protocol with the server(s).

#1 - What's a good way to set up and maintain compressed and encrypted connections from the clients to my cluster? I've been wanting to try out websockets, think that'll do?

#2 - I'm new to Haskell and would appreciate thoughts on how to store and process game state. Imagine a plain application managing 1000 tables of poker, responding to player input (that magically appears) and timer events (e.g. folding players that fail to take action). What data structures and libraries should I be looking at here? Thoughts on concurrency and how I organize program flow?

#3 - I'm thinking of ZMQ to wire components together. Would I be better off basing my cluster on Cloud Haskell or some other library for distributed work?


Would love some quick pointers and feedback here, thanks in advance,
Isak


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Wojtek Narczyński | 9 Jul 20:52 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

On 09.07.2014 19:40, Isak Hansen wrote:

Thousands of connections, your smartphone wouldn't handle, not to 
mention a server.

> #1 - What's a good way to set up and maintain compressed and encrypted 
> connections from the clients to my cluster? I've been wanting to try 
> out websockets, think that'll do?
>

http://caniuse.com/websockets - WebSockets browser compatibility, caveat 
emptor

> #2 - I'm new to Haskell and would appreciate thoughts on how to store 
> and process game state. Imagine a plain application managing 1000 
> tables of poker, responding to player input (that magically appears) 
> and timer events (e.g. folding players that fail to take action). What 
> data structures and libraries should I be looking at here? Thoughts on 
> concurrency and how I organize program flow?
>

I would spawn a process for each game, and a process for each player.

> #3 - I'm thinking of ZMQ to wire components together. Would I be 
> better off basing my cluster on Cloud Haskell or some other library 
> for distributed work?
>
Erlang/OTP is great, but I have a feeling that something was 'lost in 
translation' to Haskell, and that thing is simplicity. For example, I do 
not understand why such a small cabal package has been split into 
multiple tiny cabal packages. This is the fuel that cabal hell fire 
burns on. Despite this, I would give Cloud Haskell a try. If you don't 
you, will have to rediscover things like supervision hierarchy, not to 
mention timeouts as the first line of defense againtst all kinds of 
concurrency problems. Erlang/OTP really does get certain things right.

--

-- 
Kind regards,
Wojtek N.
Carter Schonwald | 9 Jul 23:39 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

the courier package on hackage seems to strike a better balance vs CH


On Wed, Jul 9, 2014 at 2:52 PM, Wojtek Narczyński <wojtek <at> power.com.pl> wrote:
On 09.07.2014 19:40, Isak Hansen wrote:

Thousands of connections, your smartphone wouldn't handle, not to mention a server.


#1 - What's a good way to set up and maintain compressed and encrypted connections from the clients to my cluster? I've been wanting to try out websockets, think that'll do?


http://caniuse.com/websockets - WebSockets browser compatibility, caveat emptor


#2 - I'm new to Haskell and would appreciate thoughts on how to store and process game state. Imagine a plain application managing 1000 tables of poker, responding to player input (that magically appears) and timer events (e.g. folding players that fail to take action). What data structures and libraries should I be looking at here? Thoughts on concurrency and how I organize program flow?


I would spawn a process for each game, and a process for each player.


#3 - I'm thinking of ZMQ to wire components together. Would I be better off basing my cluster on Cloud Haskell or some other library for distributed work?

Erlang/OTP is great, but I have a feeling that something was 'lost in translation' to Haskell, and that thing is simplicity. For example, I do not understand why such a small cabal package has been split into multiple tiny cabal packages. This is the fuel that cabal hell fire burns on. Despite this, I would give Cloud Haskell a try. If you don't you, will have to rediscover things like supervision hierarchy, not to mention timeouts as the first line of defense againtst all kinds of concurrency problems. Erlang/OTP really does get certain things right.

--
Kind regards,
Wojtek N.

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Wojtek Narczyński | 10 Jul 00:05 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

On 09.07.2014 23:39, Carter Schonwald wrote:
> the courier package on hackage seems to strike a better balance vs CH
> https://hackage.haskell.org/package/courier
>
Well, no.

There really is great value in the OTP part of Erlang/OTP, while this 
package only provides Erlang (messaging).

--

-- 
Wojtek
Alois Cochard | 9 Jul 20:53 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

Hi Isak,


On 9 July 2014 18:40, Isak Hansen <isak.hansen <at> gmail.com> wrote:

#2 - I'm new to Haskell and would appreciate thoughts on how to store and process game state. Imagine a plain application managing 1000 tables of poker, responding to player input (that magically appears) and timer events (e.g. folding players that fail to take action). What data structures and libraries should I be looking at here? Thoughts on concurrency and how I organize program flow?

If I had to build an system like this, the main design principle  I would adopt is to not try to share the state of games among nodes, but instead redirect players of the same game on the same node (where you basically have a thread processing the event of the game and updating the state in memory).
Then I would probably use some form of event sourcing to record game action which could be use in case of node crash to recreate a game state by replaying events.

The event source could then be used as well to generate stats/views in near real-time.


--
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Mikael Brockman | 16 Jul 14:47 2014

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

Alois Cochard <alois.cochard <at> gmail.com> writes:

> Hi Isak,
>
> On 9 July 2014 18:40, Isak Hansen <isak.hansen <at> gmail.com> wrote:
>
>     #2 - I'm new to Haskell and would appreciate thoughts on how to
>     store and process game state. Imagine a plain application managing
>     1000 tables of poker, responding to player input (that magically
>     appears) and timer events (e.g. folding players that fail to take
>     action). What data structures and libraries should I be looking at
>     here? Thoughts on concurrency and how I organize program flow?
>
> If I had to build an system like this, the main design principle I
> would adopt is to not try to share the state of games among nodes, but
> instead redirect players of the same game on the same node (where you
> basically have a thread processing the event of the game and updating
> the state in memory).
>
> Then I would probably use some form of event sourcing to record game
> action which could be use in case of node crash to recreate a game
> state by replaying events.
>
> The event source could then be used as well to generate stats/views in
> near real-time.

Some related thoughts:

For poker, it seems reasonable to organize the program around a function
like this:

  step :: Game -> Event -> (Game, Response)

This would be the entry point for the game logic.  It's obvious how to
write tests for this function, and if you're sure that it does the right
thing, then you'll be confident about your game loop.

Then you need a way to produce a sequence of events, combined from
different sources: timers, HTTP requests, etc.  If every game is fed by
a `TChan`, for example, then you can write events into that from
different sources.  But from the viewpoint of the core logic, this is
just an abstract source of events.

Given the performance of GHC's threads, I don't think thousands of
concurrent clients is that much different from dozens or hundreds -- yet
another reason to avoid premature optimization and focus on clarity and
purity!

By the way, I've had fun using `pipes`, `stm`, and `async` to implement
an event sourcing server using JSON commands from web requests.  Using
pipe combinators and queues, the data/control flow of the server can be
apparent from the code of the main function, which is something I've
rarely seen in non-Haskell servers!

--
Mikael Brockman
Alexander Kjeldaas | 9 Jul 22:48 2014
Picon

Re: Looking for architecture/infrastructure advice on coping with thousands of concurrent clients

I'd have user state serviced by a thread per user. Then have separate game state serviced by another thread. Then use zmq or cloud Haskell for interconnect between the two states.

Scalability games then boils down to what cross sectional bandwidth you can get by optimizing the interconnect and the placement/locality of the state.

Scaling the number of client connections can be done independently.

Alexander

On Jul 9, 2014 7:40 PM, "Isak Hansen" <isak.hansen <at> gmail.com> wrote:
Let's just imagine there aren't enough poker sites out there already and that it makes sense to build another one. Basically, there will be players connecting to one or more tables, most likely talking some kind of JSON based protocol with the server(s).

#1 - What's a good way to set up and maintain compressed and encrypted connections from the clients to my cluster? I've been wanting to try out websockets, think that'll do?

#2 - I'm new to Haskell and would appreciate thoughts on how to store and process game state. Imagine a plain application managing 1000 tables of poker, responding to player input (that magically appears) and timer events (e.g. folding players that fail to take action). What data structures and libraries should I be looking at here? Thoughts on concurrency and how I organize program flow?

#3 - I'm thinking of ZMQ to wire components together. Would I be better off basing my cluster on Cloud Haskell or some other library for distributed work?


Would love some quick pointers and feedback here, thanks in advance,
Isak


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane