Thatcher Ulrich | 12 Jul 2010 06:06

State Object / Process Object design

This sounds really interesting and enticing but I still have a lot of questions:

* you mention that SO's can inherit from a common interface in order to reuse PO's -- but doesn't that undermine their state-only-ness?

* it sounds like this would result in a lot of dynamic allocation of SO's and SO collections, that would not occur in a more conventional OO implementation.  Is that true, and is it a problem?

* what about a complicated persistent data structure like a spatial index, that refers to other objects in the game?  Is this allowed in an SO?  Is it an exception to this pattern?

* is there a more detailed writeup or example system (I.e. with source code available for study) that illustrates this?

Thanks for any insights!

-T

On Jun 3, 2010 11:01 AM, "Morten Brodersen" <mb <at> mbrodersen.com> wrote:
> There are obviously a lot of people on this list interested in software
> architecture/design. Especially how to do it in a pragmatic/effective
> way with language such as C++.
>
> During the 15+ years I have been working in the games industry (and the
> 30 years I have been writing software), I had a lot of opportunities to
> experiment with software architectures for games (from the engine all
> the way up). Some architectures worked well, others less so :) But after
> many interesting experiments I ended up with a non-complex way to design
> software that works well for small and large scale applications. And it
> (surprisingly) works well not only for OO style languages but also for
> functional style languages (and multi-threaded applications!).
>
> I have no idea whether the approach will work for anybody else. I am
> definitely not making that claim. But hey it might be interesting for
> you so here it is:
>
> Think about your application as consisting of 2 types of objects: State
> Objects and Process Objects.
>
> A State Object (SO) handles a state (surprise). It can be anything from
> a texture to a database to a display to a list of integers. In an OO
> language, SO's will typically be implemened as classes/objects with the
> usual set/get/add/remove methods. In a functional language, SO's are
> typically implemented as (immutable) values.
>
> The Process Objects (PO) takes as input one or more input SO's and
> transforms/outputs it to one or more output SO's. In an OO language,
> PO's are typically implemented as classes/objects/static
> methods/functions. The PO's might use temporary data while processing
> input to output but it normally doesn't maintain a state. In a
> functional language PO's are typically implemented as functions.
>
> Large scale applications are constructed by joining SO's and PO's
> together in a graph structure:
>
> S0 -> [P0] -> S0 -> [P0] -> SO -> [PO] -> SO
>
> I can of course only show a linear chain in this email so imagine a
> number of boxes (SO's) and spheres (PO's) spread around on a piece of
> paper and lines going from (input) SO's to PO's and from PO's to
> (output) SO's. The graph typically have loops making it possible to (for
> example) feed the output of a game tick into the next game tick.
>
> The SO/PO approach can be used at any abstraction level. Here is (part
> of) a very high level architecture for a game:
>
> GameConfig -> [GameStateMaker] -> GameState
> GameConfig+TestConfig -> [GameStateForTesting] -> GameState
> NetworkMsg -> [GameStateFromNetworkMsg] -> GameState
> GameState+Time -> [GameTicker] -> GameState
> GameState -> [GameTester] -> GameTestReport
> GameState -> [GameSaver] -> GameFile
> GameFile -> [GameLoader] -> GameState
> GameState -> [GameStateToNetworkMsg] -> NetworkMsg
> GameState -> [GameStateVerifier] -> GameStateErrors
> GameStateErrors -> [GameStateErrorPrinter] -> Console
> GameStateErrors -> [GameStateWindowMaker] -> Window
> etc.
>
> And some low level ones:
>
> Camera+EntityList -> [VisibilityChecker] -> EntityList
> EntityList -> [EntityRender] -> DirectX
> EntityList -> [EntityPrinter] -> Console
> etc.
>
> In C++, you could for example implement the very high level architecture
> this way:
>
> static void Game::tick(GameState* gs, float time) { ... }
>
> The reason why the SO/PO approach works well is because the application
> ends up consisting of a flat collection of self contained SO's that are
> easy to understand, test and reuse. And a collection of PO's that also
> are easy to understand because they simply transform the input SO's to
> the output SO's without relying on any internal state.
>
> And because the application is broken down into a large number of SO's,
> you can typically add new functionality by adding an extra
> serial/parallel/concurrent transformation step to the architecture
> without having to touch the already working code. You usually do not end
> up in the "grab the banana and you get the whole jungle" sitation that
> is typical of C++ software.
>
> If a number of SO's inherit from the same SO interface, you can easily
> pick the best SO for the job without having to touch any of the
> surrounding PO code that use it.
>
> If SO's are organized as lists of simple values/objects, you get the
> cache performance advantage discuss previously in this email list.
>
> If some of the PO's are implemented as threads with the input/output
> SO's implemented as semaphore protected data/queues, you get a cleanly
> architectured multi-core/threaded application.
>
> Morten
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-Gamedev <at> lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Morten Brodersen | 12 Jul 2010 13:20

Re: State Object / Process Object design

* you mention that SO's can inherit from a common interface in order to reuse PO's -- but doesn't that undermine their state-only-ness?

It is fine for an SO to inherit from a common interface. It doesn't make it a non-state object. It just allows the same PO to use different SO implementations. For example, one SO can be memory based and another can be disk/network based. You might also have a SO that logs all method calls for debugging purposes.

* it sounds like this would result in a lot of dynamic allocation of SO's and SO collections, that would not occur in a more conventional OO implementation.  Is that true, and is it a problem?

Whether you keep allocating the same SO over and over again or simply keep one alive through the whole game is up to you. It is independent of whether you use the SO/PO pattern.

* what about a complicated persistent data structure like a spatial index, that refers to other objects in the game?  Is this allowed in an SO?  Is it an exception to this pattern?

A SO can be big or small. It doesn't matter. What matters is whether it is a SO or a PO. In the extreme case the whole game state can be a single SO. And there will be a number of PO's "operating" on it. However I wouldn't recommend that architecture because the PO's will have to do a lot of pointer walking down the SO tree before they get to the object they want to operate on. That is a bad idea for a lot of reasons (cache performance, lines of code etc.) and if you then change the structure of the SO, a lot of PO's will be impacted. The ideal is to have a number of clean standalone SO's with PO's transforming between them.

* is there a more detailed writeup or example system (I.e. with source code available for study) that illustrates this?

Not anything that is easily digestable unfortunately. What I would recommend is to simply try it out. The #1 idea is to stop thinking about creating a big object tree for your application. Instead, create a flat graph of objects with PO's connecting SO's. For example, for a game you might traditionally have designed an architecture like this: Boot->Application->Game->Level->Scene->PVS->Entity->Model->VertexBuffer->Vertex->Color->Red i.e. a top-down object tree with the boot code (Boot) as its root. Try instead to take the hierarchy and flatten it out so that you have a number of SO's and PO's that process and convert them. A PO that loads from a file (SO) and creates a model (SO). A PO that modifiers the model (SO) in various ways. A PO that converts the model (SO) to a DirectX Vertex/Index Buffer (2 SO's). A PO that takes a controller input (SO) and converts it to a game specific input (SO). A PO that creates a height map (SO) from an image (SO). A PO that converts a height map (SO) to a DirectX Vertex/Index Buffer. A PO that takes Vertex/Pixel shaders (SO) and VB/IB's (SO) and draws a model on a render target (SO) etc. A good way to learn the SO/PO approach is to take an object hierarchy from one of your application or the idea for an application you want to write and convert it to a flat PO/SO structure. Just use a drawing tool (or pen and paper for that matter) and use boxes for SO's and circles for PO's with arrows connecting them. It is a fun little exercise. And hey for the more advanced practicioners, to reach level 2 and become a true SOPO Ninja, think hard about how many of the PO's you designed that can fairly easily run in its own thread. The answer might surprise you.

* Thanks for any insights!

You are welcome :-)

Morten
 
-----Original Message-----
From: sweng-gamedev-bounces <at> lists.midnightryder.com [mailto:sweng-gamedev-bounces <at> lists.midnightryder.com] On Behalf Of Thatcher Ulrich
Sent: Monday, 12 July 2010 2:06 PM
To: sweng-gamedev <at> midnightryder.com
Subject: [Sweng-Gamedev] State Object / Process Object design

This sounds really interesting and enticing but I still have a lot of questions:

* you mention that SO's can inherit from a common interface in order to reuse PO's -- but doesn't that undermine their state-only-ness?

* it sounds like this would result in a lot of dynamic allocation of SO's and SO collections, that would not occur in a more conventional OO implementation.  Is that true, and is it a problem?

* what about a complicated persistent data structure like a spatial index, that refers to other objects in the game?  Is this allowed in an SO?  Is it an exception to this pattern?

* is there a more detailed writeup or example system (I.e. with source code available for study) that illustrates this?

Thanks for any insights!

-T

On Jun 3, 2010 11:01 AM, "Morten Brodersen" <mb <at> mbrodersen.com> wrote:
> There are obviously a lot of people on this list interested in software
> architecture/design. Especially how to do it in a pragmatic/effective
> way with language such as C++.
>
> During the 15+ years I have been working in the games industry (and the
> 30 years I have been writing software), I had a lot of opportunities to
> experiment with software architectures for games (from the engine all
> the way up). Some architectures worked well, others less so :) But after
> many interesting experiments I ended up with a non-complex way to design
> software that works well for small and large scale applications. And it
> (surprisingly) works well not only for OO style languages but also for
> functional style languages (and multi-threaded applications!).
>
> I have no idea whether the approach will work for anybody else. I am
> definitely not making that claim. But hey it might be interesting for
> you so here it is:
>
> Think about your application as consisting of 2 types of objects: State
> Objects and Process Objects.
>
> A State Object (SO) handles a state (surprise). It can be anything from
> a texture to a database to a display to a list of integers. In an OO
> language, SO's will typically be implemened as classes/objects with the
> usual set/get/add/remove methods. In a functional language, SO's are
> typically implemented as (immutable) values.
>
> The Process Objects (PO) takes as input one or more input SO's and
> transforms/outputs it to one or more output SO's. In an OO language,
> PO's are typically implemented as classes/objects/static
> methods/functions. The PO's might use temporary data while processing
> input to output but it normally doesn't maintain a state. In a
> functional language PO's are typically implemented as functions.
>
> Large scale applications are constructed by joining SO's and PO's
> together in a graph structure:
>
> S0 -> [P0] -> S0 -> [P0] -> SO -> [PO] -> SO
>
> I can of course only show a linear chain in this email so imagine a
> number of boxes (SO's) and spheres (PO's) spread around on a piece of
> paper and lines going from (input) SO's to PO's and from PO's to
> (output) SO's. The graph typically have loops making it possible to (for
> example) feed the output of a game tick into the next game tick.
>
> The SO/PO approach can be used at any abstraction level. Here is (part
> of) a very high level architecture for a game:
>
> GameConfig -> [GameStateMaker] -> GameState
> GameConfig+TestConfig -> [GameStateForTesting] -> GameState
> NetworkMsg -> [GameStateFromNetworkMsg] -> GameState
> GameState+Time -> [GameTicker] -> GameState
> GameState -> [GameTester] -> GameTestReport
> GameState -> [GameSaver] -> GameFile
> GameFile -> [GameLoader] -> GameState
> GameState -> [GameStateToNetworkMsg] -> NetworkMsg
> GameState -> [GameStateVerifier] -> GameStateErrors
> GameStateErrors -> [GameStateErrorPrinter] -> Console
> GameStateErrors -> [GameStateWindowMaker] -> Window
> etc.
>
> And some low level ones:
>
> Camera+EntityList -> [VisibilityChecker] -> EntityList
> EntityList -> [EntityRender] -> DirectX
> EntityList -> [EntityPrinter] -> Console
> etc.
>
> In C++, you could for example implement the very high level architecture
> this way:
>
> static void Game::tick(GameState* gs, float time) { ... }
>
> The reason why the SO/PO approach works well is because the application
> ends up consisting of a flat collection of self contained SO's that are
> easy to understand, test and reuse. And a collection of PO's that also
> are easy to understand because they simply transform the input SO's to
> the output SO's without relying on any internal state.
>
> And because the application is broken down into a large number of SO's,
> you can typically add new functionality by adding an extra
> serial/parallel/concurrent transformation step to the architecture
> without having to touch the already working code. You usually do not end
> up in the "grab the banana and you get the whole jungle" sitation that
> is typical of C++ software.
>
> If a number of SO's inherit from the same SO interface, you can easily
> pick the best SO for the job without having to touch any of the
> surrounding PO code that use it.
>
> If SO's are organized as lists of simple values/objects, you get the
> cache performance advantage discuss previously in this email list.
>
> If some of the PO's are implemented as threads with the input/output
> SO's implemented as semaphore protected data/queues, you get a cleanly
> architectured multi-core/threaded application.
>
> Morten
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-Gamedev <at> lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Thatcher Ulrich | 12 Jul 2010 21:07

Re: State Object / Process Object design

On Mon, Jul 12, 2010 at 7:20 AM, Morten Brodersen <mb <at> mbrodersen.com> wrote:
> * you mention that SO's can inherit from a common interface in order to
> reuse PO's -- but doesn't that undermine their state-only-ness?
>
> It is fine for an SO to inherit from a common interface. It doesn't make it
> a non-state object. It just allows the same PO to use different SO
> implementations. For example, one SO can be memory based and another can be
> disk/network based. You might also have a SO that logs all method calls for
> debugging purposes.
>
> * it sounds like this would result in a lot of dynamic allocation of SO's
> and SO collections, that would not occur in a more conventional OO
> implementation.  Is that true, and is it a problem?
>
> Whether you keep allocating the same SO over and over again or simply keep
> one alive through the whole game is up to you. It is independent of whether
> you use the SO/PO pattern.
>
> * what about a complicated persistent data structure like a spatial index,
> that refers to other objects in the game?  Is this allowed in an SO?  Is it
> an exception to this pattern?
>
> A SO can be big or small. It doesn't matter. What matters is whether it is a
> SO or a PO. In the extreme case the whole game state can be a single SO. And
> there will be a number of PO's "operating" on it. However I wouldn't
> recommend that architecture because the PO's will have to do a lot of
> pointer walking down the SO tree before they get to the object they want to
> operate on. That is a bad idea for a lot of reasons (cache performance,
> lines of code etc.) and if you then change the structure of the SO, a lot of
> PO's will be impacted. The ideal is to have a number of clean standalone
> SO's with PO's transforming between them.
>
> * is there a more detailed writeup or example system (I.e. with source code
> available for study) that illustrates this?
>
> Not anything that is easily digestable unfortunately. What I would recommend
> is to simply try it out. The #1 idea is to stop thinking about creating a
> big object tree for your application. Instead, create a flat graph of
> objects with PO's connecting SO's. For example, for a game you might
> traditionally have designed an architecture like this:
> Boot->Application->Game->Level->Scene->PVS->Entity->Model->VertexBuffer->Vertex->Color->Red
> i.e. a top-down object tree with the boot code (Boot) as its root. Try
> instead to take the hierarchy and flatten it out so that you have a number
> of SO's and PO's that process and convert them. A PO that loads from a file
> (SO) and creates a model (SO). A PO that modifiers the model (SO) in various
> ways. A PO that converts the model (SO) to a DirectX Vertex/Index Buffer (2
> SO's). A PO that takes a controller input (SO) and converts it to a game
> specific input (SO). A PO that creates a height map (SO) from an image (SO).
> A PO that converts a height map (SO) to a DirectX Vertex/Index Buffer. A PO
> that takes Vertex/Pixel shaders (SO) and VB/IB's (SO) and draws a model on a
> render target (SO) etc. A good way to learn the SO/PO approach is to take an
> object hierarchy from one of your application or the idea for an application
> you want to write and convert it to a flat PO/SO structure. Just use a
> drawing tool (or pen and paper for that matter) and use boxes for SO's and
> circles for PO's with arrows connecting them. It is a fun little exercise.
> And hey for the more advanced practicioners, to reach level 2 and become a
> true SOPO Ninja, think hard about how many of the PO's you designed that can
> fairly easily run in its own thread. The answer might surprise you.

Another question that jumps to mind: is a PO the same as a non-member
function?  Or might they ever have state?

I'm leaning towards giving this a try but I'm still a little fuzzy on
the differences (if any) between this and non-OO procedural
programming, or OO programming with very flat inheritance.

-T

>
> * Thanks for any insights!
>
> You are welcome :-)
>
> Morten
>
> -----Original Message-----
> From: sweng-gamedev-bounces <at> lists.midnightryder.com
> [mailto:sweng-gamedev-bounces <at> lists.midnightryder.com] On Behalf Of Thatcher
> Ulrich
> Sent: Monday, 12 July 2010 2:06 PM
> To: sweng-gamedev <at> midnightryder.com
> Subject: [Sweng-Gamedev] State Object / Process Object design
>
> This sounds really interesting and enticing but I still have a lot of
> questions:
>
> * you mention that SO's can inherit from a common interface in order to
> reuse PO's -- but doesn't that undermine their state-only-ness?
>
> * it sounds like this would result in a lot of dynamic allocation of SO's
> and SO collections, that would not occur in a more conventional OO
> implementation.  Is that true, and is it a problem?
>
> * what about a complicated persistent data structure like a spatial index,
> that refers to other objects in the game?  Is this allowed in an SO?  Is it
> an exception to this pattern?
>
> * is there a more detailed writeup or example system (I.e. with source code
> available for study) that illustrates this?
>
> Thanks for any insights!
>
> -T
>
> On Jun 3, 2010 11:01 AM, "Morten Brodersen" <mb <at> mbrodersen.com> wrote:
>> There are obviously a lot of people on this list interested in software
>> architecture/design. Especially how to do it in a pragmatic/effective
>> way with language such as C++.
>>
>> During the 15+ years I have been working in the games industry (and the
>> 30 years I have been writing software), I had a lot of opportunities to
>> experiment with software architectures for games (from the engine all
>> the way up). Some architectures worked well, others less so :) But after
>> many interesting experiments I ended up with a non-complex way to design
>> software that works well for small and large scale applications. And it
>> (surprisingly) works well not only for OO style languages but also for
>> functional style languages (and multi-threaded applications!).
>>
>> I have no idea whether the approach will work for anybody else. I am
>> definitely not making that claim. But hey it might be interesting for
>> you so here it is:
>>
>> Think about your application as consisting of 2 types of objects: State
>> Objects and Process Objects.
>>
>> A State Object (SO) handles a state (surprise). It can be anything from
>> a texture to a database to a display to a list of integers. In an OO
>> language, SO's will typically be implemened as classes/objects with the
>> usual set/get/add/remove methods. In a functional language, SO's are
>> typically implemented as (immutable) values.
>>
>> The Process Objects (PO) takes as input one or more input SO's and
>> transforms/outputs it to one or more output SO's. In an OO language,
>> PO's are typically implemented as classes/objects/static
>> methods/functions. The PO's might use temporary data while processing
>> input to output but it normally doesn't maintain a state. In a
>> functional language PO's are typically implemented as functions.
>>
>> Large scale applications are constructed by joining SO's and PO's
>> together in a graph structure:
>>
>> S0 -> [P0] -> S0 -> [P0] -> SO -> [PO] -> SO
>>
>> I can of course only show a linear chain in this email so imagine a
>> number of boxes (SO's) and spheres (PO's) spread around on a piece of
>> paper and lines going from (input) SO's to PO's and from PO's to
>> (output) SO's. The graph typically have loops making it possible to (for
>> example) feed the output of a game tick into the next game tick.
>>
>> The SO/PO approach can be used at any abstraction level. Here is (part
>> of) a very high level architecture for a game:
>>
>> GameConfig -> [GameStateMaker] -> GameState
>> GameConfig+TestConfig -> [GameStateForTesting] -> GameState
>> NetworkMsg -> [GameStateFromNetworkMsg] -> GameState
>> GameState+Time -> [GameTicker] -> GameState
>> GameState -> [GameTester] -> GameTestReport
>> GameState -> [GameSaver] -> GameFile
>> GameFile -> [GameLoader] -> GameState
>> GameState -> [GameStateToNetworkMsg] -> NetworkMsg
>> GameState -> [GameStateVerifier] -> GameStateErrors
>> GameStateErrors -> [GameStateErrorPrinter] -> Console
>> GameStateErrors -> [GameStateWindowMaker] -> Window
>> etc.
>>
>> And some low level ones:
>>
>> Camera+EntityList -> [VisibilityChecker] -> EntityList
>> EntityList -> [EntityRender] -> DirectX
>> EntityList -> [EntityPrinter] -> Console
>> etc.
>>
>> In C++, you could for example implement the very high level architecture
>> this way:
>>
>> static void Game::tick(GameState* gs, float time) { ... }
>>
>> The reason why the SO/PO approach works well is because the application
>> ends up consisting of a flat collection of self contained SO's that are
>> easy to understand, test and reuse. And a collection of PO's that also
>> are easy to understand because they simply transform the input SO's to
>> the output SO's without relying on any internal state.
>>
>> And because the application is broken down into a large number of SO's,
>> you can typically add new functionality by adding an extra
>> serial/parallel/concurrent transformation step to the architecture
>> without having to touch the already working code. You usually do not end
>> up in the "grab the banana and you get the whole jungle" sitation that
>> is typical of C++ software.
>>
>> If a number of SO's inherit from the same SO interface, you can easily
>> pick the best SO for the job without having to touch any of the
>> surrounding PO code that use it.
>>
>> If SO's are organized as lists of simple values/objects, you get the
>> cache performance advantage discuss previously in this email list.
>>
>> If some of the PO's are implemented as threads with the input/output
>> SO's implemented as semaphore protected data/queues, you get a cleanly
>> architectured multi-core/threaded application.
>>
>> Morten
>>
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> Sweng-Gamedev <at> lists.midnightryder.com
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
> _______________________________________________
> Sweng-Gamedev mailing list
> Sweng-Gamedev <at> lists.midnightryder.com
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com


Gmane