gmane.comp.lang.smalltalk.squeak.general

Maintenance of remote references

Hi there,

	Thing. I'm working in the RemoteObjects package that is a sort of RMI or
RPC but for real objects ;) Right now is working > x2 than rST in localhost. It
was designed in a P2P fashion and it allows to have N brokers even inside the
same image. 

	Purpose. Scaling Squeak today.

	Plan. Some of you helped with ideas and I plan to publish in squeak
source (MIT) when minimally stability is reached.

	Status. Remote inspectors just started to work and I need to start
thinking about making something for the maintenance of its cache.

	Problem. A broker can publish some objects but, as the communication get
used, remote references start to be holded by them. Thats fine until you think
in how to claim space without injecting troubles.

	Reference. As benchmarking I saw the rST way and I'm not satisfied with
its model. Some objects end up being missed with time. 

	Ideas. So I though about 1 making a kind of watchdog to maintain "alive"
the remote references renovating the expiration period every time a remote
message reach the object. Also I'm thinking in 2 the idea of the session. So
when session is expired a whole group of references is claimed.

	Hears. Anyway I'm in analisys phase for this problem so I want to hear
more ideas. So thinking loud is welcome.

(Continue reading)

headers

Klaus D. Witzel | 18 Jul 20:30

Re: Maintenance of remote references

Hi Sebastian,

on Fri, 18 Jul 2008 19:09:37 +0200, you wrote:

> Hi there,
>
> 	Thing. I'm working in the RemoteObjects package that is a sort of RMI or
> RPC but for real objects ;) Right now is working > x2 than rST in  
> localhost. It
> was designed in a P2P fashion and it allows to have N brokers even  
> inside the
> same image.
>
> 	Purpose. Scaling Squeak today.

Great :)

...
> 	Hears. Anyway I'm in analisys phase for this problem so I want to hear
> more ideas. So thinking loud is welcome.

Have you considered how to avoid cyclic remote references, anything  
positive into this direction would be valuable for (currently) a small  
group of people ;) TIA.

We discuss and make plans for implementing models analogous to yours, with  
multiple Smalltalk .images, each with own native thread, on n-core  
machines. We recently found one model which apparently cannot avoid cyclic  
remote references :(

(Continue reading)

headers

Sebastian Sastre | 18 Jul 21:06

RE: Re: Maintenance of remote references


>I want to hear
> > more ideas. So thinking loud is welcome.
> 
> Have you considered how to avoid cyclic remote references, anything  
> positive into this direction would be valuable for 
> (currently) a small  
> group of people ;) TIA.
> 
> We discuss and make plans for implementing models analogous 
> to yours, with  
> multiple Smalltalk .images, each with own native thread, on n-core  
> machines. We recently found one model which apparently cannot 
> avoid cyclic  
> remote references :(
> 
> /Klaus
> 
Hi Klaus,

glad to hear that I'm not alone with this industrial crisis. The only thing I
can think of is using sessions. So when a session is closed or expires the
system notifies remote brokers telling the group of ids of objects to be
considered free for that session (and of course eventually claimed if no other
session is referencing them). Maybe you want/need a more general approach? If we
deny to give up to consistency I don't see lots of alternatives. I'm starting to
wonder what happen in real world to solve this problematic pattern-problem...
hhmmm...

Sebastian

(Continue reading)

headers

Adrian Lienhard | 18 Jul 22:10

Re: Re: Maintenance of remote references


On Jul 18, 2008, at 21:06 , Sebastian Sastre wrote:

> glad to hear that I'm not alone with this industrial crisis. The  
> only thing I
> can think of is using sessions. So when a session is closed or  
> expires the
> system notifies remote brokers telling the group of ids of objects  
> to be
> considered free for that session (and of course eventually claimed  
> if no other
> session is referencing them). Maybe you want/need a more general  
> approach? If we
> deny to give up to consistency I don't see lots of alternatives. I'm  
> starting to
> wonder what happen in real world to solve this problematic pattern- 
> problem...
> hhmmm...

For what its worth, there was a paper with the title "Programming with  
Live Distributed Objects" published at this year's ECOOP. Not sure,  
though, if/how they solve the problem.

Cheers,
Adrian

headers

Klaus D. Witzel | 19 Jul 07:00

Re: Maintenance of remote references

On Fri, 18 Jul 2008 22:10:05 +0200, Adrian Lienhard wrote:

> On Jul 18, 2008, at 21:06 , Sebastian Sastre wrote:
>
>> glad to hear that I'm not alone with this industrial crisis. The only  
>> thing I
>> can think of is using sessions. So when a session is closed or expires  
>> the
>> system notifies remote brokers telling the group of ids of objects to be
>> considered free for that session (and of course eventually claimed if  
>> no other
>> session is referencing them). Maybe you want/need a more general  
>> approach? If we
>> deny to give up to consistency I don't see lots of alternatives. I'm  
>> starting to
>> wonder what happen in real world to solve this problematic pattern- 
>> problem...
>> hhmmm...
>
> For what its worth, there was a paper with the title "Programming with  
> Live Distributed Objects" published at this year's ECOOP. Not sure,  
> though, if/how they solve the problem.

Thanks for the pointer! Unfortunately words like "reachable",  
"unreachable", "circular", etc do not appear :( FWIW, it's more about  
type-safe drag&drop for some OLE-ish development framework but in  
javaneese, e.g. if(a type is B) etc :( But I will check the referred-to  
papers as well. Thanks.

/Klaus

(Continue reading)

headers

stephane ducasse | 20 Jul 21:26

Re: Re: Maintenance of remote references

What is live distributed objects?
What was the punch line.

Stef
>> glad to hear that I'm not alone with this industrial crisis. The  
>> only thing I
>> can think of is using sessions. So when a session is closed or  
>> expires the
>> system notifies remote brokers telling the group of ids of objects  
>> to be
>> considered free for that session (and of course eventually claimed  
>> if no other
>> session is referencing them). Maybe you want/need a more general  
>> approach? If we
>> deny to give up to consistency I don't see lots of alternatives.  
>> I'm starting to
>> wonder what happen in real world to solve this problematic pattern- 
>> problem...
>> hhmmm...
>
> For what its worth, there was a paper with the title "Programming  
> with Live Distributed Objects" published at this year's ECOOP. Not  
> sure, though, if/how they solve the problem.
>
> Cheers,
> Adrian
>
>

(Continue reading)

headers

Klaus D. Witzel | 19 Jul 07:32

Re: Re: Maintenance of remote references

On Fri, 18 Jul 2008 21:06:55 +0200, Sebastian Sastre wrote:

>> I want to hear
>> > more ideas. So thinking loud is welcome.
>>
>> Have you considered how to avoid cyclic remote references, anything
>> positive into this direction would be valuable for
>> (currently) a small
>> group of people ;) TIA.
>>
>> We discuss and make plans for implementing models analogous
>> to yours, with
>> multiple Smalltalk .images, each with own native thread, on n-core
>> machines. We recently found one model which apparently cannot
>> avoid cyclic
>> remote references :(
>>
>> /Klaus
>>
> Hi Klaus,
>
> glad to hear that I'm not alone with this industrial crisis. The only  
> thing I
> can think of is using sessions. So when a session is closed or expires  
> the
> system notifies remote brokers telling the group of ids of objects to be
> considered free for that session (and of course eventually claimed if no  
> other session is referencing them).

There's one problem with session expirency: you never know why some

(Continue reading)

headers

Sebastian Sastre | 19 Jul 15:06

RE: Re: Re: Maintenance of remote references


> Interesting point. Perhaps consistency can only be related to 
> autonomy?  
> Then we'd have analogy with Http and URLs and service 
> endpoints -- most  
> often they work but are not guaranteed to always work.
>
Automomy yes. One of the principles to honour for scalable design. Due to the
CAP theorem (see http://citeseer.ist.psu.edu/544596.html) my trade off is on
availability and partition tolerance so giving up on consistency.

That relinquish of consistency cames with a price: contention measures.
So this system will work ruled by the economy of "often works and do something
on exceptions" as you well said and with flexible policies of contention so the
developer have choices.

What I do like of this is the similarity with the real world domains: we live in
societies that often works. Err.. o.O  

...ok here a less controversial one from:
http://www.enterpriseintegrationpatterns.com/ramblings/18_starbucks.html
"...If they deliver you a drink that is incorrect or nonsatisfactory they will
remake it. If the machine breaks down and they cannot make your drink they will
refund your money. Each of these scenarios describes a different, but common
error handling strategy..."

The fact is that they will often make (so sell) the right drinks.

To deliver coffe like drinks is very application level and the error handling
responsibility should be transferred there. That's why I choosed to make it as

(Continue reading)

headers

Michael van der Gulik | 30 Jul 08:37

Re: Maintenance of remote references

On Fri, 18 Jul 2008 14:09:37 -0300
"Sebastian Sastre" <ssastre <at> seaswork.com> wrote:

> Hi there,
> 
> 	Thing. I'm working in the RemoteObjects package that is a sort of RMI or
> RPC but for real objects ;) Right now is working > x2 than rST in localhost. It
> was designed in a P2P fashion and it allows to have N brokers even inside the
> same image. 
> 
> 	Purpose. Scaling Squeak today.

What do you mean by "Scaling Squeak"?

Are you talking about being able to scale the image on a single machine with multiple CPUs? Or scaling an
image to run on a cluster of servers? Or are you talking about a wide-area distributed network of some sort?

If this is a business requirement, you could look at GemStone.

> 	Problem. A broker can publish some objects but, as the communication get
> used, remote references start to be holded by them. Thats fine until you think
> in how to claim space without injecting troubles.

This is a field of research in distributed systems: "Distributed Garbage Collection".

One algorithm would be to use backtracing GC. Essentially, every remote object contains a list of other
remote objects that refer to it. It then becomes possible to determine whether a given object is garbage by
tracing back through all of the backpointers to see if you can reach a known non-garbage object. Loops can
be detected by marking the objects somehow. This GC algorithm can also have multiple garbage collectors
running on it concurrently.

(Continue reading)