Sterling Clover | 30 Aug 20:52
Picon

Where STM is unstable at the moment, and how we can fix it

This email is inspired by the discussion here: http:// 
hackage.haskell.org/trac/ghc/ticket/2401

As the ticket discusses, unsafeIOToSTM is, unlike unsafePerformIO or  
unsafeInterleaveIO, genuinely completely unsafe in that there is no  
way to use it such that a segfault or deadlock is not at least  
somewhat encouraged. The code attached to the ticket creates a  
deadlock solely through using it to write to stdout. But, for the  
same reason that unsafeIOToSTM is unstable, unsafeInterleaveIO now is  
very unstable as well -- conceivably, data generated from functions  
with lazy IO (including those in the prelude) could cause deadlocks  
within STM, and even segfaults.

In summary, a "validation" step is performed on all threads inside  
atomically blocks during garbage collection. This validation step  
will, on encountering invalid threads (i.e. ones which should be  
rolled back) immediately kill them dead and retry. This is different  
than the implementation described in the STM paper, where rollbacks  
only occur on commit. However, it does add a measure of efficiency.  
The problem is that the validation code disregards exception  
handlers, since rollback is not an exception, and so anything  
embedded in STM that brackets an IO action, for example, can be  
rolled back without the final part of the exception even being called.

As Simon M. notes, the obvious solution would be to turn rollbacks  
into regular exceptions, but this would open a number of cans of worms.

A start, though not sufficient, would be for stm validation to  
respect blocked status -- not to block on it, obviously, but simply  
to refuse to rollback a transaction within it. Validation on GC is,  
(Continue reading)

Simon Marlow | 1 Sep 11:39
Picon

Re: Where STM is unstable at the moment, and how we can fix it

Sterling Clover wrote:
> This email is inspired by the discussion here: 
> http://hackage.haskell.org/trac/ghc/ticket/2401
> 
> As the ticket discusses, unsafeIOToSTM is, unlike unsafePerformIO or 
> unsafeInterleaveIO, genuinely completely unsafe in that there is no way 
> to use it such that a segfault or deadlock is not at least somewhat 
> encouraged. The code attached to the ticket creates a deadlock solely 
> through using it to write to stdout. But, for the same reason that 
> unsafeIOToSTM is unstable, unsafeInterleaveIO now is very unstable as 
> well -- conceivably, data generated from functions with lazy IO 
> (including those in the prelude) could cause deadlocks within STM, and 
> even segfaults.
> 
> In summary, a "validation" step is performed on all threads inside 
> atomically blocks during garbage collection. This validation step will, 
> on encountering invalid threads (i.e. ones which should be rolled back) 
> immediately kill them dead and retry. This is different than the 
> implementation described in the STM paper, where rollbacks only occur on 
> commit. However, it does add a measure of efficiency.

Its not just an efficiency trick, in fact.  The validation step is 
absolutely necessary for correctness.  The problem is that a transaction 
may have seen an inconsistent view of memory, and as a result it may have 
gone into an infinite loop; the only way to catch and recover from this 
situation is to validate at regular intervals, say before a GC (this 
suffers from the problem that the transaction has to be allocating in order 
to be stopped, but that's another matter).  e.g. the code might be 
something like

(Continue reading)


Gmane