Matthew Mondor | 6 Apr 2012 14:32

Re: Locking strategy (best so far)

On Fri, 6 Apr 2012 13:02:16 +0200
Juan Jose Garcia-Ripoll <juanjose.garciaripoll@...> wrote:

> There were a bunch of problems with the previous code. From a wrong order
> in the arguments to pthread_sigmask() to the choice of signals. I have
> fixed them together with the following things
> 
> * Implemented semaphores
> * Implemented barriers (copied part of the API from Lispworks, though I
> feel it is overengineered)
> * mp:process-join relies on barriers (no trivial thing, the original code
> was a mess because of the lack of this tool)
> * fixes in external processes, locking, etc.
> 
> In particular the changes to mp:process-join are quite critical for ECL
> being able to gracefully quit when there are running threads.
> 
> I can confirm that I am able to run your server and get decent connection
> times, slightly better than on OS X (see below). However, please bear in
> mind that I am running this all inside a virtualbox which may improve the
> stability, as processors are not really independent, but bound by the
> emulator.

I confirm that these changes allow the HTTPd to run here as well,
thanks!  However, I didn't stress-test it much or try performance tests
with it yet, but that should happen soon.
--

-- 
Matt

------------------------------------------------------------------------------
(Continue reading)

Juan Jose Garcia-Ripoll | 6 Apr 2012 16:24

Re: Locking strategy (best so far)

On Fri, Apr 6, 2012 at 2:32 PM, Matthew Mondor <mm_lists <at> pulsar-zone.net> wrote:
I confirm that these changes allow the HTTPd to run here as well,
thanks!  However, I didn't stress-test it much or try performance tests
with it yet, but that should happen soon.

I am now running some stress tests and trying to hit some corner cases. There are already some subtle race conditions in the lisp part of the code. Hopefully will be easy to fix...

Juanjo

--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ecls-list mailing list
Ecls-list@...
https://lists.sourceforge.net/lists/listinfo/ecls-list
Matthew Mondor | 7 Apr 2012 19:42

Re: Locking strategy (best so far)

On Fri, 6 Apr 2012 16:24:49 +0200
Juan Jose Garcia-Ripoll <juanjose.garciaripoll@...> wrote:

> On Fri, Apr 6, 2012 at 2:32 PM, Matthew Mondor <mm_lists@...>wrote:
> 
> > I confirm that these changes allow the HTTPd to run here as well,
> > thanks!  However, I didn't stress-test it much or try performance tests
> > with it yet, but that should happen soon.
> 
> 
> I am now running some stress tests and trying to hit some corner cases.
> There are already some subtle race conditions in the lisp part of the code.
> Hopefully will be easy to fix...

With some tests outside of Slime as I have trouble with Slime on recent
HEAD:

I can use ab(8) a few runs of -c16 -n1000 with good performance (2k+
requests served per second).  After a few runs, one of the threads (the
first try it was the children manager one, at the second try it was one
of the server thread) gets stuck into a busy loop inside the GC.  I
also saw a bunch of stack overflow errors at the REPL.

A backtrace of the busy-looping thread:

#0  handle_or_queue (the_env=0x7f7ff7fc6000, signal_code=0x7f7ff7a3ccf8, code=11) at /home/mmondor/work/ecl-git/ecl/src/c/unixint.d:500
#1  <signal handler called>
#2  0x00007f7ff5e1211c in GC_mark_from () from /usr/pkg/lib/libgc.so.1
#3  0x00007f7ff5e135fd in GC_mark_some () from /usr/pkg/lib/libgc.so.1
#4  0x00007f7ff5e0bc3f in GC_stopped_mark () from /usr/pkg/lib/libgc.so.1
#5  0x00007f7ff5e0c2ea in GC_try_to_collect_inner () from /usr/pkg/lib/libgc.so.1
#6  0x00007f7ff5e0cc20 in GC_collect_or_expand () from /usr/pkg/lib/libgc.so.1
#7  0x00007f7ff5e1088c in GC_alloc_large () from /usr/pkg/lib/libgc.so.1
#8  0x00007f7ff5e11207 in GC_generic_malloc_ignore_off_page () from /usr/pkg/lib/libgc.so.1
#9  0x00007f7ff7774089 in ecl_alloc (n=16384) at /home/mmondor/work/ecl-git/ecl/src/c/alloc_2.d:717
#10 0x00007f7ff776403e in cl__make_hash_table (test=<optimized out>, size=<optimized out>,
rehash_size=0x2b38280, rehash_threshold=<optimized out>)
    at /home/mmondor/work/ecl-git/ecl/src/c/hash.d:888
#11 0x00007f7ff7764312 in cl_make_hash_table (narg=<optimized out>) at /home/mmondor/work/ecl-git/ecl/src/c/hash.d:745
#12 0x00007f7ff281031e in L47make_http_request (narg=0) at test-httpd.c:5769
#13 0x00007f7ff282125e in L68http_request_parse (V1=0x10270d71, V2=0x1ea58c0) at test-httpd.c:6845
#14 0x00007f7ff281a407 in L85http_serve (V1=0xb7d5f90) at test-httpd.c:9136
#15 0x00007f7ff241ebd3 in L57accept_loop_thread (V1=0xc409e40) at ecl-mp-server.c:6497
#16 0x00007f7ff771d04d in cl_apply (narg=<optimized out>, fun=0x1f27200, lastarg=<optimized out>)
at /home/mmondor/work/ecl-git/ecl/src/c/eval.d:166
#17 0x00007f7ff7770842 in thread_entry_point (arg=<optimized out>) at /home/mmondor/work/ecl-git/ecl/src/c/threads/process.d:256
#18 0x00007f7ff5e17fb5 in GC_inner_start_routine () from /usr/pkg/lib/libgc.so.1
#19 0x00007f7ff5e14b60 in GC_call_with_stack_base () from /usr/pkg/lib/libgc.so.1
#20 0x00007f7ff7209dbd in pthread__create_tramp (cookie=0x7f7fea800000) at /usr/src/lib/libpthread/pthread.c:492
#21 0x00007f7ff6a76ef0 in ___lwp_park50 () from /usr/lib/libc.so.12
#22 0x00007f7feac00000 in ?? ()
#23 0x00007f7ff7ff60c0 in ?? ()
#24 0x0000000111110001 in ?? ()
#25 0x0000000033330003 in ?? ()
#26 0x0000000000000000 in ?? ()

Thanks,
--

-- 
Matt

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2

Gmane