Re: Random crashing
More debug info... Sorry about the multiple emails, I'm figuring
things out as I go.
(gdb) info threads
9 Thread 0x40263950 (LWP 3271) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
8 Thread 0x404c7950 (LWP 3272) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
7 Thread 0x4072b950 (LWP 3305) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
6 Thread 0x4098f950 (LWP 3306) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
5 Thread 0x40bf3950 (LWP 3307) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
4 Thread 0x40e57950 (LWP 6093) 0x00002adafe591bfb in read () from /
lib64/libc.so.6
3 Thread 0x4131f950 (LWP 6094) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
2 Thread 0x410bb950 (LWP 6095) 0x00002adafe2ca2cb in sem_timedwait
() from /lib64/libpthread.so.0
1 Thread 0x2adafe820880 (LWP 3268) 0x00002adafe2ca2cb in
sem_timedwait () from /lib64/libpthread.so.0
(gdb) thread 4
[Switching to thread 4 (Thread 0x40e57950 (LWP 6093))]#0
0x00002adafe591bfb in read () from /lib64/libc.so.6
(gdb) bt
#0 0x00002adafe591bfb in read () from /lib64/libc.so.6
#1 0x00002adafe545553 in _IO_file_underflow () from /lib64/libc.so.6
#2 0x00002adafe545d0e in _IO_default_uflow () from /lib64/libc.so.6
#3 0x00002adafe541404 in getc () from /lib64/libc.so.6
#4 0x000000000041d43d in readc () at /usr/include/bits/stdio.h:43
#5 0x000000000041d590 in lisp_Debugger (xp=0x40e55d60,
info=0x40e56110, why=11, in_foreign_code=1, message=0x40e55b10
"Unhandled exception 11 at 0x2adafe2ca325, context->regs at
#x40e55d88") at ../lisp-debug.c:914
#6 0x000000000041a2c6 in signal_handler (signum=11, info=0x40e56110,
context=0x40e55d60, tcr=0x40e577d0, old_valence=1) at ../x86-
exceptions.c:1070
#7 <signal handler called>
#8 0x00002adafe2ca325 in sem_post () from /lib64/libpthread.so.0
#9 0x000000000041b3e2 in resume_tcr (tcr=0x415837d0) at ../
thread_manager.c:1376
#10 0x000000000041c146 in lisp_resume_tcr (tcr=0x415837d0) at ../
thread_manager.c:1418
#11 0x000000000041a0c8 in handle_exception (signum=<value optimized
out>, info=0x40e56aa0, context=0x40e566f0, tcr=0x40e577d0,
old_valence=0) at ../x86-exceptions.c:910
#12 0x000000000041a218 in signal_handler (signum=4, info=0x40e56aa0,
context=0x40e566f0, tcr=0x40e577d0, old_valence=0) at ../x86-
exceptions.c:1064
#13 <signal handler called>
#14 0x00003000400110ab in ?? ()
#15 0x000030004042660c in ?? ()
#16 0x000000000040e0ac in _SPnthrowvalues () at ../x86-spentry64.s:1404
#17 0x00002aaaad1b1110 in ?? ()
#18 0x0000000000000008 in ?? ()
#19 0x0000000000000000 in ?? ()
(gdb)
On Jul 17, 2008, at 3:54 PM, Gary Byers wrote:
>
>
> On Thu, 17 Jul 2008, Osei Poku wrote:
>
>> Hello,
>>
>> I updated today from svn but this thing happened again. Again the
>> PC was in the pthread memory region and %rdi was 0. I verified
>> that the fix (r9997 i think) was in my ccl working directory
>> (somewhere in thread_manager.c right?).
>
> Yes; there are 3 calls to pthread_kill() in that file. One of them
> (in resume_tcr()) is conditionlized out; the other two
> (in raise_thread_interrupt() and suspend_tcr()) should check
> to make sure that the thread that they'd pass as the first
> argument to pthread_kill is non-zero before doing the call.)
>
>>
>> My current version is:
>> Clozure Common Lisp Version 1.2-r10073M-RC1 (LinuxX8664)!
>>
>> Is there anything other than (rebuild-ccl :force t) that I need to
>> do to recompile the c source for the lisp kernel?
>
> As Gail just pointed out, :full t (or :kernel t) is necessary
> in order to get the kernel updated. (:force t will recompile
> FASLs even if they're newer than the corresponding source;
> that's occasionally useful, but not really what you want here.)
>
> If the kernel that you're running had its modified date change
> by the rebuild process, it likely incorporates those changes. If
> those changes didn't fix the problem, then I don't have a good
> guess as to what the problem is: there aren't too many places
> where the lisp calls into the threads library: it creates threads
> and sends them signals via pthread_kill(). (There's another place
> where a thread will send itself a signal via pthread_kill(),
> but that is pretty much guaranteed to be a valid thread ...)
>
>
>>
>> Thanks,
>> Osei
>>
>> On Jul 9, 2008, at 3:05 PM, Gary Byers wrote:
>>
>>> --On July 9, 2008 2:26:56 PM -0400 Osei Poku <osei.poku <at> gmail.com>
>>> wrote:
>>>> Hi,
>>>> It crashed again for me. This time I managed to grab the
>>>> contents of
>>>> /proc/pid/maps before I killed it. Logs of the tty session and
>>>> memory
>>>> maps are attached. I had also managed to update from the
>>>> repository to
>>>> r9890-RC1.
>>>> Osei
>>> It seems to be crashed in the threads library (libpthread.so).
>>> There's a race condition in the code which suspends threads
>>> on entry to the GC: the thread that's running the GC looks
>>> at each thread that it wants to suspend to see if it's
>>> still alive (the data structure that represents a thread
>>> might still be around, even if the OS-level thread has
>>> exited.) The suspending thread looks at the tcr->osid
>>> field of the target, notes that it's non-zero, then
>>> calls a function to send the os-level thread a signal.
>>> That function accesses the tcr->osid field again (which,
>>> when non-zero, represents a POSIX thread ID) and calls
>>> pthread_kill()).
>>> When a thread dies, it clears its tcr->osid field, so
>>> if the target thread dies between the point when the
>>> suspending thread looks and the point where it leaps,
>>> we wind up calling pthread_kill() with a first argument
>>> of 0, and it crashes. That's consistent with the
>>> register information: we're somewhere in the threads
>>> library (possibly in pthread_kill()), and the register
>>> in which C functions receive their first argument (%rdi)
>>> is 0.
>>> I'll try to check in a fix for that (look before leaping)
>>> soon. As I understand it, SLIME will sometimes (depending
>>> on the setting of a "communication style" variable)
>>> spawn a thread in which to run each form being evaluated
>>> (via C-M-x or whatever); whether that's a good idea or
>>> not, consing short-lived threads all the time is probably
>>> a good way to trigger this bug. I don't use SLIME, and
>>> don't know what the consequences of changing the communication
>>> style variable would be.
>>
RSS Feed