Announcements, discussion, questions, bug reports, or whatever related to the development of Gauche

Vitaly Magerya | 23 May 2012 15:24

Picon

Gauche fails to build on FreeBSD/i386

Hi, folks. I'm trying to build Gauche 0.9.3.2 on FreeBSD, and while
it works fine on amd64 (e.g. passes all tests [1]), it fails on i386
either like this (full log at [2]):

../../src/gosh -ftest ../../src/precomp -e -P -o gauche--collection ../../libsrc/gauche/collection.scm
Deallocating excessively large block.  Too large an allocation?
*** Signal 6

Or prints many error messages like this:

../../src/gosh -ftest ../../src/recomp [..stuff..]
GC Warning: Out of Memory! Heap size: 4 MiB. Returning NULL!
out of memory (2693136576).  aborting...

... and then:

../src/gosh -q -I../src -I../lib -lgauche-init ./extract -en -o gauche-refe.texi ./gauche-ref.texi
gosh: "error": Compile Error: Compile Error: Compile Error: Compile Error: cannot find "util/match" in
("../lib" "../src" "/usr/local/share/gauche/site/lib" "/usr/local/share/gauche/0.9/lib")
"../lib/gauche/condutil.scm":36:(define-module gauche.condutil (use  ...
"../lib/gauche/regexp.scm":106:(define-condition-type <regexp-inval ...
"../lib/gauche/parseopt.scm":34:(define-module gauche.parseopt (use  ...
"././extract":16:(use gauche.parseopt)

I'm seeing this on all FreeBSD releases (e.g. see [3]), and I can
provide full logs, or any other information on request.

Note that Gauche 0.9.2 does not work on FreeBSD 9.0 at all: it
segfaults on startup inside GC_FreeBSDGetDataStart. This is a common
problem with old bdwgc releases on FreeBSD 9.0 -- which makes me

(Continue reading)

Shiro Kawai | 23 May 2012 22:33

Re: Gauche fails to build on FreeBSD/i386

I remember I did work to make Gauche work on FreeBSD.
I think ultimately updating bdwgc to 7.2alpha6 did the
trick.  I didn't have FreeBSD-i386 platform and couldn't
check on it.  

> GC Warning: Out of Memory! Heap size: 4 MiB. Returning NULL!
> out of memory (2693136576).  aborting...

This is really weird message.  The 'Heap size' is too small,
and the number of bytes allocating is ridiculously large.
Definitely, something is really messed up.

I need the platform to dig into it.  However, a new release
of bdwgc seems close based on the conversation in gc ML, so
we can hope that it is taken care of.  (You can check it by
using git HEAD of bdwgc, but there are a few modifications
you need to apply---check the parts marked as "[SK]" in the
gc tree in Gauche.)

> [1] To be fair, it does hang on threading tests once in a while, and rfc.tls
>     test needs modifications, since "ps -ef" in killgnutls.sh does not work
>     witout procfs on FreeBSD.

Actually, gnutls isn't used at all in our test, so killgnutls.sh is
no use.  I just leave it to make changes minimum.  Does the script
made the test fail?  The I'll just remove the call to the script
altogether.  (What matters is killopenssh.sh, and for that I intercept
the original script to avoid using "ps -ef".)

For the thread tests, do you know which test is hanging?  When you

(Continue reading)

Vitaly Magerya | 24 May 2012 13:07

Picon

Re: Gauche fails to build on FreeBSD/i386

Shiro Kawai <shiro <at> lava.net> wrote:
> I remember I did work to make Gauche work on FreeBSD.
> I think ultimately updating bdwgc to 7.2alpha6 did the
> trick.

Ah, yes. I overlooked that.

> I didn't have FreeBSD-i386 platform and couldn't
> check on it.

I'm testing in VirtualBox myself; I don't use i386 daily.

>> GC Warning: Out of Memory! Heap size: 4 MiB. Returning NULL!
>> out of memory (2693136576).  aborting...
>
> [...] a new release
> of bdwgc seems close based on the conversation in gc ML, so
> we can hope that it is taken care of.  (You can check it by
> using git HEAD of bdwgc, but there are a few modifications
> you need to apply---check the parts marked as "[SK]" in the
> gc tree in Gauche.)

I just tried that with both gc-7.2 and latest git sources -- and
I hope I didn't miss any of the changes you made; it's all in
configure scripts, right? Anyway, there appears to be no difference
for i386.

>> [1] To be fair, it does hang on threading tests once in a while, and rfc.tls
>>     test needs modifications, since "ps -ef" in killgnutls.sh does not work
>>     witout procfs on FreeBSD.

(Continue reading)

Vitaly Magerya | 25 May 2012 01:33

Picon

Re: Gauche fails to build on FreeBSD/i386

Shiro Kawai <shiro <at> lava.net> wrote:
>> GC Warning: Out of Memory! Heap size: 4 MiB. Returning NULL!
>> out of memory (2693136576).  aborting...
>
> This is really weird message.  The 'Heap size' is too small,
> and the number of bytes allocating is ridiculously large.
> Definitely, something is really messed up.

I did some debugging, and the problem comes from Scm_InitParameterLoc,
where ensure_parameter_slot is called with that value as "index",
which is caused by "next_parameter_index" jumping from 1, 2, 3,
4, ... to something silly. Unfortunately hardware watchpoints
don't seem to work under VirtualBox (at least on my machine),
so I can't really determine exactly what overwrites it.

As a workaround I found that adding a dummy int variable just
before next_parameter_index is defined solves the problem... (If
you can call that a solution).

Anyway, I'll try to find real i386 hardware in the next few days
to try debugging there (unless you beat me to it).

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Shiro Kawai | 25 May 2012 01:42

Re: Gauche fails to build on FreeBSD/i386

From: Vitaly Magerya <vmagerya <at> gmail.com>
Subject: Re: [Gauche-devel] Gauche fails to build on FreeBSD/i386
Date: Fri, 25 May 2012 02:33:40 +0300

> As a workaround I found that adding a dummy int variable just
> before next_parameter_index is defined solves the problem... (If
> you can call that a solution).

Hmm... that strongly suggests that something is overrunning
the memory region, which is worrysome.

Can you "nm libgauche-0.9.so | sort" and see what's allocated
around next_parameter_index in the bss area?

(Side note: I noticed I forgot to say 'static' in parameter_mutex;
it isn't supposed to be exported.)

On linux/x86_64, it's like this:

00000000004e3e60 b keywords
00000000004e3ea0 b sigHandlers
00000000004e4360 B parameter_mutex
00000000004e4388 b next_parameter_index
00000000004e43a0 b gaucheModule
00000000004e4400 b nullModule
00000000004e4460 b schemeModule

--shiro

------------------------------------------------------------------------------

(Continue reading)

Vitaly Magerya | 25 May 2012 13:36

Picon

Re: Gauche fails to build on FreeBSD/i386

Shiro Kawai wrote:
> Can you "nm libgauche-0.9.so | sort" and see what's allocated
> around next_parameter_index in the bss area?
> 
> (Side note: I noticed I forgot to say 'static' in parameter_mutex;
> it isn't supposed to be exported.)
> 
> On linux/x86_64, it's like this:
> 
> 00000000004e3e60 b keywords
> 00000000004e3ea0 b sigHandlers
> 00000000004e4360 B parameter_mutex
> 00000000004e4388 b next_parameter_index
> 00000000004e43a0 b gaucheModule
> 00000000004e4400 b nullModule
> 00000000004e4460 b schemeModule

Here's what I have:

    00257310 b keywords
    00257320 b sigHandlers
    00257434 B parameter_mutex
    00257438 b next_parameter_index
    00257440 b nullModule
    00257480 b schemeModule
    002574c0 b gaucheModule

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and

(Continue reading)

Vitaly Magerya | 25 May 2012 18:48

Picon

Re: Gauche fails to build on FreeBSD/i386

reviously I wrote wrote:
> Here's what I have:
>
>     00257310 b keywords
>     00257320 b sigHandlers
>     00257434 B parameter_mutex
>     00257438 b next_parameter_index
>     00257440 b nullModule
>     00257480 b schemeModule
>     002574c0 b gaucheModule

And the winner is sigHandlers.

It appears that on FreeBSD NSIG is 32, but SIGRTMIN is 65, so
GAUCHE_PTHREAD_SIGNAL is 70 and when Scm_SetMasterSigmask tries
to do this:

    sigHandlers.handlers[GAUCHE_PTHREAD_SIGNAL] = DEFAULT_SIGHANDLER;

... next_parameter_index is overwritten.

It apears that you need to use _SIG_MAXSIG+1 instead of NSIG on
FreeBSD.

In summary, if I update the bundled gc to the latest git sources,
fix NSIG usage in src/signal.c and src/gauche/vm.h (are both
needed?), and comment out killgnutls.sh, Gauche builds and passes
all tests on both amd64 and i386.

So, how do we proceed? I want to update FreeBSD port of Gauche

(Continue reading)

Shiro Kawai | 26 May 2012 01:29

Re: Gauche fails to build on FreeBSD/i386

Bravo.  Thanks, Vitaly.

Changing the use of NSIG in gauche/vm.h is a bit more involved than
just replacing the constant, since doing so would break ABI.
Let me try to put a reasonable fix in git master.   I'm not sure
I can make a new release soon, for I won't have time to check
on various platforms for at least 2-3 weeks, but I can make a branch
from where you can pull.

--shiro

From: Vitaly Magerya <vmagerya <at> gmail.com>
Subject: Re: [Gauche-devel] Gauche fails to build on FreeBSD/i386
Date: Fri, 25 May 2012 19:48:02 +0300

> And the winner is sigHandlers.
> 
> It appears that on FreeBSD NSIG is 32, but SIGRTMIN is 65, so
> GAUCHE_PTHREAD_SIGNAL is 70 and when Scm_SetMasterSigmask tries
> to do this:
> 
>     sigHandlers.handlers[GAUCHE_PTHREAD_SIGNAL] = DEFAULT_SIGHANDLER;
> 
> ... next_parameter_index is overwritten.
> 
> It apears that you need to use _SIG_MAXSIG+1 instead of NSIG on
> FreeBSD.
> 
> In summary, if I update the bundled gc to the latest git sources,
> fix NSIG usage in src/signal.c and src/gauche/vm.h (are both

(Continue reading)

Shiro Kawai | 26 May 2012 14:41

Re: Gauche fails to build on FreeBSD/i386

Vitaly,

I pushed the fixes of signals and killgnutls.sh to git HEAD.
Regarding gc update---since git HEAD of bdgwc is pretty actively
changed, I'm a bit reluctant to bundle its snapshot to Gauche tarball.
For now, I updated it to 7.2b (released just two weeks ago).
Could you check if this version works on FreeBSD/i386?

If it works (and if it also works on OSX etc), it'll likely
become 0.9.3.3.

--shiro

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Vitaly Magerya | 26 May 2012 21:10

Picon

Re: Gauche fails to build on FreeBSD/i386

Shiro Kawai <shiro <at> lava.net> wrote:
> I pushed the fixes of signals and killgnutls.sh to git HEAD.
> Regarding gc update---since git HEAD of bdgwc is pretty actively
> changed, I'm a bit reluctant to bundle its snapshot to Gauche tarball.

That's probably for the better.

> For now, I updated it to 7.2b (released just two weeks ago).

Did you mean two days ago?

> Could you check if this version works on FreeBSD/i386?

It appears to work fine on i386, but it hangs randomly on threading
tests on amd64, whereas with the latest gc sources no hang occurs
on either platform. Maybe some bug was fixed in git, maybe it's
just a coincidence; I don't know.

Anyway, I think I'll package up latest Gauche git tree with the
latest bdwgc and run that through the package building server --
if it won't hang there too, I'll update the FreeBSD port to that.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

Shiro Kawai | 27 May 2012 05:42

Re: Gauche fails to build on FreeBSD/i386

Ok, then.  It would take some more time to dig into which fix
worked on FreeBSD-amd64, so please go ahead making your own package.
If possible, indicate in the version number that it's modified from
the original 0.9.3.2 release (e.g. 0.9.3.2-freebsd or something)
so that people who try to compile from the 0.9.3.2 source by themselves
won't be confused.

--shiro

From: Vitaly Magerya <vmagerya <at> gmail.com>
Subject: Re: [Gauche-devel] Gauche fails to build on FreeBSD/i386
Date: Sat, 26 May 2012 22:10:43 +0300

> Shiro Kawai <shiro <at> lava.net> wrote:
>> I pushed the fixes of signals and killgnutls.sh to git HEAD.
>> Regarding gc update---since git HEAD of bdgwc is pretty actively
>> changed, I'm a bit reluctant to bundle its snapshot to Gauche tarball.
> 
> That's probably for the better.
> 
>> For now, I updated it to 7.2b (released just two weeks ago).
> 
> Did you mean two days ago?
> 
>> Could you check if this version works on FreeBSD/i386?
> 
> It appears to work fine on i386, but it hangs randomly on threading
> tests on amd64, whereas with the latest gc sources no hang occurs
> on either platform. Maybe some bug was fixed in git, maybe it's
> just a coincidence; I don't know.

(Continue reading)

Return

Return to gmane.lisp.scheme.gauche.

Project Web Page

Announcements, discussion, questions, bug reports, or whatever related to the development of Gauche

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane