headers
Donald Bruce Stewart | 22 Oct 2005 11:54
Picon
Picon
Favicon
Gravatar

Fixing "OS allocated a heap in high memory"

Hey guys,

Recent changes to malloc on OpenBSD mean that nhc98 builds now die with
the "OS allocated a heap in high memory" error, as malloc is putting
things above the 0x80000000 line.

Switching the heap allocation to use mmap with a hint to allocate
somewhere lower fixes it at least on x86/openbsd. The ghc testsuite
runs, for example. The mmap code is stolen from MBlock.c in ghc.

Any options on this? Is it reasonable? (I'd like to use it as a custom
patch in the OpenBSD port of nhc98, but want to check I'm not doing
anything hideously wrong :)

-- Don

Here's what I'm doing:

--- src/runtime/Kernel/collector.c.orig	Tue Mar  8 03:53:38 2005
+++ src/runtime/Kernel/collector.c	Sat Oct 22 19:05:22 2005
 <at>  <at>  -7,6 +7,11  <at>  <at> 
 /*#include "runtime.h"    -- already included in node.h */
 #include "mark.h"

+#include <errno.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+
+
 /*#define HEAPSIZE 100000  -- defined in top-level Makefile at config-time */
(Continue reading)

Permalink | Reply |
headers
Malcolm Wallace | 22 Oct 2005 20:31
Picon

Re: Fixing "OS allocated a heap in high memory"

dons <at> cse.unsw.edu.au (Donald Bruce Stewart) writes:

> Switching the heap allocation to use mmap with a hint to allocate
> somewhere lower fixes it at least on x86/openbsd.

Thanks for the fix(es).  Eventually the high-memory problem will be
fixed by not using the top bit for GC, but until then, your approach
will be a useful workaround.  I'll check it into CVS.

Regards,
    Malcolm
Permalink | Reply |
headers
Donald Bruce Stewart | 23 Oct 2005 03:47
Picon
Picon
Favicon
Gravatar

Re: Fixing "OS allocated a heap in high memory"

Malcolm.Wallace:
> dons <at> cse.unsw.edu.au (Donald Bruce Stewart) writes:
> 
> > Switching the heap allocation to use mmap with a hint to allocate
> > somewhere lower fixes it at least on x86/openbsd.
> 
> Thanks for the fix(es).  Eventually the high-memory problem will be
> fixed by not using the top bit for GC, but until then, your approach
> will be a useful workaround.  I'll check it into CVS.

How do we deal with portability issues in nhc? The code I submitted will
run find on the BSDs and Linux, but for Solaris, Irix, HPUS, and Darwin
when need slight variations (which ghc currently has) (and we're stuck
with malloc on Windows).

Does nhc have some equivalent of the 
    #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS)
macros?

-- Don
Permalink | Reply |
headers
Malcolm Wallace | 23 Oct 2005 16:01
Picon

Re: Fixing "OS allocated a heap in high memory"

dons <at> cse.unsw.edu.au (Donald Bruce Stewart) writes:

> Does nhc have some equivalent of the 
>     #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS)
> macros?

Bearing in mind Sven's good advice to use feature tests in preference to
OS tests where possible, you can test any of the symbols ordinarily
defined by your C compiler.  See include/HsFFI.h and include/newmacros.h
for examples.

    #if defined(__sparc__) || defined (__sgi) || etc

Regards,
    Malcolm
Permalink | Reply |
headers
Sven Panne | 23 Oct 2005 13:34
Picon

Re: Fixing "OS allocated a heap in high memory"

Am Sonntag, 23. Oktober 2005 03:47 schrieb Donald Bruce Stewart:
> How do we deal with portability issues in nhc? The code I submitted will
> run find on the BSDs and Linux, but for Solaris, Irix, HPUS, and Darwin
> when need slight variations (which ghc currently has) (and we're stuck
> with malloc on Windows).
>
> Does nhc have some equivalent of the
>     #if defined(solaris2_HOST_OS) || defined(irix_HOST_OS)
> macros?

[ putting on my autotools hat again... ]

Unless you *really* care about the OS, never use #ifdefs like the ones above, 
they are a source of constant maintenance trouble. Feature-based #ifdefs are 
the way to go, and this is e.g. the central point of the highly successful 
autotools toolchain. In our case at hand we should check for the availability 
of mmap (and perhaps its flags) and use the code you gave if it is available, 
falling back to malloc if it is not. Even if we are lazy and don't want to do 
a real feature detection at configuration time, using feature-based #ifdefs 
in the code is still the right way to go, because then the build system can 
#define the feature-based #ifdefs in a single place depending on the OS, 
which is much easier to maintain.

Cheers,
   S.
Permalink | Reply |
headers
Donald Bruce Stewart | 22 Oct 2005 13:11
Picon
Picon
Favicon
Gravatar

Re: Fixing "OS allocated a heap in high memory"


Also, there's some gnu-isms here that add an unnecessary build
dependency. Thanks to Marc Espie for this patch.

Cheers,
   Don

--- script/tprofprel.orig	Sun Mar 13 11:56:03 2005
+++ script/tprofprel	Sun Mar 13 11:56:38 2005
 <at>  <at>  -7,20 +7,8  <at>  <at>  case $# in
   1) exit 1;;
 esac

-case $BASH in
-  "") if ( bash --version -e >/dev/null 2>&1 )
-      then exec bash $0 "$ <at> "
-      fi ;;
-  *) ;;
-esac
+increment() { echo $1 + 1 | bc; }

-
-# Nasty compatibility stuff between /bin/sh/ and bash for arithmetic.
-if sh --version 2>/dev/null | grep -i gnu >/dev/null
-then increment() { ( let x=$1+1; echo $x; ); }
-else increment() { echo $1 + 1 | bc; }
-fi
-
 BUILTIN='Builtin' #Runtime Module added to beginning
 MODNAMES=''
(Continue reading)

Permalink | Reply |

Gmane