Re: Parallel --make (GHC build times on newer MacBook Pros?)
Simon Marlow <marlowsd <at> gmail.com>
2011-09-05 12:58:44 GMT
On 03/09/2011 02:05, Evan Laforge wrote:
>>> Another way to do this would be to have GHC --make invoke itself to
>>> compile each module separately. Actually I think I prefer this method,
>>> although it might be a bit slower since each individual compilation has
>>> to read lots of interface files. The main GHC --make process would do
>>> the final link only. A fun hack for somebody?
>> this would also help building large libraries on architectures with
>> little memory, as it seems to me that when one ghc instance is compiling
>> multiple modules in a row, some leaked memory/unevaluated thunks pile up
>> and eventually cause the compilation to abort. I suspect that building
>> each file on its own avoids this issue.
> In my experience, reading all those .hi files is not so quick, about
> 1.5s for around 200 modules, on an SSD. It gets worse with a pgmF, since ghc
> wants to preprocess each file, it's a minimum of 5s given 'cat' as a
> Part of my wanting to use make instead of --make was to avoid this
> re-preprocessing delay. It's nice that it will automatically notice
> which modules to recompile if a CPP define changes, but not so nice
> that it has to take a lot of time to figure that out every single
> compile, or for a preprocessor that doesn't have the power to change
> whether the module should be recompiled or not.
Ah, but you're measuring the startup time of ghc --make, which is not
the same as the work that each individual ghc would do if ghc were
invoked separately on each module, for two reasons:
- when used in one-shot mode (i.e. without --make), ghc only reads
and processes the interface files it needs, lazilly
- the individual ghc's would not need to proprocess modules - that
would only be done once, by the master process, before starting
the subprocesses. The preprocessed source would be cached,
exactly as it is now by --make.