headers
Ian Lynagh | 27 Nov 2012 15:52
Favicon

Dynamic libraries by default and GHC 7.8


Hi all,

GHC HEAD now has support for using dynamic libraries by default (and in
particular, using dynamic libraries and the system linker in GHCi) for a
number of platforms.

This has some advantages and some disadvantages, so we need to make a
decision about what we want to do in GHC 7.8. There are also some policy
questions we need to answer about how Cabal will work with a GHC that
uses dynamic libraries by default. We would like to make these as soon
as possible, so that GHC 7.6.2 can ship with a Cabal that works
correctly.

The various issues are described in a wiki page here:
    http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault

If you have a few minutes to read it then we'd be glad to hear your
feedback, to help us in making our decisions

Thanks
Ian
--

-- 
Ian Lynagh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/
Permalink | Reply |
headers
Stephen Paul Weber | 27 Nov 2012 16:22
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Somebody claiming to be Ian Lynagh wrote:
>GHC HEAD now has support for using dynamic libraries by default (and in
>particular, using dynamic libraries and the system linker in GHCi) for a
>number of platforms.
>
>The various issues are described in a wiki page here:
>    http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault

IIRC, one of the problems with dynamic linking in GHC is that when the GHC 
version is different, the ABI can often be different enough to making such 
shared libraries incompatible with each other / binaries made on different 
GHCs.  This makes distribution a potential nightmare (where you have to 
package all the *.so files and send them along with your binary, which is 
essentially the same as static linking, but more pain).  Is this no longer 
the case, or am I completely misremembering this?

Also, you say "for a number of platforms" and that wiki page says 
"Currently, we don't know how to do dynamic-by-default on Windows in 
a satisfactory way."  So I assume Windows is not one of the platforms that 
would be seeing this change?

--

-- 
Stephen Paul Weber,  <at> singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
(Continue reading)

Permalink | Reply |
headers
Ian Lynagh | 27 Nov 2012 18:00
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 10:22:12AM -0500, Stephen Paul Weber wrote:
> Somebody claiming to be Ian Lynagh wrote:
> >GHC HEAD now has support for using dynamic libraries by default (and in
> >particular, using dynamic libraries and the system linker in GHCi) for a
> >number of platforms.
> >
> >The various issues are described in a wiki page here:
> >   http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> IIRC, one of the problems with dynamic linking in GHC is that when
> the GHC version is different, the ABI can often be different enough
> to making such shared libraries incompatible with each other /
> binaries made on different GHCs.  This makes distribution a
> potential nightmare (where you have to package all the *.so files
> and send them along with your binary, which is essentially the same
> as static linking, but more pain).  Is this no longer the case, or
> am I completely misremembering this?

That is still the case. However, if you want to distribute binaries then
you will still be able to build with -static if you don't want to have
ot bundle a load of DLLs. It's only the default that will change.

> Also, you say "for a number of platforms" and that wiki page says
> "Currently, we don't know how to do dynamic-by-default on Windows in
> a satisfactory way."  So I assume Windows is not one of the
> platforms that would be seeing this change?

We would love for Windows to be one of the platforms, but currently we
can't do it on Windows. So unless that changes, Windows will not be one
of the platforms, correct.
(Continue reading)

Permalink | Reply |
headers
Stephen Paul Weber | 27 Nov 2012 18:07
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Somebody claiming to be Ian Lynagh wrote:
>On Tue, Nov 27, 2012 at 10:22:12AM -0500, Stephen Paul Weber wrote:
>> IIRC, one of the problems with dynamic linking in GHC is that when
>> the GHC version is different, the ABI can often be different enough
>> to making such shared libraries incompatible with each other /
>> binaries made on different GHCs.  This makes distribution a
>> potential nightmare (where you have to package all the *.so files
>> and send them along with your binary, which is essentially the same
>> as static linking, but more pain).
>
>That is still the case. However, if you want to distribute binaries then
>you will still be able to build with -static if you don't want to have
>ot bundle a load of DLLs. It's only the default that will change.

If the default changes, though, that would mean that before distribution 
I would have to re-build all my cabal packages with -static?  And if 
I change my default so that cabal builds with -static GHCI would no longer 
work for me?

--

-- 
Stephen Paul Weber,  <at> singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
(Continue reading)

Permalink | Reply |
headers
Ian Lynagh | 27 Nov 2012 19:02
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 12:07:34PM -0500, Stephen Paul Weber wrote:
> Somebody claiming to be Ian Lynagh wrote:
> >On Tue, Nov 27, 2012 at 10:22:12AM -0500, Stephen Paul Weber wrote:
> >>IIRC, one of the problems with dynamic linking in GHC is that when
> >>the GHC version is different, the ABI can often be different enough
> >>to making such shared libraries incompatible with each other /
> >>binaries made on different GHCs.  This makes distribution a
> >>potential nightmare (where you have to package all the *.so files
> >>and send them along with your binary, which is essentially the same
> >>as static linking, but more pain).
> >
> >That is still the case. However, if you want to distribute binaries then
> >you will still be able to build with -static if you don't want to have
> >ot bundle a load of DLLs. It's only the default that will change.
> 
> If the default changes, though, that would mean that before
> distribution I would have to re-build all my cabal packages with
> -static?  And if I change my default so that cabal builds with
> -static GHCI would no longer work for me?

You can configure Cabal to build both static and dynamic libraries
whenever you install anything - but it'll take twice as long.

We actually have half a plan to fix this, so that a single compilation
would build both static and dynamic libraries. Most of the work
(parsing, type checking, optimising) can be shared; it's just the
codegen phase that needs to be different.

The tricky bit is that it only works if all the dependencies have been
compiled for both ways at the same time too. If any dependencies were
(Continue reading)

Permalink | Reply |
headers
Jens Petersen | 28 Nov 2012 08:00

Re: Dynamic libraries by default and GHC 7.8

On 28 November 2012 03:02, Ian Lynagh <ian <at> well-typed.com> wrote:
> We actually have half a plan to fix this, so that a single compilation
> would build both static and dynamic libraries. Most of the work
> (parsing, type checking, optimising) can be shared; it's just the
> codegen phase that needs to be different.

Perhaps this plan could be used for a transition phase?
If a simple pass were enough to generate both dyn and static
(which I could then also check off my ghc wishlist:-)
then a compromise might be to make ghc/Cabal default to generating
both until ghc8 is released?
Still defaulting to dynamic linking of executables of course.
By that time people should be sufficiently used to the change.
Such a softer transition plan might be more generally acceptable to
Linux and MacOS users?

To me being able to default ghc and Cabal to use just dyn will be big
improvement.

Could you say more about the impact to ghc-7.6.2 Cabal?

Jens
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 13:58
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 04:00:02PM +0900, Jens Petersen wrote:
> 
> Could you say more about the impact to ghc-7.6.2 Cabal?

For example, question 8 is about whether Cabal should also build static
libraries for a dynamic-by-default compiler. We would like to ship a
version of Cabal that does the right thing with GHC 7.6.2. That will
hopefully mean that by the time GHC 7.8 is released, people will have a
Haskell Platform that contains a cabal-install that knows the right way
to build things for GHC 7.8.

Thanks
Ian
Permalink | Reply |
headers
Max Bolingbroke | 6 Dec 2012 10:27
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On 27 November 2012 17:00, Ian Lynagh <ian <at> well-typed.com> wrote:
> We would love for Windows to be one of the platforms, but currently we
> can't do it on Windows. So unless that changes, Windows will not be one
> of the platforms, correct.

So if I understand correctly, the problem here arises because you
can't import from a DLL by *absolute path* but only by *relative
path*?

The tragedy here is that the Windows loader actually supports loading
from absolute paths natively, as an undocumented feature. If you build
a EXE and then hexedit the references to "library.dll" to absolute
paths "C:\library.dll" then your EXE will run completely correctly.

I'm not sure how to convince GNU ld to link us such a DLL, though. It
might be possible to hack it in by linking against
"verylongdummydllname.dll" and then having GHC search for that string
in DLL/EXE returned by LD and replace it by the null-terminated
absolute path.

Max
Permalink | Reply |
headers
Max Bolingbroke | 6 Dec 2012 11:05
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On 6 December 2012 09:27, Max Bolingbroke <batterseapower <at> hotmail.com> wrote:
> I'm not sure how to convince GNU ld to link us such a DLL, though. It
> might be possible to hack it in by linking against
> "verylongdummydllname.dll" and then having GHC search for that string
> in DLL/EXE returned by LD and replace it by the null-terminated
> absolute path.

While I was in the shower I figured out how to make this work:

 1. We need to create an *import library* for every Haskell DLL. This
import library will export all the functions exported by the normal
DLL, but we will ensure that it has an absolute path name in the
.idata section
 2. To create the import library, there are two methods:
  a) If we know all the object files that went into linking the DLL
(we don't care about other linker flags like the DLLs that the current
DLL depends on), we can just use "dlltool --output-lib foo.lib <object
files>" to generate the library.
  b) If we don't know the object files, we will instead have to create
a .def file using mingw's pexports utility/nm/dumpbin, and then run
"dlltool --output-lib foo.lib -d foo.def"
 3. In ether case, we should create the object file with --dllname
verylongdummymodulename.dll
 4. After dlltool returns, we simply go into the import library and
replace the dummy module name with the absolute path
 5. When linking against Haskell DLLs later on, simply link against
the import library rather than the DLL itself

I've tested this by hand and it seems like it works: the absolute
paths you hack into the import library are not messed with by the
(Continue reading)

Permalink | Reply |
headers
Max Bolingbroke | 6 Dec 2012 14:34
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On 6 December 2012 10:05, Max Bolingbroke <batterseapower <at> hotmail.com> wrote:
> I've tested this by hand and it seems like it works: the absolute
> paths you hack into the import library are not messed with by the
> later link step. The resulting executables run flawlessly. Job done.

To continue spamming the list, I'd just like to advertise my blog post
explaining the proposed technique, with a worked example:

http://blog.omega-prime.co.uk/?p=138

Cheers,
Max
Permalink | Reply |
headers
Nathan Howell | 7 Dec 2012 03:18
Picon

Re: Dynamic libraries by default and GHC 7.8

Would the SxS loader (with associated manifests, etc) work? I've used
it to support similar scenarios in a former job without any PE32
hacking. Unless things have changed recently, the APIs aren't super
well documented but they are supported and there are a handful of
tools for tracing the loader through various activation contexts.

On Thu, Dec 6, 2012 at 5:34 AM, Max Bolingbroke
<batterseapower <at> hotmail.com> wrote:
> On 6 December 2012 10:05, Max Bolingbroke <batterseapower <at> hotmail.com> wrote:
>> I've tested this by hand and it seems like it works: the absolute
>> paths you hack into the import library are not messed with by the
>> later link step. The resulting executables run flawlessly. Job done.
>
> To continue spamming the list, I'd just like to advertise my blog post
> explaining the proposed technique, with a worked example:
>
> http://blog.omega-prime.co.uk/?p=138
>
> Cheers,
> Max
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users <at> haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Matthias Kilian | 27 Nov 2012 20:38
Picon

Re: Dynamic libraries by default and GHC 7.8

Hi,

On Tue, Nov 27, 2012 at 02:52:48PM +0000, Ian Lynagh wrote:
> The various issues are described in a wiki page here:
>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> If you have a few minutes to read it then we'd be glad to hear your
> feedback, to help us in making our decisions

Regarding Question 7 (enable dynamic by default on other platforms)
and OpenBSD: as long as it's easy to disable it again, I'll be happy
with *any* decision.

That's partially because currently we've even a patch explicitely
disabling shared library support in our ports/packages system (last
time I tried with shared lib support, I got some segfaults in the
midst of the build, and unfortunately I'm still too short of time
to debug/fix it).

Ciao,
	Kili
Permalink | Reply |
headers
Ian Lynagh | 27 Nov 2012 21:15
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 08:38:21PM +0100, Matthias Kilian wrote:
> On Tue, Nov 27, 2012 at 02:52:48PM +0000, Ian Lynagh wrote:
> > The various issues are described in a wiki page here:
> >     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> > 
> > If you have a few minutes to read it then we'd be glad to hear your
> > feedback, to help us in making our decisions
> 
> Regarding Question 7 (enable dynamic by default on other platforms)
> and OpenBSD: as long as it's easy to disable it again, I'll be happy
> with *any* decision.

It will be easy to turn it off, but depending on the platform we might
have removed support for GHCi when it's turned off.

Does ghci work for you currently?

Is this a registerised or unregisterised build?

> That's partially because currently we've even a patch explicitely
> disabling shared library support in our ports/packages system (last
> time I tried with shared lib support, I got some segfaults in the
> midst of the build, and unfortunately I'm still too short of time
> to debug/fix it).

That's a bit bizarre. With shared libraries enabled, there still won't
be any dynamically linked programs actually run.

Thanks
Ian
(Continue reading)

Permalink | Reply |
headers
Matthias Kilian | 27 Nov 2012 23:08
Picon

Re: Dynamic libraries by default and GHC 7.8

Hi,

On Tue, Nov 27, 2012 at 08:15:59PM +0000, Ian Lynagh wrote:
> > Regarding Question 7 (enable dynamic by default on other platforms)
> > and OpenBSD: as long as it's easy to disable it again, I'll be happy
> > with *any* decision.
> 
> It will be easy to turn it off, but depending on the platform we might
> have removed support for GHCi when it's turned off.
> 
> Does ghci work for you currently?

Yes, but I'm still at ghc-7,4.2. Not enough time to catch up with
recent ghc development :-(

> Is this a registerised or unregisterised build?

Registerised, working on i386 and amd66 (or x86 and x86_64 in the
non-bsd-world).

> > That's partially because currently we've even a patch explicitely
> > disabling shared library support in our ports/packages system (last
> > time I tried with shared lib support, I got some segfaults in the
> > midst of the build, and unfortunately I'm still too short of time
> > to debug/fix it).
> 
> That's a bit bizarre. With shared libraries enabled, there still won't
> be any dynamically linked programs actually run.

No worry. There were a lot of changes in OpenBSD during the last 6
(Continue reading)

Permalink | Reply |
headers
kudah | 3 Dec 2012 18:09
Picon

Re: Dynamic libraries by default and GHC 7.8

On Tue, 27 Nov 2012 20:15:59 +0000 Ian Lynagh <ian <at> well-typed.com>
wrote:

> It will be easy to turn it off, but depending on the platform we might
> have removed support for GHCi when it's turned off.

Why not provide RTS linker as a deprecated option at least until it's
completely removed from all platforms? I mostly do static builds for
Linux/x86 and Win32, and the prospect of compiling twice as much,
possibly with a slower compiler, just to use ghci prompt(or even
Template Haskell(?)) is not exactly exciting. And introducing more
differences between Haskell on *nix and on Windows isn't going to make
cross-platform development easier.
Permalink | Reply |
headers
Evan Laforge | 27 Nov 2012 22:34
Picon

Re: Dynamic libraries by default and GHC 7.8

I don't totally understand how ghci loading would work.  I assume that
for external packages it will go load  x.so instead of x.a, but what
about local modules?  I assume ghc -c is still going to produce .o
files, so does that mean ghci will have to interpret all local moules?
 If so, is there a way around that?  For instance, I imagine I could
add an extra build step to bundle the .o files up into a .so and then
somehow get ghci to know to load that.  I guess that's like compiling
all the local modules as a separate package and then adding that to
the package db, so in principle it should be possible.

On the plus side, it seems like this would do wonders for link time,
which is currently about 15s of "machine grinds to a halt" for me.  I
didn't see mention of it on the page, but I assume linking a binary
with lots of giant packages is going to be a lot faster if they can be
loaded on demand at runtime... or is just going to make startup really
slow?

On Tue, Nov 27, 2012 at 6:52 AM, Ian Lynagh <ian <at> well-typed.com> wrote:
>
> Hi all,
>
> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.
>
> This has some advantages and some disadvantages, so we need to make a
> decision about what we want to do in GHC 7.8. There are also some policy
> questions we need to answer about how Cabal will work with a GHC that
> uses dynamic libraries by default. We would like to make these as soon
> as possible, so that GHC 7.6.2 can ship with a Cabal that works
(Continue reading)

Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 00:48
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 01:34:59PM -0800, Evan Laforge wrote:
> I don't totally understand how ghci loading would work.  I assume that
> for external packages it will go load  x.so instead of x.a, but what
> about local modules?  I assume ghc -c is still going to produce .o
> files, so does that mean ghci will have to interpret all local moules?

No, it links them into a temporary .so in /tmp and dlopens that.

> On the plus side, it seems like this would do wonders for link time,
> which is currently about 15s of "machine grinds to a halt" for me.  I
> didn't see mention of it on the page, but I assume linking a binary
> with lots of giant packages is going to be a lot faster if they can be
> loaded on demand at runtime... or is just going to make startup really
> slow?

It looks like it's fractionally slower on x86_64/Linux:

Using static libraries with ghci:
$ time echo GHC.maxPrecedence | ghc --interactive -package ghc -v0
9
1.00s user 0.12s system 100% cpu 1.122 total

Using dynamic libraries with ghci:
$ time echo GHC.maxPrecedence | ghc --interactive -package ghc -v0
9
1.01s user 0.15s system 101% cpu 1.141 total

Thanks
Ian
(Continue reading)

Permalink | Reply |
headers
Joachim Breitner | 28 Nov 2012 00:28
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Hi,

Am Dienstag, den 27.11.2012, 14:52 +0000 schrieb Ian Lynagh:
> The various issues are described in a wiki page here:
>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> If you have a few minutes to read it then we'd be glad to hear your
> feedback, to help us in making our decisions

here comes the obligatory butting in by the Debian Haskell Group:

Given the current sensitivity of the ABI hashes we really do not want to
have Programs written in Haskell have a runtime dependency on all the
included Haskell libraries. So I believe we should still link Haskell
programs statically in Debian.

Hence, Debian will continue to provide its libraries built the static
way.

Building them also in the dynamic way for the sake of GHCi users seems
possible.

Open question: What should GHC on Debian do when building binaries,
given that all libraries are likely available in both ways – shared or
static. Shared means that all locally built binaries (e.g. xmonad!) will
suddenly break when the user upgrades its Haskell packages, as the
package management is ignorant of unpackaged, locally built programs.
I’d feel more comfortable if that could not happen.

Other open question: Should we put the dynamic libraries in the normal
(Continue reading)

Permalink | Reply |
headers
Tyson Whitehead | 28 Nov 2012 03:57
Picon

Re: Dynamic libraries by default and GHC 7.8

On Wed, 2012-11-28 at 00:28 +0100, Joachim Breitner wrote:
> Am Dienstag, den 27.11.2012, 14:52 +0000 schrieb Ian Lynagh:
> > The various issues are described in a wiki page here:
> >     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> > 
> > If you have a few minutes to read it then we'd be glad to hear your
> > feedback, to help us in making our decisions
> 
> here comes the obligatory butting in by the Debian Haskell Group:
> 
> Given the current sensitivity of the ABI hashes we really do not want to
> have Programs written in Haskell have a runtime dependency on all the
> included Haskell libraries. So I believe we should still link Haskell
> programs statically in Debian.
>
> Hence, Debian will continue to provide its libraries built the static
> way.

I was so excited for a bit thinking that this would finally mean that
Debian would move to a dynamic system.  Every haskell binary being 10s
of MBs (e.g., pandoc = 25MB executable) makes it look kind of bad.

From our previous discussion on this

http://lists.debian.org/debian-haskell/2010/03/msg00120.html

it seemed a big part of the problem is you really need to be able to
have multiple versions of the same package installed.

At the time, I believe the only way to do this was to include part of
(Continue reading)

Permalink | Reply |
headers
Brandon Allbery | 28 Nov 2012 05:39
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 9:57 PM, Tyson Whitehead <twhitehead <at> gmail.com> wrote:
it seemed a big part of the problem is you really need to be able to
have multiple versions of the same package installed.

At the time, I believe the only way to do this was to include part of
the hash in the name, but that meant you had to constantly be creating
new packages which Debian didn't make easy.

Maybe it is time to explain the problem to the Debian packaging people
and see if they have any ideas from their level?

Not sure they do, because it's a GHC-specific and very nasty problem at core.

The fundamental issue is that GHC does aggressive cross-module inlining to make performance halfway reasonable -- but this means you end up requiring very precise dependencies, because different builds of the same library even with the same compiler might for some reason expose different things for inlining, and that ends up infecting the whole dependency chain.  Done statically, the result is the infamous Cabal hell; done dynamically, it means shared library hell.  I am not sure it can be made sane in either case.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net 

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Joachim Breitner | 28 Nov 2012 10:45
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Hi,

Am Dienstag, den 27.11.2012, 21:57 -0500 schrieb Tyson Whitehead:
> I was so excited for a bit thinking that this would finally mean that
> Debian would move to a dynamic system.  Every haskell binary being 10s
> of MBs (e.g., pandoc = 25MB executable) makes it look kind of bad.

its not like dynamic libraries make the bytes disappear – the
(non-Haskell-developer) user who wants to use pandoc still has to
install all these bytes, but now they just come split in a dozen of
packages.

Or gix-annex, a more and more popular Haskell application: Building it
requires 94 Haskell library packages. Now imagine this to be dynamically
built: Now installing git-annex will require 94 strage sounding packages
that the user most likely has no idea what they are about, and chances
are high that there is no other packages requiring these shared
libraries, making most of the benefit of shared libraries moot.

Now, if Haskell was as popular as C and the user _would_ run several
different processes at once that could share the shared library, this
would be interesting. At the moment, I do not see how dynamically built
Haskell programs are in the interest of our user.

> I was left with the impression that we were going to have this back in
> 2010 just as soon as squeeze got out the door...  :)

It seems that noone cared enough about that, but any help is welcome.
Two things to do: Patch haskell-devscripts to build the -dyn ways, and
manually adding the additional package stance to the debian/contol files
(if it is to be decided that the -dyn libraries should reside in
packages of their own. If we decide to include them in the regular
packages, this is not needed.)

Greetings,
Joachim

--

-- 
Joachim "nomeata" Breitner
Debian Developer
  nomeata <at> debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nomeata <at> joachim-breitner.de | http://people.debian.org/~nomeata
Permalink | Reply |
headers
Tyson Whitehead | 28 Nov 2012 17:35
Picon

Re: Dynamic libraries by default and GHC 7.8

On November 28, 2012 04:45:57 Joachim Breitner wrote:
> Am Dienstag, den 27.11.2012, 21:57 -0500 schrieb Tyson Whitehead:
> > I was so excited for a bit thinking that this would finally mean that
> > Debian would move to a dynamic system.  Every haskell binary being 10s
> > of MBs (e.g., pandoc = 25MB executable) makes it look kind of bad.
> 
> its not like dynamic libraries make the bytes disappear – the
> (non-Haskell-developer) user who wants to use pandoc still has to
> install all these bytes, but now they just come split in a dozen of
> packages.

My point was more trying to get at the idea that maybe we don't need a 
separate copy of most of the bytes in each application.

> Or gix-annex, a more and more popular Haskell application: Building it
> requires 94 Haskell library packages. Now imagine this to be dynamically
> built: Now installing git-annex will require 94 strage sounding packages
> that the user most likely has no idea what they are about, and chances
> are high that there is no other packages requiring these shared
> libraries, making most of the benefit of shared libraries moot.
> 
> Now, if Haskell was as popular as C and the user _would_ run several
> different processes at once that could share the shared library, this
> would be interesting. At the moment, I do not see how dynamically built
> Haskell programs are in the interest of our user.

I guess this is really a question of how many haskell programs are there being 
used out there.  From the looks of popcon results, there isn't a whole lot of 
take up on anything at moment apart from ghc, xmond, and pandoc.

> > I was left with the impression that we were going to have this back in
> > 2010 just as soon as squeeze got out the door...  :)
> 
> It seems that noone cared enough about that, but any help is welcome.
> Two things to do: Patch haskell-devscripts to build the -dyn ways, and
> manually adding the additional package stance to the debian/contol files
> (if it is to be decided that the -dyn libraries should reside in
> packages of their own. If we decide to include them in the regular
> packages, this is not needed.)

Fair enough.

If I was update my 2010 patch so it worked again at some point in the upcoming 
year (I don't have the time to do this at the moment), would there be a 
reasonable chance it would seem worthwhile to include it at this point?

Please feel free to say no here if that is the case.  I realize that maybe in 
a few years, when there are even more haskell applications, we can revisit the 
again, and possibly then it will make more sense.

Cheers!  -Tyson

PS:  I don't mean to be critical here.  You've done a lot of work supporting 
haskell under Debian, and it's all volunteer.  I really appreciate that.

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Joachim Breitner | 28 Nov 2012 18:30
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Hi,

Am Mittwoch, den 28.11.2012, 11:35 -0500 schrieb Tyson Whitehead:
> > > I was left with the impression that we were going to have this back in
> > > 2010 just as soon as squeeze got out the door...  :)
> > 
> > It seems that noone cared enough about that, but any help is welcome.
> > Two things to do: Patch haskell-devscripts to build the -dyn ways, and
> > manually adding the additional package stance to the debian/contol files
> > (if it is to be decided that the -dyn libraries should reside in
> > packages of their own. If we decide to include them in the regular
> > packages, this is not needed.)
> 
> Fair enough.
> 
> If I was update my 2010 patch so it worked again at some point in the upcoming 
> year (I don't have the time to do this at the moment), would there be a 
> reasonable chance it would seem worthwhile to include it at this point?
> 
> Please feel free to say no here if that is the case.  I realize that maybe in 
> a few years, when there are even more haskell applications, we can revisit the 
> again, and possibly then it will make more sense.

there was even a patch? Sorry then, this is not how contributors should
be treated.

Given that dynamic linking is becoming the default elsewhere, we should
definitely provide it with the next release. So yes, please do look
again at the issue.

We currently have one package providing dynamic libraries, and that is
ghc-dyn. It contains lots of .dyn_hi and some .so files. Which of these
are required at runtime? What is the space ratio between the .dyn_hi
(which presumably are only required at build time) and the .so files? Do
we want separate packages that contain only the files required at
runtime? What should be the filename (just libghc-foo?)? Where do
the .dyn_hi files go? If a user only build dynamically (only staticaly,
both), what packages does she needs?

Make liberal use of package-version-hash-base virtual package names
(e.g. one set of names for the build time requirements with dynamic
packages and one set of names for whats required at runtime) to ensure
that we can revise these decisions without breaking everything.

But this is getting too specific for glasgow-haskell-users, I guess.
replies on this issue please only to d-haskell.

Greetings,
Joachim

--

-- 
Joachim "nomeata" Breitner
Debian Developer
  nomeata <at> debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nomeata <at> joachim-breitner.de | http://people.debian.org/~nomeata
Permalink | Reply |
headers
Tyson Whitehead | 28 Nov 2012 19:43
Picon

Re: Dynamic libraries by default and GHC 7.8

On November 28, 2012 12:30:52 Joachim Breitner wrote:
> Am Mittwoch, den 28.11.2012, 11:35 -0500 schrieb Tyson Whitehead:
> > If I was update my 2010 patch so it worked again at some point in the
> > upcoming year (I don't have the time to do this at the moment), would
> > there be a reasonable chance it would seem worthwhile to include it at
> > this point?
> > 
> > Please feel free to say no here if that is the case.  I realize that
> > maybe in a few years, when there are even more haskell applications, we
> > can revisit the again, and possibly then it will make more sense.
> 
> there was even a patch? Sorry then, this is not how contributors should
> be treated.

Hey, no problem.  Like I said, I'm just glad we got some people interested in 
making Haskell a success under Debian.  Thanks for all your work.

The patch was attached to my original email.  It was built on top of the 0.7.5 
release of the haskell-devscripts.

http://lists.debian.org/debian-haskell/2010/03/msg00120.html

> Given that dynamic linking is becoming the default elsewhere, we should
> definitely provide it with the next release. So yes, please do look
> again at the issue.

Hopefully I'll have sometime in the new year to dust it back off again.

It won't be right away though, so, if someone else wants to take a stab at it, 
please do go ahead.

WRT the rest of your points/questions, I haven't looked at it since I wrote 
the patch, so I have to plead ignorant.

Cheers!  -Tyson

--

-- 
To UNSUBSCRIBE, email to debian-haskell-request <at> lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster <at> lists.debian.org
Archive: http://lists.debian.org/201211281343.40992.twhitehead <at> gmail.com
Permalink | Reply |
headers
Bryan O'Sullivan | 30 Nov 2012 18:07
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 1:45 AM, Joachim Breitner <nomeata <at> debian.org> wrote:

At the moment, I do not see how dynamically built
Haskell programs are in the interest of our user.

They do offer the prospect of fixing some annoying bugs for free, by offloading them to existing, working system infrastructure. For instance, we can't call C++ code from ghci right now, because ghci's loader doesn't invoke static initializers that C++ libraries tend to use heavily.

The "save some bytes" argument seems like a red herring, but it's far from the only thing to pay attention to.
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Jens Petersen | 28 Nov 2012 08:14

Re: Dynamic libraries by default and GHC 7.8

On 28 November 2012 08:28, Joachim Breitner <nomeata <at> debian.org> wrote:
> Open question: What should GHC on Debian do when building binaries,
> given that all libraries are likely available in both ways – shared or
> static. Shared means that all locally built binaries (e.g. xmonad!) will
> suddenly break when the user upgrades its Haskell packages, as the
> package management is ignorant of unpackaged, locally built programs.
> I’d feel more comfortable if that could not happen.

Right, I tried patching Fedora's xmonad for a while to use dynamic linking
(it made Mod-q almost instant! :-) but finally reverted it not to confuse people
linking their .xmonad/ to user libraries, at least until the time
ghc/Cabal support dyn by default...

Jens

--

-- 
To UNSUBSCRIBE, email to debian-haskell-request <at> lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster <at> lists.debian.org
Archive: http://lists.debian.org/CAEDGBq+CMkeyDJ_RXSUuwdxc2Q+NsdAumjF4UZGRM=+MjbC5gA <at> mail.gmail.com
Permalink | Reply |
headers
Joachim Breitner | 28 Nov 2012 10:37
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Am Mittwoch, den 28.11.2012, 16:14 +0900 schrieb Jens Petersen:
> On 28 November 2012 08:28, Joachim Breitner <nomeata <at> debian.org> wrote:
> > Open question: What should GHC on Debian do when building binaries,
> > given that all libraries are likely available in both ways – shared or
> > static. Shared means that all locally built binaries (e.g. xmonad!) will
> > suddenly break when the user upgrades its Haskell packages, as the
> > package management is ignorant of unpackaged, locally built programs.
> > I’d feel more comfortable if that could not happen.
> 
> Right, I tried patching Fedora's xmonad for a while to use dynamic linking
> (it made Mod-q almost instant! :-) but finally reverted it not to confuse people
> linking their .xmonad/ to user libraries, at least until the time
> ghc/Cabal support dyn by default...

but would that not mean that when they upgrade a library used by xmonad,
their xmonad binary would stop working and they could not even log in
any more? It is that kind of breakage that is worrying me.

Greetings,
Joachim

--

-- 
Joachim "nomeata" Breitner
Debian Developer
  nomeata <at> debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nomeata <at> joachim-breitner.de | http://people.debian.org/~nomeata
Permalink | Reply |
headers
Simon Marlow | 28 Nov 2012 10:09
Picon

Re: Dynamic libraries by default and GHC 7.8

On 27/11/12 23:28, Joachim Breitner wrote:
> Hi,
>
> Am Dienstag, den 27.11.2012, 14:52 +0000 schrieb Ian Lynagh:
>> The various issues are described in a wiki page here:
>>      http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
>>
>> If you have a few minutes to read it then we'd be glad to hear your
>> feedback, to help us in making our decisions
>
> here comes the obligatory butting in by the Debian Haskell Group:
>
> Given the current sensitivity of the ABI hashes we really do not want to
> have Programs written in Haskell have a runtime dependency on all the
> included Haskell libraries. So I believe we should still link Haskell
> programs statically in Debian.
>
> Hence, Debian will continue to provide its libraries built the static
> way.
>
> Building them also in the dynamic way for the sake of GHCi users seems
> possible.

So let me try to articulate the options, because I think there are some 
dependencies that aren't obvious here.  It's not a straightforward 
choice between -dynamic/-static being the default, because of the GHCi 
interaction.

Here are the 3 options:

(1) (the current situation) GHCi is statically linked, and -static is
     the default.  Uses the RTS linker.

(2) (the proposal, at least for some platforms) GHCi is dynamically
     linked, and -dynamic is the default.  Does not use the RTS linker.

(3) GHCi is dynamically linked, but -static is the default.  Does not
     use the RTS linker.  Packages must be installed with -dynamic,
     otherwise they cannot be loaded into GHCi, and only objects
     compiled with -dynamic can be loaded into GHCi.

You seem to be saying that Debian would do (3), but we hadn't considered 
that as a viable option because of the extra hoops that GHCi users would 
have to jump through.  We consider it a prerequisite that GHCi continues 
to work without requiring any extra flags.

Cheers,
	Simon

> Open question: What should GHC on Debian do when building binaries,
> given that all libraries are likely available in both ways – shared or
> static. Shared means that all locally built binaries (e.g. xmonad!) will
> suddenly break when the user upgrades its Haskell packages, as the
> package management is ignorant of unpackaged, locally built programs.
> I’d feel more comfortable if that could not happen.
>
> Other open question: Should we put the dynamic libraries in the normal
> libghc-*-dev package? Con: Package size doubles (and xmonad users are
> already shocked by the size of stuff they need to install). Pro: It
> cannot happen that I can build Foo.hs statically, but not load it in
> GHCi, or vice-versa.
>
> I still find it unfortunate that once cannot use the .so for static
> linking as well, but that is a problem beyond the scope of GHC.
>
> Greetings,
> Joachim
>
>
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users <at> haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 13:41
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 09:09:58AM +0000, Simon Marlow wrote:
> On 27/11/12 23:28, Joachim Breitner wrote:
> >
> >Hence, Debian will continue to provide its libraries built the static
> >way.
> 
> So let me try to articulate the options, because I think there are
> some dependencies that aren't obvious here.  It's not a
> straightforward choice between -dynamic/-static being the default,
> because of the GHCi interaction.
> 
> Here are the 3 options:
> 
> (1) (the current situation) GHCi is statically linked, and -static is
>     the default.  Uses the RTS linker.
> 
> (2) (the proposal, at least for some platforms) GHCi is dynamically
>     linked, and -dynamic is the default.  Does not use the RTS linker.
> 
> (3) GHCi is dynamically linked, but -static is the default.  Does not
>     use the RTS linker.  Packages must be installed with -dynamic,
>     otherwise they cannot be loaded into GHCi, and only objects
>     compiled with -dynamic can be loaded into GHCi.
> 
> You seem to be saying that Debian would do (3), but we hadn't
> considered that as a viable option because of the extra hoops that
> GHCi users would have to jump through.  We consider it a
> prerequisite that GHCi continues to work without requiring any extra
> flags.

I think what Joachim means is that the binaries /in Debian packages/
will be explicitly linked with -static, but the open question below is
about whether to make -static or -dynamic the default for GHC.

> >Open question: What should GHC on Debian do when building binaries,
> >given that all libraries are likely available in both ways – shared or
> >static. Shared means that all locally built binaries (e.g. xmonad!) will
> >suddenly break when the user upgrades its Haskell packages, as the
> >package management is ignorant of unpackaged, locally built programs.
> >I’d feel more comfortable if that could not happen.

Thanks
Ian

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 13:39
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 12:28:31AM +0100, Joachim Breitner wrote:
> 
> here comes the obligatory butting in by the Debian Haskell Group:
> 
> Given the current sensitivity of the ABI hashes we really do not want to
> have Programs written in Haskell have a runtime dependency on all the
> included Haskell libraries. So I believe we should still link Haskell
> programs statically in Debian.

Note that you'll have to link ghc dynamically or ghci won't work. This
won't cause dependency problems, though, as ghc comes with all the
libraries that it needs.

Thanks
Ian
Permalink | Reply |
headers
Stephen Paul Weber | 28 Nov 2012 17:04
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Somebody signing messages as Joachim Breitner wrote:
>Given the current sensitivity of the ABI hashes we really do not want to
>have Programs written in Haskell have a runtime dependency on all the
>included Haskell libraries. So I believe we should still link Haskell
>programs statically in Debian.

This is also a concern I have.

>Hence, Debian will continue to provide its libraries built the static
>way.

Right.

>Building them also in the dynamic way for the sake of GHCi users seems
>possible.

Perhaps Debian could just ship a GHCi that uses the RTS linker, as now?  The 
change is to be made for "some platforms", we could opt to have Debian not 
be such a platform.

--

-- 
Stephen Paul Weber,  <at> singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 18:04
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 11:04:44AM -0500, Stephen Paul Weber wrote:
> 
> >Building them also in the dynamic way for the sake of GHCi users seems
> >possible.
> 
> Perhaps Debian could just ship a GHCi that uses the RTS linker, as
> now?  The change is to be made for "some platforms", we could opt to
> have Debian not be such a platform.

This isn't a good long-term plan. Even if we don't actively remove
support for other platforms (which we plan to do), they'll probably
bitrot, and the current bugs won't be fixed.

Thanks
Ian
Permalink | Reply |
headers
dag.odenhall@gmail.com | 28 Nov 2012 03:11
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On Tue, Nov 27, 2012 at 3:52 PM, Ian Lynagh <ian <at> well-typed.com> wrote:
This has some advantages and some disadvantages, so we need to make a
decision about what we want to do in GHC 7.8.

When I built my CLI app with dynamic linking, the time to run and complete a command went from 20ms to 100ms, which is particularly noticeable when used via its bash completions which call the program to provide completions.
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
dag.odenhall@gmail.com | 28 Nov 2012 03:16
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

That was with GHC 7.4.1, I should add. I didn't measure overall performance difference, only the time to finish a command-line invocation.



On Wed, Nov 28, 2012 at 3:11 AM, dag.odenhall <at> gmail.com <dag.odenhall <at> gmail.com> wrote:
On Tue, Nov 27, 2012 at 3:52 PM, Ian Lynagh <ian <at> well-typed.com> wrote:
This has some advantages and some disadvantages, so we need to make a
decision about what we want to do in GHC 7.8.

When I built my CLI app with dynamic linking, the time to run and complete a command went from 20ms to 100ms, which is particularly noticeable when used via its bash completions which call the program to provide completions.

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
dag.odenhall@gmail.com | 28 Nov 2012 03:25
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

And on 32 bit Linux, which apparently is known to suffer from dynamic... Though from 20ms to 100ms is more than 30% penalty.



On Wed, Nov 28, 2012 at 3:16 AM, dag.odenhall <at> gmail.com <dag.odenhall <at> gmail.com> wrote:
That was with GHC 7.4.1, I should add. I didn't measure overall performance difference, only the time to finish a command-line invocation.



On Wed, Nov 28, 2012 at 3:11 AM, dag.odenhall <at> gmail.com <dag.odenhall <at> gmail.com> wrote:
On Tue, Nov 27, 2012 at 3:52 PM, Ian Lynagh <ian <at> well-typed.com> wrote:
This has some advantages and some disadvantages, so we need to make a
decision about what we want to do in GHC 7.8.

When I built my CLI app with dynamic linking, the time to run and complete a command went from 20ms to 100ms, which is particularly noticeable when used via its bash completions which call the program to provide completions.


_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Jens Petersen | 28 Nov 2012 04:02

Re: Dynamic libraries by default and GHC 7.8

Hi,

> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.

I am very happy to hear this news.

I have long been a quiet proponent for defaulting ghc and Cabal to
shared libraries and dynamic linking for Linux.
I think Fedora was probably the first Linux distro to enable ghc
shared libs and use dynamic linking for executables in our packages.
Having ghci and cabal linking benefiting from dynamic linking seems a
positive step forward indeed IMHO.
I hope we will see ghc shared libs support spreading also to other
Linux archs (particularly ARM).

Sure it will change deployment workflow somewhat but that is what
Linux distros are for. :)

I am looking forward to test this with Fedora's Haskell packages very much.

Jens

ps Btw RHEL is also increasingly moving to 64bit.
_______________________________________________
haskell mailing list
haskell <at> lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/haskell
Permalink | Reply |
headers
Tyson Whitehead | 28 Nov 2012 17:42
Picon

Re: Dynamic libraries by default and GHC 7.8

On November 27, 2012 22:02:33 Jens Petersen wrote:
> > GHC HEAD now has support for using dynamic libraries by default (and in
> > particular, using dynamic libraries and the system linker in GHCi) for a
> > number of platforms.
> 
> I am very happy to hear this news.
> 
> I have long been a quiet proponent for defaulting ghc and Cabal to
> shared libraries and dynamic linking for Linux.
> I think Fedora was probably the first Linux distro to enable ghc
> shared libs and use dynamic linking for executables in our packages.
> Having ghci and cabal linking benefiting from dynamic linking seems a
> positive step forward indeed IMHO.
> I hope we will see ghc shared libs support spreading also to other
> Linux archs (particularly ARM).

Hi Jens,

I was curious how Fedora has managed to go dynamic given the various issues 
that the Debian team has brought up.

Is the situation on Fedora somehow different?  Is there something clever you 
are doing that gets around this issues?

Thanks!  -Tyson
Permalink | Reply |
headers
Ganesh Sittampalam | 28 Nov 2012 07:43
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On 27/11/2012 14:52, Ian Lynagh wrote:

> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.
> 
> This has some advantages and some disadvantages, so we need to make a
> decision about what we want to do in GHC 7.8. There are also some policy
> questions we need to answer about how Cabal will work with a GHC that
> uses dynamic libraries by default. We would like to make these as soon
> as possible, so that GHC 7.6.2 can ship with a Cabal that works
> correctly.
> 
> The various issues are described in a wiki page here:
>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault

If I understand the problem on Windows correctly, you can use dynamic
libraries for ghci, but using them for compiled executables is difficult
because there's no good solution for having the compiled exe find its DLLs.

If so, does that mean that you intend to switch over ghci to dynamic and
build static+dynamic by default, or are you going to leave ghci as static?

My general feeling about Windows is that it's ok for the end result to
be a little annoying, because Windows DLLs *are* annoying and it's
nothing to do with GHC.

In particular I think in practice most Windows developers will have
admin rights and could live with the ghc installation and cabal install
having to be done as elevated operations. Where they weren't done with
admin rights, then ghc -o could warn the user that the DLLs need to be
copied locally (or even copy them itself and tell the user it happened).

More generally, if you can implement the "half a plan" you mentioned
elsewhere in the thread for quickly building both static and dynamic
ways, then the combination of the ABI and performance issues mean that
I'm marginally in favour of keeping static linking as the default for
executables on all platforms, but building the dynamic libraries for ghci.

Cheers,

Ganesh
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 14:13
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 06:43:09AM +0000, Ganesh Sittampalam wrote:
> On 27/11/2012 14:52, Ian Lynagh wrote:
> 
> > GHC HEAD now has support for using dynamic libraries by default (and in
> > particular, using dynamic libraries and the system linker in GHCi) for a
> > number of platforms.
> > 
> > This has some advantages and some disadvantages, so we need to make a
> > decision about what we want to do in GHC 7.8. There are also some policy
> > questions we need to answer about how Cabal will work with a GHC that
> > uses dynamic libraries by default. We would like to make these as soon
> > as possible, so that GHC 7.6.2 can ship with a Cabal that works
> > correctly.
> > 
> > The various issues are described in a wiki page here:
> >     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> If I understand the problem on Windows correctly, you can use dynamic
> libraries for ghci, but using them for compiled executables is difficult
> because there's no good solution for having the compiled exe find its DLLs.

Right.

> If so, does that mean that you intend to switch over ghci to dynamic and
> build static+dynamic by default, or are you going to leave ghci as static?

Leave ghci as static.

> My general feeling about Windows is that it's ok for the end result to
> be a little annoying, because Windows DLLs *are* annoying and it's
> nothing to do with GHC.
> 
> In particular I think in practice most Windows developers will have
> admin rights and could live with the ghc installation and cabal install
> having to be done as elevated operations. Where they weren't done with
> admin rights, then ghc -o could warn the user that the DLLs need to be
> copied locally (or even copy them itself and tell the user it happened).

Personally, I would prefer the "C stub" option to that.

> More generally, if you can implement the "half a plan" you mentioned
> elsewhere in the thread for quickly building both static and dynamic
> ways, then the combination of the ABI and performance issues mean that
> I'm marginally in favour of keeping static linking as the default for
> executables on all platforms, but building the dynamic libraries for ghci.

That would solve the "installing libraries takes twice as long" problem,
but not the "ghci can't load modules compiled with ghc -c" problem.

Thanks
Ian
Permalink | Reply |
headers
Ganesh Sittampalam | 28 Nov 2012 14:34
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On 28/11/2012 13:13, Ian Lynagh wrote:

>> My general feeling about Windows is that it's ok for the end result to
>> be a little annoying, because Windows DLLs *are* annoying and it's
>> nothing to do with GHC.
>>
>> In particular I think in practice most Windows developers will have
>> admin rights and could live with the ghc installation and cabal install
>> having to be done as elevated operations. Where they weren't done with
>> admin rights, then ghc -o could warn the user that the DLLs need to be
>> copied locally (or even copy them itself and tell the user it happened).
> 
> Personally, I would prefer the "C stub" option to that.

I think that one would be ok too, but I somewhat prefer SxS simply
because of the long-term costs of being non-standard.

>> More generally, if you can implement the "half a plan" you mentioned
>> elsewhere in the thread for quickly building both static and dynamic
>> ways, then the combination of the ABI and performance issues mean that
>> I'm marginally in favour of keeping static linking as the default for
>> executables on all platforms, but building the dynamic libraries for ghci.
> 
> That would solve the "installing libraries takes twice as long" problem,
> but not the "ghci can't load modules compiled with ghc -c" problem.

Can't ghc -c also produce both static and dynamic objects? I guess only
one of the two could be called sourcefile.o

Cheers,

Ganesh
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 17:11
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 01:34:22PM +0000, Ganesh Sittampalam wrote:
> On 28/11/2012 13:13, Ian Lynagh wrote:
> 
> >> More generally, if you can implement the "half a plan" you mentioned
> >> elsewhere in the thread for quickly building both static and dynamic
> >> ways, then the combination of the ABI and performance issues mean that
> >> I'm marginally in favour of keeping static linking as the default for
> >> executables on all platforms, but building the dynamic libraries for ghci.
> > 
> > That would solve the "installing libraries takes twice as long" problem,
> > but not the "ghci can't load modules compiled with ghc -c" problem.
> 
> Can't ghc -c also produce both static and dynamic objects?

That's true, it could. Simon will point out that build systems will
need updating if we start generating more/different files, but perhaps
the pain would be worthwhile.

I've been thinking about how to actually implement the half-plan. How
about this? (don't worry about the names I've used for things for now):

* Add a new way, "dynstatic".
  Compiling Foo this way makes
    Foo.ds_hi
    Foo.ds_s_o -- static
    Foo.ds_d_o -- dynamic
  and by default links statically. There's a flag to make it link
  dynamically instead.

* When compiling things the dynstatic way, all dependencies must be
  compiled the dynstatic way
* When compiling things the static way, dependencies can be
  compiled either the static or the dynstatic way
* When compiling things the dynamic way, dependencies can be
  compiled either the dynamic or the dynstatic way

* Cabal compiles dynstatic by default
* ghc compiles dynstatic (linking statically) by default
* ghci uses dynamic libraries on all platforms

* If we are worried about performance of the compiler, then ship ghc as
  a static binary and ghci as a separate dynamic binary. The size of the
  two combined is only 2-3% larger than just the static binary.

Thanks
Ian
Permalink | Reply |
headers
Simon Marlow | 28 Nov 2012 10:20
Picon

Re: Dynamic libraries by default and GHC 7.8

On 27/11/12 14:52, Ian Lynagh wrote:
>
> Hi all,
>
> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.
>
> This has some advantages and some disadvantages, so we need to make a
> decision about what we want to do in GHC 7.8. There are also some policy
> questions we need to answer about how Cabal will work with a GHC that
> uses dynamic libraries by default. We would like to make these as soon
> as possible, so that GHC 7.6.2 can ship with a Cabal that works
> correctly.
>
> The various issues are described in a wiki page here:
>      http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault

Thanks for doing all the experiments and putting this page together, it
certainly helps us to make a more informed decision.

> If you have a few minutes to read it then we'd be glad to hear your
> feedback, to help us in making our decisions

My personal opinion is that we should switch to dynamic-by-default on 
all x86_64 platforms, and OS X x86. The performance penalty for 
x86/Linux is too high (30%), and there are fewer bugs affecting the 
linker on that platform than OS X.

I am slightly concerned about the GC overhead on x86_64/Linux (8%), but 
I think the benefits outweigh the penalty there, and I can probably 
investigate to find out where the overhead is coming from.

Cheers,
	Simon
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 13:48
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 09:20:57AM +0000, Simon Marlow wrote:
> 
> My personal opinion is that we should switch to dynamic-by-default
> on all x86_64 platforms, and OS X x86. The performance penalty for
> x86/Linux is too high (30%),

FWIW, if they're able to move from x86 static to x86_64 dynamic then
there's only a ~15% difference overall:

Run Time
-1 s.d. -----   -18.7%
+1 s.d. -----   +60.5%
Average -----   +14.2%

Mutator Time
-1 s.d. -----   -29.0%
+1 s.d. -----   +33.7%
Average -----   -2.6%

GC Time
-1 s.d. -----   +22.0%
+1 s.d. -----   +116.1%
Average -----   +62.4%

> I am slightly concerned about the GC overhead on x86_64/Linux (8%),
> but I think the benefits outweigh the penalty there, and I can
> probably investigate to find out where the overhead is coming from.

Improving this would also help the above, of course.

Thanks
Ian
Permalink | Reply |
headers
Simon Marlow | 28 Nov 2012 14:28
Picon

Re: Dynamic libraries by default and GHC 7.8

On 28/11/12 12:48, Ian Lynagh wrote:
> On Wed, Nov 28, 2012 at 09:20:57AM +0000, Simon Marlow wrote:
>>
>> My personal opinion is that we should switch to dynamic-by-default
>> on all x86_64 platforms, and OS X x86. The performance penalty for
>> x86/Linux is too high (30%),
>
> FWIW, if they're able to move from x86 static to x86_64 dynamic then
> there's only a ~15% difference overall:
>
> Run Time
> -1 s.d. -----   -18.7%
> +1 s.d. -----   +60.5%
> Average -----   +14.2%
>
> Mutator Time
> -1 s.d. -----   -29.0%
> +1 s.d. -----   +33.7%
> Average -----   -2.6%
>
> GC Time
> -1 s.d. -----   +22.0%
> +1 s.d. -----   +116.1%
> Average -----   +62.4%

The figures on the wiki are different: x86 static -> x86_64 dynamic has 
+2.3% runtime. What's going on here?

I'm not sure I buy the argument that it's ok to penalise x86/Linux users 
by 30% because they can use x86_64 instead, which is only 15% slower. 
Unlike OS X, Linux users using the 32-bit binaries probably have a 
32-bit Linux installation, which can't run 64-bit binaries (32-bit is 
still the recommended Ubuntu installation for desktops, FWIW).

Cheers,
	Simon
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 14:43
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 01:28:54PM +0000, Simon Marlow wrote:
> On 28/11/12 12:48, Ian Lynagh wrote:
> >On Wed, Nov 28, 2012 at 09:20:57AM +0000, Simon Marlow wrote:
> >>
> >>My personal opinion is that we should switch to dynamic-by-default
> >>on all x86_64 platforms, and OS X x86.

I should have deleted the above sentence.

> >>The performance penalty for
> >>x86/Linux is too high (30%),
> >
> >FWIW, if they're able to move from x86 static to x86_64 dynamic then
> >there's only a ~15% difference overall:
> >
> >Run Time
> >-1 s.d. -----   -18.7%
> >+1 s.d. -----   +60.5%
> >Average -----   +14.2%
> >
> >Mutator Time
> >-1 s.d. -----   -29.0%
> >+1 s.d. -----   +33.7%
> >Average -----   -2.6%
> >
> >GC Time
> >-1 s.d. -----   +22.0%
> >+1 s.d. -----   +116.1%
> >Average -----   +62.4%
> 
> The figures on the wiki are different: x86 static -> x86_64 dynamic
> has +2.3% runtime. What's going on here?

+2.3% on OS X, +14.2% on Linux.

Thanks
Ian
Permalink | Reply |
headers
Manuel M T Chakravarty | 5 Dec 2012 03:33
Picon
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Simon Marlow <marlowsd <at> gmail.com>:
>> This has some advantages and some disadvantages, so we need to make a
>> decision about what we want to do in GHC 7.8. There are also some policy
>> questions we need to answer about how Cabal will work with a GHC that
>> uses dynamic libraries by default. We would like to make these as soon
>> as possible, so that GHC 7.6.2 can ship with a Cabal that works
>> correctly.
>> 
>> The various issues are described in a wiki page here:
>>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> Thanks for doing all the experiments and putting this page together, it
> certainly helps us to make a more informed decision.
> 
>> If you have a few minutes to read it then we'd be glad to hear your
>> feedback, to help us in making our decisions
> 
> My personal opinion is that we should switch to dynamic-by-default on all x86_64 platforms, and OS X x86.
The performance penalty for x86/Linux is too high (30%), and there are fewer bugs affecting the linker on
that platform than OS X.
> 
> I am slightly concerned about the GC overhead on x86_64/Linux (8%), but I think the benefits outweigh the
penalty there, and I can probably investigate to find out where the overhead is coming from.

I agree with your opinion.

Firstly, correctness is more important than performance. (It's Haskell, after all.) Secondly, the RTS
linker is an unnecessary time sink, and developer cycles are precious. Thirdly, when shipping
production systems, people can always link statically to get best performance.

Manuel
Permalink | Reply |
headers
Carter Schonwald | 5 Dec 2012 04:33
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Agreed. I'd much rather never again problems in ghci + linking,

 Its one of the biggest sources of exotic bugs for new haskellers on OS X, for all that i've never quite managed to ever pin down a simple test case to replicate those problems

point being: ghci should do normal dynamic linking for reduced engineering complexity + an end to "it doesn't work in ghci" woes that some folks periodically enncount

+++++1

-Carter


On Tue, Dec 4, 2012 at 9:33 PM, Manuel M T Chakravarty <chak <at> cse.unsw.edu.au> wrote:
Simon Marlow <marlowsd <at> gmail.com>:
>> This has some advantages and some disadvantages, so we need to make a
>> decision about what we want to do in GHC 7.8. There are also some policy
>> questions we need to answer about how Cabal will work with a GHC that
>> uses dynamic libraries by default. We would like to make these as soon
>> as possible, so that GHC 7.6.2 can ship with a Cabal that works
>> correctly.
>>
>> The various issues are described in a wiki page here:
>>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
>
> Thanks for doing all the experiments and putting this page together, it
> certainly helps us to make a more informed decision.
>
>> If you have a few minutes to read it then we'd be glad to hear your
>> feedback, to help us in making our decisions
>
> My personal opinion is that we should switch to dynamic-by-default on all x86_64 platforms, and OS X x86. The performance penalty for x86/Linux is too high (30%), and there are fewer bugs affecting the linker on that platform than OS X.
>
> I am slightly concerned about the GC overhead on x86_64/Linux (8%), but I think the benefits outweigh the penalty there, and I can probably investigate to find out where the overhead is coming from.

I agree with your opinion.

Firstly, correctness is more important than performance. (It's Haskell, after all.) Secondly, the RTS linker is an unnecessary time sink, and developer cycles are precious. Thirdly, when shipping production systems, people can always link statically to get best performance.

Manuel


_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Chris Smith | 5 Dec 2012 06:03
Picon

Re: Dynamic libraries by default and GHC 7.8

I'm curious how much of the "compile twice" situation for static and dynamic libraries could actually be shared.  Even if it's not likely to be implemented in the next year or two, IMO it would make a big difference if it were feasible to generate both static and dynamic libraries at the same time and share some large percent of the compile time.


It seems to me that making GHCi always dynamic but GHC static by default, with the default being for Cabal to build both static and dynamic libraries, is the right answer.  After all, static binaries are the right answer for binaries on everything but a few resource-limited mobile platforms anyway.  Doubling the compile times is harsh, but I think it's particularly justifiable if it were something that eventually, if it annoys someone enough, they will pound out the code to fix it.

Or... how hard would it be to make GHCi fall back to interpreting modules from any packages that don't have dynamic libraries?
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Brandon Allbery | 5 Dec 2012 16:17
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On Wed, Dec 5, 2012 at 12:03 AM, Chris Smith <cdsmith <at> gmail.com> wrote:
I'm curious how much of the "compile twice" situation for static and dynamic libraries could actually be shared.  

Probably none; on most platforms you're actually generating different code (dynamic libraries require generation of position-independent code).  That said, the PPC ABI uses position-independent code even for static libraries and I think Apple decided to go that route on Intel as well rather than change their build system ... but if you do this then linking to other platform-native libraries may be more difficult.  Not a problem for Apple since they control the ABI, but not something ghc can force on libc or random libraries someone might want to use FFI with.

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net 

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Simon Marlow | 6 Dec 2012 22:04
Picon

Re: Dynamic libraries by default and GHC 7.8

On 05/12/12 15:17, Brandon Allbery wrote:
> On Wed, Dec 5, 2012 at 12:03 AM, Chris Smith <cdsmith <at> gmail.com
> <mailto:cdsmith <at> gmail.com>> wrote:
>
>     I'm curious how much of the "compile twice" situation for static and
>     dynamic libraries could actually be shared.
>
>
> Probably none; on most platforms you're actually generating different
> code (dynamic libraries require generation of position-independent
> code).  That said, the PPC ABI uses position-independent code even for
> static libraries and I think Apple decided to go that route on Intel as
> well rather than change their build system ... but if you do this then
> linking to other platform-native libraries may be more difficult.  Not a
> problem for Apple since they control the ABI, but not something ghc can
> force on libc or random libraries someone might want to use FFI with.

Sure there's a lot of differences in the generated code, but inside GHC 
these differences only appear at the very last stage of the pipeline, 
native code generation (or LLVM).  All the stages up to that can be 
shared, which accounts for roughly 80% of compilation time (IIRC).

Cheers,
	Simon
Permalink | Reply |
headers
Brandon Allbery | 6 Dec 2012 22:35
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

On Thu, Dec 6, 2012 at 4:04 PM, Simon Marlow <marlowsd <at> gmail.com> wrote:
On 05/12/12 15:17, Brandon Allbery wrote:
Probably none; on most platforms you're actually generating different
code (dynamic libraries require generation of position-independent
Sure there's a lot of differences in the generated code, but inside GHC these differences only appear at the very last stage of the pipeline, native code generation (or LLVM).  All the stages up to that can be shared, which accounts for roughly 80% of compilation time (IIRC).

I was assuming it would be difficult to separate those stages of the internal compilation pipeline out, given previous discussions of how said pipeline works.  (In particular I was under the impression saving/restoring state in the pipeline to rerun the final phase with multiple code generators was not really possible, and multithreading them concurrently even less so.)

--
brandon s allbery kf8nh                               sine nomine associates
allbery.b <at> gmail.com                                  ballbery <at> sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net 

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Simon Marlow | 6 Dec 2012 22:39
Picon

Re: Dynamic libraries by default and GHC 7.8

On 06/12/12 21:35, Brandon Allbery wrote:
> On Thu, Dec 6, 2012 at 4:04 PM, Simon Marlow <marlowsd <at> gmail.com
> <mailto:marlowsd <at> gmail.com>> wrote:
>
>     On 05/12/12 15:17, Brandon Allbery wrote:
>
>         Probably none; on most platforms you're actually generating
>         different
>         code (dynamic libraries require generation of position-independent
>
>     Sure there's a lot of differences in the generated code, but inside
>     GHC these differences only appear at the very last stage of the
>     pipeline, native code generation (or LLVM).  All the stages up to
>     that can be shared, which accounts for roughly 80% of compilation
>     time (IIRC).
>
>
> I was assuming it would be difficult to separate those stages of the
> internal compilation pipeline out, given previous discussions of how
> said pipeline works.  (In particular I was under the impression
> saving/restoring state in the pipeline to rerun the final phase with
> multiple code generators was not really possible, and multithreading
> them concurrently even less so.)

I don't think there's any problem (unless I've forgotten something).  In 
fact, the current architecture should let us compile one function at a 
time both ways, so we don't get a space leak by retaining all the Cmm code.

Cheers,
	Simon
Permalink | Reply |
headers
Herbert Valerio Riedel | 28 Nov 2012 12:34
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Ian Lynagh <ian <at> well-typed.com> writes:

[...]

> There are also some policy questions we need to answer about how Cabal
> will work with a GHC that uses dynamic libraries by default.

btw, how is it planned to have .so libraries interact with the
soon-to-be-released cabal-install sandbox feature?
Permalink | Reply |
headers
Ian Lynagh | 28 Nov 2012 13:30
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Wed, Nov 28, 2012 at 12:34:03PM +0100, Herbert Valerio Riedel wrote:
> Ian Lynagh <ian <at> well-typed.com> writes:
> 
> [...]
> 
> > There are also some policy questions we need to answer about how Cabal
> > will work with a GHC that uses dynamic libraries by default.
> 
> btw, how is it planned to have .so libraries interact with the
> soon-to-be-released cabal-install sandbox feature?

I don't think anything special needs to be done. The RPATHs will still
point to the right DLL if it's in a sandbox.

Please let me know if there's a problem that I'm missing, though!

Thanks
Ian
Permalink | Reply |
headers
Herbert Valerio Riedel | 28 Nov 2012 22:53
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Ian Lynagh <ian <at> well-typed.com> writes:

> On Wed, Nov 28, 2012 at 12:34:03PM +0100, Herbert Valerio Riedel wrote:
>> Ian Lynagh <ian <at> well-typed.com> writes:
>> 
>> [...]
>> 
>> > There are also some policy questions we need to answer about how Cabal
>> > will work with a GHC that uses dynamic libraries by default.
>> 
>> btw, how is it planned to have .so libraries interact with the
>> soon-to-be-released cabal-install sandbox feature?
>
> I don't think anything special needs to be done. The RPATHs will still
> point to the right DLL if it's in a sandbox.
>
> Please let me know if there's a problem that I'm missing, though!

I'm not sure yet if there's a problem, but doesn't using RPATH cause
the cabal sandbox path getting hardcoded into the compiled
binaries? I'm just wondering if such a sandboxed build would be
deployable/redistributable then.

For instance, IIRC, autotools/libtools based packages use
wrapper-scripts to point the dynamic linker to the build-local (i.e. not
installed yet) shared objects via env-vars and (used to) set the RPATH
to the target install path where the shared objects will be found if the
package is actually installed.
Permalink | Reply |
headers
Johan Tibell | 29 Nov 2012 00:15
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

What does gcc do? Does it link statically or dynamically by default?
Does it depend on if it can find a dynamic version of libraries or
not?

-- Johan
Permalink | Reply |
headers
Simon Marlow | 29 Nov 2012 09:29
Picon

Re: Dynamic libraries by default and GHC 7.8

On 28/11/12 23:15, Johan Tibell wrote:
> What does gcc do? Does it link statically or dynamically by default?
> Does it depend on if it can find a dynamic version of libraries or
> not?

If it finds a dynamic library first, it links against that.

Unlike GHC, with gcc you do not have to choose at compile-time whether 
you are later going to link statically or dynamically, although you do 
choose at compile-time to make an object for a shared library (-fPIC is 
needed).

When gcc links dynamically, it assumes the binary will be able to find 
its libraries at runtime, because they're usually in /lib or /usr/lib. 
Apps that ship with their own shared libraries and don't install into 
the standard locations typically have a wrapper script that sets 
LD_LIBRARY_PATH, or they use RPATH with $ORIGIN (a better solution).

Cheers,
	Simon
Permalink | Reply |
headers
Simon Marlow | 30 Nov 2012 13:28
Picon

Re: Dynamic libraries by default and GHC 7.8

On 27/11/12 14:52, Ian Lynagh wrote:
> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.
>
> This has some advantages and some disadvantages, so we need to make a
> decision about what we want to do in GHC 7.8. There are also some policy
> questions we need to answer about how Cabal will work with a GHC that
> uses dynamic libraries by default. We would like to make these as soon
> as possible, so that GHC 7.6.2 can ship with a Cabal that works
> correctly.
>
> The various issues are described in a wiki page here:
>      http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
>
> If you have a few minutes to read it then we'd be glad to hear your
> feedback, to help us in making our decisions

It's hard to know what the best course of action is, because all the 
options have downsides.

Current situation:
  * fast code and compiler
  * but there are bugs in GHCi that are hard to fix, and an ongoing
    maintenance problem (the RTS linker).
  * binaries are not broken by library updates

Switching to dynamic:
  * slower code and compiler (by varying amounts depending
    on the platform)
  * but several bugs in GHCi are fixed, no RTS linker needed
  * binaries can be broken by library updates
  * can't do it on Windows (as far as we know)

Perhaps we should look again at the option that we discarded: making 
-static the default, and require a special option to build objects for 
use in GHCi.  If we also build packages both static+dynamic at the same 
time in Cabal, this might be a good compromise.

Static by default, GHCi is dynamic:
  * fast code and compiler
  * GHCi bugs are fixed, no maintenance problems
  * binaries not broken by library updates
  * we have to build packages twice in Cabal (but can improve GHC to
    emit both objects from a single compilation)
  * BUT, objects built with 'ghc -c' cannot be loaded into GHCi unless
    also built with -dynamic.
  * still can't do this on Windows

Cheers,
	Simon
Permalink | Reply |
headers
Herbert Valerio Riedel | 30 Nov 2012 13:38
Picon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Simon Marlow <marlowsd <at> gmail.com> writes:

[...]

> Perhaps we should look again at the option that we discarded: making
> -static the default, and require a special option to build objects for
> use in GHCi.  If we also build packages both static+dynamic at the
> same time in Cabal, this might be a good compromise.
>
> Static by default, GHCi is dynamic:
[...]

+1
Permalink | Reply |
headers
Ian Lynagh | 30 Nov 2012 13:43
Favicon

Re: Dynamic libraries by default and GHC 7.8

On Fri, Nov 30, 2012 at 12:28:41PM +0000, Simon Marlow wrote:
> 
> Static by default, GHCi is dynamic:
>  * still can't do this on Windows

We can do it on Windows: We can use side-by-side assemblies.

(well, assuming we fix #5987).

Thanks
Ian
Permalink | Reply |
headers
Joachim Breitner | 30 Nov 2012 18:10
Picon
Favicon
Gravatar

Re: Dynamic libraries by default and GHC 7.8

Hi,

Am Freitag, den 30.11.2012, 12:28 +0000 schrieb Simon Marlow:
> 
> Static by default, GHCi is dynamic:
>   * fast code and compiler
>   * GHCi bugs are fixed, no maintenance problems
>   * binaries not broken by library updates
>   * we have to build packages twice in Cabal (but can improve GHC to
>     emit both objects from a single compilation)
>   * BUT, objects built with 'ghc -c' cannot be loaded into GHCi unless
>     also built with -dynamic.
>   * still can't do this on Windows 

from Debians pov this would be most welcome.

Greetings,
Joachim

--

-- 
Joachim "nomeata" Breitner
Debian Developer
  nomeata <at> debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
  JID: nomeata <at> joachim-breitner.de | http://people.debian.org/~nomeata

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |
headers
Christian Hoener zu Siederdissen | 30 Nov 2012 16:42
Picon
Picon
Favicon

Re: Dynamic libraries by default and GHC 7.8

Hi,

a bit late but here are my comments:

- my main (and in a way ``only'') concern is speed. At some point I'd
  like my programs to consistently beat the pants off C ...
  This won't happen soon, but when comparing to C it makes a difference
  being able to say Haskell is x1.3 slower or x1.4 slower.

- BUT performance is required only for the end product, where I would --
  in addition -- prefer statically built binaries. Getting other people
  to install GHC 7.6 (platform is not good enough!) and do the whole
  installation process won't work.

- development is done using cabal-dev, so I am compiling libraries 10-20
  times anyway. Once more for dynamic/statis wouldn't matter that much.

Gruss,
Christian

* Ian Lynagh <ian <at> well-typed.com> [27.11.2012 15:55]:
> 
> Hi all,
> 
> GHC HEAD now has support for using dynamic libraries by default (and in
> particular, using dynamic libraries and the system linker in GHCi) for a
> number of platforms.
> 
> This has some advantages and some disadvantages, so we need to make a
> decision about what we want to do in GHC 7.8. There are also some policy
> questions we need to answer about how Cabal will work with a GHC that
> uses dynamic libraries by default. We would like to make these as soon
> as possible, so that GHC 7.6.2 can ship with a Cabal that works
> correctly.
> 
> The various issues are described in a wiki page here:
>     http://hackage.haskell.org/trac/ghc/wiki/DynamicByDefault
> 
> If you have a few minutes to read it then we'd be glad to hear your
> feedback, to help us in making our decisions
> 
> 
> Thanks
> Ian
> -- 
> Ian Lynagh, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
> 
> 
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users <at> haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Permalink | Reply |

Gmane