Bryan O'Sullivan | 30 Aug 07:26 2012

Build regressions due to GHC 7.6

Since the release of the GHC 7.6 RC, I've been going through my packages and fixing up build problems so that people who upgrade to 7.6 will have a smooth ride.


Sad to say, my experience of 7.6 is that it has felt like a particularly rough release for backwards incompatibility. I wanted to quantify the pain, so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have needed updates due to the GHC 7.6 release:
  • base16-bytestring
  • blaze-textual
  • bloomfilter
  • configurator
  • criterion
  • double-conversion
  • filemanip
  • HDBC-mysql
  • mwc-random
  • pcap
  • pool
  • riak-haskell-client
  • snappy
  • text
  • text-format
  • text-icu
That's 16 out of 25 packages I've had to update. I've also either reported bugs on, or had to fix, several other people's packages along the way (maybe four?). So let's say I've run into problems with 20 out of the combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:
  • Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change.
  • The FFI now requires constructors to be visible, so "CInt" has to be imported as "CInt(..)".
  • bytestring finally got bumped to 0.10, so many upper bounds had to be relaxed (cf my suggestion that the upper-bounds-by-default policy is destructive).
It has been a lot of work to test 29 packages, and then modify, rebuild, and release 20 of them. It has consumed most of my limited free time for almost two weeks. Worse, this has felt like make-work, of no practical benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the platform by tidying things up, and yet just two seemingly innocuous changes (catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to estimate the downstream effects of what look like small changes. And so I'm not exactly complaining. Call this an unhappy data point.
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
MightyByte | 30 Aug 08:18 2012
Picon

Re: Build regressions due to GHC 7.6

Interesting data point.  I think my initial thoughts can be summarized with the suggestion that this thread would be better served by a little irony and a new subject: "Reuse Considered Harmful".


On Thu, Aug 30, 2012 at 1:26 AM, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
Since the release of the GHC 7.6 RC, I've been going through my packages and fixing up build problems so that people who upgrade to 7.6 will have a smooth ride.

Sad to say, my experience of 7.6 is that it has felt like a particularly rough release for backwards incompatibility. I wanted to quantify the pain, so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have needed updates due to the GHC 7.6 release:
  • base16-bytestring
  • blaze-textual
  • bloomfilter
  • configurator
  • criterion
  • double-conversion
  • filemanip
  • HDBC-mysql
  • mwc-random
  • pcap
  • pool
  • riak-haskell-client
  • snappy
  • text
  • text-format
  • text-icu
That's 16 out of 25 packages I've had to update. I've also either reported bugs on, or had to fix, several other people's packages along the way (maybe four?). So let's say I've run into problems with 20 out of the combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:
  • Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change.
  • The FFI now requires constructors to be visible, so "CInt" has to be imported as "CInt(..)".
  • bytestring finally got bumped to 0.10, so many upper bounds had to be relaxed (cf my suggestion that the upper-bounds-by-default policy is destructive).
It has been a lot of work to test 29 packages, and then modify, rebuild, and release 20 of them. It has consumed most of my limited free time for almost two weeks. Worse, this has felt like make-work, of no practical benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the platform by tidying things up, and yet just two seemingly innocuous changes (catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to estimate the downstream effects of what look like small changes. And so I'm not exactly complaining. Call this an unhappy data point.

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Ivan Lazar Miljenovic | 30 Aug 08:42 2012
Picon

Re: Build regressions due to GHC 7.6

On 30 August 2012 15:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
> The reasons for these problems fall into three bins:
>
> Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)"
> had to change.

It looks like this might be fixed before the release:
http://hackage.haskell.org/trac/ghc/ticket/7167

--

-- 
Ivan Lazar Miljenovic
Ivan.Miljenovic <at> gmail.com
http://IvanMiljenovic.wordpress.com
Richard O'Keefe | 30 Aug 10:17 2012
Picon

Re: Build regressions due to GHC 7.6


On 30/08/2012, at 5:26 PM, Bryan O'Sullivan wrote:
> The reasons for these problems fall into three bins:
> 	• Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change.

This could have been avoided if

	import <module> hiding (<importables>)

were interpreted simply as a requiring that the specified <importables>
not be imported *whether they could have been or not* rather than as
requiring that they exist to be sneered at.  It seems rather odd that
the removable of something that a module insists it doesn't want should
break that module.
Erik Hesselink | 30 Aug 12:18 2012
Picon

Re: Build regressions due to GHC 7.6

On Thu, Aug 30, 2012 at 7:26 AM, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
> The FFI now requires constructors to be visible, so "CInt" has to be
> imported as "CInt(..)".

I think there was already a warning about this one in GHC 7.4, so
there was more time to fix it. Not to say I don't feel your pain, but
if deprecating/warning and then removing in the next release isn't
good enough, nothing can ever be changed.

Erik
Alexander Kjeldaas | 30 Aug 15:03 2012
Picon

Re: Build regressions due to GHC 7.6

This is very unfortunate, but this is crucially a tooling issue.  I am going to wave my hands, but..

Ignore the mapreduce in the following video, but look at the use of clang to do automatic refactoring of C++.  This is *incredibly* powerful in dealing with updates to APIs.


But without all that fancy tech, *just* having all of Hackage source code in one repository and using perl/regexps, fixing these types of issues is O(1) instead of O(n).

All of the issues you mention seems to be fixable by a few lines of perl *if we had the repository*.

[a few hours later]

Actually, I went and downloaded all of hackage, put it into a git repository and fixed these issues:

Fix catch
perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep 'import Prelude hiding (catch)')

Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed though)
perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/import Foreign.C.Types${1}CInt(..)${1}/g' $(git grep -l '^import.*CInt')

Fix bytestring versioning
perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([ ]*)0.10/bytestring$1>=$2<${3}0.11/g' $(git grep 'bytestring.*< *0\.')

Patch to hackage:

I understand that this doesn't help anyone, but if there was a way fix, upload, and get *consensus* on a few regexps like this, then doing API changes wouldn't be such a headache.

Alexander

On 30 August 2012 07:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
Since the release of the GHC 7.6 RC, I've been going through my packages and fixing up build problems so that people who upgrade to 7.6 will have a smooth ride.

Sad to say, my experience of 7.6 is that it has felt like a particularly rough release for backwards incompatibility. I wanted to quantify the pain, so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have needed updates due to the GHC 7.6 release:
  • base16-bytestring
  • blaze-textual
  • bloomfilter
  • configurator
  • criterion
  • double-conversion
  • filemanip
  • HDBC-mysql
  • mwc-random
  • pcap
  • pool
  • riak-haskell-client
  • snappy
  • text
  • text-format
  • text-icu
That's 16 out of 25 packages I've had to update. I've also either reported bugs on, or had to fix, several other people's packages along the way (maybe four?). So let's say I've run into problems with 20 out of the combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:
  • Prelude no longer exports catch, so a lot of "import Prelude hiding (catch)" had to change.
  • The FFI now requires constructors to be visible, so "CInt" has to be imported as "CInt(..)".
  • bytestring finally got bumped to 0.10, so many upper bounds had to be relaxed (cf my suggestion that the upper-bounds-by-default policy is destructive).
It has been a lot of work to test 29 packages, and then modify, rebuild, and release 20 of them. It has consumed most of my limited free time for almost two weeks. Worse, this has felt like make-work, of no practical benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the platform by tidying things up, and yet just two seemingly innocuous changes (catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to estimate the downstream effects of what look like small changes. And so I'm not exactly complaining. Call this an unhappy data point.

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Alexander Bernauer | 30 Aug 15:34 2012
Picon

Re: Build regressions due to GHC 7.6

Hi

I agree that automatic code migration can solve this issue in large
parts. The Python folks have done this to mitigate the transition from
version 2 to version 3 [1].

On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:
> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
> 'import Prelude hiding (catch)')

I don't think regular expressions are powerful enough. This example
does not match on hiding multiple names, for instance.

But writing proper 'HsModule -> HsModule' functions should be doable.

And when each release comes with a bunch of such functions, packages
could be automatically migrated.

Greetings

Alex

[1] http://docs.python.org/library/2to3.html
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Alexander Kjeldaas | 30 Aug 15:47 2012
Picon

Re: Build regressions due to GHC 7.6

On 30 August 2012 15:34, Alexander Bernauer <alex-haskell <at> copton.net> wrote:
Hi

I agree that automatic code migration can solve this issue in large
parts. The Python folks have done this to mitigate the transition from
version 2 to version 3 [1].

On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:
> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
> 'import Prelude hiding (catch)')

I don't think regular expressions are powerful enough. This example
does not match on hiding multiple names, for instance.


This was just an example, the CInt regexp handles multiple names, so regexps have no problems handling this.

And it's a simple "git grep 'import.*Prelude.*catch'" to see if this actually is a problem or not.

My point is that this works, fixes 99% of the cases, and is 1000x less work overall.

Alexander

 
But writing proper 'HsModule -> HsModule' functions should be doable.

And when each release comes with a bunch of such functions, packages
could be automatically migrated.

Greetings

Alex

[1] http://docs.python.org/library/2to3.html

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAlA/a/oACgkQevm6Dd/q44nJXQCffaxEJ/NZEftgoZ7viAWMuBO3
+jkAnRTw+VCMQn1k9NibyKpkGMtwvrQw
=ds3M
-----END PGP SIGNATURE-----

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Michael Sloan | 30 Aug 21:16 2012
Picon

Re: Build regressions due to GHC 7.6

A good way to specify such refactorings is as a Haskell module.  For example:

module PreludePre_7_6 where

import Prelude hiding ( catch )

Or, an example of avoiding the Eq / Show / Num debacle:

module PreludePre_7_4 (module Prelude, Num) where

import Prelude hiding ( Num )
import qualified Prelude as P

instance template (Eq a, Show a) => Num a where
  instance P.Num a

(Uses my instance template proposal - though with different keywords -
https://github.com/mgsloan/instance-templates )

Let's say we wanted to rename "catch" to "unsafeCatch"

module PreludePreUnsafeCatch (module Prelude, catch) where

import Prelude hiding (unsafeCatch)

catch = unsafeCatch

Then, an automated refactoring system is "simply" a sourcecode-based
module inliner.  Quite a bit of effort, certainly, but it's much
better than coming up with a whole new notation for refactorings.

-Michael

On Thu, Aug 30, 2012 at 6:34 AM, Alexander Bernauer
<alex-haskell <at> copton.net> wrote:
> Hi
>
> I agree that automatic code migration can solve this issue in large
> parts. The Python folks have done this to mitigate the transition from
> version 2 to version 3 [1].
>
> On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:
>> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
>> 'import Prelude hiding (catch)')
>
> I don't think regular expressions are powerful enough. This example
> does not match on hiding multiple names, for instance.
>
> But writing proper 'HsModule -> HsModule' functions should be doable.
>
> And when each release comes with a bunch of such functions, packages
> could be automatically migrated.
>
> Greetings
>
> Alex
>
> [1] http://docs.python.org/library/2to3.html
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAlA/a/oACgkQevm6Dd/q44nJXQCffaxEJ/NZEftgoZ7viAWMuBO3
> +jkAnRTw+VCMQn1k9NibyKpkGMtwvrQw
> =ds3M
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
Michael Sloan | 30 Aug 21:19 2012
Picon

Re: Build regressions due to GHC 7.6

Whoops, I messed up that first example:

module PreludePre_7_6 (module Prelude, catch) where

import Prelude
import qualified System.IO.Error as E

catch = E.catch

On Thu, Aug 30, 2012 at 12:16 PM, Michael Sloan <mgsloan <at> gmail.com> wrote:
> A good way to specify such refactorings is as a Haskell module.  For example:
>
>
> module PreludePre_7_6 where
>
> import Prelude hiding ( catch )
>
>
> Or, an example of avoiding the Eq / Show / Num debacle:
>
> module PreludePre_7_4 (module Prelude, Num) where
>
> import Prelude hiding ( Num )
> import qualified Prelude as P
>
> instance template (Eq a, Show a) => Num a where
>   instance P.Num a
>
>
> (Uses my instance template proposal - though with different keywords -
> https://github.com/mgsloan/instance-templates )
>
>
> Let's say we wanted to rename "catch" to "unsafeCatch"
>
> module PreludePreUnsafeCatch (module Prelude, catch) where
>
> import Prelude hiding (unsafeCatch)
>
> catch = unsafeCatch
>
>
> Then, an automated refactoring system is "simply" a sourcecode-based
> module inliner.  Quite a bit of effort, certainly, but it's much
> better than coming up with a whole new notation for refactorings.
>
> -Michael
>
> On Thu, Aug 30, 2012 at 6:34 AM, Alexander Bernauer
> <alex-haskell <at> copton.net> wrote:
>> Hi
>>
>> I agree that automatic code migration can solve this issue in large
>> parts. The Python folks have done this to mitigate the transition from
>> version 2 to version 3 [1].
>>
>> On Thu, Aug 30, 2012 at 03:03:05PM +0200, Alexander Kjeldaas wrote:
>>> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
>>> 'import Prelude hiding (catch)')
>>
>> I don't think regular expressions are powerful enough. This example
>> does not match on hiding multiple names, for instance.
>>
>> But writing proper 'HsModule -> HsModule' functions should be doable.
>>
>> And when each release comes with a bunch of such functions, packages
>> could be automatically migrated.
>>
>> Greetings
>>
>> Alex
>>
>> [1] http://docs.python.org/library/2to3.html
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.10 (GNU/Linux)
>>
>> iEYEARECAAYFAlA/a/oACgkQevm6Dd/q44nJXQCffaxEJ/NZEftgoZ7viAWMuBO3
>> +jkAnRTw+VCMQn1k9NibyKpkGMtwvrQw
>> =ds3M
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe <at> haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
Erik Hesselink | 30 Aug 16:26 2012
Picon

Re: Build regressions due to GHC 7.6

Note that this does not work if you want to support multiple versions
of GHC, and might not work in general, since

* The hiding of catch is needed for preludes that still have it, since
otherwise it will probably conflict with the one from
Control.Exception.
* Older versions do not have constructors for the FFI types, leading
to warnings (or errors, I'm not sure).
* Packages might not work with the new bytestring version, since the
API has breaking changes (although they're supposed to be minor).

For the first two, you need to add some CPP, for the last, you need
actual testing.

Erik

On Thu, Aug 30, 2012 at 3:03 PM, Alexander Kjeldaas
<alexander.kjeldaas <at> gmail.com> wrote:
> This is very unfortunate, but this is crucially a tooling issue.  I am going
> to wave my hands, but..
>
> Ignore the mapreduce in the following video, but look at the use of clang to
> do automatic refactoring of C++.  This is *incredibly* powerful in dealing
> with updates to APIs.
>
> http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4
>
> But without all that fancy tech, *just* having all of Hackage source code in
> one repository and using perl/regexps, fixing these types of issues is O(1)
> instead of O(n).
>
> All of the issues you mention seems to be fixable by a few lines of perl *if
> we had the repository*.
>
> [a few hours later]
>
> Actually, I went and downloaded all of hackage, put it into a git repository
> and fixed these issues:
>
> Fix catch
> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
> 'import Prelude hiding (catch)')
>
> Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed
> though)
> perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/import
> Foreign.C.Types${1}CInt(..)${1}/g' $(git grep -l '^import.*CInt')
>
> Fix bytestring versioning
> perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([
> ]*)0.10/bytestring$1>=$2<${3}0.11/g' $(git grep 'bytestring.*< *0\.')
>
> Patch to hackage:
> http://ge.tt/6Cb5ErM/v/0
>
> I understand that this doesn't help anyone, but if there was a way fix,
> upload, and get *consensus* on a few regexps like this, then doing API
> changes wouldn't be such a headache.
>
> Alexander
>
> On 30 August 2012 07:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
>>
>> Since the release of the GHC 7.6 RC, I've been going through my packages
>> and fixing up build problems so that people who upgrade to 7.6 will have a
>> smooth ride.
>>
>> Sad to say, my experience of 7.6 is that it has felt like a particularly
>> rough release for backwards incompatibility. I wanted to quantify the pain,
>> so I did some research, and here's what I found.
>>
>> I maintain 25 open source Haskell packages. Of these, the majority have
>> needed updates due to the GHC 7.6 release:
>>
>> base16-bytestring
>> blaze-textual
>> bloomfilter
>> configurator
>> criterion
>> double-conversion
>> filemanip
>> HDBC-mysql
>> mwc-random
>> pcap
>> pool
>> riak-haskell-client
>> snappy
>> text
>> text-format
>> text-icu
>>
>> That's 16 out of 25 packages I've had to update. I've also either reported
>> bugs on, or had to fix, several other people's packages along the way (maybe
>> four?). So let's say I've run into problems with 20 out of the combined 29
>> packages of mine and my upstreams.
>>
>> The reasons for these problems fall into three bins:
>>
>> Prelude no longer exports catch, so a lot of "import Prelude hiding
>> (catch)" had to change.
>> The FFI now requires constructors to be visible, so "CInt" has to be
>> imported as "CInt(..)".
>> bytestring finally got bumped to 0.10, so many upper bounds had to be
>> relaxed (cf my suggestion that the upper-bounds-by-default policy is
>> destructive).
>>
>> It has been a lot of work to test 29 packages, and then modify, rebuild,
>> and release 20 of them. It has consumed most of my limited free time for
>> almost two weeks. Worse, this has felt like make-work, of no practical
>> benefit to anyone beyond scrambling to restore the status quo ante.
>>
>> If over half of my packages needed fixing, I'm alarmed at the thought of
>> the effects on the rest of Hackage.
>>
>> I'm torn over this. I understand and agree with the impetus to improve the
>> platform by tidying things up, and yet just two seemingly innocuous changes
>> (catch and FFI) have forced me to do a bunch of running to stand still.
>>
>> I don't have any suggestions about what to do; I know that it's hard to
>> estimate the downstream effects of what look like small changes. And so I'm
>> not exactly complaining. Call this an unhappy data point.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe <at> haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
wren ng thornton | 30 Aug 19:24 2012

Re: Build regressions due to GHC 7.6

On 8/30/12 10:26 AM, Erik Hesselink wrote:
> Note that this does not work if you want to support multiple versions
> of GHC, and might not work in general, since
>
> * The hiding of catch is needed for preludes that still have it, since
> otherwise it will probably conflict with the one from
> Control.Exception.
> * Older versions do not have constructors for the FFI types, leading
> to warnings (or errors, I'm not sure).

In 6.12, they're warnings. And those warnings only show up with -Wall 
enabled. Though there does not appear to be an appropriate -fno-warn-foo 
to disable only these warnings while leaving the rest of -Wall intact. 
(If anyone knows of such a flag in 6.12, I'd be much obliged.)

> * Packages might not work with the new bytestring version, since the
> API has breaking changes (although they're supposed to be minor).
>
> For the first two, you need to add some CPP, for the last, you need
> actual testing.

Actually, that can be more problematic than you suspect. In particular, 
hsc2hs does not play nicely with Cabal's MIN_VERSION_foo(1,2,3) macros. 
So if you're using hsc2hs, which is common for certain FFI uses, then 
you have to put up with the warnings or rely on the __GLASGOW_HASKELL__ 
macro in lieu of the MIN_VERSION_foo(1,2,3) macros. It's doable, but 
it's uglier and more hackish than it ought to be.

--

-- 
Live well,
~wren
Erik Hesselink | 30 Aug 21:41 2012
Picon

Re: Build regressions due to GHC 7.6

On Thu, Aug 30, 2012 at 7:24 PM, wren ng thornton <wren <at> freegeek.org> wrote:
> On 8/30/12 10:26 AM, Erik Hesselink wrote:
>> * Packages might not work with the new bytestring version, since the
>> API has breaking changes (although they're supposed to be minor).
>>
>> For the first two, you need to add some CPP, for the last, you need
>> actual testing.
>
> Actually, that can be more problematic than you suspect. In particular,
> hsc2hs does not play nicely with Cabal's MIN_VERSION_foo(1,2,3) macros. So
> if you're using hsc2hs, which is common for certain FFI uses, then you have
> to put up with the warnings or rely on the __GLASGOW_HASKELL__ macro in lieu
> of the MIN_VERSION_foo(1,2,3) macros. It's doable, but it's uglier and more
> hackish than it ought to be.

Ah yes, I found this out a while ago on one of my packages. As a tip
for those doing this: the value of __GLASGOW_HASKELL__ is xyy, with x
the first version component, and y the second. So, say, 7.4.2 has a
value of 704, not 742.

Erik
Jay Sulzberger | 30 Aug 22:26 2012
Picon

Re: Build regressions due to GHC 7.6


On Thu, 30 Aug 2012, Alexander Kjeldaas <alexander.kjeldaas <at> gmail.com> wrote:

> This is very unfortunate, but this is crucially a tooling issue.  I am
> going to wave my hands, but..
>
> Ignore the mapreduce in the following video, but look at the use of clang
> to do automatic refactoring of C++.  This is *incredibly* powerful in
> dealing with updates to APIs.
>
> http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4
>
> But without all that fancy tech, *just* having all of Hackage source code
> in one repository and using perl/regexps, fixing these types of issues is
> O(1) instead of O(n).
>
> All of the issues you mention seems to be fixable by a few lines of perl
> *if we had the repository*.

Better to do this with sexps.

ad repositories: Part of the general problem of managing a
repository is close to the problem of inferring a good type for
(the value of) an expression.  The style of constraints is
similar.  Now the design problem is:

1. Arrange a general system for the specification of the
    constraints.

2. Design a systematic way of giving both advice and direct
    commands to the system.  This subsystem would be used to set
    up the default policy.

3. Choose a constraint solver.

Maybe worth looking at:

   http://en.wikipedia.org/wiki/Nix_package_manager
   [page was last modified on 17 July 2012 at 20:20]

oo--JS.

>
> [a few hours later]
>
> Actually, I went and downloaded all of hackage, put it into a git
> repository and fixed these issues:
>
> Fix catch
> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
> 'import Prelude hiding (catch)')
>
> Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed
> though)
> perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/import
> Foreign.C.Types${1}CInt(..)${1}/g' $(git grep -l '^import.*CInt')
>
> Fix bytestring versioning
> perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([
> ]*)0.10/bytestring$1>=$2<${3}0.11/g' $(git grep 'bytestring.*< *0\.')
>
> Patch to hackage:
> http://ge.tt/6Cb5ErM/v/0
>
> I understand that this doesn't help anyone, but if there was a way fix,
> upload, and get *consensus* on a few regexps like this, then doing API
> changes wouldn't be such a headache.
>
> Alexander
>
> On 30 August 2012 07:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
>
>> Since the release of the GHC 7.6 RC, I've been going through my packages
>> and fixing up build problems so that people who upgrade to 7.6 will have a
>> smooth ride.
>>
>> Sad to say, my experience of 7.6 is that it has felt like a particularly
>> rough release for backwards incompatibility. I wanted to quantify the pain,
>> so I did some research, and here's what I found.
>>
>> I maintain 25 open source Haskell packages. Of these, the majority have
>> needed updates due to the GHC 7.6 release:
>>
>>    - base16-bytestring
>>    - blaze-textual
>>    - bloomfilter
>>    - configurator
>>    - criterion
>>    - double-conversion
>>    - filemanip
>>    - HDBC-mysql
>>    - mwc-random
>>    - pcap
>>    - pool
>>    - riak-haskell-client
>>    - snappy
>>    - text
>>    - text-format
>>    - text-icu
>>
>> That's 16 out of 25 packages I've had to update. I've also either reported
>> bugs on, or had to fix, several other people's packages along the way
>> (maybe four?). So let's say I've run into problems with 20 out of the
>> combined 29 packages of mine and my upstreams.
>>
>> The reasons for these problems fall into three bins:
>>
>>    - Prelude no longer exports catch, so a lot of "import Prelude hiding
>>    (catch)" had to change.
>>    - The FFI now requires constructors to be visible, so "CInt" has to be
>>    imported as "CInt(..)".
>>    - bytestring finally got bumped to 0.10, so many upper bounds had to
>>    be relaxed (*cf* my suggestion that the upper-bounds-by-default policy
>>    is destructive).
>>
>> It has been a lot of work to test 29 packages, and then modify, rebuild,
>> and release 20 of them. It has consumed most of my limited free time for
>> almost two weeks. Worse, this has felt like make-work, of no practical
>> benefit to anyone beyond scrambling to restore the status quo ante.
>>
>> If over half of my packages needed fixing, I'm alarmed at the thought of
>> the effects on the rest of Hackage.
>>
>> I'm torn over this. I understand and agree with the impetus to improve the
>> platform by tidying things up, and yet just two seemingly innocuous changes
>> (catch and FFI) have forced me to do a bunch of running to stand still.
>>
>> I don't have any suggestions about what to do; I know that it's hard to
>> estimate the downstream effects of what look like small changes. And so I'm
>> not exactly complaining. Call this an unhappy data point.
>>
>> _______________________________________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe <at> haskell.org
>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>
>>
>
Alexander Kjeldaas | 31 Aug 09:22 2012
Picon

Re: Build regressions due to GHC 7.6


I think you're making this way harder than it really is.

What 99% of people need is that hackage packages builds with the latest haskell platform, and/or with bleeding edge ghc, and with the latest versions of its dependencies.

Thus for every dependency there is only one possible version - the latest one, and there are only a couple of compilers.

Having "special interest groups" for ghc 6.12 support and old versions of text is fine, but I think it is a pretty uninteresting problem to solve.

Likewise, supporting/fixing packages where the author for some reason *requires* use of a non-current version of some other package is also quite uninteresting (or at least outside the scope of my needs).   Such a package is basically just a relic.

Alexander

On 30 August 2012 22:26, Jay Sulzberger <jays <at> panix.com> wrote:


On Thu, 30 Aug 2012, Alexander Kjeldaas <alexander.kjeldaas <at> gmail.com> wrote:

This is very unfortunate, but this is crucially a tooling issue.  I am
going to wave my hands, but..

Ignore the mapreduce in the following video, but look at the use of clang
to do automatic refactoring of C++.  This is *incredibly* powerful in
dealing with updates to APIs.

http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4

But without all that fancy tech, *just* having all of Hackage source code
in one repository and using perl/regexps, fixing these types of issues is
O(1) instead of O(n).

All of the issues you mention seems to be fixable by a few lines of perl
*if we had the repository*.

Better to do this with sexps.

ad repositories: Part of the general problem of managing a
repository is close to the problem of inferring a good type for
(the value of) an expression.  The style of constraints is
similar.  Now the design problem is:

1. Arrange a general system for the specification of the
   constraints.

2. Design a systematic way of giving both advice and direct
   commands to the system.  This subsystem would be used to set
   up the default policy.

3. Choose a constraint solver.

Maybe worth looking at:

  http://en.wikipedia.org/wiki/Nix_package_manager
  [page was last modified on 17 July 2012 at 20:20]

oo--JS.



[a few hours later]

Actually, I went and downloaded all of hackage, put it into a git
repository and fixed these issues:

Fix catch
perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
'import Prelude hiding (catch)')

Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed
though)
perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/import
Foreign.C.Types${1}CInt(..)${1}/g' $(git grep -l '^import.*CInt')

Fix bytestring versioning
perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([
]*)0.10/bytestring$1>=$2<${3}0.11/g' $(git grep 'bytestring.*< *0\.')

Patch to hackage:
http://ge.tt/6Cb5ErM/v/0

I understand that this doesn't help anyone, but if there was a way fix,
upload, and get *consensus* on a few regexps like this, then doing API
changes wouldn't be such a headache.

Alexander

On 30 August 2012 07:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:

Since the release of the GHC 7.6 RC, I've been going through my packages
and fixing up build problems so that people who upgrade to 7.6 will have a
smooth ride.

Sad to say, my experience of 7.6 is that it has felt like a particularly
rough release for backwards incompatibility. I wanted to quantify the pain,
so I did some research, and here's what I found.

I maintain 25 open source Haskell packages. Of these, the majority have
needed updates due to the GHC 7.6 release:

   - base16-bytestring
   - blaze-textual
   - bloomfilter
   - configurator
   - criterion
   - double-conversion
   - filemanip
   - HDBC-mysql
   - mwc-random
   - pcap
   - pool
   - riak-haskell-client
   - snappy
   - text
   - text-format
   - text-icu


That's 16 out of 25 packages I've had to update. I've also either reported
bugs on, or had to fix, several other people's packages along the way
(maybe four?). So let's say I've run into problems with 20 out of the
combined 29 packages of mine and my upstreams.

The reasons for these problems fall into three bins:

   - Prelude no longer exports catch, so a lot of "import Prelude hiding
   (catch)" had to change.
   - The FFI now requires constructors to be visible, so "CInt" has to be
   imported as "CInt(..)".
   - bytestring finally got bumped to 0.10, so many upper bounds had to
   be relaxed (*cf* my suggestion that the upper-bounds-by-default policy

   is destructive).

It has been a lot of work to test 29 packages, and then modify, rebuild,
and release 20 of them. It has consumed most of my limited free time for
almost two weeks. Worse, this has felt like make-work, of no practical
benefit to anyone beyond scrambling to restore the status quo ante.

If over half of my packages needed fixing, I'm alarmed at the thought of
the effects on the rest of Hackage.

I'm torn over this. I understand and agree with the impetus to improve the
platform by tidying things up, and yet just two seemingly innocuous changes
(catch and FFI) have forced me to do a bunch of running to stand still.

I don't have any suggestions about what to do; I know that it's hard to
estimate the downstream effects of what look like small changes. And so I'm
not exactly complaining. Call this an unhappy data point.

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Jay Sulzberger | 1 Sep 02:56 2012
Picon

Re: Build regressions due to GHC 7.6


On Fri, 31 Aug 2012, Alexander Kjeldaas <alexander.kjeldaas <at> gmail.com> wrote:

> I think you're making this way harder than it really is.
>
> What 99% of people need is that hackage packages builds with the latest
> haskell platform, and/or with bleeding edge ghc, and with the latest
> versions of its dependencies.
>
> Thus for every dependency there is only one possible version - the latest
> one, and there are only a couple of compilers.
>
> Having "special interest groups" for ghc 6.12 support and old versions of
> text is fine, but I think it is a pretty uninteresting problem to solve.
>
> Likewise, supporting/fixing packages where the author for some reason
> *requires* use of a non-current version of some other package is also quite
> uninteresting (or at least outside the scope of my needs).   Such a package
> is basically just a relic.
>
> Alexander

If that is the set of constraints you must meet, namely that for
every library you wish to use, the same platform is specified as
the only platform you want, yes, I agree.  I am also sympathetic
to imposing such tight management of the repository.  (This tight
management could consist of just well checked annotations as to
what works with what.)  But there are other cases, for example,
testing various combinations of libraries for speed and memory
use.  And such a flexible tool will be of use when the Move To
One Platform comes.  Being able to pick up code and modify code,
and to write code to do this, is part of the tradition today
called "functional programming".  The Cut Over would be done with
a Big Haskell Program that would test/re-write/test etc. until
every line of code was updated and a database of interactions
produced and made available in convenient form.

oo--JS.

>
> On 30 August 2012 22:26, Jay Sulzberger <jays <at> panix.com> wrote:
>
>>
>>
>> On Thu, 30 Aug 2012, Alexander Kjeldaas <alexander.kjeldaas <at> gmail.com>
>> wrote:
>>
>>  This is very unfortunate, but this is crucially a tooling issue.  I am
>>> going to wave my hands, but..
>>>
>>> Ignore the mapreduce in the following video, but look at the use of clang
>>> to do automatic refactoring of C++.  This is *incredibly* powerful in
>>> dealing with updates to APIs.
>>>
>>> http://www.llvm.org/devmtg/**2011-11/videos/Carruth_**
>>> ClangMapReduce-desktop.mp4<http://www.llvm.org/devmtg/2011-11/videos/Carruth_ClangMapReduce-desktop.mp4>
>>>
>>> But without all that fancy tech, *just* having all of Hackage source code
>>> in one repository and using perl/regexps, fixing these types of issues is
>>> O(1) instead of O(n).
>>>
>>> All of the issues you mention seems to be fixable by a few lines of perl
>>> *if we had the repository*.
>>>
>>
>> Better to do this with sexps.
>>
>> ad repositories: Part of the general problem of managing a
>> repository is close to the problem of inferring a good type for
>> (the value of) an expression.  The style of constraints is
>> similar.  Now the design problem is:
>>
>> 1. Arrange a general system for the specification of the
>>    constraints.
>>
>> 2. Design a systematic way of giving both advice and direct
>>    commands to the system.  This subsystem would be used to set
>>    up the default policy.
>>
>> 3. Choose a constraint solver.
>>
>> Maybe worth looking at:
>>
>>   http://en.wikipedia.org/wiki/**Nix_package_manager<http://en.wikipedia.org/wiki/Nix_package_manager>
>>   [page was last modified on 17 July 2012 at 20:20]
>>
>> oo--JS.
>>
>>
>>
>>> [a few hours later]
>>>
>>> Actually, I went and downloaded all of hackage, put it into a git
>>> repository and fixed these issues:
>>>
>>> Fix catch
>>> perl -ni -e 'print unless /import Prelude hiding \(catch\)/' $(git grep
>>> 'import Prelude hiding (catch)')
>>>
>>> Fix CInt constructors (lots of other stuff from Foreign.C.Types not fixed
>>> though)
>>> perl -p -i -e 's/^import Foreign.C.Types(.*)CInt([^(])/**import
>>> Foreign.C.Types${1}CInt(..)${**1}/g' $(git grep -l '^import.*CInt')
>>>
>>> Fix bytestring versioning
>>> perl -p -i -e 's/bytestring( +)>=([0-9. &]+)<([
>>> ]*)0.10/bytestring$1>=$2<${3}**0.11/g' $(git grep 'bytestring.*< *0\.')
>>>
>>> Patch to hackage:
>>> http://ge.tt/6Cb5ErM/v/0
>>>
>>> I understand that this doesn't help anyone, but if there was a way fix,
>>> upload, and get *consensus* on a few regexps like this, then doing API
>>> changes wouldn't be such a headache.
>>>
>>> Alexander
>>>
>>> On 30 August 2012 07:26, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
>>>
>>>  Since the release of the GHC 7.6 RC, I've been going through my packages
>>>> and fixing up build problems so that people who upgrade to 7.6 will have
>>>> a
>>>> smooth ride.
>>>>
>>>> Sad to say, my experience of 7.6 is that it has felt like a particularly
>>>> rough release for backwards incompatibility. I wanted to quantify the
>>>> pain,
>>>> so I did some research, and here's what I found.
>>>>
>>>> I maintain 25 open source Haskell packages. Of these, the majority have
>>>> needed updates due to the GHC 7.6 release:
>>>>
>>>>    - base16-bytestring
>>>>    - blaze-textual
>>>>    - bloomfilter
>>>>    - configurator
>>>>    - criterion
>>>>    - double-conversion
>>>>    - filemanip
>>>>    - HDBC-mysql
>>>>    - mwc-random
>>>>    - pcap
>>>>    - pool
>>>>    - riak-haskell-client
>>>>    - snappy
>>>>    - text
>>>>    - text-format
>>>>    - text-icu
>>>>
>>>>
>>>> That's 16 out of 25 packages I've had to update. I've also either
>>>> reported
>>>> bugs on, or had to fix, several other people's packages along the way
>>>> (maybe four?). So let's say I've run into problems with 20 out of the
>>>> combined 29 packages of mine and my upstreams.
>>>>
>>>> The reasons for these problems fall into three bins:
>>>>
>>>>    - Prelude no longer exports catch, so a lot of "import Prelude hiding
>>>>    (catch)" had to change.
>>>>    - The FFI now requires constructors to be visible, so "CInt" has to be
>>>>    imported as "CInt(..)".
>>>>    - bytestring finally got bumped to 0.10, so many upper bounds had to
>>>>    be relaxed (*cf* my suggestion that the upper-bounds-by-default policy
>>>>
>>>>    is destructive).
>>>>
>>>> It has been a lot of work to test 29 packages, and then modify, rebuild,
>>>> and release 20 of them. It has consumed most of my limited free time for
>>>> almost two weeks. Worse, this has felt like make-work, of no practical
>>>> benefit to anyone beyond scrambling to restore the status quo ante.
>>>>
>>>> If over half of my packages needed fixing, I'm alarmed at the thought of
>>>> the effects on the rest of Hackage.
>>>>
>>>> I'm torn over this. I understand and agree with the impetus to improve
>>>> the
>>>> platform by tidying things up, and yet just two seemingly innocuous
>>>> changes
>>>> (catch and FFI) have forced me to do a bunch of running to stand still.
>>>>
>>>> I don't have any suggestions about what to do; I know that it's hard to
>>>> estimate the downstream effects of what look like small changes. And so
>>>> I'm
>>>> not exactly complaining. Call this an unhappy data point.
>>>>
>>>> ______________________________**_________________
>>>> Haskell-Cafe mailing list
>>>> Haskell-Cafe <at> haskell.org
>>>> http://www.haskell.org/**mailman/listinfo/haskell-cafe<http://www.haskell.org/mailman/listinfo/haskell-cafe>
>>>>
>>>>
>>>>
>>>
>> ______________________________**_________________
>> Haskell-Cafe mailing list
>> Haskell-Cafe <at> haskell.org
>> http://www.haskell.org/**mailman/listinfo/haskell-cafe<http://www.haskell.org/mailman/listinfo/haskell-cafe>
>>
>

Gmane