Roman Cheplyaka | 18 May 18:25 2013

Non-deterministic behaviour of aeson's parser

I am observing a non-deterministic behaviour of aeson's parser.

I'm writing here in addition to filing a bug report [1] to draw
attention to this (pretty scary) problem.

To try to reproduce this problem, do this:

  git clone https://gist.github.com/5604887.git aeson
  cd aeson
  ghc aeson.hs
  ./aeson | sort | uniq -c

This is my result:

    32 Left "key \"module\" not present"
    55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present."
  1913 Right ()

Can others reproduce this in their environments?

Does anyone have ideas about where the bug may lie?
Many aeson's dependencies do unsafe IO stuff that could lead to
such consequences.

Roman

[1]: https://github.com/bos/aeson/issues/125
Mateusz Kowalczyk | 18 May 19:03 2013
Picon

Re: Non-deterministic behaviour of aeson's parser


On 18/05/13 17:25, Roman Cheplyaka wrote:
> I am observing a non-deterministic behaviour of aeson's parser.
> 
> I'm writing here in addition to filing a bug report [1] to draw 
> attention to this (pretty scary) problem.
> 
> To try to reproduce this problem, do this:
> 
> git clone https://gist.github.com/5604887.git aeson cd aeson ghc
> aeson.hs ./aeson | sort | uniq -c
> 
> This is my result:
> 
> 32 Left "key \"module\" not present" 55 Left "When parsing the
> record SymValue of type Main.SymValueInfo the key fixity was not
> present." 1913 Right ()
> 
> Can others reproduce this in their environments?
> 
> Does anyone have ideas about where the bug may lie? Many aeson's
> dependencies do unsafe IO stuff that could lead to such
> consequences.
> 
> Roman
> 
> [1]: https://github.com/bos/aeson/issues/125
> 
> _______________________________________________ Haskell-Cafe
> mailing list Haskell-Cafe <at> haskell.org 
(Continue reading)

MigMit | 18 May 19:17 2013
Picon

Re: Non-deterministic behaviour of aeson's parser

My result: 2000 lines of "Right ()"

ghc-pkg list aeson says "aeson-0.6.1.0"

On May 18, 2013, at 8:25 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:

> I am observing a non-deterministic behaviour of aeson's parser.
> 
> I'm writing here in addition to filing a bug report [1] to draw
> attention to this (pretty scary) problem.
> 
> To try to reproduce this problem, do this:
> 
>  git clone https://gist.github.com/5604887.git aeson
>  cd aeson
>  ghc aeson.hs
>  ./aeson | sort | uniq -c
> 
> This is my result:
> 
>    32 Left "key \"module\" not present"
>    55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present."
>  1913 Right ()
> 
> Can others reproduce this in their environments?
> 
> Does anyone have ideas about where the bug may lie?
> Many aeson's dependencies do unsafe IO stuff that could lead to
> such consequences.
> 
(Continue reading)

Niklas Hambüchen | 18 May 19:23 2013

Re: Non-deterministic behaviour of aeson's parser

Can't reproduce:

% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()

Time 100:

% ./aeson | sort | uniq -c
 200000 Right ()

My packages:

% ghc-pkg list
/var/lib/ghc/package.conf.d
   Cabal-1.16.0
   array-0.4.0.1
   base-4.6.0.1
   bin-package-db-0.0.0.0
   binary-0.5.1.1
   bytestring-0.10.0.2
   containers-0.5.0.0
   deepseq-1.3.0.1
(Continue reading)

Clark Gaebel | 18 May 20:21 2013
Picon
Picon

Re: Non-deterministic behaviour of aeson's parser

CNR with aeson 0.6.1.0 and ghc 7.6.3.


pkg-list output can be found at http://pastebin.com/Zuuujcaz

On Saturday, May 18, 2013, Niklas Hambüchen wrote:
Can't reproduce:

% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()
% ./aeson | sort | uniq -c
   2000 Right ()

Time 100:

% ./aeson | sort | uniq -c
 200000 Right ()


My packages:

% ghc-pkg list
/var/lib/ghc/package.conf.d
   Cabal-1.16.0
   array-0.4.0.1
   base-4.6.0.1
   bin-package-db-0.0.0.0
   binary-0.5.1.1
   bytestring-0.10.0.2
   containers-0.5.0.0
   deepseq-1.3.0.1
   directory-1.2.0.1
   filepath-1.3.0.1
   ghc-7.6.2
   ghc-prim-0.3.0.0
   haskell2010-1.1.1.0
   haskell98-2.0.0.2
   hoopl-3.9.0.0
   hpc-0.6.0.0
   integer-gmp-0.5.0.0
   old-locale-1.0.0.5
   old-time-1.1.0.1
   pretty-1.1.1.0
   process-1.1.0.2
   rts-1.0
   template-haskell-2.8.0.0
   time-1.4.0.1
   unix-2.6.0.1
/home/niklas/.ghc/x86_64-linux-7.6.2/package.conf.d
   HTTP-4000.2.8
   HUnit-1.2.5.2
   QuickCheck-2.6
   Xauth-0.1
   aeson-0.6.1.0
   ansi-terminal-0.6
   attoparsec-0.10.4.0
   attoparsec-binary-0.2
   base-unicode-symbols-0.2.2.4
   blaze-builder-0.3.1.1
   byteorder-1.0.4
   cipher-aes-0.1.8
   convertible-1.0.11.1
   cpphs-1.16
   dlist-0.5
   ghc-paths-0.1.0.9
   ghc-syb-utils-0.2.1.1
   hashable-1.2.0.6
   haskell-lexer-1.0
   haskell-src-exts-1.13.5
   hidapi-1.0
   hlint-1.8.44
   hscolour-1.20.3
   hspec-1.5.4
   hspec-expectations-0.3.2
   io-choice-0.0.3
   lifted-base-0.2.0.4
   monad-control-0.3.2.1
   mtl-2.1.2
   network-2.4.1.2
   parsec-3.1.3
   pretty-show-1.5
   primitive-0.5.0.1
   quickcheck-io-0.1.0
   random-1.0.1.1
   robot-1.0.1.1
   robot-1.1
   setenv-0.1.0
   stm-2.4.2
   storable-record-0.0.2.5
   syb-0.4.0
   text-0.11.3.0
   transformers-0.3.0.0
   transformers-base-0.4.1
   uniplate-1.6.10
   unordered-containers-0.2.3.1
   utility-ht-0.0.9
   vector-0.10.0.1
   vector-th-unbox-0.2.0.1
   xhb-0.5.2012.11.23
   zlib-0.5.4.1


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Gregory Collins | 18 May 22:19 2013
Picon

Re: Non-deterministic behaviour of aeson's parser

First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".

I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:


and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).

G

On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.

I'm writing here in addition to filing a bug report [1] to draw
attention to this (pretty scary) problem.

To try to reproduce this problem, do this:

  git clone https://gist.github.com/5604887.git aeson
  cd aeson
  ghc aeson.hs
  ./aeson | sort | uniq -c

This is my result:

    32 Left "key \"module\" not present"
    55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present."
  1913 Right ()

Can others reproduce this in their environments?

Does anyone have ideas about where the bug may lie?
Many aeson's dependencies do unsafe IO stuff that could lead to
such consequences.

Roman

[1]: https://github.com/bos/aeson/issues/125

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



--
Gregory Collins <greg <at> gregorycollins.net>
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Clark Gaebel | 18 May 22:47 2013
Picon
Picon

Re: Non-deterministic behaviour of aeson's parser

$ uname -a

Linux clark-laptop 3.9.0-2-ARCH #1 SMP PREEMPT Tue Apr 30 09:48:29 CEST 2013 x86_64 GNU/Linux

It'd take too long for my helpfulness to build with cabal install -fsse2 hashable and rebuild an environment.

If someone writes a bash script to do it (using cabal-dev please!), I'd be more than happy to run it and post the results.

  - Clark

On Saturday, May 18, 2013, Gregory Collins wrote:
First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".

I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:


and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).

G

On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.

I'm writing here in addition to filing a bug report [1] to draw
attention to this (pretty scary) problem.

To try to reproduce this problem, do this:

  git clone https://gist.github.com/5604887.git aeson
  cd aeson
  ghc aeson.hs
  ./aeson | sort | uniq -c

This is my result:

    32 Left "key \"module\" not present"
    55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present."
  1913 Right ()

Can others reproduce this in their environments?

Does anyone have ideas about where the bug may lie?
Many aeson's dependencies do unsafe IO stuff that could lead to
such consequences.

Roman

[1]: https://github.com/bos/aeson/issues/125

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



--
Gregory Collins <greg <at> gregorycollins.net>
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 18 May 23:38 2013

Re: Non-deterministic behaviour of aeson's parser

Indeed it looks like a bug in hashable — it goes away with hashable-1.1.2.5.

Building with -f-sse2 results in a linker error

Loading package hashable-1.2.0.7 ... linking ... ghc:
/home/feuerbach/tmp/aeson/.cabal-sandbox/lib/i386-linux-ghc-7.6.3/hashable-1.2.0.7/libHShashable-1.2.0.7.a:
unknown symbol `hashable_siphash24_sse2'
ghc: unable to load package `hashable-1.2.0.7'

Roman

* Gregory Collins <greg <at> gregorycollins.net> [2013-05-18 22:19:38+0200]
> First off, everyone reporting results to this thread: your bug report would
> be much more helpful if you included your OS/architecture/GHC version
> combo, as well as the results of re-running the tests if you build
> "hashable" with "cabal install -f-sse2".
> 
> I have a funny feeling that this is a bug in hashable or
> unordered-containers. I'm guessing hashable, especially because of this:
> 
>     https://github.com/tibbe/hashable/issues/66
> 
> and because hashable has had subtle bugs in its C code before (see
> https://github.com/tibbe/hashable/issues/60).
> 
> G
> 
> On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
> 
> > I am observing a non-deterministic behaviour of aeson's parser.
> >
> > I'm writing here in addition to filing a bug report [1] to draw
> > attention to this (pretty scary) problem.
> >
> > To try to reproduce this problem, do this:
> >
> >   git clone https://gist.github.com/5604887.git aeson
> >   cd aeson
> >   ghc aeson.hs
> >   ./aeson | sort | uniq -c
> >
> > This is my result:
> >
> >     32 Left "key \"module\" not present"
> >     55 Left "When parsing the record SymValue of type Main.SymValueInfo
> > the key fixity was not present."
> >   1913 Right ()
> >
> > Can others reproduce this in their environments?
> >
> > Does anyone have ideas about where the bug may lie?
> > Many aeson's dependencies do unsafe IO stuff that could lead to
> > such consequences.
> >
> > Roman
> >
> > [1]: https://github.com/bos/aeson/issues/125
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > Haskell-Cafe <at> haskell.org
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
> 
> 
> 
> -- 
> Gregory Collins <greg <at> gregorycollins.net>

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Gregory Collins | 10 Jun 14:48 2013
Picon

Re: Non-deterministic behaviour of aeson's parser

It's definitely hashable. Here's a minimal failing test case: https://gist.github.com/gregorycollins/5748445


On Sat, May 18, 2013 at 11:38 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
Indeed it looks like a bug in hashable — it goes away with hashable-1.1.2.5.

Building with -f-sse2 results in a linker error

Loading package hashable-1.2.0.7 ... linking ... ghc: /home/feuerbach/tmp/aeson/.cabal-sandbox/lib/i386-linux-ghc-7.6.3/hashable-1.2.0.7/libHShashable-1.2.0.7.a: unknown symbol `hashable_siphash24_sse2'
ghc: unable to load package `hashable-1.2.0.7'

Roman

* Gregory Collins <greg <at> gregorycollins.net> [2013-05-18 22:19:38+0200]
> First off, everyone reporting results to this thread: your bug report would
> be much more helpful if you included your OS/architecture/GHC version
> combo, as well as the results of re-running the tests if you build
> "hashable" with "cabal install -f-sse2".
>
> I have a funny feeling that this is a bug in hashable or
> unordered-containers. I'm guessing hashable, especially because of this:
>
>     https://github.com/tibbe/hashable/issues/66
>
> and because hashable has had subtle bugs in its C code before (see
> https://github.com/tibbe/hashable/issues/60).
>
> G
>
> On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
>
> > I am observing a non-deterministic behaviour of aeson's parser.
> >
> > I'm writing here in addition to filing a bug report [1] to draw
> > attention to this (pretty scary) problem.
> >
> > To try to reproduce this problem, do this:
> >
> >   git clone https://gist.github.com/5604887.git aeson
> >   cd aeson
> >   ghc aeson.hs
> >   ./aeson | sort | uniq -c
> >
> > This is my result:
> >
> >     32 Left "key \"module\" not present"
> >     55 Left "When parsing the record SymValue of type Main.SymValueInfo
> > the key fixity was not present."
> >   1913 Right ()
> >
> > Can others reproduce this in their environments?
> >
> > Does anyone have ideas about where the bug may lie?
> > Many aeson's dependencies do unsafe IO stuff that could lead to
> > such consequences.
> >
> > Roman
> >
> > [1]: https://github.com/bos/aeson/issues/125
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > Haskell-Cafe <at> haskell.org
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
>
>
>
> --
> Gregory Collins <greg <at> gregorycollins.net>



--
Gregory Collins <greg <at> gregorycollins.net>
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane