graham | 17 Dec 16:50 2012

HTF & Quickcheck

Getting test failures, I believe, due to using conditional properties
within quickckeck etc

[TEST] RecFunSpec:map (test/RecFunSpec.hs:48)
Gave up! Passed only 20 tests.
*** Failed! (2ms)

Is there a way for HTF/Quickcheck to specify these are not failed i.e.
only fail if there is an actual error ?

Thanks
Roman Cheplyaka | 17 Dec 17:04 2012

Re: HTF & Quickcheck

Unfortunately, I don't know the answer to your question.

However, if you don't find a solution, I suggest using SmallCheck
instead of QuickCheck — it works better when you have many unsuitable
cases.
https://github.com/feuerbach/smallcheck/wiki/Comparison-with-QuickCheck

As far as I know, SmallCheck is not supported by HTF, but it is
supported by test-framework.

Roman

* graham <at> fatlazycat.com <graham <at> fatlazycat.com> [2012-12-17 15:50:32+0000]
> Getting test failures, I believe, due to using conditional properties
> within quickckeck etc
> 
> [TEST] RecFunSpec:map (test/RecFunSpec.hs:48)
> Gave up! Passed only 20 tests.
> *** Failed! (2ms)
> 
> Is there a way for HTF/Quickcheck to specify these are not failed i.e.
> only fail if there is an actual error ?
> 
> Thanks

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
(Continue reading)

Simon Hengel | 18 Dec 00:20 2012
Picon

Re: HTF & Quickcheck

On Mon, Dec 17, 2012 at 06:04:15PM +0200, Roman Cheplyaka wrote:
> Unfortunately, I don't know the answer to your question.
> 
> However, if you don't find a solution, I suggest using SmallCheck
> instead of QuickCheck — it works better when you have many unsuitable
> cases.
> https://github.com/feuerbach/smallcheck/wiki/Comparison-with-QuickCheck
> 
> As far as I know, SmallCheck is not supported by HTF, but it is
> supported by test-framework.

test-framework also sometimes thinks a "Gave up!" is not a fail, which I
think is a bug.  But I gave up on trying to get that fixed [1], e.g.
this (admittedly contrived) property is still a pass with
test-framework:

    defaultMain [testProperty "foo" $ \x -> x == 23 ==> True]

Cheers,
Simon

[1] https://github.com/batterseapower/test-framework/issues/16

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
graham | 18 Dec 08:01 2012

Re: HTF & Quickcheck

So what use are conditional properties meant to be used for ??

If you just want 2 integers that are not equal, it would seem a lot
simpler to do this as a conditional rather than constructing some pair
in an arbitrary instance ?!?

Thanks

On Mon, Dec 17, 2012, at 11:20 PM, Simon Hengel wrote:
> On Mon, Dec 17, 2012 at 06:04:15PM +0200, Roman Cheplyaka wrote:
> > Unfortunately, I don't know the answer to your question.
> > 
> > However, if you don't find a solution, I suggest using SmallCheck
> > instead of QuickCheck — it works better when you have many unsuitable
> > cases.
> > https://github.com/feuerbach/smallcheck/wiki/Comparison-with-QuickCheck
> > 
> > As far as I know, SmallCheck is not supported by HTF, but it is
> > supported by test-framework.
> 
> test-framework also sometimes thinks a "Gave up!" is not a fail, which I
> think is a bug.  But I gave up on trying to get that fixed [1], e.g.
> this (admittedly contrived) property is still a pass with
> test-framework:
> 
>     defaultMain [testProperty "foo" $ \x -> x == 23 ==> True]
> 
> Cheers,
> Simon
> 
(Continue reading)

graham | 18 Dec 18:25 2012

Re: HTF & Quickcheck

Are there any libraries that define various common generators ?

What would be the cleanest way to define two positive integers below
1000 that are different ? Seems relatively easy with conditionals.

Thanks

On Tue, Dec 18, 2012, at 07:01 AM, graham <at> fatlazycat.com wrote:
> So what use are conditional properties meant to be used for ??
> 
> If you just want 2 integers that are not equal, it would seem a lot
> simpler to do this as a conditional rather than constructing some pair
> in an arbitrary instance ?!?
> 
> Thanks
> 
> On Mon, Dec 17, 2012, at 11:20 PM, Simon Hengel wrote:
> > On Mon, Dec 17, 2012 at 06:04:15PM +0200, Roman Cheplyaka wrote:
> > > Unfortunately, I don't know the answer to your question.
> > > 
> > > However, if you don't find a solution, I suggest using SmallCheck
> > > instead of QuickCheck — it works better when you have many unsuitable
> > > cases.
> > > https://github.com/feuerbach/smallcheck/wiki/Comparison-with-QuickCheck
> > > 
> > > As far as I know, SmallCheck is not supported by HTF, but it is
> > > supported by test-framework.
> > 
> > test-framework also sometimes thinks a "Gave up!" is not a fail, which I
> > think is a bug.  But I gave up on trying to get that fixed [1], e.g.
(Continue reading)

Simon Hengel | 18 Dec 19:04 2012
Picon

Re: HTF & Quickcheck

On Tue, Dec 18, 2012 at 05:25:41PM +0000, graham <at> fatlazycat.com wrote:
> Are there any libraries that define various common generators ?
> 
> What would be the cleanest way to define two positive integers below
> 1000 that are different ? Seems relatively easy with conditionals.

You can still use (==>) to ensure that two numbers are different.  I
would use something like this:

    newtype Small = Small Int
      deriving Show

    instance Arbitrary Small where
      arbitrary = Small . (`mod` 1000) <$> arbitrary

    prop_foo (Small x) (Small y) = x /= y ==> ...

Cheers,
Simon
graham | 18 Dec 19:38 2012

Re: HTF & Quickcheck

Thanks, how does using /= not cause test failures ? Just the sample rate
will be high enough ?

On Tue, Dec 18, 2012, at 06:04 PM, Simon Hengel wrote:
> On Tue, Dec 18, 2012 at 05:25:41PM +0000, graham <at> fatlazycat.com wrote:
> > Are there any libraries that define various common generators ?
> > 
> > What would be the cleanest way to define two positive integers below
> > 1000 that are different ? Seems relatively easy with conditionals.
> 
> You can still use (==>) to ensure that two numbers are different.  I
> would use something like this:
> 
>     newtype Small = Small Int
>       deriving Show
> 
>     instance Arbitrary Small where
>       arbitrary = Small . (`mod` 1000) <$> arbitrary
> 
>     prop_foo (Small x) (Small y) = x /= y ==> ...
> 
> Cheers,
> Simon
Simon Hengel | 17 Dec 17:08 2012
Picon

Re: HTF & Quickcheck

On Mon, Dec 17, 2012 at 03:50:32PM +0000, graham <at> fatlazycat.com wrote:
> Getting test failures, I believe, due to using conditional properties
> within quickckeck etc
> 
> [TEST] RecFunSpec:map (test/RecFunSpec.hs:48)
> Gave up! Passed only 20 tests.
> *** Failed! (2ms)
> 
> Is there a way for HTF/Quickcheck to specify these are not failed i.e.
> only fail if there is an actual error ?

I have no idea about HTF, but QuickCheck thinks those are failures, too.
If I run into something like this, it  indicates a "bug" in my arbitrary
instance.  Here I would just adapt it so that it generates valid test
data (possibly with the help of a newtype wrap).

Cheers,
Simon

Gmane