Roman Cheplyaka | 5 Aug 20:48 2013

ANNOUNCE: tasty, a new testing framework

I am pleased to announce the first release of tasty, a new testing
framework for Haskell. It is meant to be a successor to test-framework
(which is unmaintained).

Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of
the box (through the standard packages), but it is very extensible, so
you can write your own test providers.

Please see the home page for more information:
http://documentup.com/feuerbach/tasty

Roman
Andrey Chudnov | 5 Aug 21:31 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
> (which is unmaintained).
   Has this been confirmed by the author/maintainer?
> Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of
> the box (through the standard packages), but it is very extensible, so
> you can write your own test providers.
>
> Please see the home page for more information:
> http://documentup.com/feuerbach/tasty
Is it a drop-in replacement for test-framework, e.g. if I substitute 
"test-framework" for "tasty" in my .cabal files, will it work? If not, 
could you provide a quick guide for porting? Also, is the current 
version (0.1) recommended for general use?
Carter Schonwald | 5 Aug 22:01 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

Likewise, is test-framework explicitly unmaintained? I'd generally think a testing tool should stabilize after a while... though I guess that test-framework's author is pretty busy with some other work this year, but I could be wrong.


On Mon, Aug 5, 2013 at 3:31 PM, Andrey Chudnov <achudnov <at> gmail.com> wrote:
On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
(which is unmaintained).
  Has this been confirmed by the author/maintainer?

Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of
the box (through the standard packages), but it is very extensible, so
you can write your own test providers.

Please see the home page for more information:
http://documentup.com/feuerbach/tasty
Is it a drop-in replacement for test-framework, e.g. if I substitute "test-framework" for "tasty" in my .cabal files, will it work? If not, could you provide a quick guide for porting? Also, is the current version (0.1) recommended for general use?


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 5 Aug 22:02 2013

Re: ANNOUNCE: tasty, a new testing framework

* Andrey Chudnov <achudnov <at> gmail.com> [2013-08-05 15:31:16-0400]
> On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
> >(which is unmaintained).
>   Has this been confirmed by the author/maintainer?

I've sent a couple of emails to Max (one in January, one in April) and
haven't heard anything from him. My patches, which he applied in
January, are still not released. That's why I regard test-framework as
unmaintained.

You can also make your own impression by browsing github (commit
activity, outstanding pull requests, open issues...)

> Is it a drop-in replacement for test-framework, e.g. if I substitute
> "test-framework" for "tasty" in my .cabal files, will it work?
> If not, could you provide a quick guide for porting?

Not quite.

At the very least, you'll have to change module names
(Test.Framework -> Test.Tasty,
 Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...),
and wrap the top-level list of tests into a testGroup.

If you have type signatures, you'll need to rename Test to TestTree.

That should be enough in most cases. If you use plusTestOptions, you'll
need to look up appropriate functions from Test.Tasty.Options.

> Also, is the
> current version (0.1) recommended for general use?

I'd love to see people using it. But you should treat this as beta
software. I am in the process of migrating my own packages to use Tasty.

Roman
Jan Stolarek | 5 Aug 22:29 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

Oh, a new testing framework - I'm always interested in that :)

> At the very least, you'll have to change module names
> (Test.Framework -> Test.Tasty,
>  Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...),
> and wrap the top-level list of tests into a testGroup.
>
> If you have type signatures, you'll need to rename Test to TestTree.
>
> That should be enough in most cases. If you use plusTestOptions, you'll
> need to look up appropriate functions from Test.Tasty.Options.
Making a tutorial would probably be a good idea here.

Janek
Carter Schonwald | 5 Aug 22:58 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

fair enough. I take it that you're also (implicitly) committing to maintaining this for the next few years? :) 


On Mon, Aug 5, 2013 at 4:02 PM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
* Andrey Chudnov <achudnov <at> gmail.com> [2013-08-05 15:31:16-0400]
> On 08/05/2013 02:48 PM, Roman Cheplyaka wrote:
> >(which is unmaintained).
>   Has this been confirmed by the author/maintainer?

I've sent a couple of emails to Max (one in January, one in April) and
haven't heard anything from him. My patches, which he applied in
January, are still not released. That's why I regard test-framework as
unmaintained.

You can also make your own impression by browsing github (commit
activity, outstanding pull requests, open issues...)

> Is it a drop-in replacement for test-framework, e.g. if I substitute
> "test-framework" for "tasty" in my .cabal files, will it work?
> If not, could you provide a quick guide for porting?

Not quite.

At the very least, you'll have to change module names
(Test.Framework -> Test.Tasty,
 Test.Framework.Providers.HUnit -> Test.Tasty.HUnit, ...),
and wrap the top-level list of tests into a testGroup.

If you have type signatures, you'll need to rename Test to TestTree.

That should be enough in most cases. If you use plusTestOptions, you'll
need to look up appropriate functions from Test.Tasty.Options.

> Also, is the
> current version (0.1) recommended for general use?

I'd love to see people using it. But you should treat this as beta
software. I am in the process of migrating my own packages to use Tasty.

Roman

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 6 Aug 07:53 2013

Re: ANNOUNCE: tasty, a new testing framework

* Carter Schonwald <carter.schonwald <at> gmail.com> [2013-08-05 16:58:37-0400]
> fair enough. I take it that you're also (implicitly) committing to
> maintaining this for the next few years? :)

That's correct.

Roman
Tikhon Jelvis | 6 Aug 15:29 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

Could you add some documentation on how to use this with cabal? I've found integrating tests with cabal unintuitive and poorly documented--to the point where I haven't really bothered! I've gotten it working before, but I would have to look it up again in the future. (I also didn't use a framework.)

It would be awesome to see an example .cabal file along with your example test cases.

thanks,
-tikhon


On Tue, Aug 6, 2013 at 1:53 AM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
* Carter Schonwald <carter.schonwald <at> gmail.com> [2013-08-05 16:58:37-0400]
> fair enough. I take it that you're also (implicitly) committing to
> maintaining this for the next few years? :)

That's correct.

Roman

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 6 Aug 15:50 2013

Re: ANNOUNCE: tasty, a new testing framework

You can find an example here:
https://github.com/feuerbach/regex-applicative/blob/master/regex-applicative.cabal#L89

If you'd like to contribute a short README section based on that, please
go ahead! :)

Roman

* Tikhon Jelvis <tikhon <at> jelv.is> [2013-08-06 09:29:21-0400]
> Could you add some documentation on how to use this with cabal? I've found
> integrating tests with cabal unintuitive and poorly documented--to the
> point where I haven't really bothered! I've gotten it working before, but I
> would have to look it up again in the future. (I also didn't use a
> framework.)
> 
> It would be awesome to see an example .cabal file along with your example
> test cases.
> 
> thanks,
> -tikhon
> 
> 
> On Tue, Aug 6, 2013 at 1:53 AM, Roman Cheplyaka <roma <at> ro-che.info> wrote:
> 
> > * Carter Schonwald <carter.schonwald <at> gmail.com> [2013-08-05 16:58:37-0400]
> > > fair enough. I take it that you're also (implicitly) committing to
> > > maintaining this for the next few years? :)
> >
> > That's correct.
> >
> > Roman
> >
> > _______________________________________________
> > Haskell-Cafe mailing list
> > Haskell-Cafe <at> haskell.org
> > http://www.haskell.org/mailman/listinfo/haskell-cafe
> >
Carter Schonwald | 6 Aug 19:30 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

Awesome.  Ill take a look at tasty sometime this month.  Thanks for taking the time to patiently answer all of our questions.  

On Tuesday, August 6, 2013, Roman Cheplyaka wrote:

* Carter Schonwald <carter.schonwald <at> gmail.com> [2013-08-05 16:58:37-0400]
> fair enough. I take it that you're also (implicitly) committing to
> maintaining this for the next few years? :)

That's correct.

Roman
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Joe Quinn | 5 Aug 22:41 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

On 8/5/2013 2:48 PM, Roman Cheplyaka wrote:
I am pleased to announce the first release of tasty, a new testing framework for Haskell. It is meant to be a successor to test-framework (which is unmaintained). Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out of the box (through the standard packages), but it is very extensible, so you can write your own test providers. Please see the home page for more information: http://documentup.com/feuerbach/tasty Roman _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe <at> haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Will tasty be available through yum?

*ducks*
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
John Wiegley | 6 Aug 20:40 2013

Re: ANNOUNCE: tasty, a new testing framework

>>>>> Roman Cheplyaka <roma <at> ro-che.info> writes:

> I am pleased to announce the first release of tasty, a new testing framework
> for Haskell. It is meant to be a successor to test-framework (which is
> unmaintained).

It would be nice to see a comparison of the various test frameworks and why
one might select one over another.  I use hspec currently (which also
integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what
tasty might offer.  And I particularly dislike writing tests inside of a
gigantic list; I much prefer the "monadic" style of hspec.

--

-- 
John Wiegley
FP Complete                         Haskell tools, training and consulting
http://fpcomplete.com               johnw on #haskell/irc.freenode.net
Roman Cheplyaka | 6 Aug 23:51 2013

Re: ANNOUNCE: tasty, a new testing framework

* John Wiegley <johnw <at> fpcomplete.com> [2013-08-06 13:40:50-0500]
> >>>>> Roman Cheplyaka <roma <at> ro-che.info> writes:
> 
> > I am pleased to announce the first release of tasty, a new testing framework
> > for Haskell. It is meant to be a successor to test-framework (which is
> > unmaintained).
> 
> It would be nice to see a comparison of the various test frameworks and why
> one might select one over another.  I use hspec currently (which also
> integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what
> tasty might offer.  And I particularly dislike writing tests inside of a
> gigantic list; I much prefer the "monadic" style of hspec.

This has been discussed on reddit here:
http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framework_successor_to/cbhiz40

Roman
Jan Stolarek | 7 Aug 09:58 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

I admit I haven't yet had the time to try out testy, but there's one thing I'm curious about. QuickCheck can
classify tests:
Jan Stolarek | 7 Aug 10:00 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:

  +++ OK, passed 100 tests (29% Short)

Can tasty display this classification info? That was a thing I missed a lot in test-framework and would
probably motivate me to switch to tasty.

Janek

----- Oryginalna wiadomość -----
Od: "Roman Cheplyaka" <roma <at> ro-che.info>
Do: haskell-cafe <at> haskell.org
Wysłane: wtorek, 6 sierpień 2013 22:51:57
Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework

* John Wiegley <johnw <at> fpcomplete.com> [2013-08-06 13:40:50-0500]
> >>>>> Roman Cheplyaka <roma <at> ro-che.info> writes:
> 
> > I am pleased to announce the first release of tasty, a new testing framework
> > for Haskell. It is meant to be a successor to test-framework (which is
> > unmaintained).
> 
> It would be nice to see a comparison of the various test frameworks and why
> one might select one over another.  I use hspec currently (which also
> integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what
> tasty might offer.  And I particularly dislike writing tests inside of a
> gigantic list; I much prefer the "monadic" style of hspec.

This has been discussed on reddit here:
http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framework_successor_to/cbhiz40

Roman

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 7 Aug 11:56 2013

Re: ANNOUNCE: tasty, a new testing framework

It certainly can, but it doesn't do that yet. Should be very easy to
fix, though. Patches are welcome.

Roman

* Jan Stolarek <jan.stolarek <at> p.lodz.pl> [2013-08-07 10:00:36+0200]
> Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:
> 
>   +++ OK, passed 100 tests (29% Short)
> 
> Can tasty display this classification info? That was a thing I missed a lot in test-framework and would
probably motivate me to switch to tasty.
> 
> Janek
> 
> ----- Oryginalna wiadomość -----
> Od: "Roman Cheplyaka" <roma <at> ro-che.info>
> Do: haskell-cafe <at> haskell.org
> Wysłane: wtorek, 6 sierpień 2013 22:51:57
> Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework
> 
> * John Wiegley <johnw <at> fpcomplete.com> [2013-08-06 13:40:50-0500]
> > >>>>> Roman Cheplyaka <roma <at> ro-che.info> writes:
> > 
> > > I am pleased to announce the first release of tasty, a new testing framework
> > > for Haskell. It is meant to be a successor to test-framework (which is
> > > unmaintained).
> > 
> > It would be nice to see a comparison of the various test frameworks and why
> > one might select one over another.  I use hspec currently (which also
> > integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what
> > tasty might offer.  And I particularly dislike writing tests inside of a
> > gigantic list; I much prefer the "monadic" style of hspec.
> 
> This has been discussed on reddit here:
> http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framework_successor_to/cbhiz40
> 
> Roman
> 
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
> 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Jan Stolarek | 7 Aug 12:00 2013
Picon

Re: ANNOUNCE: tasty, a new testing framework

I'll add that as an issue on github then. I tried implementing this for test-framework and failed - the code
was just too complicatd for me. It'll be intereseting to see whether tasty has simpler implementation :)

Janek

----- Oryginalna wiadomość -----
Od: "Roman Cheplyaka" <roma <at> ro-che.info>
Do: "Jan Stolarek" <jan.stolarek <at> p.lodz.pl>
DW: haskell-cafe <at> haskell.org
Wysłane: środa, 7 sierpień 2013 10:56:21
Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework

It certainly can, but it doesn't do that yet. Should be very easy to
fix, though. Patches are welcome.

Roman

* Jan Stolarek <jan.stolarek <at> p.lodz.pl> [2013-08-07 10:00:36+0200]
> Sorry, my last email got sent too quickly. Anyway, continuing my thought. So QuickCheck can classify tests:
> 
>   +++ OK, passed 100 tests (29% Short)
> 
> Can tasty display this classification info? That was a thing I missed a lot in test-framework and would
probably motivate me to switch to tasty.
> 
> Janek
> 
> ----- Oryginalna wiadomość -----
> Od: "Roman Cheplyaka" <roma <at> ro-che.info>
> Do: haskell-cafe <at> haskell.org
> Wysłane: wtorek, 6 sierpień 2013 22:51:57
> Temat: Re: [Haskell-cafe] ANNOUNCE: tasty, a new testing framework
> 
> * John Wiegley <johnw <at> fpcomplete.com> [2013-08-06 13:40:50-0500]
> > >>>>> Roman Cheplyaka <roma <at> ro-che.info> writes:
> > 
> > > I am pleased to announce the first release of tasty, a new testing framework
> > > for Haskell. It is meant to be a successor to test-framework (which is
> > > unmaintained).
> > 
> > It would be nice to see a comparison of the various test frameworks and why
> > one might select one over another.  I use hspec currently (which also
> > integrates with HUnit, QuickCheck, etc.), and couldn't tell at a glance what
> > tasty might offer.  And I particularly dislike writing tests inside of a
> > gigantic list; I much prefer the "monadic" style of hspec.
> 
> This has been discussed on reddit here:
> http://www.reddit.com/r/haskell/comments/1jr8lb/tasty_a_new_testing_framework_successor_to/cbhiz40
> 
> Roman
> 
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
> 

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Magnus Therning | 8 Aug 07:59 2013

Re: ANNOUNCE: tasty, a new testing framework

On Mon, Aug 05, 2013 at 09:48:39PM +0300, Roman Cheplyaka wrote:
> I am pleased to announce the first release of tasty, a new testing
> framework for Haskell. It is meant to be a successor to
> test-framework (which is unmaintained).
> 
> Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out
> of the box (through the standard packages), but it is very
> extensible, so you can write your own test providers.
> 
> Please see the home page for more information:
> http://documentup.com/feuerbach/tasty

Are there plans an equivalent of test-framework-th too?

/M

--

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus <at> therning.org   jabber: magnus <at> therning.org
twitter: magthe               http://therning.org/magnus

I invented the term Object-Oriented, and I can tell you I did not have
C++ in mind.
     -- Alan Kay
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 8 Aug 08:30 2013

Re: ANNOUNCE: tasty, a new testing framework

* Magnus Therning <magnus <at> therning.org> [2013-08-08 07:59:37+0200]
> On Mon, Aug 05, 2013 at 09:48:39PM +0300, Roman Cheplyaka wrote:
> > I am pleased to announce the first release of tasty, a new testing
> > framework for Haskell. It is meant to be a successor to
> > test-framework (which is unmaintained).
> > 
> > Tasty supports HUnit, SmallCheck, QuickCheck, and golden tests out
> > of the box (through the standard packages), but it is very
> > extensible, so you can write your own test providers.
> > 
> > Please see the home page for more information:
> > http://documentup.com/feuerbach/tasty
> 
> Are there plans an equivalent of test-framework-th too?

I don't have any plans to do that myself, but I welcome anyone who cares
to create and maintain such a package.

donri on reddit said that porting test-framework-th should be a simple
matter of changing one import.

Roman
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane