Jan Stolarek | 23 Sep 11:25 2012
Picon

Organizaing tests in Haskell

Dear list,

during many years of Java programming I've been faithful to TDD methology. Recently I've been 
trying to figure out how to do tests in Haskell. Thanks to RWH and help from great folks at 
#haskell I've managed to get on my feet. There is however one issue I wasn't able to solve.

In Java it is very easy to separate test code from the actual source. A Java project simply 
contains two folders: src and tests. These two are treated as the source directories. Both these 
directories have the same subdirectory structure. For example I have a source file 
src/myPackage/mySubpackage/MyClass.java and test for it are kept in file 
tests/myPackage/mySubpackage/MyClassTest.java. These two files are considered by Java to be in 
the same package. Here's the main trick: fileds and methods that are marked as protected in Java 
are accessible to other classes in the same package. This allows to test internal methods of a 
class by marking them as protected instead of private. This breaks encapsulation but only within 
a package, which is acceptable.

Now I'd like to achieve something similar in Haskell. I'm using cabal's support for testing. I 
created separate src and tests directories, both with the same subdirectory structure. I keep 
tests for each module in a separate file (e.g. I have src/Math/MyModule.hs and 
tests/Math/MyModuleTest.hs) and I have one file that assembles all the tests into a single test 
suite (I use test-framework for that). The only problem is that in order to test some function 
from a module I have to expose that function, which pretty much forces me to give up on 
encapsulation.

Is there any better solution to organize tests in Haskell? Should I just give up on module 
encapsulation, or should I only test functions exposed by the module and don't worry about 
internal functions? Perhaps I should use some different approach?

Jan
(Continue reading)

Simon Hengel | 23 Sep 12:06 2012
Picon

Re: Organizaing tests in Haskell

Hi,

> Is there any better solution to organize tests in Haskell?

(Disclaimer: I'm the maintainer of Hspec ;)

If you use Hspec[1] for testing, you do not have to assemble your
individual tests manually into a test suit; hspec-discover[2] takes care
of that.

There is no comprehensive user's guide for Hspec yet, but a basic
introduction is at [3].  If you have any questions, feel free to join in
at #hspec on freenode.

> Should I just give up on module encapsulation, or should I only test
> functions exposed by the module and don't worry about internal
> functions?

You can do it with CPP.  Say, if you have a module Foo, with functions
foo, bar and baz, where baz  is not part of the public interface, then
the export list becomes:

    {-# LANGUAGE CPP #-}
    module Foo where (
      foo
    , bar
    #ifdef TEST
    , baz
    #endif
    )
(Continue reading)

Matthew West | 23 Sep 12:16 2012
Picon

Re: Organizaing tests in Haskell


On 23 Sep 2012, at 10:25, Jan Stolarek wrote:

> Dear list,
> 
> during many years of Java programming I've been faithful to TDD methology. Recently I've been 
> trying to figure out how to do tests in Haskell. Thanks to RWH and help from great folks at 
> #haskell I've managed to get on my feet. There is however one issue I wasn't able to solve.
> 
> In Java it is very easy to separate test code from the actual source. A Java project simply 
> contains two folders: src and tests. These two are treated as the source directories. Both these 
> directories have the same subdirectory structure. For example I have a source file 
> src/myPackage/mySubpackage/MyClass.java and test for it are kept in file 
> tests/myPackage/mySubpackage/MyClassTest.java. These two files are considered by Java to be in 
> the same package. Here's the main trick: fileds and methods that are marked as protected in Java 
> are accessible to other classes in the same package. This allows to test internal methods of a 
> class by marking them as protected instead of private. This breaks encapsulation but only within 
> a package, which is acceptable.
> 
> Now I'd like to achieve something similar in Haskell. I'm using cabal's support for testing. I 
> created separate src and tests directories, both with the same subdirectory structure. I keep 
> tests for each module in a separate file (e.g. I have src/Math/MyModule.hs and 
> tests/Math/MyModuleTest.hs) and I have one file that assembles all the tests into a single test 
> suite (I use test-framework for that). The only problem is that in order to test some function 
> from a module I have to expose that function, which pretty much forces me to give up on 
> encapsulation.
> 
> Is there any better solution to organize tests in Haskell? Should I just give up on module 
> encapsulation, or should I only test functions exposed by the module and don't worry about 
> internal functions? Perhaps I should use some different approach?
(Continue reading)

Simon Hengel | 23 Sep 12:28 2012
Picon

Re: Organizaing tests in Haskell

> Of course others are still able to import your Internal modules

That is not necessarily true.  For libraries, you can list internal
modules as other-modules (in contrast to exposed-modules) in you Cabal
file.  That way they are not part of the public interface of your
library.

However, that approach does not work if you want to do encapsulation
within a single project.  I think in that case you most likely end up
with CPP (it's ugly, but it works).

Cheers,
Simon
Jan Stolarek | 23 Sep 16:10 2012
Picon

Re: Organizaing tests in Haskell

Thanks for replies. CPP approach seems to be what I would like to achieve, but it looks more like 
a hack than a real solution. That said, I like the idea of creating a module that acts as an 
external interface to the library and I I don't mind sacrificing encapsulation within the package 
itself. If it works for project as big as Yesod it should work for me.

> If you use Hspec[1] for testing, you do not have to assemble your
> individual tests manually into a test suit; hspec-discover[2] takes care
> of that.
I guess that I like to have my tests organized manually. It takes a bit of more work and there's a 
risk that I forget to add some test to the suite, but I'm willing to accept these drawbacks and 
get more fine-grained control in return.

Jan
Simon Hengel | 23 Sep 19:11 2012
Picon

Re: Organizaing tests in Haskell

On Sun, Sep 23, 2012 at 04:10:56PM +0200, Jan Stolarek wrote:
> I don't mind sacrificing encapsulation within the package itself. If
> it works for project as big as Yesod it should work for me.

Yesod uses the CPP solution, too (e.g. [1]).

Cheers,
Simon

[1] https://github.com/yesodweb/shakespeare/blob/master/shakespeare/Text/Shakespeare.hs
Heinrich Apfelmus | 23 Sep 18:11 2012
Picon

Re: Organizaing tests in Haskell

Simon Hengel wrote:
>> Of course others are still able to import your Internal modules
> 
> That is not necessarily true.  For libraries, you can list internal
> modules as other-modules (in contrast to exposed-modules) in you Cabal
> file.  That way they are not part of the public interface of your
> library.

How do I access internal modules with  cabal test , though? Last time I 
tried, I could not find a way to expose in the test section of the cabal 
file.

Best regards,
Heinrich Apfelmus

--
http://apfelmus.nfshost.com
Simon Hengel | 23 Sep 19:00 2012
Picon

Re: Organizaing tests in Haskell

On Sun, Sep 23, 2012 at 06:11:59PM +0200, Heinrich Apfelmus wrote:
> Simon Hengel wrote:
> >>Of course others are still able to import your Internal modules
> >
> >That is not necessarily true.  For libraries, you can list internal
> >modules as other-modules (in contrast to exposed-modules) in you Cabal
> >file.  That way they are not part of the public interface of your
> >library.
> 
> How do I access internal modules with  cabal test , though? Last
> time I tried, I could not find a way to expose in the test section
> of the cabal file.

It works, if you add the source directory to hs-source-dirs of the test
suite (in contrast to depending on the library!), e.g.:

  hs-source-dirs: test, src

or

  hs-source-dirs: test, .

This still has the disadvantage, that the sources are compiled twice.
But I'm not aware of a better way to do it.  If you mostly use GHCi for
development, it's not a big issue.

Cheers,
Simon
Heinrich Apfelmus | 24 Sep 15:46 2012
Picon

Re: Organizaing tests in Haskell

Simon Hengel wrote:
> On Sun, Sep 23, 2012 at 06:11:59PM +0200, Heinrich Apfelmus wrote:
>>
>> How do I access internal modules with  cabal test , though? Last
>> time I tried, I could not find a way to expose in the test section
>> of the cabal file.
> 
> It works, if you add the source directory to hs-source-dirs of the test
> suite (in contrast to depending on the library!), e.g.:
> 
>   hs-source-dirs: test, src
> 
> or
> 
>   hs-source-dirs: test, .
> 
> This still has the disadvantage, that the sources are compiled twice.
> But I'm not aware of a better way to do it.  If you mostly use GHCi for
> development, it's not a big issue.

I got it to work, thanks!

I also had to duplicate the dependency information, but that's alright.

Best regards,
Heinrich Apfelmus

--
http://apfelmus.nfshost.com
(Continue reading)


Gmane