headers
Mauricio Herrera Cuadra | 30 Sep 23:54

Test coverage for BioPerl now available

Hi all,

Daily-updated test coverage reports are now available for those BioPerl 
packages which make use of the Build.PL mechanism (except bioperl-db):

http://bioperl.org/test-coverage/bioperl-live/
http://bioperl.org/test-coverage/bioperl-network/
http://bioperl.org/test-coverage/bioperl-run/

These reports will help us to know the current 'quality' of the code in 
SVN for most of the BioPerl modules. This idea was started by Nathan 
Haigh and Sendu a long time ago and it was my fault to not implement on 
time the necessary script to run the process on a daily basis, so 
apologies for that.

There are still a few things to be done in order to have this working as 
it should:

- Nathan, current Devel::Cover module from CPAN doesn't include the JS 
modifications to make table columns sortable. Do you know what happened 
to the code you contributed to the author for that?

- Reports could be generated for the rest of the BioPerl packages as 
soon as they're migrated to the Build.PL infrastructure. Anyone up for that?

- bioperl-db tests require BioSQL to be setup in the webserver machine, 
and the same goes for bioperl-run's tests with ALL of its dependencies. 
The bioperl.org site is co-hosted with all of the other OBF projects and 
that machine also takes care of other things (mailing lists, etc), so I 
would like your feedback on possible workarounds to not overload the
(Continue reading)

Permalink | Reply |
headers
Heikki Lehvaslaiho | 1 Oct 09:31

Re: Test coverage for BioPerl now available

Cool!

I have trouble understanding the values in different columns. Could you whip 
together a wiki page that explains in plain English how to read them?

  -Heikki

On Tuesday 30 September 2008 23:56:04 Mauricio Herrera Cuadra wrote:
> Hi all,
>
> Daily-updated test coverage reports are now available for those BioPerl
> packages which make use of the Build.PL mechanism (except bioperl-db):
>
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/
>
> These reports will help us to know the current 'quality' of the code in
> SVN for most of the BioPerl modules. This idea was started by Nathan
> Haigh and Sendu a long time ago and it was my fault to not implement on
> time the necessary script to run the process on a daily basis, so
> apologies for that.
>
> There are still a few things to be done in order to have this working as
> it should:
>
> - Nathan, current Devel::Cover module from CPAN doesn't include the JS
> modifications to make table columns sortable. Do you know what happened
> to the code you contributed to the author for that?
>
(Continue reading)

Permalink | Reply |
headers
Nathan S. Watson-Haigh | 6 Oct 04:39
Favicon

Re: Test coverage for BioPerl now available

Firstly, thanks Mauricio for getting this set up....it should be really
useful for everyone and make adding tests so much easier by focusing
attention on problem areas! In fact, it could help users make the transition
from using Bioperl to contributing code by giving them an entry point into
understanding the modules. Increasing test coverage in a group of modules
would make a good student project!

Heikki,

You should find this page useful:
http://search.cpan.org/~pjcj/Devel-Cover-0.64/lib/Devel/Cover/Tutorial.pod

I use the following as a list of priorities for code coverage:
1) Get the "subroutine" metric to 100% - every subroutine should have at
least 1 test to check it's returning the correct value/object given at least
one set of arguments.
2) Get the "statement" metric as close to 100% as possible by providing
various inputs so that each statement is tested for BOTH the true and false
possibilities.
3) Work on the "branch" and "path" metrics if after performing the above 2,
these are still in low coverage. You'll find that performing the above
steps, will have a knock-on effect of increasing these coverage's at the
same time.

NOTE:
Testing never proves the absence of faults, it only shows their presence.

I think as a result of better test coverage, there will be a need to
formalise/discuss the intended behaviour of some modules, as well as the
choice over default values. Also, Don't forget to test for situations where
(Continue reading)

Permalink | Reply |
headers
Jason Stajich | 1 Oct 10:07

Re: Test coverage for BioPerl now available

Thanks for doing this Mauricio!  Great to have this resource and to  
follow up on the excellent efforts by Nathan and Sendu to get this  
ball rolling..

We have a couple of Virtual Hosts through Chris Dagdigian and  
BioTeam's donated resources that we can setup postgres and mysql  
instances for biosql and even Bio::DB::GFF & Bio::DB::SeqFeature  
testing. Let's see what Chris's plans are for the current VM instance  
- we have talked about also starting to port some of the websites to  
separate instances of the VM to spread the load a little bit more.

One idea that can follow out of doing this work is some sort of  
testable reference servers for some of the bio{*} tools to access some  
basic datasets and hosting.   Maybe with a simple Swissprot instance  
and a slice of a genbank division so that working gbrowse backend &  
biosql instances can be used for code testing and development purposes.

-jason
On Sep 30, 2008, at 2:56 PM, Mauricio Herrera Cuadra wrote:

> Hi all,
>
> Daily-updated test coverage reports are now available for those  
> BioPerl packages which make use of the Build.PL mechanism (except  
> bioperl-db):
>
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/
>
(Continue reading)

Permalink | Reply |
headers
Chris Fields | 1 Oct 15:36
Gravatar

Re: Test coverage for BioPerl now available

Speaking of databases and testing, one thing I would add to the list is a test aggregation server of some sort (maybe using Smolder).  If a VM is set up for BioSQL/Gbrowse-related tests it might be worth adding this in when we have the tuits.

chris

On Oct 1, 2008, at 3:07 AM, Jason Stajich wrote:

Thanks for doing this Mauricio!  Great to have this resource and to follow up on the excellent efforts by Nathan and Sendu to get this ball rolling..

We have a couple of Virtual Hosts through Chris Dagdigian and BioTeam's donated resources that we can setup postgres and mysql instances for biosql and even Bio::DB::GFF & Bio::DB::SeqFeature testing. Let's see what Chris's plans are for the current VM instance - we have talked about also starting to port some of the websites to separate instances of the VM to spread the load a little bit more.

One idea that can follow out of doing this work is some sort of testable reference servers for some of the bio{*} tools to access some basic datasets and hosting.   Maybe with a simple Swissprot instance and a slice of a genbank division so that working gbrowse backend & biosql instances can be used for code testing and development purposes.

-jason
On Sep 30, 2008, at 2:56 PM, Mauricio Herrera Cuadra wrote:

Hi all,

Daily-updated test coverage reports are now available for those BioPerl packages which make use of the Build.PL mechanism (except bioperl-db):

http://bioperl.org/test-coverage/bioperl-live/
http://bioperl.org/test-coverage/bioperl-network/
http://bioperl.org/test-coverage/bioperl-run/

These reports will help us to know the current 'quality' of the code in SVN for most of the BioPerl modules. This idea was started by Nathan Haigh and Sendu a long time ago and it was my fault to not implement on time the necessary script to run the process on a daily basis, so apologies for that.

There are still a few things to be done in order to have this working as it should:

- Nathan, current Devel::Cover module from CPAN doesn't include the JS modifications to make table columns sortable. Do you know what happened to the code you contributed to the author for that?

- Reports could be generated for the rest of the BioPerl packages as soon as they're migrated to the Build.PL infrastructure. Anyone up for that?

- bioperl-db tests require BioSQL to be setup in the webserver machine, and the same goes for bioperl-run's tests with ALL of its dependencies. The bioperl.org site is co-hosted with all of the other OBF projects and that machine also takes care of other things (mailing lists, etc), so I would like your feedback on possible workarounds to not overload the server if we want to setup such test reports.

Thanks & regards,
Mauricio.
_______________________________________________
Bioperl-l mailing list
Bioperl-l <at> lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/bioperl-l

Jason Stajich
jason <at> bioperl.org



_______________________________________________
Bioperl-l mailing list
Bioperl-l <at> lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/bioperl-l

Christopher Fields
Postdoctoral Researcher
Lab of Dr. Marie-Claude Hofmann
College of Veterinary Medicine
University of Illinois Urbana-Champaign




_______________________________________________
Bioperl-l mailing list
Bioperl-l <at> lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/bioperl-l
Permalink | Reply |
headers
Lincoln Stein | 2 Oct 17:41

Re: Test coverage for BioPerl now available

Actually I have a family of kvm images with a host of gbrowse databases (of
various types, including Bio::DB::GFF and Bio::SeqFeature::Store) running in
it. I would be very happy to contribute this to the cause.

Lincoln

On Wed, Oct 1, 2008 at 9:36 AM, Chris Fields <cjfields <at> illinois.edu> wrote:

> Speaking of databases and testing, one thing I would add to the list is a
> test aggregation server of some sort (maybe using Smolder).  If a VM is set
> up for BioSQL/Gbrowse-related tests it might be worth adding this in when we
> have the tuits.
>
> chris
>
>
> On Oct 1, 2008, at 3:07 AM, Jason Stajich wrote:
>
> Thanks for doing this Mauricio!  Great to have this resource and to follow
> up on the excellent efforts by Nathan and Sendu to get this ball rolling..
>
>
> We have a couple of Virtual Hosts through Chris Dagdigian and BioTeam's
> donated resources that we can setup postgres and mysql instances for biosql
> and even Bio::DB::GFF & Bio::DB::SeqFeature testing. Let's see what Chris's
> plans are for the current VM instance - we have talked about also starting
> to port some of the websites to separate instances of the VM to spread the
> load a little bit more.
>
>
> One idea that can follow out of doing this work is some sort of testable
> reference servers for some of the bio{*} tools to access some basic datasets
> and hosting.   Maybe with a simple Swissprot instance and a slice of a
> genbank division so that working gbrowse backend & biosql instances can be
> used for code testing and development purposes.
>
>
> -jason
>
> On Sep 30, 2008, at 2:56 PM, Mauricio Herrera Cuadra wrote:
>
>
> Hi all,
>
>
> Daily-updated test coverage reports are now available for those BioPerl
> packages which make use of the Build.PL mechanism (except bioperl-db):
>
>
> http://bioperl.org/test-coverage/bioperl-live/
>
> http://bioperl.org/test-coverage/bioperl-network/
>
> http://bioperl.org/test-coverage/bioperl-run/
>
>
> These reports will help us to know the current 'quality' of the code in SVN
> for most of the BioPerl modules. This idea was started by Nathan Haigh and
> Sendu a long time ago and it was my fault to not implement on time the
> necessary script to run the process on a daily basis, so apologies for that.
>
>
> There are still a few things to be done in order to have this working as it
> should:
>
>
> - Nathan, current Devel::Cover module from CPAN doesn't include the JS
> modifications to make table columns sortable. Do you know what happened to
> the code you contributed to the author for that?
>
>
> - Reports could be generated for the rest of the BioPerl packages as soon
> as they're migrated to the Build.PL infrastructure. Anyone up for that?
>
>
> - bioperl-db tests require BioSQL to be setup in the webserver machine, and
> the same goes for bioperl-run's tests with ALL of its dependencies. The
> bioperl.org site is co-hosted with all of the other OBF projects and that
> machine also takes care of other things (mailing lists, etc), so I would
> like your feedback on possible workarounds to not overload the server if we
> want to setup such test reports.
>
>
> Thanks & regards,
>
> Mauricio.
>
> _______________________________________________
>
> Bioperl-l mailing list
>
> Bioperl-l <at> lists.open-bio.org
>
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>
>
> Jason Stajich
>
> jason <at> bioperl.org
>
>
>
>
> _______________________________________________
>
> Bioperl-l mailing list
>
> Bioperl-l <at> lists.open-bio.org
>
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>
>
> Christopher Fields
> Postdoctoral Researcher
> Lab of Dr. Marie-Claude Hofmann
> College of Veterinary Medicine
> University of Illinois Urbana-Champaign
>
>
>
>
>
> _______________________________________________
> Bioperl-l mailing list
> Bioperl-l <at> lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>

--

-- 
Lincoln D. Stein

Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Stacey Quinn <Stacey.Quinn <at> oicr.on.ca>

Cold Spring Harbor Laboratory
1 Bungtown Road
Cold Spring Harbor, NY 11724 USA
(516) 367-8380
Assistant: Sandra Michelsen <michelse <at> cshl.edu>
Permalink | Reply |
headers
Sendu Bala | 1 Oct 10:43
Favicon

Re: Test coverage for BioPerl now available

Mauricio Herrera Cuadra wrote:
> Hi all,
> 
> Daily-updated test coverage reports are now available for those BioPerl 
> packages which make use of the Build.PL mechanism (except bioperl-db):
> 
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/

Brilliant, thanks for doing this Mauricio.

> - Nathan, current Devel::Cover module from CPAN doesn't include the JS 
> modifications to make table columns sortable. Do you know what happened 
> to the code you contributed to the author for that?

Could you, in any case, put Nathan's version in the private module area 
of the user running Build.PL, just so we have it in the meantime?
Permalink | Reply |
headers
Nathan S. Watson-Haigh | 2 Oct 07:07
Favicon

Re: Test coverage for BioPerl now available

Hi Sendu,

The JS modifications made it into Devel::Cover 0.62, so no need for my
modified Devel::Cover code now:
http://search.cpan.org/src/PJCJ/Devel-Cover-0.62/CHANGES

The latest version is now 0.64.

Nath

-----Original Message-----
From: Sendu Bala [mailto:bix <at> sendu.me.uk] 
Sent: Wednesday, 1 October 2008 6:43 PM
To: Mauricio Herrera Cuadra
Cc: bioperl-l; Nathan Haigh
Subject: Re: Test coverage for BioPerl now available

Mauricio Herrera Cuadra wrote:
> Hi all,
> 
> Daily-updated test coverage reports are now available for those BioPerl 
> packages which make use of the Build.PL mechanism (except bioperl-db):
> 
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/

Brilliant, thanks for doing this Mauricio.

> - Nathan, current Devel::Cover module from CPAN doesn't include the JS 
> modifications to make table columns sortable. Do you know what happened 
> to the code you contributed to the author for that?

Could you, in any case, put Nathan's version in the private module area 
of the user running Build.PL, just so we have it in the meantime?
Permalink | Reply |
headers
Mauricio Herrera Cuadra | 2 Oct 16:30

Re: Test coverage for BioPerl now available

We are using 0.64 to generate the reports. Maybe the feature is not 
enabled by default? Any hints on that?

Mauricio.

Nathan S. Watson-Haigh wrote:
> Hi Sendu,
> 
> The JS modifications made it into Devel::Cover 0.62, so no need for my
> modified Devel::Cover code now:
> http://search.cpan.org/src/PJCJ/Devel-Cover-0.62/CHANGES
> 
> The latest version is now 0.64.
> 
> Nath
> 
> -----Original Message-----
> From: Sendu Bala [mailto:bix <at> sendu.me.uk] 
> Sent: Wednesday, 1 October 2008 6:43 PM
> To: Mauricio Herrera Cuadra
> Cc: bioperl-l; Nathan Haigh
> Subject: Re: Test coverage for BioPerl now available
> 
> Mauricio Herrera Cuadra wrote:
>> Hi all,
>>
>> Daily-updated test coverage reports are now available for those BioPerl 
>> packages which make use of the Build.PL mechanism (except bioperl-db):
>>
>> http://bioperl.org/test-coverage/bioperl-live/
>> http://bioperl.org/test-coverage/bioperl-network/
>> http://bioperl.org/test-coverage/bioperl-run/
> 
> Brilliant, thanks for doing this Mauricio.
> 
> 
>> - Nathan, current Devel::Cover module from CPAN doesn't include the JS 
>> modifications to make table columns sortable. Do you know what happened 
>> to the code you contributed to the author for that?
> 
> Could you, in any case, put Nathan's version in the private module area 
> of the user running Build.PL, just so we have it in the meantime?
> 
> 
> 
>
Permalink | Reply |
headers
Mauricio Herrera Cuadra | 2 Oct 17:56

Re: Test coverage for BioPerl now available

Disregard that, Adam gave me the tip on the '-report html_basic' option 
for `cover`, which generates the tables as we want. Reports are now 
updated to sortable views.

Mauricio.

Mauricio Herrera Cuadra wrote:
> We are using 0.64 to generate the reports. Maybe the feature is not 
> enabled by default? Any hints on that?
> 
> Mauricio.
> 
> 
> Nathan S. Watson-Haigh wrote:
>> Hi Sendu,
>>
>> The JS modifications made it into Devel::Cover 0.62, so no need for my
>> modified Devel::Cover code now:
>> http://search.cpan.org/src/PJCJ/Devel-Cover-0.62/CHANGES
>>
>> The latest version is now 0.64.
>>
>> Nath
>>
>> -----Original Message-----
>> From: Sendu Bala [mailto:bix <at> sendu.me.uk] Sent: Wednesday, 1 October 
>> 2008 6:43 PM
>> To: Mauricio Herrera Cuadra
>> Cc: bioperl-l; Nathan Haigh
>> Subject: Re: Test coverage for BioPerl now available
>>
>> Mauricio Herrera Cuadra wrote:
>>> Hi all,
>>>
>>> Daily-updated test coverage reports are now available for those 
>>> BioPerl packages which make use of the Build.PL mechanism (except 
>>> bioperl-db):
>>>
>>> http://bioperl.org/test-coverage/bioperl-live/
>>> http://bioperl.org/test-coverage/bioperl-network/
>>> http://bioperl.org/test-coverage/bioperl-run/
>>
>> Brilliant, thanks for doing this Mauricio.
>>
>>
>>> - Nathan, current Devel::Cover module from CPAN doesn't include the 
>>> JS modifications to make table columns sortable. Do you know what 
>>> happened to the code you contributed to the author for that?
>>
>> Could you, in any case, put Nathan's version in the private module 
>> area of the user running Build.PL, just so we have it in the meantime?
>>
>>
>>
>>
>
Permalink | Reply |
headers
Dave Messina | 1 Oct 11:09
Favicon

Re: Test coverage for BioPerl now available

Great, Mauricio! This should be a big help in encouraging everyone to
contribute tests.

Thanks for taking the time to do this.

Dave
Permalink | Reply |
headers
Spiros Denaxas | 1 Oct 11:13
Gravatar

Re: Test coverage for BioPerl now available

Awesome work Mauricio, thanks for taking the time to do this. im sure
it will greatly benefit us.

Spiros

On Tue, Sep 30, 2008 at 10:56 PM, Mauricio Herrera Cuadra
<mauricio <at> open-bio.org> wrote:
> Hi all,
>
> Daily-updated test coverage reports are now available for those BioPerl
> packages which make use of the Build.PL mechanism (except bioperl-db):
>
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/
>
> These reports will help us to know the current 'quality' of the code in SVN
> for most of the BioPerl modules. This idea was started by Nathan Haigh and
> Sendu a long time ago and it was my fault to not implement on time the
> necessary script to run the process on a daily basis, so apologies for that.
>
> There are still a few things to be done in order to have this working as it
> should:
>
> - Nathan, current Devel::Cover module from CPAN doesn't include the JS
> modifications to make table columns sortable. Do you know what happened to
> the code you contributed to the author for that?
>
> - Reports could be generated for the rest of the BioPerl packages as soon as
> they're migrated to the Build.PL infrastructure. Anyone up for that?
>
> - bioperl-db tests require BioSQL to be setup in the webserver machine, and
> the same goes for bioperl-run's tests with ALL of its dependencies. The
> bioperl.org site is co-hosted with all of the other OBF projects and that
> machine also takes care of other things (mailing lists, etc), so I would
> like your feedback on possible workarounds to not overload the server if we
> want to setup such test reports.
>
> Thanks & regards,
> Mauricio.
> _______________________________________________
> Bioperl-l mailing list
> Bioperl-l <at> lists.open-bio.org
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
>
Permalink | Reply |
headers
Chris Fields | 1 Oct 14:54
Gravatar

Re: Test coverage for BioPerl now available

On Sep 30, 2008, at 4:56 PM, Mauricio Herrera Cuadra wrote:

> Hi all,
>
> Daily-updated test coverage reports are now available for those  
> BioPerl packages which make use of the Build.PL mechanism (except  
> bioperl-db):
>
> http://bioperl.org/test-coverage/bioperl-live/
> http://bioperl.org/test-coverage/bioperl-network/
> http://bioperl.org/test-coverage/bioperl-run/
>
> These reports will help us to know the current 'quality' of the code  
> in SVN for most of the BioPerl modules. This idea was started by  
> Nathan Haigh and Sendu a long time ago and it was my fault to not  
> implement on time the necessary script to run the process on a daily  
> basis, so apologies for that.
>
> There are still a few things to be done in order to have this  
> working as it should:
>
> - Nathan, current Devel::Cover module from CPAN doesn't include the  
> JS modifications to make table columns sortable. Do you know what  
> happened to the code you contributed to the author for that?
>
> - Reports could be generated for the rest of the BioPerl packages as  
> soon as they're migrated to the Build.PL infrastructure. Anyone up  
> for that?

Beyond bioperl-db (which Jason mentioned a solution for) and bioperl- 
pedigree which other distributions would we be talking about?  I think  
bioperl-ext would be too problematic under the current build scheme.

> - bioperl-db tests require BioSQL to be setup in the webserver  
> machine, and the same goes for bioperl-run's tests with ALL of its  
> dependencies. The bioperl.org site is co-hosted with all of the  
> other OBF projects and that machine also takes care of other things  
> (mailing lists, etc), so I would like your feedback on possible  
> workarounds to not overload the server if we want to setup such test  
> reports.

I think Jason answered that one.

> Thanks & regards,
> Mauricio.

Fantastic work Mauricio, thanks for taking the time to do this!

chris
Permalink | Reply |
headers
George Hartzell | 1 Oct 18:33
Gravatar

Re: Test coverage for BioPerl now available


Awesome!  Thanks for pushing this forward.

g.
Permalink | Reply |

Gmane