Wednesday, November 14, 2007

Test Suite Problems

One of the projects I work on has a reasonable number of JUnit 3 tests (2500). We also use Cruise Control to automate the build cycle. Each package has a suite which defines all the tests to be run, but this has become more and more error prone over time. Developers forget to add tests to the suites, leaving orphaned tests etc.

The obvious solution is to use the batchtest mechanism provided in Apache Ant. However this produces an XML file for each test run, containing about 20K of setup information for the tests.
Cruise will then attempt to merge all these XML files for its build reports, which takes a lot of time and a *lot* of RAM.

So the core issue is that I want to run a single test suite, without having to maintain the suite methods.

Turns out that using the JUnit-addons allows us to do this pretty easily :

public class AutoTests
{

public static Test suite() throws Exception
{
DirectorySuiteBuilder builder = new DirectorySuiteBuilder();

builder.setFilter(new SimpleTestFilter()
{
public boolean include(Class classs)
{
return super.include(class);
}
});
return builder.suite("classes");
}
}



This will then select any class that ends Test to a suite. You just call the test as per any JUnit test :


<test outfile="${junit.log.location}/odyssey/Tests" name="com.AutoTests" />



I didn't like the fact that you have to know the classes directory in the above example, a really simple (slightly hacky) way around this is to do the following :

String testClassesLocation=System.getProperty("junit.test.class.location", "classes");
return builder.suite(testClassesLocation);


then in the ant build file define the class location like this :

<jvmarg value="-Djunit.test.class.location=classes"/>
<test outfile="${junit.log.location}/Test" name="com.AutoTests" />


At least this way the junit class location can change without changing the source code for the tests.