Wednesday, November 14, 2007

Test Suite Problems

One of the projects I work on has a reasonable number of JUnit 3 tests (2500). We also use Cruise Control to automate the build cycle. Each package has a suite which defines all the tests to be run, but this has become more and more error prone over time. Developers forget to add tests to the suites, leaving orphaned tests etc.

The obvious solution is to use the batchtest mechanism provided in Apache Ant. However this produces an XML file for each test run, containing about 20K of setup information for the tests.
Cruise will then attempt to merge all these XML files for its build reports, which takes a lot of time and a *lot* of RAM.

So the core issue is that I want to run a single test suite, without having to maintain the suite methods.

Turns out that using the JUnit-addons allows us to do this pretty easily :

public class AutoTests

public static Test suite() throws Exception
DirectorySuiteBuilder builder = new DirectorySuiteBuilder();

builder.setFilter(new SimpleTestFilter()
public boolean include(Class classs)
return super.include(class);
return builder.suite("classes");

This will then select any class that ends Test to a suite. You just call the test as per any JUnit test :

<test outfile="${junit.log.location}/odyssey/Tests" name="com.AutoTests" />

I didn't like the fact that you have to know the classes directory in the above example, a really simple (slightly hacky) way around this is to do the following :

String testClassesLocation=System.getProperty("junit.test.class.location", "classes");
return builder.suite(testClassesLocation);

then in the ant build file define the class location like this :

<jvmarg value="-Djunit.test.class.location=classes"/>
<test outfile="${junit.log.location}/Test" name="com.AutoTests" />

At least this way the junit class location can change without changing the source code for the tests.

Monday, May 28, 2007

Passing Parameters To JMeter In Ant

As well as using JChav as part of our deployment cycle, I have additionally been using it to monitor some production servers. For the production servers you obviously don't run them at any load that is likely to affect the real users. However 1 thread running a set of tests once an hour (thanks to cron) can give a good view of performance over time. I am fully aware of other software that can be used to monitor my sites, but given that I already have a JMeter script designed to test key points of the application it seems like the right solution.

The production server I am using is a cluster placed behind a load balancer and I want to be able to run the tests against each instance. The application already has a URI to manually set each server to use (i.e. the load balancer server affinity), my issue is how do I maintain a single JMeter script but be able to set domain/server specific details?

The great news is that the good folks responsible for JMeter have already thought this through and added a property system.

So given an HTTPSampler that looks like this :

<HTTPSampler guiclass="HttpTestSampleGui" testclass="HTTPSampler" testname="SetServer to server1 " enabled="true">

it can be replaced with :

<HTTPSampler guiclass="HttpTestSampleGui" testclass="HTTPSampler" testname="SetServer to ${__property(affinity)} " enabled="true">

The JMeter Ant task also allows you to set the property, so you can do the following :

<jmeter jmeterhome="${jmeter.install.dir}" testplan="${jmeter.testplan}" resultlog="${jmeter.result.file}"> <property name="affinity" value="server1"/> <property name="" value="xml"/> </jmeter>

This will call my JMeter script passing a parameter named affinity with the value server1.

I want to call my script with several values and store them in different output directories etc. i.e. call my ant script with the property affinitty set to server1, then server2 etc etc.

I've come up with a hack that works for me, but I'm sure that a better way exists. What I have done is to separate my ant builds into 2 files. The first builds a single instance, and looks like this :

<jmeter jmeterhome="${jmeter.install.dir}" testplan="${jmeter.testplan}" resultlog="${jmeter.result.file}"> <property name="affinity" value="${servername}"/> <property name="" value="xml"/> </jmeter>
servername is now a parameter into this script, and all the other directory dependencies etc use that parameter too.

I then create a master build which calls the build single for each value :

<echo>Testing server1.</echo> <ant antfile="build-singleone.xml"> <property name="servername" value="server1"/> </ant> <echo>Testing server2.</echo> <ant antfile="build-singleone.xml"> <property name="servername" value="server2"/> </ant> <echo>Testing server3.</echo> <ant antfile="build-singleone.xml"> <property name="servername" value="server3"/> </ant>

I don't like the repetition in the script, but it does work.

Thursday, May 17, 2007

Clues on getting started with GWT under Eclipse

After an interesting JavaOne session on GWT, I thought I would have a quick play with the toolkit. I hit a little roadblock initially which after discussions with other colleagues they had problems with too.

In the Getting Started guide the section on building an Eclipse project skips one step that caught me out. Do not create the project in your Eclipse workspace, instead perform the two steps in a temporary area. Then import that temporary project using Eclipse/import/Existing files into workspace - be sure to click the copy source.

I found if I didn't do that I get errors about the project I was trying to import being already inside the workspace.

My first impressions of the toolkit are that it is excellent. This all looks much easier than supporting a big ball of JavaScript, and some of the 1.4 features described in the JavaOne talk really seemed to have been well thought out. In particular the idea of combining many small graphics into one large and clipping for view(automatically) will cut down lots of http requests and speed up application start.