Agile Stack

Building better development techniques

Jan 09

How to be agile when all about you are not – part 4

In part 3 of this series, I explained how automated testing can be used in the build process to greatly reduce the time taken to verify that new code is correct. I also introduced the idea of refactoring to make code easier to test. In this article, I shall go on to explain how to further improve the build process with continuous integration.

The idea behind continuous integration is simple, and stems directly from automated testing. There should be an automated process that continuously checks out the latest code and attempts to build and test it, reporting any failures automatically to the person who last made a commit since they are most likely to have broken the build.

Although this sounds daunting, it’s actually trivial to set up. All that is required (and again I speak from a Java based perspective) is:
1) the free and open source application Hudson
2) the free and open source servlet container Tomcat
3) the free and open source build tool Ant

You’ll doubtless have noticed a distinct bias on my behalf on free and open source software. If you’d like more information on why this is a Good Thing then check out this article about it.

All of those applications are a snap to install and get running. Once Hudson is up and running (it’s packaged as a self-contained WAR file), it is merely a matter of creating a job and configuring it with the version control details and the location of the build script file. I would recommend, though, that the machine on which these applications are installed (I’ll call it the continuous integration server, or CI server for short) is fairly beefy since it will very quickly start to be used by everybody for everything. And therein lies a problem.

At this point it is difficult to introduce the Agile methodology by stealth, since you’re going to need a server somewhere and that means involving other people. Of course, you could just install said CI server on your own machine and let it run overnight which is perfectly acceptable up to a point. Out of the box, Hudson will perform a fairly basic set of operations: check out the code at a particular time (or interval); attempt to run the build script; note any test failures and report back with a green light if all is well.

However, Hudson has an extensible architecture that allows plugins to be created and there is a rich tapestry of possible add-ons available from the supporting website. There is a plugin for the FindBugs application which can identify code that is likely to cause problems by an examination of the byte code. Another keeps track of code coverage so that weak tests can be readily identified. Still another identifies long term trends associated with the number of tests and number of failures.

The list is long and before long the CI server will become as vital to the build process as the version control system, and clearly it will need to be given the same level of support from the network administrators. All being well, if you adopt the nightly build approach based on your own machine, then it won’t be long before you are starting to gather very useful data about the build process. Managers love this kind of information so it should be a fairly painless transition to move Hudson from your development machine to a managed server.

So by now the gradual changes to the development process have started to show real results and others are starting to take notice. It seems that your code is getting created faster than everyone else’s and it almost always works first time. And you’re identifying weak spots in the main application that no-one has ever seen before. Not bad, but surely there is more to it.

In part 5 of this series I will explain the next major shift: automating everything.

Tagged with: agile • hudson • testing

Jan 06

Avoiding duplicate test results with Hudson

workaround No Comments »

I’m a big fan of automated testing, in fact automated everything since I quickly lose interest in repetitive tasks. As a result of this I’m drawn to build systems like Maven which provide a lot of good conventions and automation right out of the box.

There are a million articles out there about Maven so I’m not going to repeat what they say here, but I would like to comment about a particularly tricky problem that I encountered a little while ago which ended up having a very neat solution.

Hudson duplicates the results of multiple executions of a test suite through Maven so that a failure in, say, the first run is overwritten by a success in the second. Hudson signals the failure in the build report and the test report contains multiple entries with the same class and method name, one marked as a failure the other as a success and no reliable stack trace highlighting which of the executions was responsible for the failure. How can Hudson be made to provide separate test reports for each test suite execution?

Firstly, it should be noted that Hudson runs Maven in a special way which allows it to trigger operations when plugins complete. In the case of automated testing Hudson is able to slip in between plugin executions and extract the test results before another plugin is able to execute. This rules out a plugin based solution (although I did have some fun writing one only for it to fail to solve the problem).

The way to solve the problem is to change the JUnit4ClassRunner that is being used by JUnit to one of your own making which is able to modify the returned name of the test method that has been executed. This allows a test method to have, for example, a different suffix depending on an environment variable that can be passed in by Maven. This feature is easy to implement if you are using JUnit4 (which is backwards compatible with JUnit3 so there’s no excuse not to upgrade).

In Maven the surefire plugin provides access to JUnit tests and thanks to the <executions> configuration element the plugin can be repeated with different settings. For example,

<executions>
  <execution>
    <id>1_FF3</id>
    <phase>integration-test</phase>
    <goals>
      <goal>test</goal>
    </goals>
    <configuration>
      <environmentVariables>
        <browser>ff3</browser>
      </environmentVariables>
    </configuration>
  </execution>
  <execution>
    <id>2_IE6</id>
    <phase>integration-test</phase>
    <goals>
      <goal>test</goal>
    </goals>
    <configuration>
      <environmentVariables>
        <browser>ie6/browser>
      </environmentVariables>
    </configuration>
  </execution>
</executions>

The above, if placed within a surefire plugin declaration, will run the test suite twice, once passing in “ff3″ as an environment variable (“browser”) and the second “ie6″.

Typically, most JUnit test cases inherit from some common base class that provides common methods, so I’ll assume that you’ve got something like AbstractWebTestCase below:

@RunWith(JUnit4ClassRunner.class)
public abstract class AbstractWebTestCase {}

The next step is to write your own replacement for JUnit4ClassRunner so that the Description object is constructed with slightly altered values. This is actually pretty trivial since you only need to override a single method:

public class SuffixEnabledJUnit4ClassRunner extends JUnit4ClassRunner {
    public SuffixEnabledJUnit4ClassRunner(Class clazz) throws InitializationError {
        super(clazz);
    }
    @Override
    protected Description methodDescription(Method method) {
        String suffix = System.getenv("browser");
        return Description.createTestDescription(
          getTestClass().getJavaClass(),
          testName(method)+suffix,
          testAnnotations(method));
    }  
}

Using the above example, you should be able to construct your own JUnit4ClassRunner which will modify your method name descriptions in the manner of your choosing. Hudson will merrily use whatever name is given out by your ClassRunner so feel free to experiment.

Tagged with: hudson • maven • testing • workaround