What’s the problem?

It was written in the days when we would use a pre-processing stage to instrument code and then execute tests. This is fine if I wanted to work at the command line and use ant/maven to build. However, I want to work in an IDE that makes running unit tests easy (Eclipse, though they all do it now). But if I have to remember to instrument my classes before running my tests, that leads to human error. I don’t want that, I want to just run my unit tests and have my classes dynamically instrumented. (This is not speculation, this comes from working in a group consisting of multiple teams, all of which were using some Aspects written using AspectJ and running unit tests in Eclipse – when we introduced dynamic instrumentation – pre JDK 1.5 and even working in WebSphere 5.x), it improved productivity.

What about a plug-in? Sure, there’s one for Eclipse but I’ve not been able to download it. It probably works fine – after a class is compiled, it gets instrumented – but if I’m able to write a simple Java Agent, I can create a JVM configuration with a few parameters and every time I run my tests, viola, dynamic instrumentation with only a little one-time environment configuration. Also, I don’t have to worry about waiting for a plug-in update to continue using the tool.

Will my tests run slower? Probably, but until I know how much slower, I’m willing to risk it. (I’ve used dynamic instrumentation when running over 1,000 tests on a workspace with > 1 Million lines of code, it was fine.) If it’s an issue, I can imagine using a combination of annotations and the Java agent. Something like:

@TestInstrument
public class SomeClassThatUsesThreading {}

This would allow the Java Agent to only instrument some classes, rather than all classes. This could cause problems if I forget the annotation, but it’s an option if speed is an issue.

I would not do that unless it was necessary. First, most of my tests would be testing code that is not thread-related. Those tests would not require instrumentation. The tests that require instrumentation, would be somewhere else, and I’d run them with a different frequency; I’d run them longer, with more configurations and iterations, to increase my chances of finding threading-related issues.

There’s another option. Copy what ConTest is already doing using AOP. I tried, and I cannot select the correct point-cuts using traditional point-cut syntax – try selecting synchronized blocks and then every line within the block, that’s not a typical point-cut usage scenario. I considered a combination of hand instrumentation and AOP – it’ll work but it makes the code ugly. I even considered using asm or cglib directly and at that point I knew it was more time than I wanted to spend when the developers of ConTest have years of experience instrumenting classes.

Anyway, I’m hoping the team working on this tool will publish an API soon so I can give that a try. They mentioned they would at some point.

If you want more information on writing a Java Agent (and the class that actually registeres it), have a look at Brett’s Java Agent Notes.