For me, this is further evidence of why the practices of Extreme Programming (XP), especially Test-Driven Development (TDD), are so important. To a high degree, TDD ensures that the software is really working and that regressions are caught and fixed quickly. TDD also provides an unambiguous definition of “done”; do the automated tests pass? The feedback on which stories are really done tells the team its true velocity and hence how many story points will be finished by a given target date. Also, you spend far less unplanned and unpredictable time debugging problems.

Hence, productivity and schedule accuracy tend to be much better for XP teams. Over time, as the software grows in size and complexity, the automated test suite will keep delivering “dividends”, by continuing to catch regressions and by empowering the team to do the aggressive refactorings that are necessary to keep the software evolving to meet new requirements.

The SD Times article goes on to say that

... two-thirds of the responding managers indicated that a solution that reduced the time spent on resolving application problems would be of interest if it created significant efficiencies and improved quality.

The article concludes with a quote that automated testing is one solution. I agree. Just make sure it is the XP kind of automated testing.

In Extreme Programming (XP), a team’s velocity is defined as the number of story points completed in an iteration. We can track the velocity for each iteration and calculate the average velocity over several iterations. If we know how many story points comprise an application, we can even project when an application can be released . Velocity is a very useful measure of a team’s capabilities. I have found it to be a much more reliable (make that infinitely more reliable) number to be used for project planning than asking developers for time estimates based on requirements.

Because velocity correlates to a team’s productivity, there is a tendency for managers to try to “goose” the velocity. The technical term for this is a stretch goal. Just like the coach of a sport’s team trying to get the team to play better, I can imagine project managers saying at a team meeting at an iteration boundary, “Come on team – you did 22 points last week, what do you say we try for 25 this week?” This same manager probably buys those motivation posters of eagles soaring and hangs them all around the team room.

The programming team takes pride in their achievements and they also like to see their velocity increasing. They will try to persuade the Customer to give them points for any bit of work they do. I can imagine a developer putting in a few minutes work to change the position of a GUI button on a screen and begging the Customer, “Can I get a point for that?”

XP teams can become obsessed with their velocity, but increasing velocity isn’t always a good thing. A team can inadvertently become so focused on getting points done that they sometimes forget the real goal – to deliver a quality product.

This became apparent to me recently when I visited a client to see how they were doing in their transition to XP. Their velocity had skyrocketed since I last saw them – and it was definitely not a good thing.

When Object Mentor does an XP transition with a client, we start with our XP Immersion course to get everybody on the same page about what our goals are. Ideally, we use two instructors, one to train the programmers in topics such as Test Driven Development and Refactoring, and the other coach teaches story writing, iteration planning, and release planning to the customer team. We then bring the two groups together for a day and have the customer team drive the programming team on a classroom exercise so everybody can experience how the process works. The instructors then stay and work with the team for a few iterations, coaching them on how to use what they learned in class on their own project.

Such was the case with the client I mentioned earlier. I was working with the programmers and the other coach had the customer team. When it came time to estimate stories, the other coach suggested using an open-ended scale. Where I like to use a scale of 1-5, this team was using a 1-infinity scale. That’s fine, it doesn’t really matter which scale you use, as long as whatever scale you choose is consistent. Stories were estimated according to their relative complexities, iterations were planned, and the programmers started cranking out code. After a few iterations the team’s velocity settled to a value of about 14 points. The team was doing fine and it was time for me to move on and let them do it alone.

When I returned to do a process checkup, their velocity had climbed to 48 points. Wow. This new process must really be resonating with the team. I purposely timed my visit to occur on an iteration boundary and we conducted a retrospection on how well the team was adhering the the XP practices. This turned out to be the bad news.

With a focus on getting more points done, it seemed that the team had abandoned many of the practices. Programmers weren’t pairing, weren’t writing unit tests, and weren’t refactoring their code. Customers were trying to stay ahead of the programmer’s ever increasing productivity abandoned writing automated acceptance tests in FitNesse and were now back to manual testing. I was heartbroken.

Beck teaches that one of the things teams have to do is to adapt the practices to the organization. Perhaps what they were doing was adapting. Adapting to the point that they weren’t even close to doing XP anymore, but adapting nonetheless. I wondered how that was working for them.

I observed the iteration planning meeting for the upcoming iteration and I noticed that all of the user stories were bug fixes from the previous iteration. No new functionality was being added to the application, the team was essentially spinning its wheels. So even though the velocity indicated a very productive team, the actual productivity was essentially zero. There must be a lesson here.

Velocity is what it is. You must take great care when encouraging a team to increase its velocity because you will always get whatever you’re trying to maximize. You have to be careful that you are maximizing the right thing. Want to maximize number of lines of code produced? Well you’re most likely going to get a lot of really crappy code. Want to maximize the number of unit tests that developers write? You’ll get a lot of them, but they’ll probably be worthless. Applying pressure to increase the velocity might make developers to subconsciously inflate their estimates or worse, abandon good development practices in order to get the numbers up.