Object Mentor Blog: Why you have time for TDD (but may not know it yet...)

Why you have time for TDD (but may not know it yet...)

Dean Wampler — Sun, 30 Sep 2007 20:01:02 +0000

Note: Updated 9/30/2007 to improve the graphs and to clarify the content.

A common objection to TDD is this; “We don’t have time to write so many tests. We don’t even have enough time to write features!”

Here’s why people who say this probably already have enough time in the (real) schedule, they just don’t know it yet.

Let’s start with an idealized Scrum-style “burn-down chart” for a fictional project run in a “traditional” way (even though traditional projects don’t use burn-down charts…).

We have time increasing on the x axis and the number of “features” remaining to implement on the y axis (it could also be hours or “story points” remaining). During a project, a nice feature of burn-down charts is that you can extend the line to see where it intersects the x axis, which is a rough indicator of when you’ll actually finish.

The optimistic planners for our fictional project plan to give the software to QA near the end of the project. They expect QA to find nothing serious, so the release will occur soon thereafter on date T0.

Of course, it never works out that way:

The red line is the actual effort for our fictional project. It’s quite natural for the planned list of features to change as the team reacts to market changes, etc.. This is why the line goes up sometimes (in “good” projects, too!). Since this is a “traditional” project, I’m assuming that there are no automated tests that actually prove that a given feature is really done. We’re effectively running “open loop”, without the feedback of tests.

Inevitably, the project goes over budget and th planned QA drop comes late. Then things get ugly. Without our automated unit tests, there are lots of little bugs in the code. Without our automated integration tests, there are problems when the subsystems are run together. Without our acceptance tests, the implemented features don’t quite match the actual requirements for them.

Hence, a chaotic, end-of-project “birthing” period ensues, where QA reports a list of big and small problems, followed by a frantic effort (usually involving weekends…) by the developers to address the problems, followed by another QA drop, followed by…, and so forth.

Finally, out of exhaustion and because everyone else is angry at the painful schedule slip, the team declares “victory” and ships it, at time T1.

We’ve all lived through projects like this one.

Now, if you remember your calculus classes (sorry to bring up painful memories), you will recall that the area under the curve is the total quantity of whatever the curve represents. So, the actual total feature work required for our project corresponds to the area under the red line, while the planned work corresponds to the area under the black line. So, we really did have more time than we originally thought.

Now consider a Test-Driven Development (TDD) project [1]:

Here, the blue line is similar to the red line, at least early in the project. Now we have frequent “milestones” where we verify the state of the project with the three kinds of automated tests mentioned above. Each milestone is the end of an iteration (usually 1-4 weeks apart). Not shown are the 5-minute TDD cycles and the feedback from the continuous integration process that does our builds and runs all our tests after every block of commits to version control (many times a day).

The graph suggests that the total amount of effort will be higher than the expected effort without tests, which may be true [2]. However, because of the constant feedback during the whole life of the project, we really know where we actually are at any time. By measuring our progress in this way, we will know early whether or not we can meet the target date with the planned feature set. With early warnings, we can adjust accordingly, either dropping features or moving the target date, with relatively little pain. Whereas, without this feedback, we really don’t know what’s done until something, e.g., the QA process, gives us that feedback. Hence, at time T0, just before the big QA drop, the traditional project has little certainty about what features are really completed.

So, we’ll experience less of the traditional end-of-project chaos, because there will be fewer surprises. Without the feedback from automated tests, QA is find lots of problems, causing the chaotic and painful end-of-project experience. Finding and trying to fix major problems late in the game can even kill a project.

So, TDD converts that unknown schedule time at the end into known time early in the project. You really do have time for automated tests and your tests will make your projects more predictable and less painful at the end.

Note: I appreciate the early comments and questions that helped me clarify this post.

[1] As one commenter remarked, this post doesn’t actually make the case for TDD itself vs. alternative “test-heavy” strategies, but I think it’s pretty clear that TDD is the best of the known test-heavy strategies, as argued elsewhere.

[2] There is some evidence that TDD and pair programming lead to smaller applications, because they help avoid unnecessary features. Also, they provide constant feedback to the team, including the stake holders, on what the feature set should really be and which features are most important to complete first.

"Why you have time for TDD (but may not know it yet...)" by Dean Wampler

Thu, 04 Oct 2007 08:18:17 +0000

@Carfield,

I’m not sure what exactly you have in mind about the end of a project. It’s true that near the end, you have less uncertainty about when you’ll finish and what work remains, assuming you have done some testing, automated or manual, so that big surprises are less likely. Also, even in a well-run TDD project, you may encounter some volatility near the end of a release cycle as the team works through final integration issues, last minute story changes, etc. Often, being close to a major event like that clarifies your thinking somewhat.

"Why you have time for TDD (but may not know it yet...)" by Carfield Yim

Thu, 04 Oct 2007 02:47:51 +0000

Thanks for the nice article and article, it probably true when the project is just start. However, if the project is near “finish” ( in term of schedule and function implemented ) , do you think is it still valid?

"Why you have time for TDD (but may not know it yet...)" by Dean Wampler

Wed, 03 Oct 2007 11:03:23 +0000

@KPSeal,

I’ve greatly simplified a complex, human process, of course ;)

I agree that a comprehensive economic analysis is important. I’ve read that a significant amount of a project’s total life-time return on investment is lost when the schedule slips. So, if TDD (including all the different kinds of tests, as Bob said…) allows you to release a limited subset of the planned features early, your return on investment might be significantly better. Just having the greater schedule predictability, i.e., avoiding the end-of-project chaos, empowers the stakeholders to make good decisions at appropriate times that will benefit everyone.

Concerning hiring the best people (which we all want to do, of course), Kent Beck said that XP was designed so that average developers can succeed, not just the best developers.

"Why you have time for TDD (but may not know it yet...)" by KPSeal

Wed, 03 Oct 2007 10:18:13 +0000

What about the iron triangle?

What you’ve demonstrated is that the total feature effort is less, most likely by virtue of a higher quality (ie, TDD) implementation. This is, to my mind, as a result of having more money spent either through having more skilled developers or more QA team involvement throughout implementation. You’ve not shown that the real effort, that all important bottom line, is less.

Were I worried solely about a timely delivery I can see that TDD would give me a better chance. But then so might hiring the 6 best developers in the world.

Based on personal experience I believe you, by the way, but I can imagine the raised eyebrows if I tried to use this as a justification for a change in methodology!

"Why you have time for TDD (but may not know it yet...)" by unclebob

Wed, 03 Oct 2007 07:27:19 +0000

We need to be clear that TDD includes automated unit tests, and automated acceptance tests. Both are written before the code that they specify. Unit tests are written seconds before, and acceptance tests are written hours or days before.

It is possible to gain the same kinds of benefits using a “test-after” approach. However it is much more difficult. Tests that are written first aren’t as influenced by the implementation as late tests are. Early acceptance tests are specifications of what should be. Late acceptance tests are often specifications of what is.

"Why you have time for TDD (but may not know it yet...)" by Johannes Brodwall

Tue, 02 Oct 2007 12:00:50 +0000

Much clearer. Thank you.

"Why you have time for TDD (but may not know it yet...)" by Peter Wood

Mon, 01 Oct 2007 10:24:11 +0000

I was reading about the Law of Big Numbers in ‘An Introduction To General Systems Theory’ on the train this morning. Basically, the bigger the sample the more accurate the estimate.

Prediction works better with bigger numbers; lots of iterations, technology spikes, tests, milestones, user stories. There will be a point at which the cost of creating these will not outweigh the worth of the new accuracy of estimate. So it’s not simply as many of the above as possible but as many as is useful in improving your predictions.

"Why you have time for TDD (but may not know it yet...)" by Dean Wampler

Sun, 30 Sep 2007 16:20:03 +0000

Yea, I see your confusion about the sort-of parabola. It’s meant to suggest that the “effort”, i.e., number of concurrent development tasks starts out low, then climbs as more people get involved, but the meaning of the peak and the end point aren’t that obvious.

I think a simpler Scrum-style “burn-down chart” would have been better, where it’s plotting the number of features that are left to implement versus time. If you’re not familiar with them, they (should) trend down and hit zero when you’re down. So, earlier in the project, if you extend the existing line down it gives you a rough measure of when you’ll finish, i.e., when the line intersects the X axis. In fact, the graphs to the right of the peaks are basically burn-downs.

I think I’ll edit the blog to use this approach instead.

"Why you have time for TDD (but may not know it yet...)" by Johannes Brodwall

Sun, 30 Sep 2007 16:09:34 +0000

I think that for TDD definition that means more than just automated unit tests, this could work. But it still requires a lot of discipline and understanding. Especially regarding acceptance tests, and in my experience: deployment and (lightweight) configuration management.

However, I don’t understand whats up with the parabola? What is it supposed to represent? The amount of work remaining? Wouldn’t that be an “S-curve”? Or the amount of effort at any point in time? So we’re staffing up the hypothetical project? Why?

So: I might be persuaded by your argument, but I don’t really understand it. Am I being slow, or should you rething the information design here?

"Why you have time for TDD (but may not know it yet...)" by Dean Wampler

Sun, 30 Sep 2007 15:33:00 +0000

This is a summary of my cumulative experience.

I did say that you should use continuous integration. Also, it’s true that you may get a similar curve with “Test-After Development”, but the design won’t be as good and you’ll find yourself doing more heavy-duty refactoring to retrofit the tests (e.g., breaking dependencies that make testing hard). Hence, you’ll be closer to the “bad” curve, probably.

"Why you have time for TDD (but may not know it yet...)" by Arnon Rotem-Gal-Oz

Sat, 29 Sep 2007 17:58:20 +0000

I am sorry, but you are just a lot of hand waiving here. There’s nothing here that explains why TDD will cause this effect for instance if you do TDD but not integrate often you’d just have a lot of tested units and you’d still fail if you do code first and write unit tests after + continuous integration and system tests you should probably get a similar curve etc.

Arnon