Search | Blog | Contact

Why you have time for TDD (but may not know it yet...) 44

Posted by Dean Wampler Mon, 01 Oct 2007 01:01:02 GMT

Note: Updated 9/30/2007 to improve the graphs and to clarify the content.

A common objection to TDD is this; “We don’t have time to write so many tests. We don’t even have enough time to write features!”

Here’s why people who say this probably already have enough time in the (real) schedule, they just don’t know it yet.

Let’s start with an idealized Scrum-style “burn-down chart” for a fictional project run in a “traditional” way (even though traditional projects don’t use burn-down charts…).

ideal_timeline2.png

We have time increasing on the x axis and the number of “features” remaining to implement on the y axis (it could also be hours or “story points” remaining). During a project, a nice feature of burn-down charts is that you can extend the line to see where it intersects the x axis, which is a rough indicator of when you’ll actually finish.

The optimistic planners for our fictional project plan to give the software to QA near the end of the project. They expect QA to find nothing serious, so the release will occur soon thereafter on date T0.

Of course, it never works out that way:

actual_timeline2.png

The red line is the actual effort for our fictional project. It’s quite natural for the planned list of features to change as the team reacts to market changes, etc.. This is why the line goes up sometimes (in “good” projects, too!). Since this is a “traditional” project, I’m assuming that there are no automated tests that actually prove that a given feature is really done. We’re effectively running “open loop”, without the feedback of tests.

Inevitably, the project goes over budget and th planned QA drop comes late. Then things get ugly. Without our automated unit tests, there are lots of little bugs in the code. Without our automated integration tests, there are problems when the subsystems are run together. Without our acceptance tests, the implemented features don’t quite match the actual requirements for them.

Hence, a chaotic, end-of-project “birthing” period ensues, where QA reports a list of big and small problems, followed by a frantic effort (usually involving weekends…) by the developers to address the problems, followed by another QA drop, followed by…, and so forth.

Finally, out of exhaustion and because everyone else is angry at the painful schedule slip, the team declares “victory” and ships it, at time T1.

We’ve all lived through projects like this one.

Now, if you remember your calculus classes (sorry to bring up painful memories), you will recall that the area under the curve is the total quantity of whatever the curve represents. So, the actual total feature work required for our project corresponds to the area under the red line, while the planned work corresponds to the area under the black line. So, we really did have more time than we originally thought.

Now consider a Test-Driven Development (TDD) project [1]:

tdd_timeline2.png

Here, the blue line is similar to the red line, at least early in the project. Now we have frequent “milestones” where we verify the state of the project with the three kinds of automated tests mentioned above. Each milestone is the end of an iteration (usually 1-4 weeks apart). Not shown are the 5-minute TDD cycles and the feedback from the continuous integration process that does our builds and runs all our tests after every block of commits to version control (many times a day).

The graph suggests that the total amount of effort will be higher than the expected effort without tests, which may be true [2]. However, because of the constant feedback during the whole life of the project, we really know where we actually are at any time. By measuring our progress in this way, we will know early whether or not we can meet the target date with the planned feature set. With early warnings, we can adjust accordingly, either dropping features or moving the target date, with relatively little pain. Whereas, without this feedback, we really don’t know what’s done until something, e.g., the QA process, gives us that feedback. Hence, at time T0, just before the big QA drop, the traditional project has little certainty about what features are really completed.

So, we’ll experience less of the traditional end-of-project chaos, because there will be fewer surprises. Without the feedback from automated tests, QA is find lots of problems, causing the chaotic and painful end-of-project experience. Finding and trying to fix major problems late in the game can even kill a project.

So, TDD converts that unknown schedule time at the end into known time early in the project. You really do have time for automated tests and your tests will make your projects more predictable and less painful at the end.

Note: I appreciate the early comments and questions that helped me clarify this post.

[1] As one commenter remarked, this post doesn’t actually make the case for TDD itself vs. alternative “test-heavy” strategies, but I think it’s pretty clear that TDD is the best of the known test-heavy strategies, as argued elsewhere.

[2] There is some evidence that TDD and pair programming lead to smaller applications, because they help avoid unnecessary features. Also, they provide constant feedback to the team, including the stake holders, on what the feature set should really be and which features are most important to complete first.

Comments

Leave a response

  1. Avatar
    Arnon Rotem-Gal-Oz 1 day later:

    I am sorry, but you are just a lot of hand waiving here. There’s nothing here that explains why TDD will cause this effect for instance if you do TDD but not integrate often you’d just have a lot of tested units and you’d still fail if you do code first and write unit tests after + continuous integration and system tests you should probably get a similar curve etc.

    Arnon

  2. Avatar
    Dean Wampler about 4 hours later:

    This is a summary of my cumulative experience.

    I did say that you should use continuous integration. Also, it’s true that you may get a similar curve with “Test-After Development”, but the design won’t be as good and you’ll find yourself doing more heavy-duty refactoring to retrofit the tests (e.g., breaking dependencies that make testing hard). Hence, you’ll be closer to the “bad” curve, probably.

  3. Avatar
    Johannes Brodwall about 4 hours later:

    I think that for TDD definition that means more than just automated unit tests, this could work. But it still requires a lot of discipline and understanding. Especially regarding acceptance tests, and in my experience: deployment and (lightweight) configuration management.

    However, I don’t understand whats up with the parabola? What is it supposed to represent? The amount of work remaining? Wouldn’t that be an “S-curve”? Or the amount of effort at any point in time? So we’re staffing up the hypothetical project? Why?

    So: I might be persuaded by your argument, but I don’t really understand it. Am I being slow, or should you rething the information design here?

  4. Avatar
    Dean Wampler about 4 hours later:

    Yea, I see your confusion about the sort-of parabola. It’s meant to suggest that the “effort”, i.e., number of concurrent development tasks starts out low, then climbs as more people get involved, but the meaning of the peak and the end point aren’t that obvious.

    I think a simpler Scrum-style “burn-down chart” would have been better, where it’s plotting the number of features that are left to implement versus time. If you’re not familiar with them, they (should) trend down and hit zero when you’re down. So, earlier in the project, if you extend the existing line down it gives you a rough measure of when you’ll finish, i.e., when the line intersects the X axis. In fact, the graphs to the right of the peaks are basically burn-downs.

    I think I’ll edit the blog to use this approach instead.

  5. Avatar
    Peter Wood about 14 hours later:

    I was reading about the Law of Big Numbers in ‘An Introduction To General Systems Theory’ on the train this morning. Basically, the bigger the sample the more accurate the estimate.

    Prediction works better with bigger numbers; lots of iterations, technology spikes, tests, milestones, user stories. There will be a point at which the cost of creating these will not outweigh the worth of the new accuracy of estimate. So it’s not simply as many of the above as possible but as many as is useful in improving your predictions.

  6. Avatar
    Johannes Brodwall 1 day later:

    Much clearer. Thank you.

  7. Avatar
    unclebob 2 days later:

    We need to be clear that TDD includes automated unit tests, and automated acceptance tests. Both are written before the code that they specify. Unit tests are written seconds before, and acceptance tests are written hours or days before.

    It is possible to gain the same kinds of benefits using a “test-after” approach. However it is much more difficult. Tests that are written first aren’t as influenced by the implementation as late tests are. Early acceptance tests are specifications of what should be. Late acceptance tests are often specifications of what is.

  8. Avatar
    KPSeal 3 days later:

    What about the iron triangle?

    What you’ve demonstrated is that the total feature effort is less, most likely by virtue of a higher quality (ie, TDD) implementation. This is, to my mind, as a result of having more money spent either through having more skilled developers or more QA team involvement throughout implementation. You’ve not shown that the real effort, that all important bottom line, is less.

    Were I worried solely about a timely delivery I can see that TDD would give me a better chance. But then so might hiring the 6 best developers in the world.

    Based on personal experience I believe you, by the way, but I can imagine the raised eyebrows if I tried to use this as a justification for a change in methodology!

  9. Avatar
    Dean Wampler 3 days later:

    @KPSeal,

    I’ve greatly simplified a complex, human process, of course ;)

    I agree that a comprehensive economic analysis is important. I’ve read that a significant amount of a project’s total life-time return on investment is lost when the schedule slips. So, if TDD (including all the different kinds of tests, as Bob said…) allows you to release a limited subset of the planned features early, your return on investment might be significantly better. Just having the greater schedule predictability, i.e., avoiding the end-of-project chaos, empowers the stakeholders to make good decisions at appropriate times that will benefit everyone.

    Concerning hiring the best people (which we all want to do, of course), Kent Beck said that XP was designed so that average developers can succeed, not just the best developers.

  10. Avatar
    Carfield Yim 3 days later:

    Thanks for the nice article and article, it probably true when the project is just start. However, if the project is near “finish” ( in term of schedule and function implemented ) , do you think is it still valid?

  11. Avatar
    Dean Wampler 4 days later:

    @Carfield,

    I’m not sure what exactly you have in mind about the end of a project. It’s true that near the end, you have less uncertainty about when you’ll finish and what work remains, assuming you have done some testing, automated or manual, so that big surprises are less likely. Also, even in a well-run TDD project, you may encounter some volatility near the end of a release cycle as the team works through final integration issues, last minute story changes, etc. Often, being close to a major event like that clarifies your thinking somewhat.

  12. Avatar
    Designer Bags over 3 years later:

    Thanks for ur nice sharing!!It help a lot..

  13. Avatar
    Backup iPhone SMS over 3 years later:

    Why not study it, we can get more from it.

  14. Avatar
    iPod to iPod over 3 years later:

    Learn more about it, we can do more with this tool.

  15. Avatar
    iPod to iPod over 3 years later:

    If you doesn’t have enough time, save you time to do it.

  16. Avatar
    iPod to iPod over 3 years later:

    If you doesn’t have enough time, save you time to do it.

  17. Avatar
    http://www.blacktowhiteiphone4.com over 3 years later:

    Want to buy the hottest white iphone 4 conversion kit? Highest quality white iphone 4 conversion kit is waiting for your choice! Come and take it home!

  18. Avatar
    Silicone Molding over 3 years later:

    Intertech Machinery Inc. provides the most precise Plastic Injection Mold and Rubber Molds from Taiwan. With applying excellent unscrewing device in molds, Intertech is also very professional for making flip top Cap Molds in the world.

    (2).

  19. Avatar
    iPad PDF Transfer for Mac over 3 years later:

    I really like this essay. Thank you for writing it so seriously. I want to recommend it for my friends strongly. iPad PDF Transfer for Mac can help you transfer ebooks in PDF format from ipad to mac/iTunes.

  20. Avatar
    iPhone Contacts Backup For Mac over 3 years later:

    Thanks for shareing! I agree with you. The artical improve me so much! I will come here frequently. iPhone Contacts Backup For Mac can help you backup iphone contact to mac.

  21. Avatar
    iPhone Contacts Backup For Mac over 3 years later:

    Thanks for shareing! I agree with you. The artical improve me so much! I will come here frequently. iPhone Contacts Backup For Mac can help you backup iphone contact to mac.

  22. Avatar
    Criminal Records over 3 years later:

    So, the actual total feature work required for our project corresponds to the area under the red line, while the planned work corresponds to the area under the black line.

  23. Avatar
    ugg boots over 3 years later:

    This article is truly relevant to my study at this moment, and I am really happy I discovered your website.

  24. Avatar
    nike shoes over 3 years later:

    while the planned work corresponds to the area under the black line.

  25. Avatar
    Sunglass over 3 years later:

    Buy $10 Replica Designer Sunglasses with 3-day FREE SHIPPING

  26. Avatar
    tiffany outlet over 4 years later:

    thank you for nice information.. I am going to bookmark this page. and i will also suggest my friends about this post.

  27. Avatar
    okey oyunu oyna over 4 years later:

    nice information. Thanks

    internette görüntülü olarak okey oyunu oyna, gerçek kisilerle tanis, turnuva heyecanini yasa.

  28. Avatar
    coach purses over 4 years later:

    Mr Coates coach purses is the longest U.S. market popular with one of the most successful leather brand. Mr Coates coach purses store represents the most admirable American fashion innovative style and traditional skills . Mr Coates coach bags have durable quality and exquisite technology, Conspicuous Coach Heels in the female consumers have good reputation. Welcome to our shop Elegant Coach Purses

  29. Avatar
    Jewellery over 4 years later:

    thnnnnnklka yoiu

  30. Avatar
    beats by dr dre headphones over 4 years later:

    I attempted these beats by dr dre studio out in several genres thinking about which i listen to an eclectic mix Beats By Dr Dre. a washing cloth as well as the manual. Do not purchase any beats by dr dre solo purple products inside the internet unless you’re getting from an Authorized internet DealerBeats By Dre Just Solo. We are reliable provide good beats by dr dre pro black by reduced price.

  31. Avatar
    beats by dre store over 4 years later:

    iPhone Contacts Backup For Mac can help you backup iphone contact to mac.high quality headphones new design headphones

  32. Avatar
    Tips For Bowling over 4 years later:

    Everybody has a high point and a lot goes into that: timing, situation, general consciousness.

  33. Avatar
    christian louboutin over 4 years later:

    The professional design make you foot more comfortable. Even more tantalizing,this pattern make your legs look as long as you can,it will make you looked more attractive.Moveover,it has reasonable price.If you are a popular woman,do not miss it.

    Technical details of Christian Louboutin Velours Scrunch Suede Boots Coffee:

    Color: Coffee
Material: Suede
4(100mm) heel
Signature red sole x

    Fashion, delicate, luxurious Christian louboutins shoes on sale, one of its series is Christian Louboutin Tall Boots, is urbanism collocation. This Christian louboutins shoes design makes people new and refreshing. Red soles shoes is personality, your charm will be wonderful performance.

  34. Avatar
    <a href="http://www.airmaxbwltd.com/"> pas cher nike air max Tn </a> over 4 years later:

    Il a ajouté: “La croissance va se poursuivre nike tn , l’innovation des produits de l’entreprise continuera à tout faire … Il devrait être prêt pour les vacances périodes victoire.” Au 31 août pour le deuxième trimestre, le bénéfice net a bondi de Nike de 15% à 645 millions de dollars un an plus tôt était de 559 millions de dollars. Son chiffre d’affaires a bondi de 18% à 60,8 milliards de dollars américains, est de 5,18 milliards de dollars un an plus tôt. En outre, la société possède également des Converse basket tn , Cole Haan, Umbro et Hurley et d’autres marques, la société a déclaré des revenus a augmenté de 35 pour cent dans les marchés émergents dans la Grande Chine ont augmenté de 15%. Nike revenus dans ces secteurs représentaient 23% des recettes totales pas cher nike air max Tn . Nord-américaine des revenus a augmenté de 16%.

  35. Avatar
    beats by dr dre over 4 years later:

    A university studentbeats by dr dre caught by the enemy, the enemy tied him at the poles,just beats solo headphones purple and then asked him: say, where are you? You do not say it electrocuted! Scheap dr.dre beats studio headphones balck/yellowtudents back to the enemy a word, the result was electrocuted, he said: I am TVU.Hot sale beats by dr dre pro headphones

  36. Avatar
    Breguet watches over 4 years later:

    Then stick i

  37. Avatar
    louboutin sales over 5 years later:

    Why you have time for TDD (but may not know it yet…) 36 hoo,good article!!I like the post!44

  38. Avatar
    bladeless fans over 5 years later:

    Why you have time for TDD (but may not know it yet…) 37 good post164

  39. Avatar
    cheap adidas f50 over 5 years later:

    She kept true to the party’s nationalist theme, but, in an Cheap Adidas Adipure attempt to widen her electoral base, vowed to tone down the xenophobic legacy of her father. http://www.cheapadidasf50.com/ CF

  40. Avatar
    Christian Louboutin sneakers over 5 years later:

    The “Frye Paige Tall Boot” is more than one style Christian Louboutin sneakers – in fact there are a number of styles to choose Christian Louboutin prive pump from among the Paige collection that could be Christian Louboutin espadrille wedge classified as falling under the “tall” category. Christian Louboutin peep toe This fact may or may not be known by women or shoppers looking for these boots.

  41. Avatar
    handbags over 5 years later:

    Nowadays, wherever we go and whatever we do, we need a bag accompanying us for different usages as well as purposes?

  42. Avatar
    cosplay deviants over 5 years later:

    http://www.outfitscosplay.com/cosplay-accessories/the-prince-of-tennis-accessories Deluxe The Prince Of Tennis Accessories for Sale.Find your favorite characters and cosplay outfits from all the popular anime and games.

  43. Avatar
    http://maillotsbontique.com over 5 years later:

    J’ai vraiment trouvé votre poste à titre informatif et cela m’a obligé à visiter votre blog maintes et maintes fois. Par souci de pertinence, je tiens à vous remercier pour vos efforts dans la diffusion de l’information académique. Cordialement.maillots de foot

  44. Avatar
    http://maillotsbontique.com over 5 years later:

    quelle belle et en temps opportun être entré sur le lwa! hier j’ai posté la feuille Charles et Ray Eames timbre sur mon blog et découvert ma cachette de feuilles commémoratives et scanné tous l’intention de les poster.maillots pas cher

Comments