Object Mentor Blog: Echoes from the Stone Age

Echoes from the Stone Age

Uncle Bob — Tue, 06 Oct 2009 11:07:29 -0500

The echoes from Joel Spolsky’s Duct Tape blog continue to bounce off the blogosphere and twitterverse. Tim Bray and Peter Seibel have both written responses to Joel, me, and each other.

Here are some stray thoughts…

TDD

Anyone who continues to think that TDD slows you down is living in the stone age. Sorry, that’s just the truth. TDD does not slow you down, it speeds you up.

Look, TDD is not my religion, it is one of my disciplines. It’s like dual entry bookkeeping for accountants, or sterile procedure for surgeons. Professionals adopt such disciplines because they understand the theory behind them, and have directly experienced the benefits of using them.

I have experienced the tremendous benefit that TDD has had in my work, and I have observed it in others. I have seen and experienced the way that TDD helps programmers conceive their designs. I have seen and experienced the way it documents their decisions. I have seen and experienced the decouplings imposed by the tests, and I have seen and experienced the fearlessness with which TDDers can change and clean their code.

To be fair, I don’t think TDD is always appropriate. There are situations when I break the discipline and write code before tests. I’ll write about these situations in another blog. However, these situations are few and far between. In general, for me and many others, TDD is a way to go fast, well, and sure.

The upshot of all this is simple. TDD is a professional discipline. TDD works. TDD makes you faster. TDD is not going away. And anyone who has not really tried it, and yet claims that it would slow them down, is simply being willfully ignorant. I don’t care if your name is Don Knuth, Jamie Zawinski, Peter Seibel, or Peter Pan. Give it a real try, and then you have the right to comment.

Let me put this another way. And now I’m talking directly to those who make the claim that TDD would slow them down. Are you really such a good programmer that you don’t need to thoroughly check your work? Can you conceive of a better way to check your work than to express your intent in terms of an executable test? And can you think of a better way to ensure that you can write that test other than to write it first?

If you can, then I want to hear all about it. but I don’t want to hear that you write a few unit tests after the fact. I don’t want to hear that you manually check your code. I don’t want to hear that you do design and therefore don’t need to write tests. Those are all stone-age concepts. I know. I’ve been there.

So there. <grin>

The Design Pattern Religion

Tim Bray said:

My experience suggests that there are few surer ways to doom a big software project than via the Design Patterns religion.

He’s right of course. The Design Patterns religion is a foul bird that ravages teams and cuts down young projects in their prime. But let’s be clear about what that religion is. The Design Patterns religion is the ardent belief that the use of design patterns is good.

Here’s a clue. Design Patterns aren’t good. They also aren’t bad. They just are. Given a particular software design situation, there may be a pattern that fits and is beneficial. There may also be patterns that would be detrimental. It’s quite possible that none of the currently documented patterns are appropriate and that you should close the book and just solve the problem.

Here’s another clue. You don’t use patterns. You don’t apply patterns. Patterns just are. If a particular pattern is appropriate to solve a given problem, then it will be obvious. Indeed it is often so obvious that you don’t realize that the pattern is in place until you are done. You look back at your code and realize: “Oh, that’s a Decorator!”.

So am I saying that Design Patterns are useless?

NO! I want you to read the patterns books. I want you to know those patterns inside and out. If I point at you and say “Visitor” I want you at the board drawing all the different variants of the pattern without hesitation. I want you to get all the names and roles right. I want you to know patterns.

But I don’t want you to use patterns. I don’t want you to believe in patterns. I don’t want you to make patterns into a religion. Rather I want you to be able to recognize them when they appear, and to regularize them in your code so that others can recognize them too.

Design Patterns have a huge benefit. They have names. If you are reading code, and you see the word “Composite”, and if the author took care to regularize the code to the accepted names and roles of the “Composite” pattern, then you will know what that part of the code is doing instantly. And that is powerful!

Minimizing Concurrency.

In my first Duct Tape blog I made the statement:

I found myself annoyed at Joel’s notion that most programmers aren’t smart enough to use templates, design patterns, multi-threading, COM, etc. I don’t think that’s the case. I think that any programmer that’s not smart enough to use tools like that is probably not smart enough to be a programmer period.

Tim responds with:

...multi-threading is part of the problem, not part of the solution; that essentially no application programmer understands threads well enough to avoid deadlocks and races and horrible non-repeatable bugs. And that COM was one of the most colossal piles of crap my profession ever foisted on itself.

Is concurrency really part of the problem? Yes! Concurrency is a really big part of the problem. Indeed, the first rule of concurrency is: DON’T. The second rule is: REALLY, DON’T.

The problem is that some times you have no choice. And in those situations, where you absolutely must use concurrency, you should know it inside and out!

I completely and utterly reject the notion that ignorance is the best defense. I reject that lack of skill can ever be an advantage. So I want you to know concurrency. I want to shout “Dining Philosophers” and have you run to the board without hesitation and show me all the different solutions. If I holler “Deadlock”, I want you to quickly identify the causes and solutions.

Here’s a clue. If you want to avoid using something, know that something cold.

Sudoku

At the end of his blog, Peter jumps on the pile of bodies already crushing Ron Jeffries regarding the Sudoku problem from July of 2006.

I find the pile-up disturbing. Ron had the courage to fail in public. Indeed he announced up front that he might “crash and burn”. And yet he got lambasted for it by people who hid behind someone else’s work. The responses to Ron’s tutorial blogs were completely unfair because the authors of those blogs had everything worked out for them by Dr. Peter Norvig before they published their screeds. They were comparing apples to oranges because their responses were about the solution whereas Ron’s blogs were about the process.

Which one of us has not gone down a rat-hole when hunting for a solution to a complex problem? Let that person write the first blog. Everyone else ought to be a bit more humble.

Do the people on the pile think that Ron is unable to solve the Sudoku problem? (Some have said as much.) Then they don’t know Ron very well. Ron could code them all under the table with one hand tied behind his back.

Personal issues aside, I find the discussion fascinating in it’s own right. Ron had attempted to solve the Sudoku problem by gaining insight into that problem through the process of coding intermediate solutions. This is a common enough TDD approach. Indeed, the Bowling Game and the Prime Factors Kata are both examples where this approach can work reasonably well.

This approach follows the advice of no less than Grady Booch who (quoting Heinlein) said: “when faced with a problem you do not understand, do any part of it you do understand, then look at it again.“

Ron was attempting to use TDD to probe into the problem to see if he could gain any insight. This technique often bears fruit. Sometimes it does not.

Here is a classic example. Imagine you were going to write a sort algorithm test first:

Test 1: Sort an empty array. Solution: Return the input array.

Test 2: Sort an array with one element. Solution: Return the input array.

Test 3: Sort an array with two elements. Solution: Compare the two elements and swap if out of order. Return the result.

Test 4: Sort an array with three elements. Solution: Compare the first two and swap if out of order. Compare the second two and swap if out of order. Compare the first two again and swap if out of order. Return the result.

Test 5: Sort an array with four elements. Solution: Put the compare and swap operations into a nested loop. Return the result.

The end result is a bubble sort. The algorithm virtually self assembles. If you had never heard of a bubble sort before, this simple set of tests would have driven you to implement it naturally.

Problems like Bowling, Prime Factors, and Bubble Sort hold out the interesting promise that TDD may be a way to derive algorithmms from first principles!

On the other hand, what set of tests would drive you to implement a QuickSort? There are none that I know of. QuickSort and Sudoku may require a serious amount of introspection and concentrated thought before the solution is apparent. They may belong to a class of algorithms that do not self-assemble like Bowling, Prime Factors, and Bubble Sort.

This blog by Kurt Christensen provides all the links to the various Sudoku articles, and sums it up this way.

TDD may not be the best tool for inventing new algorithms, it may very well be the best tool for applying those algorithms to the problem at hand.

Actually I think TDD is a good way to find out if an algorithm will self-assemble or not. It usually doesn’t take a lot of time to figure out which it’s going to be.

"Echoes from the Stone Age" by joseph

Tue, 09 Mar 2010 02:56:39 -0600

Thanks for sharing this great article! That is very interesting Smile I love reading and I am always searching for informative information like this.

"Echoes from the Stone Age" by Easy Date

Sun, 07 Mar 2010 06:11:55 -0600

I wish things were that easy!

"Echoes from the Stone Age" by Monoket

Wed, 17 Feb 2010 07:44:10 -0600

Vote for TDD’s big future!

"Echoes from the Stone Age" by Astrologer

Mon, 15 Feb 2010 14:02:38 -0600

Bookmarked this site and emailed it to a few friends, your post was that great, keep it up

"Echoes from the Stone Age" by digital camera brisbane

Wed, 27 Jan 2010 01:51:30 -0600

I’m impressed, you know what you’re talking about

"Echoes from the Stone Age" by lami

Mon, 18 Jan 2010 02:48:18 -0600

http://www.sextoy7.com

shop sex!

"Echoes from the Stone Age" by ??????????????

Sun, 03 Jan 2010 09:59:32 -0600

dumys

"Echoes from the Stone Age" by Adrian

Thu, 05 Nov 2009 02:23:30 -0600

My remarks regarding:

“How do you respond to this Microsoft study saying that TDD results in fewer bugs, but longer development time”

TDD = longer development time (~40%) = fewer defects = less time for bug fixing = The total time spent in developing an US is smaller because what we lose in development we gain by spending less time in bug fixing.

And think about your reputation when the client sees an “null reference set to an instance of an object” exception.

"Echoes from the Stone Age" by Adrian

Thu, 05 Nov 2009 02:17:15 -0600

I don;t understand why we speak about developing speed when we speak about TDD.

In my opinion, the main goal of TDD is not “increase productivity”. I never think at TDD as a productivity booster. For me TDD is a guarantee offered to myself and to the client that my work is well done. Is a way to protect my reputation.

So, in my opinion, speeding or slowing the development process when we speak about TDD is meaningless.

"Echoes from the Stone Age" by Mohammad Azam

Wed, 04 Nov 2009 13:17:06 -0600

TDD is awesome and there is no doubt about it. It gives you confidence to make changes and in the long run it proves to be very useful.

But then there are some straight forward projects like “Building Discussion Boards”, “Building Knowledge based WebSite” to host articles and stuff that does not require TDD approach.

The reason that they don’t require TDD is that there is no complexity in the code. Everything is so straight forward.

Bottom line is that the domain will dictate if TDD is required or not. For complex domain TDD should be mandatory but for simple projects it MIGHT be an overkill.

Thanks, Azam

"Echoes from the Stone Age" by maddin

Thu, 29 Oct 2009 04:28:20 -0500

if you compare TDD with the hygiene practices a surgeon does, i think its not entirely fair. If a surgeon does not wash his hands before a surgery, the chance is almost 100% the patient will suffer. However, i have written lots of code and it works till today and i did not do TDD as it did not exist in its current form. So, absence of TDD is not life threatening, you can succeed without it.

Bottom Line: TDD is a tool that i use when appropriate. There are cases when its not appropriate. I expect an real professional to look at the problem and pick the right tools to solve the problem most effectively. And i don’t agree that TDD makes you a better programmer. Its the other way round, if you are a good programmer you adopt techniques like TDD. If you are a lame coder no tool nor technique will save you.

just my 2 cent.

"Echoes from the Stone Age" by Norbert

Thu, 29 Oct 2009 04:06:12 -0500

I read Spolkys article and thought, well, he has some points. I’d never thought what kind of blogstorm he’d raise with this.

Folks, please, do not be religious about this. If TDD helps you, great! Keep it up. Personally, I try to work like Peter Norvig and Knuth—though I know I’ll never play at their level. To me, TDD (yes, I tried) seems kinda cool if your code can be handled like a black box with nice interfaces that are not crossing system boundaries or user interfaces and if you are not trying develop an algorithm. You can only solve problems that you really do understand.

For all those TDD evangelists out there who cannot stop saving the poor sheep out there (like myself), why don’t you guys gather the evidence that TDD is really, truly producing more stable and better code? Your claims should be measurable. Repeating them does not help. You should have some statistics that count applications crashes and whatnot in order to solidify your claims. Remember: People like me do not try to convince you that TDD is bad, but that we simply do not buy into this as religiously as you do. And we do not call you names, ok?

Until then, pretty please, stop preaching and state the facts and your conclusions. You never know, you actually might win me over, but, sorry to say, the tone in these discussions is revolting.

"Echoes from the Stone Age" by Antony G

Wed, 14 Oct 2009 04:59:49 -0500

Wow – feel the love in these comments.

The simplest thing I could suggest to people detracting the effectiveness of TDD is to simply try it. Then try to live without it.

I can only speak out of personal experience, but to me this is the clincher.

"Echoes from the Stone Age" by Christopher Svec

Sun, 11 Oct 2009 23:05:25 -0500

How do you respond to this Microsoft study saying that TDD results in fewer bugs, but longer development time:

http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf

They say they saw 40-90% lower defect density with TDD, but 15-35% increase in initial development time.

(though the development time number was merely an estimate, as opposed to a measured number)

"Echoes from the Stone Age" by Jake

Fri, 09 Oct 2009 17:12:49 -0500

“I think that any programmer that’s not smart enough to use tools like that is probably not smart enough to be a programmer period.”

I totally agree with you on that. However, we can’t ignore reality. How many programmers are out there? and how many of them are really good at those tools? So if we kick them out of the industry just because they aren’t smart enough to be a prammer, this industry will collapse…

"Echoes from the Stone Age" by Cedric

Fri, 09 Oct 2009 15:15:56 -0500

James, you seem to assume that there are only two states for code: “written with TDD” and “with no tests”.

There is a third state, “written with tests last”, and my claim is simply that millions of lines of code were written this way and that this code is working just fine.

Mocking developers and companies that write code this way and calling them “unprofessional” is precisely what makes people think that there is a disconnect between the TDD crowd and the rest of the world.

"Echoes from the Stone Age" by Cedric

Fri, 09 Oct 2009 14:58:35 -0500

By the way, thanks for rectifying the tone in your second post.

The constant use of deprecating adjectives for people who don’t use TDD (unprofessional, cowboy coding, etc…) doesn’t really help when you are trying to convince.

"Echoes from the Stone Age" by Cedric

Fri, 09 Oct 2009 14:56:00 -0500

James, you seem to assume that there are only two states for code: “written with TDD” and “with no tests”.

There is a third state, “written with tests last”, and my claim is simply that millions of lines of code were written this way and that this code is working just fine.

"Echoes from the Stone Age" by James Carr

Fri, 09 Oct 2009 13:51:29 -0500

I might have came off in kind of a condescending tone. My point is “my” experience has taught me to value TDD over DDT. Maybe this is the opposite for you, but I know that it works well for me and the code I have written without doing TDD is always a bit of a mess.

"Echoes from the Stone Age" by James Carr

Fri, 09 Oct 2009 12:02:28 -0500

Cedric: What!? How can you prove to me your code is even needed if you haven’t proved the need for it to exist with a well defined example? How can you refactor without having to worry about breaking some unapparent piece of functionality that will cost a company millions of dollars?

I work both on new projects and legacy projects all the time, some done with TDD and some that were built by hacking away with no tests. I find the latter to be a lot more difficult to work with.

Maybe you just don’t have the industry experience that has taught me that code written test first is always a lot more cleaner and easier to work with than code that is not.

Or maybe I should just listen to you and see the beautiful light of cowboy coding? :)

"Echoes from the Stone Age" by Cedric

Fri, 09 Oct 2009 11:10:32 -0500

James: thousands of companies are producing very good software every day without using TDD (and sometimes without even using automated testing).

Calling them unprofessional is not just disrespectful, it makes you sound like a zealot…

And I understand that you and Rob live off the TDD industry, and as such, you are surrounded with companies that do use TDD, but hopefully you realize that you are seeing the inside of a bubble.

TDD is still far from being mainstream and so far, remains confined to conferences, books and a very small subset of companies.

"Echoes from the Stone Age" by James Carr

Fri, 09 Oct 2009 08:17:48 -0500

Good response Uncle Bob. How people can continue to think that TDD is some silly fad continues to amaze me. Anyone with an ounce of experience would know that TDD helps produce clean, professional code that has been proven that it needs to exist thanks to well defined examples.

It isn’t just silly, but unprofessional imho not to do TDD. Deal with tons of legacy code packed with 2000 line classes and no tests around it and please someone try to tell me that TDD is a bad thing.

"Echoes from the Stone Age" by Stone Age Programmer

Fri, 09 Oct 2009 07:37:11 -0500

Re 42’s comments: Yes, this sounds like a discussion between evolutionists and the creationist, with the difference that the TDD programmers are playing the role of crearionists – asking everybody to “faithfully” accept their claims that TDD is indeed better. Evolutionists (stone age programmers) want to see evidence.

If you knew Uncle Bob's political and religious views, it would be clear to you that he falls solidly on the side of the conservatives.

"Echoes from the Stone Age" by Paul Thor

Thu, 08 Oct 2009 15:36:02 -0500

its to easy to be negative! When someone is trying to do something positive.

"Echoes from the Stone Age" by 42

Thu, 08 Oct 2009 04:35:58 -0500

Reading all these comments, it reminds me of an evolutionist trying to explain the concepts of Darwin to creationists. TDD is there to protect us! If you are lucky enough to have a decent spec, that will NEVER change, code without it! But the rest of us have to deal with horrible, ever changing specs. Agile and tdd is a worm digging through the path of least resistance (see YAGNI). Non-TDD coders use a bulldozer, leaves gaping holes everywhere and someone else ends up rewriting it all.