"Quality Doesn't Matter That Much" -- Jeff and Joel 169
I was riding my exercise bike, listening to Stack Overflow #38 when I heard Jeff Atwood and Joel Spolsky say “Quality just doesn’t matter that much.” I nearly fell off my bike.
I mean this is Joel Spolsky right? His buddy Jeff makes this incredibly dumb statement and Joel grunts his approval. WTF?
This is the kind of idiotic statement I would expect to hear from junior hackers, or people who haven’t written very much code. You simply cannot deliver software products for more than a few years and think that quality doesn’t matter very much.
Now I need to be fair. I don’t think Jeff meant to say this exactly the way it came out. Just before he said it, he said that lately he had been getting less and less concerned with code hygiene. So possibly he was trying to say that code quality just doesn’t matter that much.
But that doesn’t make me feel any better; because one thing you learn after four decades of coding is that code quality matters one hell of a lot. So to respond to Jeff I’ll simply fall back on the old saw “Faithful in little, faithful in much.” In other words, if you can’t keep your code clean, no way can you keep your systems clean.
So I think the fairest thing I can say about this is that Jeff was probably saying things he didn’t really mean because he felt his mentor, Joel, was approving.
To make matters worse, all of this was preceded by a truncated rant by Joel about TDD. Joel and Jeff were agreeing that unit testing was really not very important and that testing is often a waste of time. (I’m starting to wonder if anyone should be using any of their products.)
Again, to be fair I think Joel was playing devil’s advocate a bit. He was saying that it’s a waste to write tests for every last little thing. This is certainly true. However the discipline of TDD does not recommend the position that Joel was resisting. TDD specifically tells us not to bother testing getters and setters and itty bitty methods that simply can’t fail. So Joel clearly didn’t understand what he was arguing against.
Joel went on to say that 100% code coverage by unit tests is a waste. I agree. I think that 100% code coverage is unachievable. However, I think that it is not a waste to drive that number up as high as practicable. For example, I keep FitNesse at about 90%. This requires very little effort on my part and pays back enormous dividends. Again, I think Joel was arguing without reasoning. Yes, 100% is silly; but 90%, or even 95%, is not.
Joel went on to complain about the fragile test problem which is that as you make changes to your code, some of your tests naturally fail. Joel quoted the statistic 10%. That’s laughable, and indicates that Joel has never tried to to gain anything beyond a surface understanding of TDD (a failing typical of business wonks). If you design your tests such that 10% of them fail due to a single point change, then you need a new career. Test design, like any other part of software design, is about decoupling.
I’m not religious about TDD. I consider it a discipline worth following. I don’t write all my tests first. Some small number of them are simply more convenient to write after the code. There is even some code I don’t write tests for at all, because it’s just not worth it. But those are exceptions to the rule. The vast majority of the code I write, I write test first. (And no, Joel, this isn’t a waste of my time.)
At some point in the discussion, Joel took on the SOLID principles. He said that these principles are so extremely bureaucratic that they must have come from the mind of someone who doesn’t write a lot of code. He also used the word “idiotic”.
Again, this is a symptom of someone who has focussed on business and lost his technical chops. Euclid once told a king, “There is no royal road to Geometry.” There is no executive summary that explains the SOLID principles well enough that you can comment on them critically.
I won’t go into his exegesis of the principles because it’s, frankly, embarrassing. I’ll simply say that I’ve been a coder for 40 years. I’ve written many systems taking them from inception to delivery and into maintenance. I have shipped one hell of a lot of product. And if anyone wants to know how much code I’ve written over the last six months, they are free to inspect my github repository.
Joel said that the SOLID principle aren’t “agile”. (sigh). Everybody and his uncle thinks he knows what the term “agile” means. But I’m the guy who called the meeting where the name “agile” was picked. I’ve been writing about Agile development since the term Agile development was created. I think I know what is Agile and what isn’t. And I think I have the authority to override Joel on this one. Joel, the SOLID principles are agile.
I think that maintaining your technical chops is a full time job. For that reason I have avoided becoming a business wonk. I hire people to do that for me so I can keep my technical skills as sharp as possible and remain relevant to my profession. I don’t believe I can offer technical advice unless I am living that technical advice.
You see, I think quality matters. I think the quality of my code matters, even at the smallest scale. I think the quality of my systems matters. I think the quality of my tests matters. And… I think the quality of my advice matters.

I’d been following your tweets, dying to hear the story. I’m half tempted to listen to the podcast but frankly, it doesn’t sound like it would be good for my blood pressure.
Not an HOUR ago I had another Mike Feathers “When would I possibly have found that?” moment during a TDD session.
et tu Spolsky?
sigh
“I think that maintaining your technical chops is a full time job.”
Thanks for making that statement. Technical mastery requires constant practice. Masters need to be living in the trenches where business and technology clash and/or mesh every day.
“I think that maintaining your technical chops is a full time job.”
Thanks for making that statement. Technical mastery requires constant practice. Masters need to be living in the trenches where business and technology clash and/or mesh every day.
Glad to read your response!
The podcast was rather hilarious (if depressing due to the complete lack of understanding.) I can only hope that Mr. Spolsky made his statements out of an urge to stir the debate – otherwise he really, really need to check his facts…
I was listening to the podcast during my way to work when they started talking about TDD and quality, and I the same WTF?? feeling. Honestly, I started laughing in the bus..
It’s great to know I was not alone. Thanks for the response, Bob.
Ooo ooo blog fight! Joel and Jeff both mentioned during the podcast that they knew they would bring the ire of many, so not entirely unexpected.
Rather than take this as a personal attack, or an affront to SOLID principles (although I’d say there’s justification for both) you could take this as an opportunity to realize that the marketing of the principles is lacking, and could do with some improvement. Hell, record a quick rebuttal and submit it to Joel for possible inclusion in the next podcast.
Dear Uncle Bob,
Thank you for setting the record straight. Statements like, “Quality just doesn’t matter that much” shouldn’t go unaddressed.
Personally, SOLID has transformed my career. SOLID has made my career as a software developer more enjoyable. I speak from experience when I say, maintaining a system that wasn’t written with SOLID in mind is the opposite of enjoyable.
I may not always be SOLID compliant but that’s why TDD is so important. TDD is my truth serum. TDD ensures I move towards SOLID and not away from SOLID.
It frightens me to think that new developers are listening to Jeff and Joel and not listening Uncle Bob Martin.
- Emilio
I’ve listened to all the StackOverflow podcasts, and I’ve heard them make comments that made me think that they really didn’t understand TDD or Agile. In podcast #38 they overtly revealed their ignorance.
I now firmly believe they have never tried TDD (or BDD). I have never met anyone that has honestly given it a fair trail (20 days or so) and not become addicted for life.
While I was shocked at Joel’s rejection of SOLID principles and can’t agree with him there, I think I understand his point. Some zealots are giving ahering strictly to dogmatic methodologies (not necesarily SOLID) a higher priority than delivering software and satisfying customers. If your code is incredibly clean, but doesn’t meet the user’s needs, it is useless. Of course, ideally, principles and methodolgies are used pragmatically in delivering software that does meet customer’s needs but I have seen developers forget the objective of the system they were working while obsessing on the details of the implementation.
I agree completely with the thoughts, and I wanted to emphasize something you said:
“I hire people to do that for me so I can keep my technical skills as sharp as possible”
I think it is very important for people to realize that there are too many things out there to try to do them all. Find what you want to get good at, then pay someone to do the other things. Same reason that I pay someone to change the oil in my car.
In this landscape, there appears to be a many who believe that they can understand and make valid judgments about principles and practices out of superficial aquaintance.
Although this is about a different set of practices, I find it revealing that one recent post on a blog about Scrum ( http://agilesoftwaredevelopment.com/blog/janusz-gorycki/dont-use-scrum ) says you just need to “familiarize yourself” with its principles and then you can pick and choose. Well, you need to do more than that. You need to actually learn the skills that are necessary to practice them, and then you can try them out. Otherwise, you’re just dabbbling and you have no basis for an independent opinion. You may think you can form an opinion by observing others who are following the practices, but you really can’t know whether they are doing it correctly.
He (Spolsky) also backs off on his criticism of SOLID and admits he didn’t know what he was talking about in Stackover #39.
Fantastic, I agree completely.
Thanks for this, Bob. Great stuff.
I think you’re right that maintaining your technical chops is a full-time job. At least. And I think that’s so because it’s a Red Queen problem.
In my view, the basic limit on technical innovation isn’t our collective ability to come up with new things. It’s our ability to absorb them. That means that the field is changing about as fast as most developers can handle things. For most people, taking time off means you slip behind, your instincts and tricks becoming less and less relevant.
And it’s even worse than that. We aren’t a unified profession; each tribe (e.g., .NET vs Java vs Rails) is moving ahead as fast as they can, but in somewhat different directions. So consider a manager, exec, or pundit with experience five years stale. If they convince themselves that they understand what a mixed set of teams is really doing today, they’re fooling themselves twice over.
Thanks for this, Bob. Great stuff.
I think you’re right that maintaining your technical chops is a full-time job. At least. And I think that’s so because it’s a Red Queen problem.
In my view, the basic limit on technical innovation isn’t our collective ability to come up with new things. It’s our ability to absorb them. That means that the field is changing about as fast as most developers can handle things. For most people, taking time off means you slip behind, your instincts and tricks becoming less and less relevant.
And it’s even worse than that. We aren’t a unified profession; each tribe (e.g., .NET vs Java vs Rails) is moving ahead as fast as they can, but in somewhat different directions. So consider a manager, exec, or pundit with experience five years stale. If they convince themselves that they understand what a mixed set of teams is really doing today, they’re fooling themselves twice over.
Writing quality code demonstrates more than just discipline. It displays your integrity, compassion for the team, and commitment to the project as well. Taking the time to produce quality code is something that we should all do every time we put our fingers to the keyboard. Our teammates will thank us for it, we’ll feel better when we finish, and even business wonks might thank us in the end.
Sounds like a Gerald Ratner moment. I hope not for Joel’s sake.
Always glad to see some Jeff bashing. ;)
Always glad to see some Jeff bashing. ;)
So wait, you’re saying that someone who sells bug tracking software is encouraging programmers to slack off on quality? Sounds like a clever marketing campaign to me.
Burn them at the stake!
Ironically, the two products they are famous for are best in class. What have you guys done?
Uncle Bob -
Reading your post, I didn’t even recognize who you were arguing with here. I don’t think your memory of the podcast exactly corresponds to what you’re objecting to, so I transcribed the whole 10 minutes where we had that conversation:
http://www.joelonsoftware.com/items/2009/01/31.html
I’m convinced that you’re misrepresenting Jeff’s views. He said pretty clearly, “Anything that improves quality is good.”
I think you’re misrepresenting my views, too. Check the transcript (I edited it a little bit for clarity, but not for meaning) and I think you’ll find that we’re not really disagreeing about that much.
“But that doesn’t make me feel any better; because one thing you learn after four decades of coding is that code quality matters one hell of a lot.”
This is true, but having a business to work at matters even more. They are not arguing against code quality as much as TDD, which they say helps mediocre programmers write better code. Writing tests that intentionally break, and then writing code that makes the tests pass is one step too many for some people, especially if the code they are writing already would pass muster (I would argue that most good developers already write code good enough).
“For example, I keep FitNesse at about 90%. This requires very little effort on my part and pays back enormous dividends.”
What dividends? What metrics are you using? Are you that bad of a software developer that 90% of the code you write is not to be trusted unless MORE code is written around the original code?
“Again, this is a symptom of someone who has focussed on business and lost his technical chops.”
Baloney. Just because a another developer doesn’t agree with your style of development doesn’t make their technical chops less than yours. These guys have written software the a LARGE number of people have used and loved.
“You see, I think quality matters.”
Who is really arguing this? I think they see less value in TDD than you do. TDD is A* way of getting quality better—it is not the *only or even best way.
“But I’m the guy who called the meeting where the name “agile” was picked.”
Appeal to Authority is a fallacy. I’ve heard agile thrown around so often that it has been disabused of any “right” idea that some guy in a room came up with. Like most words and ideas it has grown up with a life of its own.
@Josh – Ironically, those designations mean nothing in software as there is no governing body to decide what “best in class” means and who and how they are compared. How long have you been doing this?
Everyone else. Testing is important. Automated testing and TDD are very, very important. No unit tests, functional tests or integration tests? Just stupid. Not testing at all? I don’t even know what that means.
Jeff and Joel both write well and can tell funny stories. That’s it. They are entertainers in the field of software. That’s the only reason to ever read or listen to them, to be entertained. For good advice, look elsewhere.
Its hard to imagine that anyone in today’s day-and-age would argue against the value of unit tests in anything but the most trivial of systems; whether we are talking TDD or test-after is another value debate, but we cannot seriously be asking ‘is testing valuable’, can we?
Not writing tests AT ALL is like saying “I can read the code and deduce its behavior better than the compiler can” and although I’ve met a number of very good software engineers in my life, NONE of them read code as well as the compiler.
Re: Jeff and Joel having running software without tests (success being the only useful metric considered here—BTW a tremendous fallacy, but a debate for another thread), I think we should consider that both of their ‘successes’ are in fact quite simple software systems that may in fact lack the complexity of behavior that would benefit significantly from unit tests. Both appear to largely be forms-over-data apps with little business rules/behavior worth testing (at least at first glance to me).
“People that say things like this have just never written a heck of a lot of code. Because what they’re doing is spending an enormous amount of time writing a lot of extra code, a lot of verbiage, a lot of files, and a million little classes that don’t do anything and thousands of little interface classes and a lot of robustness to make each of these classes individually armed to go out into the world alone and do things, and you’re not going to need it.”
Funny, this comment from the transcript smacks to me of someone who has written a lot of unnecessary code. I’ve done a lot of TDD and found that it enables the ability to get rid of all that excess code that programming geniuses like Spolsky slop into their code. Spolsky apparently doesn’t understand incremental refactoring or TDD.
I learned about the SOLID principles around 1998-1999 by reading your articles, and then applying that knowledge. The effect on the quality of my code was immediate, literally. When I heard this podcast I was sickened and embarrassed.
The remarks made with regard to TDD, Agile, and SOLID have to be the most ridiculous things I have heard in recent memory. I just hope there weren’t any impressionable newcomers to our profession who took anything they said to heart.
Joel doesn’t seem to fully understand TDD.
In his example of testing the JPEG compression rate for CoPilot he cites that the setup code for the test would be large, and take more time than it was worth.
If he fully understood TDD he would know that large setup is a code smell of the God object (an object that tries to do too much). Refactoring the objects responsibilities into other classes would make it more testable.
Best post in awhile. Definitely gonna skip #38. Thanks for sharing your thoughts.
Keep ‘em in coming.
~L
This blog post reads like a rant. I question, now, how much any affirmer in the comments honestly values software over software-the-way-they’ve-been-told-is-right.
A million systems without unit tests function gloriously. Take your head out of the sand and look at code that’s stood the test of time. Statistics are on Joel’s side of the argument.
James about 4 hours later:
Keeping your code clean is a good way to help you keep a business to work at.
Jeff and Joel are arguing from a position if ignorance. They don’t know what TDD is. This is clearly evident from their naive discussion of the topic.
I would argue that few programmers write code well enough to work the first time. If there is a chance that your code has a bug, then you need to test it somehow. Testing it manually (again and again at every release) is horrifically inefficient. Writing your tests up front is simple, and has a profound beneficial effect.
I push a button and 35 seconds later I have executed 90% of the code in the entire system, and seen it work. This means I have a very simple way to ensure that any change I make has not broken anything. If you can’t work out the benefits that obtain from that, I’m sorry.
I’ve written lots of very profitable software too. So what? The point is that their arguments are non-sequitur because they don’t know what they are talking about. They have taken a position based, primarily, on ignorance. That they are ignorant of TDD is evident on the surface of their arguments.
So, it’s not that they disagree with TDD, they don’t know TDD and have taken a public position based on that ignorance.
Jeff Atwood, and by his silence, apparently Joel. Listen to the podcast, or read the transcript.
I have no problem when someone presents a reasoned critique of TDD. There’s plenty to criticize. TDD doesn’t solve world hunger. Unfortunately, Joel and Jeff presented no such reasoned critique.
Not in this case, since I am one of the authors.
And that is unfortunately due to unreasoned and ignorant arguments made by people like Jeff and Joel.
joran about 6 hours later:
It was a rant. Joel called his own words in the podcast “a rant”. One good rant deserves another.
I doubt it’s a million. There have, however, been many good systems written without unit tests. Many patients were before sterile procedure was invented too. So what? There are no statistics on Joel’s side, because Joel doesn’t have a side that’s based on any real experience with TDD.
You’re naive. This industry runs on bad code, bad code makes a lot more money than good code. Bad code means quicker to market, bad code means lock-in, bad code means more billable hours. Sure it hurts down the road, but by then your “good code” competitors have bankrupted themselves.
Quality doesn’t matter. I don’t like it, but that’s how it is.
This discussion spawned a blog post for me:
http://cwash.org/2009/01/31/in-response-to-stackoverflow-38quality-doesnt-matter-that-much-jeff-and-joel/
“I push a button and 35 seconds later I have executed 90% of the code in the entire system, and seen it work.”
You have seen it work for test cases that you yourself have come up with. Most of the problems I have seem come from what a programmer HASN’T been able to forsee. I agree that putting tests in for parts of code that see a lot of change or have certain kinds of logic makes sense, but putting a test around everything strikes me as naive and a waste of resources. I’ve done TDD on a few projects now, usually at the behest of some junior Java programmer who had the religion and talked the rest of us into it. Taking the full time and resources spent on the tests and comparing it to the very few times where any of them have caught problems, and I just don’t see the value. I’d rather have a team of guys that use their brains and write tests where they are obviously needed instead of writing tests for every little thing.
Uncle Bob, I’m confident that you’re misrepresenting what Jeff and I said. I put a transcript of our conversation up on Joel on Software and you can judge for yourself. We’d love to have you on the Stack Overflow podcast.
Kelvin-
I do agree that this is the prevalent trend, such as the gaming industry where throw-away code is written because RTM date is a key factor.
There have been many a research paper that attempts to gauge various measures of code quality directly to cost-effectiveness.
Kelvin there are many places where code quality will literally make or break the company.
Most companies have no iota of a clue how much “bad software” is costing them.
My only real question is who or what is validating your tests? If you write code that needs that level of testing then you probably shouldn’t be writing the tests. [for the record, I’m not claiming that my code doesn’t need that level of testing]
The other obvious question is “Do you actually understand the problem you are trying to solve?” – in other words, how many “bugs” have come about due to poor understanding vs. poor code?
“If you design your tests such that 10% of them fail due to a single point change, then you need a new career”
This statement destroys your credibility for me. It’s too extreme to mark you out as anything other than an ideologue.
People who’ve made a successful career building software and getting paid for it are by definition not in need of a new career; to claim otherwise, you’re trying to argue against the existence of facts.
“If you design your tests such that 10% of them fail due to a single point change, then you need a new career”
This statement destroys your credibility for me. It’s too extreme to mark you out as anything other than an ideologue.
People who’ve made a successful career building software and getting paid for it are by definition not in need of a new career; to claim otherwise, you’re trying to argue against the existence of facts.
(I don’t know if your tests have caught it, but the user experience for posting comments from within Firefox 3 is not good – it’s very susceptible to duplicates!)
In my experience it is true that, “Quality just doesn’t matter that much.” After 9 years of supporting a government agency, I look back to the tremendous increase in productivity my websites have provided the workers that formerly had to exchange spreadsheets to do their job. I have always provided updated capabilities in response to their requirements in a timely manner. The quality of my code is poor but it serves the purpose.
If you’d ever used Fogbugz, I don’t think you’d be surprised.
It’s horrible. I think it took them something like 2 years to fix the PHP quote escaping problem.
Guess they were too busy writing cross-language compilers or something.
I simply cannot understand why people assume that Joel Spolsky is a beacon of quality and reasonableness. With the bizarro cross-language-compiling stuff they do, the horribleness of his main product, and the inane and incorrect things he writes about interface design… it all adds up to a message that says “DON’T TRUST!” to me.
So he wrote some apparently great articles on software pricing. That’s nice. They clearly make money. But it doesn’t make him somebody I’d listen to about other topics.
This article reminds in some ways of a typical Slashdot linux rant. There is a lot of appeal to authority, holier-than-thou “you-just-don’t-get-it-’s-your-problem”, deep-seated anger and defensiveness but ultimately the casual reader comes a way thinking “meh, so what?”.
Actually you know what it reminds me more of – the arguments that guitar players have – who is better, Joe Satriani or Kurt Cobain? Joe Satriani clearly is technically superior, but Kurt Cobain was the one who revolutionized music. Satriani didn’t.
Stackoverflow is probably one of the best websites for regular-programmers ever. Possibly the most revolutionary tool to come along in years, at least when you take into account its scope and reach. Plus it does so many things better than anyone else out there.
On planet earth, people with creative ideas and creative energy set the standard, change the way we do things, make piles of money, get the glory, get the women. People with only technical chops work 9-5 to someone else’s vision.
If TDD and SOLID is too obscure that regular-intelligent people don’t get it, then the problem is with TDD and SOLID, not with the regular-intelligent people. People have limited time to invest. I would rather Joel and Jeff’s invested in something with the chops they already have – made something useful for ME. I don’t care how they do it, just do it and make it good for ME.
Kelvin about 7 hours later:
I prefer my naivete to your cynicism. And besides, you’re just wrong. Bad code always takes longer to create than good code. If you don’t believe that, you need to change professions.
Uncle James about 7 hours later:
Sure. That’s why we have BA/QA folks write automated acceptance tests. However, there is a powerful benefit when software developers state their intention twice. It’s rather like accountants who practice dual-entry bookkeeping. If you say it twice in two different ways, you have a good chance of avoiding stupid mistakes. And if the tests are automated, you have a very good chance of avoiding future breakage.
Of course. TDD is not about testing everything. TDD isn’t actually about testing at all. TDD is a discipline for designing, developing, and preserving software.
.bq. I’ve done TDD on a few projects now, usually at the behest of some junior Java programmer who had the religion and talked the rest of us into it. Taking the full time and resources spent on the tests and comparing it to the very few times where any of them have caught problems, and I just don’t see the value.
I’m sorry. What? You don’t see the value in hitting a button and quickly knowing that the system works. If you are seriously telling me that, then I have to doubt that you are a software developer. Most software developers would give their right arm to have a button like that.
A team that truly uses it’s brains adopts repeatable disciplines.
Joel Spolsky about 7 hours later:
I look forward to it. You can contact me at unclebob@objectmentor.com.
Kevin about 7 hours later:
When you say something twice, in two different ways, you are far more likely to understand it well, and detect errors.
Much bad software is due to poorly understood requirements. Acceptance tests can help to clarify requirements, but do little to ensure they are the right requirements. Nevertheless, TDD is a powerful way to understand and preserve the solution you are creating.
Barry Kelly about 7 hours later:
If a software developer creates a design that is so fragile that 10% of it breaks due to a single point change, then that developer needs significant remedial training.
Actually, there are quite a few developers who collect a paycheck and do more damage than good. Indeed, that’s one of our bigger problems.
Frankly, I think there’s some rather dishonest selective quotation going on in this post. For anyone who is interested in a bit more context, the entire exchange is available on the Stack Overflow Podcast transcript (https://stackoverflow.fogbugz.com/default.asp?pg=pgWiki&;command=view&ixWikiPage=29025). I’ve reproduced Jeff and Joel’s remarks before and after the snipped that Bob quoted below.
Atwood: Right. The longer I think about this, the less I care about code hygiene issues (laughs). Because the unit of measure that really matters to me is, how quickly you can respond to real business needs with your code. And by that I mean, how well are you serving the people that are using your code. To me that’s what it’s all about. Anything that gets in the way of you fixing your code or improving your code in a way that your customers can appreciate, is a negative. If that means using Ruby, or having lots of unit tests: whatever’s working for you: do that. But if it’s getting in the way, if it becomes friction, like, “I’d love to have this great new feature but I’d have to have 1000 unit tests,” that’s a negative.
Spolsky: Yeah. And the worst thing that happens is that you get people that just stop thinking about what they’re doing. “This is the principle, to always write unit tests, so I’m always going to write unit tests,” and then they’re just not thinking about how they’re spending their time, and they wind up wasting a lot of it.
Atwood: Yeah, it’s a balancing act. And I don’t want to come out and say I’m against [unit] testing, because I’m really not. Anything that improves quality is good. But there’s multiple axes you’re working on here; quality is just one axis. And I find, sadly, to be completely honest with everybody listening, quality really doesn’t matter that much, in the big scheme of things… There was this quote from Frank Zappa: “Nobody gives a crap if we’re great musicians.” And it really is true. The people that appreciate Frank Zappa’s music aren’t going, “that guitar was really off.” They’re hearing the whole song; they’re hearing the music, they’re not really worried whether your code has the correct object interfaces, or if it’s developed in a pure way, or written in Ruby or PHP… they don’t really care about that stuff. We do internally, but it’s important to balance that, I think, and I think that gets missed a lot, which is, maybe, the point you’re getting at.
Spolsky: Yeah.
Atwood: I think over time, more and more, I’ve become really lax on my thinking about this, because what matters is what you deliver to the customer, and how happy the customer is with what you’ve delivered. There’s many, many ways to get there.
I leave it to you to figure out why that paragraph is an oxymoron.
Software quality is important because the most valuable attribute of software is it’s ability to be changed and adapted to the changing needs of the users. Bad software is hard to change. Indeed, that’s a good working definition of low quality software.
TDD makes software much easier to change.
CentrifugalBumblepuppy about 9 hours later:
Do you always use five paragraphs of appeals to authority to say “meh?”
Chris about 11 hours later:
Chris, please point out the dishonesty. Be specific please.
this discussion has rapidly descended into self-assured sniping on both sides. let’s face it: as people proud of our work, we want to write unit tests, we want to use our favorite language, we want to do all the things for the sake of future generations.
but so much of this code does not get reused and so much of its value is time-sensitive. we can still artisan great software if we cut corners for the greater good.
though they might love to enskeleton every structure with admantium, even great engineers understand that the cost of doing so exceeds any value it delivers. (not to mention demolition costs down the line!). they hold to tolerances of failure and yielding stress.
what joel is saying is that an emphasis on “high-quality” software can be both over-engineering and masturbatory. for those with half a century of experience unit testing and SOLIDifying, it might be trivial to always do so, but for the majority of software writers, in the plurality of cases, it’s not.
like appropriate technology, there is appropriate coding
I’m stunned at the number of people who have come out in the comments against TDD. It has obvious benefits for crafting better code but you have to not be so lazy to ignore them. Larry Wall once stated that programmers have the trait of being lazy and this is probably the main roadblock that stops people from taking the extra time to write a unit test first.
“Chris, please point out the dishonesty. Be specific please.”
You claimed Jeff and Joel were saying, flat out, software quality doesn’t matter. What they actually said was that software quality is one element that has to be traded off against others: time to market, features delivered to the customer, etc. To expand the quote from less than one sentence to two: “But there’s multiple axes you’re working on here; quality is just one axis. And I find, sadly, to be completely honest with everybody listening, quality really doesn’t matter that much, in the big scheme of things”
You took the smallest excerpt you could (not even a whole sentence) and turned it into a strawman to argue against.
Many assume automatically that writing unit tests translates into additional time spent. It’s a natural and intutitive assumption, but it’s incorrect.
The key is that when you practice TDD, you spend more time writing tests, but far less time debugging (and in particular: searching for bugs). Having more robust and flexible software afterwards is a bonus.
“Chris, please point out the dishonesty. Be specific please.”
You claim that Joel and Jeff said flat out that, “Quality just doesn’t matter that much.” What they actually said was that quality was one element that has to be traded off against things like time to market, and delivering features to customers. To expand the quoted section from less than one sentence to two: “But there’s multiple axes you’re working on here; quality is just one axis. And I find, sadly, to be completely honest with everybody listening, quality really doesn’t matter that much, in the big scheme of things.”
You took the smallest bit you could (less than one sentence) and used it as a strawman to argue against.
This is very similar to a recent conference panel that Uncle Bob was on.
http://www.dotnetrocks.com/default.aspx?showNum=358
It is interesting that this whole ‘is quality important’ thing keeps coming up. When listening to the various arguments it is important to know that no one is saying that bad quality software is good and no one is saying that you should pursue quality at the expence of all else – the argument is about where you stand on the continuum.
Sorry about the long sentence; like Mark Twain I hadn’t enough time to make it shorter.
Don’t be too harsh with them, one is a manager of a small company and the other a blog writer. Neither is a full-time developer, and both seem to have little experience with larger projects (their blogs are good read – but not for programming reasons – perhaps except the leaky abstractions and iceberg principle parts :-)
Stephan
— Programming is hard – http://blog.codemonkeyism.com http://twitter.com/codemonkeyism
I have to admit to being surprised but some of the content of the SO podcasts, especially since I hear so much about these guys in respect to them pushing the engineering field forward.
I suppose this really comes down to the fact that the podcast is pretty much ‘made up’ as they go along, something that maybe should be a little more planned?
Anyway, I stopped listening to SO when Jeff linked to Wil Shipley’s rant on Unit Testing (on Twitter) but wouldn’t give any real justification as to why. It just seemed like stirring for no good reason, something I think was coming across in the podcasts – which is probably exactly why they said what they said.
Shame really, as this whole series could have been a much more focused cast and it seems to be moving away from that.
I stopped listening to StackOverflow a long time ago. The overly authoritative yet unqualified tone by Mr. Spolsky in particular, no matter the topic, is especially annoying to listen to and strikes me as potentially dangerous. I’m only surprised I haven’t seen more critical entries like this one before.
joran about 12 hours later:
I disagree that this is what they were saying. What Jeff said was “And I find, sadly, to be completely honest with everybody listening, quality really doesn’t matter that much, in the big scheme of things…” This is dead wrong. Quality matters a lot. Joel, on the other hand, was talking about TDD and SOLID in a manner that made it clear he didn’t know much about either.
That’s a convenient excuse, but it doesn’t fly. It’s like saying: “Doctors with 50 years experience with sterile procedure can afford it, but most doctors don’t have time to wash their hands.”
Certainly. But it is never appropriate to make a mess.
Chris about 13 hours later:
It wasn’t a claim, it was a statement of fact. Jeff said: ”...quality really doesn’t matter that much…” He’s wrong. He’s dead wrong. And so is Joel for not jumping on that statement with both feet.
They said that too, but they are wrong about that as well. Quality vs. Speed is a false dichotomy. You don’t go faster by making a mess. THAT is my real beef with this whole thing. If you want to go fast, you do the best job you can! High quality is a lubricant not an impediment.
Dead wrong. Quality is not an axis. Quality applies to all axes.
If you read my whole blog you’ll find that I actually talked about quite a bit of the podcast. Nor do I think I pulled anything out of context. In any case, the word “dishonest” is a very strong word to use, and you should take care with it.
‘It wasn’t a claim, it was a statement of fact. Jeff said: ”…quality really doesn’t matter that much…”’
Now this is the way you should have written it in the first place. Those ellipsis and either end of the quote indicate to the reader that you’ve plucked out a portion of a sentence rather than an entire statement.
Unfortunately, for the masses, I don’t think quality does really matter that much. The problem is that poor quality in software has become the norm, accepted practice. When you’re talking to the sales guy and he says “Yeah, our system does that”. You make the bit purchase, and then find that the feature is unusable because there are 25 defects in it. It’s akin to being assured that “you can drive this baby to and from work every day” by the car dealer but, upon purchase you find that on any commute of over 15 miles the car shuts just stops running. This is not acceptable in any business but software… Sad…
The argument doesn’t seem to be over whether crappy code is good. We can see systems getting built all the time on crappy code, and they deliver business value. The question seems to be whether quality, crafted code is better than what the industry generally does now.
Could we go faster if we wrote unit tests and used SOLID principles? Would we have a competitive advantage? Can we discuss tradeoffs authoritatively if we don’t have experiences from both ends of the spectrum?
Brandon Carlson about 22 hours later:
It’s not acceptable in software either.
After reading the transcript, I think what they’re trying to say, as far as tradeoff between quality and other things, is the 80/20 rule. There is a point of diminishing of return, like in trying to get absolute 100% code coverage.
They worded it poorly, but I think that’s what they were trying to say.
I’m with Uncle Bob on the matter of code quality.
If Jeff and Joel want to say that they don’t use TDD, or that they don’t follow the SOLID principles, then that’s fine. If they were to say that they achived 100% coverage once, but it took a long time to get there, then that’s fine. However, asserting that you are likely to often “make a change to your code that, somehow, breaks 10% of your unit tests” isn’t a given: it only means you’ve got a bad design and you should start refactoring so that doesn’t happen again.
I’ve done a small project where I could get code coverage metrics. They were in the high 90’s because I’d written the code using TDD and only took a short while to get 100% coverage. If I’d had the metrics available from the start I’d have been able to get continuous 100%. I’ve just started a few Python projects and I’ve built in coverage stats, so I work with 100% only.
Joel comments on the ISP, but doesn’t understand why you need an interface with just the six methods your own class uses – Joel, it means that you can pass in ANY other class that implements those six methods and you’re not limited to passing the original class that has 40 methods. It makes it easier to re-use your class.
The discussion triggered me to write this blog post:
Not doing something costs time and money, too
Keep up the good work Robert, and please don’t fall of your bike!
So who test the TEST cases?
...Since Joel doesn’t allow comments, I guess this goes here…
They’re saying a lot of things about unit testing that I said before I really got serious about testing and tried TDD. For example, this issue of “time, time, time” is the big thing that kept me from jumping head first into it in the first place. It’s counter-intuitive to someone that hasn’t actually tried it.
The two things I can say that I think of when reading their transcript are:
1) Take the “time” it will spend you to write unit tests up front and realize that all of that time is spent writing code instead of stepping through your debugger. Think of the tests as a chance to practice before you write the real production code. 2) TDD is almost equally about design as it is about testing. My code looks so much better when the tests are written first. It doesn’t get over-engineered, I’m forced to break things up into smaller, more cohesive methods/classes/modules, etc.
So with all of the emotions left aside, I think Jeff and Joel sound like the lucky guys who are good enough to be successful without relying on unit tests and code quality very much. For the rest of us, though (or at least for me) TDD is just as essential a skill as OOAD.
...And quality does matter. Broken windows, man…broken windows…
Quality is important. No argument there. BUT, who will test and validate the unit TEST scripts?
....anyone?
TDD is not just about “unit testing”. TDD is much more! It’s about Design as well. Joel somehow implied TDD as a Testing and QA tool but it is way more then that.
I am big fan of Joel, but I don’t take this rant. I think it is insulting to call someone “You don’t know because you haven’t coded enough”. Somehow Joel implied that referring to Uncle Bob. Which is sad and cheap means of publicity stunt.
@Chris 2 see this please: http://butunclebob.com/ArticleS.UncleBob.TheSensitivityProblem
I think Joel and Jeff’s views on quality make sense if you think that TDD/BDD is a testing activity. It makes no sense from the standpoint of a design activity.
So let me be clear; TDD is a design activity, TDD is a design activity, TDD is a design activity!!!
@dhui: Yes that article described the importance of TDD however that doesn’t answer my question at all. So again, who tests the TESTS?
WOW, you guys are really brainwashed. I know its easier to repeat than to think, but you guys dont even fight the argument, just say its bad :(
the funny thing is that this blog has repeated posts, so why TDD was not used here? Maybe 100% code coverage just doesnt work?
@Joel
To start with, I must stress that I have not listened to the podcast on Hanselminutes, so this comment is based purely on the content of Robert Martin’s book: Agile Software Development – Principles, Patterns, and Practices.some of Robert Martin’s books and papers (which I higly recommend to any serious software devleoper).
You say: Last week I was listening to a podcast on Hanselminutes, with Robert Martin talking about the SOLID principles. (That’s a real easy-to-Google term!) It’s object-oriented design, and they’re calling it agile design, which it really, really isn’t. It’s principles for how to design your classes, and how they should work.
Either you have not read this book, or you didn’t digest its content. His definition of agile design makes it clear that he does not claim that the principles themselves are agile design:
So, what is agile design? Agile design is a process, not an event. It’s the continous application of principles, patterns, and practices to improve the structure and readability of the software. It is the dedication to keep the design of the system as simple, clean, and expressive as possible at all times.
The solid principles are an instrument, just like design patterns and agile practices, whose continuous application constitute the process of agile design.
By clean code, Martin means code that doesn’t smell:
How do we know how whether the design of a software system is good? Here are some symptoms of poor design (design smells): Rigidity, Fragility, Immobility, Viscosity, Needless complexity, Needless repetition, Opacity. Often, the smell is caused by the violation of one of more design principles:
[the] object-oriented design principles … help developers eliminate the symptoms of poor design smells and build the best designs for the current set of features.
By the way, when you say that the SOLID principles are OO design, you are actually agreeing with Martin, amongst whose definitions of the principles is the following:
[the principles are] the hard-won product of decades of experience in software engineering. They are not the product of a single mind but represent the integration of the thoughts and writings of a large number of software developers and researchers. Although they are presented here as principles of object-oriented design, they are really special cases of long-standing principles of software engineering.
You may ask: what is agile about about applying the SOLID principles? I’ll come to that when I look at your next few comments:
People that say things like this have just never written a heck of a lot of code. Because what they’re doing is spending an enormous amount of time writing a lot of extra code, a lot of verbiage, a lot of files, and a million little classes that don’t do anything and thousands of little interface classes and a lot of robustness to make each of these classes individually armed to go out into the world alone and do things, and you’re not going to need it. You’re spending a lot of time in advance writing code that is just not going to be relevant, it’s not going to be important. It could, theoretically, protect you against things, but, how about waiting until those things happen before you protect yourself against them?
This is the big mistake you are making: you are telling one of the fathers of Agile design to adhere to YAGNI, and you don’t realise that while you accuse him of advocating the application of the design principles in a Big Design Up Front(BDUF) fashion, you are unaware that his key message is that:
...agile developers do not apply [the] principles and patterns [of s/w design] to a big, up-front design. Rather, they are applied from iteration to iteration in an attempt to keep the code, and the design it embodies, clean.
And again:
Agile teams apply principles only to solve [design] smells; they don’t apply principles when there are no smells. It would be a mistake to unconditionally conform to a principle just because it is a principle. The principles are there to help us eliminate bad smells. They are not a perfume to be liberally scattered all over the system. Over-conformance to the principles leads to the design smell of needless complexity.
Of the OCP, which is the most important of the principles, and from which the others are derived, Martin says that
In general, no matter how “closed” a module is, there will always be some kind of change against which it is not closed. ...
Since closure cannot be complete, it must be strategic. That is, the designer must choose the kinds of changes against which to close the design, must guess at the kinds of changes that are most likely, and then construct abstractions to protect against those changes. ...
conforming to OCP is expensive. It takes development time and effort to create the appropriate abstractions. Those abstractions also increase the complexity of the software design. There is a limit to the amount of abstraction that the developers can afford. Clearly, we want to limit the application of OCP to changes that are likely.
How do we know which changes are likely? We do the appropriate research, we ask the appropriate questions, and we use our experience and common sense. And after all that, we wait until the changes happen! ...
You read that right! What you are recommending is exactly what he advocates:
“Fool me once, shame on you. Fool me twice, shame on me.” This is a powerful attitude in software design. To keep from loading our software with needless complexity, we may permit ourselves to be fooled once. This means that we initially write our code expecting it not to change. When a change occurs, we implement the abstractions that protect us from future changes of that kind. In short, we take the first bullet and then make sure that we are protected from any more bullets coming from that particular gun.
@Chris 2: The Code and The TESTs test each other.
I think quality is key! I also think that in order to live from a product, and get market share, you can’t obsess on quality, you should be delivering versions over time (although of course, you don’t want to deliver a version with bugs or crashes).
Probably there should be always sort of balance between quality and quantity, but I agree that quality matters a lot, and as you said, over time it pays it’s dividends ;)
I would put it this way:
Let’s suppose you have a system of a high quality. Then you have some requirement for that system that need to be addressed. You have two choices here:
1. Implement it as fast as you can to please your users. Don’t care about quality of design, just about how quick your response to the requirements are.
2. Implement it a bit slower but in TDD fashion (TDD != just tests, or as Monti pointed out TDD is a design activity). So do care about design quality and tests.
From user point of view it’s clear that you should choose (1) The only case when it makes sense for you as a developer to chose (1) is if that is the last change you’ll ever make, which sure is not. An here is why:
Suppose that after you’re done, another requirement, that is somehow related to previous one, comes in. Again you have the same two choices as in previous case. But the difference now is that you don’t have the high quality system (which you traded for speed) anymore. So, since this new requirement is related to the previous one which is not well designed, it’s likely that it will be harder to adapt it to new changes, as opposed with well designed system. Essentially what you have done when responding to the first requirement fast, apart from trading quality for speed, you stole the time from the next related requirement. Suddenly choice (1) is not so fast anymore, and time accumulates as you keep choosing it, thus your response time is getting slower and slower. By sticking to (2), that time is relatively constant. So you end up with faster response time, better quality/design, and you have “the button” that Uncle Bob mentioned.
Now which one you choose?
I’ve pretty much given up on Joel. Rarely does he say something intelligent about software development anymore. I think you’ve hit the nail on the head about how he is now a business person and no longer a developer.
If quality really mattered, Bill Gates wouldn’t be one of the richest men in the world.
As techies, we love quality and often believe that it is the most significant quality, but in the world around us “good-enough” is often the baseline. Things just need to work well enough, to get the job done. You don’t need a Rolls Royce to get to work, or 100% guaranteed connection to ensure your email arrives. Quality is a noble goal, but in many ways it’s become strangely anti-capitalist. Competition drives down quality, it’s a Walmart effect of some kind. Sadly, we’ve moved into a period of cheap and disposable goods, which has obviously effected software too (although I think software got there first).
Paul.
I’ve never spent any time at a shop where they lost productivity because of quality. I’ve been at plenty where productivity was in the toilet because of the lack of quality. In the face of that, how does anyone say that quality is unimportant?
Unless, of course, they only work on their own code, so it’s other people suffering from their mistakes and not vice-versa.
I think the root cause of this dispute is that you and Joel inhabit different worlds. While he is into product development, you are into consulting. So, he is more concerned with faster releases and less into adhering strictly to design & test considerations.
But remember, Joel hires developers of the highest calibre (you can read about his hiring process on his website), so the typical code that they produce is already of very high quality right out of the gate. So typical problems that you may see with less capable organizations would not be there with FogCreek.
Interesting timing … I’ve been trimming the fat from a (company internal) blog post that directly relates to this very issue.
Various studies suggest between 60-95% of software development costs are incurred post initial product delivery. Up to 60-80% of that cost is spent enhancing product (as opposed to defect remediation). In other words, well over half of your development dollars are spent trying to change software you already delivered.
It seems to me, from a purely dollars and cents standpoint, it makes sense to write your code from a change friendly perspective (Your customers still want version 2.0 just as fast, right?). Anything else simply begs the scenario Joel railed at in an early blog of his own.
Change is the very sweet-spot TDD and SOLID principals are pointed at.
“I think the root cause of this dispute is that you and Joel inhabit different worlds. While he is into product development, you are into consulting. So, he is more concerned with faster releases and less into adhering strictly to design & test considerations.”
I agree. Jeff and Joel run small, self-funded startups, run by (and largely for) programmers, focused on producing web or shrinkwrap software directly to customers. I don’t know what sort of business the people participating in this discussion are in, but from comments like the one I quoted above lead me to believe that some of the folks here are work in much larger organizations where their project (or programming in general) is just a small part of the larger scheme of things, or in consulting shops that get their direction from a customer who’s paying them to write code. This may account for some of the very different views on things like SOLID and TDD.
For instance, SOLID seem to emphasize modularity quite a bit. The ability to change or rip out any portion of your code in response to some external directive makes a lot of sense in a context where the developers don’t have control over the requirements for or future development direction of the software.
Jeff and Joel, on the other hand, have much more control over the products they’re writing. They still respond to market forces and customer needs, of course, but they’re the ones who make the decision of which direction to go and what is going in the product. There’s not nearly as much incentive to modularize everywhere. They can concentrate their modularity efforts in areas they plan to change in the future.
Even if this is true (and I don’t accept that axiom), things change . Requirements, understanding, business needs… When this happens, software will need to change. Even so-called “highest calibre” individuals producing initially high-quality code will have problems without support for change.
Some amount of automated support is definitely a boon to address change. This can be in the form of unit tests, acceptance tests, smoke tests, load tests, integration tests, ... However, for any system of any complexity, lack of automation leads to an increasing cost to release (release debt?) – or releases with more and more bugs.
TDD is one way (and a very good one if not done badly) to achieve that support structure. It is not enough. TDD is not an either-or choice with, say, acceptance tests. They accomplish different goals.
I’ve been involved in several startups, and my impression is the opposite. A startup that ignores their customers’ needs will go bankrupt. And there is no way to know those needs with any precision before implementation, so there really is no such thing as “control over requirements”. When the users get their hands on working software, they discover what they really want. A large organization has the same problem in principle, but will at least survive if a project or two isn’t agile enough.
The business/mangers don’t not care about code quality, just the schedule and functionality…
I try to slap it out quickly, then give me time to re factor/clean-it-up, then the code should be ready for the next last minute requirements due to poor planning…
The more experienced the developer/s the quicker this cycle can be.
Schneider
I actually wrote about something similar recently, but more on the side of “worse code is sometimes better”. Weird.
http://www.jdconley.com/blog/archive/2009/01/26/put-down-the-abstract-factory-and-get-something-done.aspx
Chris 2 1 day later:
When you say something twice, in two different ways, you are much less likely make an error. So, the code tests the tests, and the tests test the code.
This is the same technique that accountants use to protect themselves from error. They say everything twice, once as a debit, and once as a credit. These entries follow separate mathematical pathways until they yield a zero on the balance sheet. It is possible that two complementary errors could be made, but it’s highly unlikely.
“I’ve been involved in several startups, and my impression is the opposite. A startup that ignores their customers’ needs will go bankrupt. And there is no way to know those needs with any precision before implementation, so there really is no such thing as “control over requirements”. When the users get their hands on working software, they discover what they really want. A large organization has the same problem in principle, but will at least survive if a project or two isn’t agile enough.”
True.
TDD has become to commercialized that they focus too much on the quality of the code and forgot what their customer/users want.
@dhui: Writing tests is also coding in C#/VB which, most likely, will contain bugs too. And, writing a wrong test code that doesn’t fail will obviously pass unit testing.
So again, who validates these tests?
@Chris 2: The wrong code passing the wrong test is possible, but highly unlikely. If someone always do this, he need to change professions.
@unclebob: Accountants doesn’t have QA. And coding “twice” is expensive.
@dhui: That is why it is called a “bug”. So are you saying that you write 100% bug-free tests?
All tests and experiments could contain Type I or Type II errors. Your assumptions can only be based on that which has convergent validity and is consistent both empirically and logically.
The more tests you have telling you the same thing, the more consistency and validity they have. Any one logical error will expose a systematic inconsistency.
@Chris 2: TDD is not a way to get 100% bug-free tests, it’s the best and fastest way to get high-quality product. So do you know some other way to get 100% bug-free tests? Or does such a way really exist ? In practical sense, I think TDD is the best.
@Chris 2 Also, see this section from Pragmatic Unit Testing:
@Chris 2> ‘And coding “twice” is expensive.’
Yeah, why would you do that? I wouldn’t. Mostly I use TDD to enable refactoring, so that I write a boatload less production code (and test code) than I otherwise would have. I’ve seen good-sized, “typical” (i.e. poor quality) systems shrunk to 1/3 original size by virtue of adding tests and refactoring, and it didn’t surprise me one bit. And the cool part is that I end up with this side effect that the code is really well-tested and documented.
In contrast, I’m dealing with a “brilliant” system right now that was developed by people too smart to spend all that effort on doing TDD right. Four or so years in, and >1 million lines of code later, everything takes 5 to 10 times longer than it should, and everyone fears changing the code. Half the time is spent just understanding the impact to even the smallest changes.
“TDD has become to commercialized that they focus too much on the quality of the code and forgot what their customer/users want.”
I’m sure that’s happened somewhere, but a) it’s got nothing to do with TDD being “to [sic] commercialized,” b) it’s not typical, and c) once you start criticizing something because of the people who do it poorly, you’re just trying to make yourself feel better about not doing it.
Codinq twice is, in fact, rather inexpensive when compared to debugging. If coding twice can eliminate 50% of your debugging time (a vast underestimate) then it’s already paid for itself.
Yes, accountants don’t have QA. Have you asked yourself why? Could it be because the dual entry bookkeeping practice keeps them from making so many errors?
As you probably know I am the prime maintainer of FitNesse. I release it a few times per year. It’s 50K lines of code. I have no QA. The QA cycle is to run the 90 second script that executes all the tests. If they pass, I ship. Despite thousands of users I have a very small bug list which you can see here scattered amongst the new features and stories.
The test code is also often very straightforward, so making mistakes because of the complexity of the code is very rare. A typical test has three parts:
1. Initialize the system under test (SUT). In unit tests the SUT is an object or a couple of closely related objects.
2. Perform some action on the SUT.
3. Check that the results are what expected. Often this is done by asserting what is the expected value and comparing it to the actual value in the SUT.
On the other hand, the production code has loops, conditions, mutable variables etc. which makes it more complex and very much different from the test code, even though both the production and the test code state the same intention. So you might have one complex algorithm in the production code, and many simple tests for that algorithm.
This means that:
- You state the same intention in two very different ways, so if you make an error in either one of them, you will find it out.
- The test code is very simple compared to the production code, so the probability of making errors in test code is lower.
- Also because most of the test code is very simple, writing it is fast, so it does not slow you down. In the time that it takes for you to think about the requirements and what to do next, in the same time you can also write down the requirements as a test case. And when you have written the requirements as tests, they will also serve as detailed documentation of the requirements, when you or others later work on the same code. So you will not face situations where you do not know why some code exists – if you see such production code, just delete it and see which tests fail (if no tests fail, the code was not needed anyways).
When I started doing TDD, I measured how many lines of production code per hour I produced, when including debugging time etc. Both before and after I started using TDD, I wrote in average about 20 lines per hour. So TDD did not slow me down. Instead it made coding more fun, because (1) I very rarely need to debug the code and look for hard-to-find bugs, and (2) I have all the time a sense of accomplishment and progress when I get a new test to pass every couple of minutes.
Of course there are some things that are not noticed by TDD:
- When there is a corner case that did not come into your mind, then by definition you have not written a test for it. But later on, when you notice that corner case (perhaps because of a bug), then you can write a test case for it and the same bug will never again happen.
- When the requirements are wrong, or you understood them in a wrong way, then the intention of both your tests and your code will be wrong. But when the requirements change (whether they were understood right or wrong), changing the system is not too hard, because of using TDD you have a large regression test suite and your code is decoupled (TDD forces you to write decoupled code, because otherwise you could not test the code).
To test the requirements, other methods are needed, for example close collaboration with the customer and acceptance tests. (Myself I do user interface design, and I use paper prototypes and some user interface testing methods (with and without users) to test the functional requirements of the system.)
on related note
http://stackoverflow.com/questions/507077/testing-a-test
Bob! It’s great how you defend your position but that’s a battle you cannot win with logical arguing (although your arguments make perfect sense by the way).
Remember Dilbert’s famous words of wisdom:
Never argue with an idiot—they drag you down to their level, then beat you with experience.
Thanks for setting the record straight UncleBob. I was just about to try to post a comment to Joel’s blog setting the record straight (as if he really cared about the facts, given that he’s already made his mind up on the subject – and based on nothing but his own inexperience with TDD) when I came upon your post. You said everything I wanted to, and much more eloquently and authoritatively.
BTW, apparently Joel doesn’t allow comments on his blog. I guess it’s better for your career to try to show off how good a pundit you are by loudly proclaiming an opinionated Gospel According To Me and being so over-confident of your own brilliance that you’re not interested in any evidence to the contrary.
“BTW, apparently Joel doesn’t allow comments on his blog.”
He has discussion forums instead.
Another part of the answer to “who tests the test cases?”—running the tests before writing the code to pass them. I ask students why anyone would possibly want to do that, when you know the tests will fail. They get it pretty quickly, but they have to be told it’s not a typo first.
Get over yourselves.
“Accountants doesn’t have QA. “
Isn’t that what auditors do? (Theoretically at least.)
@Chris
In his first edition of Extreme Programming Explained – Embrace Change Kent Beck said:
but he believed the following:
In the second edition of the book, Beck changed his mind about quality being a control variable:
@Chris
In his first edition of Extreme Programming Explained – Embrace Change Kent Beck said:
but he believed the following:
In the second edition of the book, Beck changed his mind about quality being a control variable:
I think it’s Spolsky’s suggestion that people like Robert Martin don’t write much code which bugs Martin the most, and I’m not surprised. Listening to the podcast, I didn’t hear anyone actually say “Quality doesn’t matter”. Maybe I didn’t listen long enough, but I wonder if it’s actually Martin’s spin on what’s been said. Robert Martin is a fervent promoter of code quality.
So finally we all can take from this that “quality matters”
I’ve never really read or listened to anything from Joel – just the following debates over his opinions.
“Quality” is a very overloaded word (more overloaded than Agile!).
“Quality” to a manager means working software. To a developer it means (or at least should mean) well written software.
There’s lots of instances where working software is poorly written, and I’m sure there’s instances where faulty software has been written well (even when it’s written in an Agile way!).
IMO, I would say that, overall, concentrating on “Software” quality results in “Product” quality. Managers (most) recognise just one facet of quality – the “Product” quality. Much the same as I recognise a car that’s reliable and smooth whereas would recognise the same car uses a specific way to manufacture pistons and valves. Then again, I wouldn’t go shouting my mouth off on the Automotive Engineer Monthly Podcast saying that any old pistons and valves will do!
I’ve never really read or listened to anything from Joel – just the following debates over his opinions.
“Quality” is a very overloaded word (more overloaded than Agile!).
“Quality” to a manager means working software. To a developer it means (or at least should mean) well written software.
There’s lots of instances where working software is poorly written, and I’m sure there’s instances where faulty software has been written well (even when it’s written in an Agile way!).
IMO, I would say that, overall, concentrating on “Software” quality results in “Product” quality. Managers (most) recognise just one facet of quality – the “Product” quality. Much the same as I recognise a car that’s reliable and smooth whereas a auto engineer would recognise that the same car uses pistons and valves that are manufactured a specific way.
Then again, I wouldn’t go shouting my mouth off on the Automotive Engineer Monthly Podcast saying that any old pistons and valves will do!
I’m not sure who “your” refers to in your first sentence. You do realize this is not an OM product. The blog is an “OM product” but the blog software is something we (and a lot of organizations) use.
Hi Uncle Bob
Just thought I’d voice my support for your position. Some of the people who commented on this thread clearly did not listen to the same podcast I did. Now there seems to be a bit of damage control going on by Joel. He can deny it and twist your words and his all he wants. He ends up sounding like John Dvorak more and more each day.
BTW: Accountants do have occasional “QA” – they are called “auditors”.
Joel is a businessman who created a business (apparently a very successful business) around the idea that his company hired only the best and brightest programmers in the world. He popularized the phrase “Rock star programmer” in his books and blog posts about how to hire Fogbugz caliber developers. He’s made a small fortune around this premise.
He can not afford to accept a development methodology if he and his team of rock star programmers don’t understand it. It undermines the premise on which his company was founded and the catalysis for his internet celebrity. His opinion is always going to be that the best way to write code is whatever approach his team takes. Anything more complex is the work “architecture astronauts”, anything less complex is the work the non-rock stars who can’t grok programming.
I’d just like to respond to the “Who tests the tests?” questions. Bob responded to this well, but I’d like to emphasize the fact that YOU test the tests. When a test breaks, you either realize your design didn’t take something into account, you realize there’s a bug in the test, or you realize there’s a bug in your code.
Having said that, if the code DOESN’T break and your tests didn’t cover the full gambit of scenarios that you were likely to face, I’d chalk that up to a learning experience. It’s those times when you know there’s an issue, but that has not been accounted for by tests, that you really learn how to be a programmer and debugger. I still think debugging is one of the most fundamental skills a programmer can have, but frankly, if you’re spending time debugging your code and not actually writing your code, then you’re not a programmer. There’s no question about that. That’s not to say that if you don’t write tests first (or last) that you’re not a programmer – rather, not writing tests first (or last, even) makes you irresponsible and foolish.
tdd might be sterile procedure
but it also might be the overuse of antibiotics
Any suggestions on how I can effectively communicate that build-and-fix is not actually an Agile methodology?
@AndrewSeven
You could use Fowler’s excellent article Is Design Dead? to explain that code-and-fix was the common, but disastrous, form of evolutionary design, whereas XP is one of the Agile methodologies that made evolutionary design a viable strategy again.
I’m working for an organization in trying to get them to adopt TDD. This week there is a whole bunch of buzz around “industry experts” saying that TDD is a waste of time.
Glad to finally know where all of this garbage is coming from. Great response uncle Bob, and like you, I have to fight to keep my technical chops, they can disappear so fast.
Some “business wonks” do still get it, and listen to the advice you give them.
Regards Jeff
@AndrewSeven One method I use is to talk about Waterfall/Code-n-fix/XP as three corners of a triangle: http://groups.google.co.uk/group/comp.software-eng/browse_thread/thread/52b0dbd2a208df35?hl=en&;ie=UTF-8&q=phlip+methodology+triangle#4a9d73a126c3231c
See phlip’s description
Hi, Uncle Bob,
You might be interested in My comments on the current contraversy, as well as Jurgen Appelo’s comments about a similar issue.
This discussion has actually caused some problems for those of us (guilty!) who Believe in agile methodologies, but who believe that the current Authority First discussions devalue the general commentary.
I hope your readership values our opinion (coming from the Pragmatic side of the agile-vs-Agile spectrum).
Kirk Wylie
Joel will say anything to get more publicity that attracts traffic in some form to one of his web presences – be it to JoelOnSoftware – so he can sell his job ads and so on. Just about every discussion ends up with a plug for FogBugz in some form.
It doesnt really matter who is right or wrong in this case – becaue he has achieved his goal. What is important is did Joel get some eyeballs to one of his sites so he could advertise ?
Correct me if I’m wrong, but it seems that Spolsky expects from the SOLID principles to be some kind of hard-and-fast rules, that you can learn from a presentation in half and hour, and go blindly implementing them.
I’m not a terribly experienced coder, but I think all of the principles need critical thinking to be useful. Take SRP for example. We all know that the words ‘single’ and ‘one’ are relative. A house can be said that is a ‘single’ object. But you can divide it into rooms. And you can divide them into furnitures. And you can divide them into pieces of wood,fibers, molecules,atoms,etc.
It all depends how you group things, and what the granularity of your system should be. That means that, when implementing SRP, one must know to keep the correct balance for the system being implemented and where to stop. How does one learn that? Experience, and lots of coding, that’s how. There’s really no shortcut here.
@Joel
You said:
In the next sentence you really did butcher the SRP by suggesting that the reasons for change are changes in the values of fields:
But what Uncle Bob said in the Hanselminutes interview is that the reasons for change are changes in the logic associated with the fields:
Regarding your objection to ‘millions of tiny little classes’, here are some wise words from Jeff Langr (in Clean Code, Uncle Bob’s latest book):
Insults. A whole pile of insults.
While they may have insulted a principle that you value, you, in turn, directly insulted Jeff and Joel.
You could have disagreed without starting a flame war. Why make it personal?
Bob -
You said “this is a symptom of someone who has focussed on business and lost his technical chops.” And quite frankly, that scares me.
Admittedly I haven’t worked at a product-oriented business; my experience in software development can be summed up in two groups: open-source projects and service-oriented agency work. If the entire world were built on open-source software, then scorning someone for losing his technical chops would probably be a valid way to judge someone’s fitness for survival in the software development landscape.
But, at least in the agency/services business, everything has to come from business needs. Everything we do is driven by our ability to be responsive to the customer, and that’s how we succeed. Now that’s not to say that we don’t adapt processes and techniques to ensure that we’re going to be successful – but I’m not going to institute scrum in a project that’ll take two staff-months to complete within a team of four people.
There has to be a balance, but at the end of the day, it’s the business – not our technical chops – that pay the bills.
Bob -
You said “this is a symptom of someone who has focussed on business and lost his technical chops.” And quite frankly, that scares me.
Admittedly I haven’t worked at a product-oriented business; my experience in software development can be summed up in two groups: open-source projects and service-oriented agency work. If the entire world were built on open-source software, then scorning someone for losing his technical chops would probably be a valid way to judge someone’s fitness for survival in the software development landscape.
But, at least in the agency/services business, everything has to come from business needs. Everything we do is driven by our ability to be responsive to the customer, and that’s how we succeed. Now that’s not to say that we don’t adapt processes and techniques to ensure that we’re going to be successful – but I’m not going to institute scrum in a project that’ll take two staff-months to complete within a team of four people.
Bugger, I must have missed the last part of that (your page mis-fired last time) -
There’s got to be a balance between technical correctness and business needs. But at the end of the day, it’s the business that pays the bills.
I am wearing a “Code Clean” band on my wrist. I truly believe TDD based design and development improves quality. But I also used to respect Joel. This is a sad day.
Why is everyone saying that the business need is for bad software, and that working software is a disservice to the business? I’m so lost here.
I think that there is no basic tenet of business that the code must be crap, must be increasingly bad crap, and must remain crap forever. I don’t know how that is good business.
I understand choosing to not have the most exquisite, elaborate, intricate, gold-plated software. All the exquisite gold-plated stuff I’ve ever seen/writen has been a wad of crap on a stick. But that’s not what we’re talking about. We’re not even talking about someone’s personal preferences.
Good software (passes tests, internally clean, minimal) is not something that disables business. If it is, then all those IT shops need to be closed now, and all the competent developers must be let go. If good software is bad business, then there’s no reason to ever do programming again.
As far as “Believe” and “Faith” are concerned, those words are easily replaced with “trust”. I trust that my TDD code is going to be better than my non-TDD code, and I trust it because it has always been so in the past not because someone told me to. There need be no blindness in faith. There does have to be some honest testing of the waters.
I think the straw men are piled high enough for us to start a nice bonfire. Can we drop the nonsense and talk about whether quality helps us deliver more, better software and whether it’s true as Deming said that quality is free (and the lack of it is expensive).
Since no one has the authority to define OO, I’ll even venture to say that “Employee” is not OO.
Wow ! What a lot of hot air. So let’s add to it … ;-)
I do think it was unfair of Joel to make a comment such as “they all sounded to me like extremely bureaucratic programming that came from the mind of somebody that has not written a lot of code” with regards Bob’s SOLID principles. This is a little bit of a personal attack and not really called for.
With regards their comments on unit testing – hey if their way of working works for them then who are we to mock them. If your way works best for you then use it and stop worrying about what others say about your way of working.
At the end of the day users, our customers, are generally more interested on whether a product works than what level of testing or what “quality” metrics have been applied to the code or the architecture used. What matters to the end user is whether they have working piece of software suitable for their needs. Are people really saying that if they knew that software X had no unit tests they wouldn’t touch it ? Ofcourse the Zappa quote in Jeff & Joel’s transcript really says that all far better than me :-)
Surprising as it may seem, people have been developing successful software for decades without TDD or agile principles and I’m sure many will develop software going forward with or without TDD, agile or other principles. These are all tools, pure and simple – these are not the holy grail of software development – I’ve been around too long and have experienced too many things that were the next great thing in software development. Some things will survive with time and some will fall by the side.
With regards the religious fever that comes with questioning TDD & unit testing – I’ve seen dreadful code that’s got a high unit test coverage figure – does this mean it’s good quality code ? Of course not! Just because somebody says they do TDD or unit testing doesn’t make them a good programmer and conversely if somebody says they do not do TDD it doesn’t make them a bad programmer.
Oh and for full disclosure I do write unit tests but having programmed for some twenty odd years I have also written successful programs using assembler, C/C++, Java & C# without a single unit test.
Way to go Bob. I’ve hated the idea of “stackoverflow” since the day it was announced. How can you trust an affiliate blogger and a capitalist?
I think you are missing the point. Quality of code matters but not to the extent of being the sole driving factor. Business should be the primary driving factor. Most proponents of 100% code coverage do not realize the reducing ratio of benefits versus cost as coverage ratio goes up, often exponentially. You talk about de-coupling tests. It is possible to some extent in most cases but it doesn’t come for free. It costs valuable programmer cycles often going around hoops to implement a simple test which would otherwise take just 5 minutes. So you see there has to be a balancing act.
I agree with Joel & Jeff on this. It is right time someone highlighted the obvious shortcomings.
There is always a cost benefit to everything. 100% quality control in a low end market would make a product unprofitable. But, in a high end market quality is a must. As Angusman said it is a balance. But, there will always be purists that say there should be 100% quality control
You can never have 100% quality control. That would be an unrealistic requirement. But, there should be a self regulated quality control done as the work is being done. This will reduce the overall need for generalized quality control parameters implemented after the project is completed
Very nice article it really catched my own idea of blogging. I really don’t care if how many people are reading my posts as long as I am happy and satisfied of what I want to share to my readers. That’s is why Im not good in writing serious articles, that your acting like your someone who knew something big but actually not. Just being true to your self is the best thing in blogging.
Well, what do you expect when you get a guy that looks like a nerd (Jeff) and a guy that needs a woman (Joel)- stupid stuff will eventually come out no matter how hard they suppress it.
Thanks for sharing these info with us! I was reading something similar on another website that i was researching. I will be sure to look around more. thanks…
A bit late, but still can’t believe some people can justify that sort of a position. Without quality (in every sense of the word), things just don’t work/last. Quality comes from the bottom up, no other way around it.
Great article. Thanks.
In my view, the basic limit on technical innovation isn’t our collective ability to come up with new things. It’s our ability to absorb them. That means that the field is changing about as fast as most developers can handle things. For most people, taking time off means you slip behind, your instincts and tricks becoming less and less relevant.
I agree completely with the thoughts, and I wanted to emphasize something you said:
“I hire people to do that for me so I can keep my technical skills as sharp as possible”
co i dado ,gracis por todos.
Reusing or copying code, though in some ways unlawful, I believe is common practice in software development,” said one freelance developer who participated. “Most developers that I come in contact with (including myself) reuse, copy, or even reverse-engineer code to make it work better or to include it in an application that we are programming.
I haven’t met a developer who wants to hide his/her code,” said one developer. “Developers are proud of their code and want other developers to see its brilliance (and feel proud if others use it). It’s companies and managers who care about copyrighting code
Great writ-up. Thanks for this deeply insight on this matter.
I think the straw men are piled high enough for us to start a nice bonfire. Can we drop the nonsense and talk about whether quality helps us deliver more, better software and whether it’s true as Deming said that quality is free (and the lack of it is expensive).
Surprising as it may seem, people have been developing successful software for decades without TDD or agile principles and I’m sure many will develop software going forward with or without TDD, agile or other principles. These are all tools, pure and simple – these are not the holy grail of software development – I’ve been around too long and have experienced too many things that were the next great thing in software development. Some things will survive with time and some will fall by the side.
Thanks for sharing this great article! That is very interesting Smile I love reading and I am always searching for informative information like this.
Great writ-up. Thanks for this deeply insight on this matter.
Great post. Thanks for sharing.
Been a reader for some time now, keep up the great work and hope you have a wonderful new year to come!
Jenny M K
Great writ-up. Happy new year and best Regards MJ
This is an awesome post, I like quality.
In our business it is everything.
Way to go Bob. I’ve hated the idea of “stackoverflow” since the day it was announced. How can you trust an affiliate blogger and a capitalist?
This is an awesome post, I like quality.