A Mess is not a Technical Debt. 475
The term Technical Debt was created by Ward Cunningham to describe the engineering trade-off’s that software developers and business stakeholders must often make in order to meet schedules and customer expectations. In short, you may need to use suboptimal designs in the short term, because the schedule does not allow longer term designs to be used. As a simple example, your initial website design may need to be frames based because you don’t have time to build an Ajax framework.
Clearly this causes a debt. If the customer is looking for a web 2.0 system, then frames just aren’t going to cut it for long. So time is going to have to be carved out of a future schedule to refit the system with an Ajax solution.
In short, the business has decided that it can afford to delay release 2 in order to accelerate release 1. Is this wise?
Businesses make this kind of trade-off all the time; and there’s nothing inherently unwise about it. If the early release of 1.0 drives the business that pays for the development of 2.0 then the business has won. So this kind of reasoned technical debt may indeed be appropriate.
Unfortunately there is another situation that is sometimes called “technical debt” but that is neither reasoned nor wise. A mess.
Technical debt may be necessary, but it had also better be clean! If you are going to implement a frames solution instead of an AJAX solution, then make sure that the workmanship of the frames solution is top-notch. Make sure the design is well balanced, and the code is clean. If you make a mess while implementing that frames solution, you’ll never be able to replace it with an AJAX framework. The mess will impede your progress forever.
A mess is not a technical debt. A mess is just a mess. Technical debt decisions are made based on real project constraints. They are risky, but they can be beneficial. The decision to make a mess is never rational, is always based on laziness and unprofessionalism, and has no chance of paying of in the future. A mess is always a loss.
When you buy a house and take on a big mortgage debt, you tighten up all your spending and accounting. You clean up your books and your budgets. You behave with increased discipline. The same is true of technical debt. The more technical debt you take on, the tighter your disciplines need to be. You should do more testing, and more pairing and more refactoring. Technical debt is not a license to make a mess. Technical debt creates the need for even greater cleanliness.
When you decide to take on a technical debt, you had better make sure that your code stays squeaky clean. Keeping the system clean is the only way you will pay down that debt.
One Thing: Extract till you Drop. 203
For years authors and consultants (like me) have been telling us that functions should do one thing. They should do it well. They should do it only.
The question is: What the hell does “one thing” mean?
After all, one man’s “one thing” might be someone else’s “two things”.
Consider this class:class SymbolReplacer { protected String stringToReplace; protected List<String> alreadyReplaced = new ArrayList<String>(); SymbolReplacer(String s) { this.stringToReplace = s; } String replace() { Pattern symbolPattern = Pattern.compile("\\$([a-zA-Z]\\w*)"); Matcher symbolMatcher = symbolPattern.matcher(stringToReplace); while (symbolMatcher.find()) { String symbolName = symbolMatcher.group(1); if (getSymbol(symbolName) != null && !alreadyReplaced.contains(symbolName)) { alreadyReplaced.add(symbolName); stringToReplace = stringToReplace.replace("$" + symbolName, translate(symbolName)); } } return stringToReplace; } protected String translate(String symbolName) { return getSymbol(symbolName); } }
It’s not too hard to understand. The replace() function searches through a string looking for $NAME and replaces each instance with the appropriate translation of NAME. It also makes sure that it doesn’t replace a name more than once. Simple.
Of course the words “It also…” pretty much proves that this function does more than one thing. So we can probably split the function up into two functions as follows:
String replace() { Pattern symbolPattern = Pattern.compile("\\$([a-zA-Z]\\w*)"); Matcher symbolMatcher = symbolPattern.matcher(stringToReplace); while (symbolMatcher.find()) { String symbolName = symbolMatcher.group(1); replaceAllInstances(symbolName); } return stringToReplace; } private void replaceAllInstances(String symbolName) { if (getSymbol(symbolName) != null && !alreadyReplaced.contains(symbolName)) { alreadyReplaced.add(symbolName); stringToReplace = stringToReplace.replace("$" + symbolName, translate(symbolName)); } }
OK, so now the replace() function simply finds all the symbols that need replacing, and the replaceAllInstances() function replaces them if they haven’t already been replaced. So do these function do one thing each?
Well, the replace() compiles the pattern and build the Matcher() Maybe those actions should be moved into the constructor?
class SymbolReplacer { protected String stringToReplace; protected List<String> alreadyReplaced = new ArrayList<String>(); private Matcher symbolMatcher; private final Pattern symbolPattern = Pattern.compile("\\$([a-zA-Z]\\w*)"); SymbolReplacer(String s) { this.stringToReplace = s; symbolMatcher = symbolPattern.matcher(s); } String replace() { while (symbolMatcher.find()) { String symbolName = symbolMatcher.group(1); replaceAllInstances(symbolName); } return stringToReplace; } private void replaceAllInstances(String symbolName) { if (getSymbol(symbolName) != null && !alreadyReplaced.contains(symbolName)) { alreadyReplaced.add(symbolName); stringToReplace = stringToReplace.replace("$" + symbolName, translate(symbolName)); } } protected String translate(String symbolName) { return getSymbol(symbolName); } }
OK, so now certainly the replace() function is doing one thing? Ah, but I see at least two. It loops, extracts the symbolName and then does the replace. OK, so how about this?
String replace() { for (String symbolName = nextSymbol(); symbolName != null; symbolName = nextSymbol()) replaceAllInstances(symbolName); return stringToReplace; } private String nextSymbol() { return symbolMatcher.find() ? symbolMatcher.group(1) : null; }
I had to restructure things a little bit. The loop is a bit ugly. I wish I could have said for (String symbolName : symbolMatcher) but I guess Matchers don’t work that way.
I kind of like the nextSymbol() function. It gets the Matcher nicely out of the way.
So now the replace() and nextSymbol() functions are certainly doing one thing. Aren’t they?
Well, I suppose I could separate the loop from the return in replace().
String replace() { replaceAllSymbols(); return stringToReplace; } private void replaceAllSymbols() { for (String symbolName = nextSymbol(); symbolName != null; symbolName = nextSymbol()) replaceAllInstances(symbolName); }
I don’t see how I could make these functions smaller. They must be doing one thing. There’s no way to extract any other functions from them!
Uh… Wait. Is that the definition of one thing? Is a function doing one thing if, and only if, you simply cannot extract any other functions from it? What else could “one thing” mean? After all, If I can extract one function out of another, the original function must have been doing more than one thing.
So does that mean that for all these years the authors and consultants (like me) have been telling us to extract until you can’t extract anymore?
Let’s try that with the rest of this class and see what it looks like…class SymbolReplacer { protected String stringToReplace; protected List<String> alreadyReplaced = new ArrayList<String>(); private Matcher symbolMatcher; private final Pattern symbolPattern = Pattern.compile("\\$([a-zA-Z]\\w*)"); SymbolReplacer(String s) { this.stringToReplace = s; symbolMatcher = symbolPattern.matcher(s); } String replace() { replaceAllSymbols(); return stringToReplace; } private void replaceAllSymbols() { for (String symbolName = nextSymbol(); symbolName != null; symbolName = nextSymbol()) replaceAllInstances(symbolName); } private String nextSymbol() { return symbolMatcher.find() ? symbolMatcher.group(1) : null; } private void replaceAllInstances(String symbolName) { if (shouldReplaceSymbol(symbolName)) replaceSymbol(symbolName); } private boolean shouldReplaceSymbol(String symbolName) { return getSymbol(symbolName) != null && !alreadyReplaced.contains(symbolName); } private void replaceSymbol(String symbolName) { alreadyReplaced.add(symbolName); stringToReplace = stringToReplace.replace( symbolExpression(symbolName), translate(symbolName)); } private String symbolExpression(String symbolName) { return "$" + symbolName; } protected String translate(String symbolName) { return getSymbol(symbolName); } }
Well, I think it’s pretty clear that each of these functions is doing one thing. I’m not sure how I’d extract anything further from any of them.
Perhaps you think this is taking things too far. I used to think so too. But after programming for over 40+ years, I’m beginning to come to the conclusion that this level of extraction is not taking things too far at all. In fact, to me, it looks just about right.
So, my advice: Extract till you just can’t extract any more. Extract till you drop.
After all, with modern tools it takes very little time. It makes each function almost trivial. The code reads very nicely. It forces you to put little snippets of code into nicely named functions. And, well gosh, extracting till you drop is kind of fun!
As the tests get more specific, the code gets more generic. 98
I tweeted this not too long ago. The basic idea is that as you add tests, the tests get more and more specific. This makes sense since tests are, after all, specifications. The more specifications you have, the more specific the whole body of specifications becomes.
As a general rule, good design dictates that the more specific your requirements become, the more general your code needs to be. This is saying roughly the same thing as Greenspun’s Tenth Rule of Programming: “Any sufficiently complicated [...] program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp.” Or rather, as more and more constraints pile upon a program, the designers look for ways to push those constraints out of the program itself and into the data.
In return for my tweet people asked for examples.
One of the better examples (though perhaps a bit trivial) is the Prime Factors Kata. This lovely little experiment grows and grows as you add test cases, and then suddenly collapses into an elegant three line algorithm.
The tests continue to become ever more specific. The production code starts out just as specific as the tests. But with the second or third test the programmer must make a decision. He can write the production code to mirror the tests (i.e. writing it as an if/else statement that detects which test is running and supplying the expected answer) or he can come up with some kind of more general algorithm that satisfies the tests without looking anything like them.
The algorithm grows and warps and twists; and then, just when it looks like it’s destined to become a wretched mess; it simply evaporates into a lovely little three line nested loop.
We see the principle at work in other ways as well. Often the programmers have a whole list of tests that they know must pass. As they write them one by one, they write the production code that satisfies them. Then, as in the Bowling Game Kata the tests start to pass unexpectedly. You were done with the code, and you weren’t aware of it. You continue writing tests, expecting one to fail, but they all pass. The test code grows, but the production code remains the same.
Sometimes this happens in a less surprising way. Sometimes you know that you have implemented an algorithm that will pass all remaining tests, but you write those tests anyway because they are part of the specification
The point is that test code and production code do not grow at the same rate. Indeed, as the application increases in complexity, the test code grows at a rate that is faster than the production code. Sometimes the production code actually shrinks as the test code grows because the programmers moved a load of functionality out of the code and into the data.
Consider FitNesse. A year ago there were 45,000 lines of code, of which 15,000 were tests, so 33% of the total were tests.
Now Fitnesse is 58,000 lines of code of which 26,000 are tests. We added 13,000 lines of code overall, but 8,000 (61%), are tests! The tests have grown to over 44% of the total.
Uncle Bob, JSPS: Learning Clojure 120
(JSPS: Just Some Poor Schmuck)
I’m trying to learn Clojure, and I’m finding the effort challenging. Perhaps you can help me.
Programming in Clojure (A derivative of Lisp) requires a certain “inside-out” thinking. I’m finding the combination of the shift in thought process and TDD quite difficult to resolve. So I thought I’d ask for your help.
Below is the Bowling Game written in Clojure complete with unit tests. I’m not at all pleased with the result. The tests, especially, look bizzare and backwards to me.
Please post your own solutions to this so that we can all compare and contrast them.
Tests
(ns bowling-game) (use 'clojure.contrib.test-is) (use 'bowling-game) (defn roll-list [game list] (if (empty? list) game (roll-list (roll game (first list)) (rest list)) )) (defn roll-many [game n pins] (loop [i n g game] (if (zero? i) g (recur (dec i) (roll g pins))))) (defn gutter-game [game] (roll-many game 20 0)) (deftest can-create-game (is (not (nil? (new-game))))) (deftest gutter-game-should-score-0 (is (= 0 (score (gutter-game (new-game)))))) (deftest all-ones-should-score-20 (is (= 20 (score (roll-many (new-game) 20 1))))) (deftest one-spare (is (= 16 (score (roll-many (roll-list (new-game) [5 5 3]) 17 0))))) (deftest one_strike (is (= 24 (score (roll-many (roll-list (new-game) [10 3 4]) 16 0))))) (deftest perfect-game (is (= 300 (score (roll-many (new-game) 12 10))))) (run-tests 'bowling-game)
Code
(ns bowling-game) (defn new-game [] []) (defn next-two-balls [rolls] (+ (rolls 0) (rolls 1))) (defn score-strike [rolls] [1 (+ 10 (+ (rolls 1) (rolls 2)))]) (defn score-spare [rolls] [2 (+ 10 (rolls 2))]) (defn score-no-mark [rolls] [2 (next-two-balls rolls)]) (defn score-next-frame [rolls] (if (= 10 (first rolls)) (score-strike rolls) (if (= 10 (next-two-balls rolls)) (score-spare rolls) (score-no-mark rolls)))) (defn score [game] (loop [frame 1 rolls game score 0] (if (> frame 10) score (let [frame-score (score-next-frame rolls)] (recur (inc frame) (subvec rolls (frame-score 0)) (+ score (frame-score 1))))))) (defn roll [game pins] (conj game pins))
The Rush 62
There’s nothing like the feeling of achievement when you get a complex software system working. It’s the feeling of the hunter making a hard fought kill. By the sheer power of your intellect you have imposed your will upon inanimate nature, and forced it to do your bidding. You feel the rush of power, the satisfaction of victory. You are the master!
That’s the good news.
The bad news is that you’ve made a mess of things along the way.
This is inevitable. It takes a great deal of focus and endurance to get a system working just right. While you are consumed by that focus, you have little room for niceties like small functions, nice names, decoupling, and cleanliness.
My apprentice and I just finished achieving a major milestone in FitNesse. It used to be that each new release of FitNesse was contained in a zip file that had all the files and directories laid out just perfectly. This is great for a brand new installation, but doesn’t work so well when you are upgrading. What we managed to get working last night was the ability for FitNesse to self-install from it’s jar file. Now, whether you are installing for the first time, or just upgrading, FitNesse will install itself into your environment appropriately.
If you’ve ever written a self-install like this, you know there are lots of niggling little details to attend to. While the problem is conceptually simple, the realization is a maze of complexity.
Last night, with dinner waiting on the table and getting cold, we got the last little problem licked, and saw the whole self-installation work as we wanted. We congratulated each other with a high-five, and then went upstairs to eat a well-deserved meal. Later that evening I went down to the office and checked in the code.
This morning I woke up, and while in the shower, I realized that we had a lot of work left to do. We need to go back over all that code and clean it up. I’m sure we left a swath of detritus while on the hunt.
The rush of victory should never be confused with the cold and crystalline concept of done. Once you get your systems to work, you still have to go back and clean up the wreckage left behind by the victorious battle. You are not done until the victorious code has been cleaned, polished, and oiled.
Clean Code and Battle Scarred Architecture 89
If you go here you’ll see that I struggle to keep the CRAP out of FitNesse. Despite the whimsical name of this metric (which I take a perverse delight in), I find it to be remarkably useful.
CRAP is a metric that is applied to each function in the system. The formula for CRAP is: CRAP = comp(f)2. X (1 – cov(f)/100)3. + comp(f)
Where comp(f) = the cyclomatic complexity of the function f. and cov(f) = the unit test coverage of function f.
So a function’s CRAP will be small iff the cyclomatic complexity is low and the test coverage is high. CRAP will be huge if cyclomatic complexity is high, and there is no coverage.
What does this have to do with architecture? Read on…
I work very hard to keep the ratio of crappy methods near .1%. Of the 5643 methods in FitNesse, only 6 are crappy, and five of those I have no control over.
If you study the graph you can see how quickly I react to even the slightest uptick in crap. I don’t tolerate it because it means that either I’ve got a horrifically complex method that needs to be refactored, or (and this is far more likely) I’ve got a method that isn’t sufficiently tested.
Why am I so fastidious about this? Why am I so concerned about keeping the crap out of FitNesse? The reason is pretty simple. It’s the least I can do.
If you look inside of FitNesse, you’ll find that there are lots of structures and decisions that don’t seem to make a lot of sense at first reading. There are complexities and abstractions that will leave you shaking your head.
For example. We generate all our HTML in code. Yes, you read that correctly. We write Java code that constructs HTML. And yes, that means we are slinging angle brackets around.
To be fair, we’ve managed to move most of the angle bracket slingers into a single module that hides the HTML construction behind an abstraction barrier. This helps a lot, but cripe who would sling angle brackets when template system are so prevalent? I hope nobody. But FitNesse was not conceived at a time when template systems made sense (at least to us).
Fear not, I am working through the Fitnesse code replacing the HTML generation with Velocity templates. It’ll take some time, but I’ll get it done. The point is, that just like every other software system you’ve seen, FitNesse is a collection of historical compromises. The architecture shows the scars of many decisions that have since had to be reversed or deeply modified.
What does this have to do with CRAP? Simply this. The battle scarred architecture is something that will never really go away. I can stop the bleeding, and disinfect the wounds, but there will always be evidence of the battle.
That scarring makes the system hard to understand. It complicates the job of adding features and fixing bugs. It decreases the effectiveness of the developers who work on FitNesse. And though I work hard to massage the scars and bandage the wounds, the war goes on.
But I can keep the CRAP out! I can keep the code so clean and simple at the micro level, that the poor folks who try to make sense out of the macro scale aren’t impeded by huge deeply nested functions that aren’t tested!
Think of it this way. CRAP is disease at the cellular level. CRAP is a rot so pervasive that it can infest every nook and cranny of the system. My system may have scars, but it’s not diseased! In fact, despite the evidence of battles long past, the FitNesse code is very healthy.
And I aim to keep it that way.
Crap Code Inevitable? Rumblings from ACCU. 379
I gave the opening Keynote at ACCU 2009 on Wednesday. It was entitled: The Birth of Craftsmanship. Nicolai Josuttis finshed the day with the closing keynote: Welcome Crappy Code – The Death of Code Quality. It was like a blow to the gut.
In my keynote I attempted to show the historical trajectory that has led to the emergence of the software craftsmanship movement. My argument was that since the business practices of SCRUM have been widely adopted, and since teams who follow those practices but do not follow the technical practices of XP experience a relentless decrease in velocity, and since that decrease in velocity is exposed by the transparency of scrum, then if follows that the eventual adoption of those technical XP practices is virtually assured. My conclusion was that Craftsmanship was the “next big thing” (tm) that would capture the attention of our industry for the next few years, driven by the business need to increase velocity. (See Martin Fowler’s blog on Flaccid Scrum) In short, we are on a trajectory towards a higher degree of professionalism and craftsmanship.
Nicolai’s thesis was the exact opposite of mine. His argument was that we are all ruled by marketing and that businesses will do whatever it takes to cut costs and increase revenue, and therefore businesses will drive software quality inexorably downward. He stipulated that this will necessarily create a crisis as the defect rates and deadline slips increased, but that all attempts to improve quality would be short lived and followed by a larger drive to decrease quality even further.
Josuttis’ talk was an hour of highly depressing rhetoric couched in articulate delivery and brilliant humor. One of the more memorable moments came when he playacted how a manger would respond to a developer’s plea to let them write clean code like Uncle Bob says. The manager replies: “I don’t care what Uncle Bob says, and if you don’t like it you can leave and take Uncle Bob with you.”
One of the funnier moments came when Josuttis came up with his rules for crap code, one of which was “Praise Copy and Paste”. Here he showed the evolution of a module from the viewpoint of clean code, and then from the viewpoint of copy-paste. His conclusion, delivered with a lovely irony, was the the copy-paste solution was more maintainable because it was clear which code belonged to which version.
It was at this point that I thought that this whole talk was a ribald joke, an elaborate spoof. I predicted that he was about to turn the tables on everyone and ringingly endorse the craftsmanship movement.
Alas, it was not so. In the end he said that he was serious about his claims, and that he was convinced that crap code would dominate our future. And then he gave his closing plea which went like this:
We finally accepted that requirements change, and so we invented Agile.
We must finally accept that code will be crap and so we must ???
He left the question marks on the screen and closed the talk.
This was like a blow to the gut. The mood of the conference changed, at least for me, from a high of enthralled geekery, to depths of hoplessness and feelings of futile striving against the inevitable. Our cause was lost. Defeat was imminent. There was no hope.
Bulls Bollocks!
To his credit, there are a few things that Josuttis got right. There is a lot of crap code out there. And there is a growing cohort of crappy coders writing that crap code.
But the solution to that is not to give up and become one of them. The solution to that is to design our systems so that they don’t require an army of maintainers slinging code. Instead we need to design our systems such that the vast majority of changes can be implemented in DSLs that are tuned to business needs, and do not require “programmers” to maintain.
The thing that Josuttis got completely wrong, in my mildly arrogant opinion, is the notion that low quality code is cheaper than high quality code. Low quality code is not cheaper; it is vastly more expensive, even in the short term. Bad code slows everyone down from the minute that it is written. It creates a continuous and copious drag on further progress. It requires armies of coders to overcome that drag; and those armies must grow exponentially to maintain constant velocity against that drag.
This strikes at the very heart of Josuttis’ argument. His claim that crappy code is inevitable is based on the notion that crappy code is cheaper than clean code, and that therefore businesses will demand the crap every time. But it has generally not been business that has demanded crappy code. Rather it has been developers who mistakenly thought that the business’ need for speed meant that they had to produce crappy code. Once we, as professional developers, realize that the only way to go fast is to create clean and well designed code, then we will see the business’ need for speed as a demand for high quality code.
My vision of the future is quite different from Josuttis’. I see software developers working together to create a discipline of craftsmanship, professionalism, and quality similar to the way that doctors, lawyers, architects, and many other professionals and artisans have done. I see a future where team velocities increase while development costs decrease because of the steadily increasing skill of the teams. I see a future where large software systems are engineered by relatively small teams of craftsmen, and are configured and customized by business people using DSLs tuned to their needs.
I see a future of Clean Code, Craftsmanship, Professionalism, and an overriding imperative for Code Quality.
Is the Supremacy of Object-Oriented Programming Over? 230
I never expected to see this. When I started my career, Object-Oriented Programming (OOP) was going mainstream. For many problems, it was and still is a natural way to modularize an application. It grew to (mostly) rule the world. Now it seems that the supremacy of objects may be coming to an end, of sorts.
I say this because of recent trends in our industry and my hands-on experience with many enterprise and Internet applications, mostly at client sites. You might be thinking that I’m referring to the mainstream breakout of Functional Programming (FP), which is happening right now. The killer app for FP is concurrency. We’ve all heard that more and more applications must be concurrent these days (which doesn’t necessarily mean multithreaded). When we remove side effects from functions and disallow mutable variables, our concurrency issues largely go away. The success of the Actor model of concurrency, as used to great effect in Erlang, is one example of a functional-style approach. The rise of map-reduce computations is another example of a functional technique going mainstream. A related phenomenon is the emergence of key-value store databases, like BigTable and CouchDB, is a reaction to the overhead of SQL databases, when the performance cost of the Relational Model isn’t justified. These databases are typically managed with functional techniques, like map-reduce.
But actually, I’m thinking of something else. Hybrid languages like Scala, F#, and OCaml have demonstrated that OOP and FP can complement each other. In a given context, they let you use the idioms that make the most sense for your particular needs. For example, immutable “objects” and functional-style pattern matching is a killer combination.
What’s really got me thinking that objects are losing their supremacy is a very mundane problem. It’s a problem that isn’t new, but like concurrency, it just seems to grow worse and worse.
The problem is that there is never a stable, clear object model in applications any more. What constitutes a BankAccount
or Customer
or whatever is fluid. It changes with each iteration. It’s different from one subsystem to another even within the same iteration! I see a lot of misfit object models that try to be all things to all people, so they are bloated and the teams that own them can’t be agile. The other extreme is “balkanization”, where each subsystem has its own model. We tend to think the latter case is bad. However, is lean and mean, but non-standard, worse than bloated, yet standardized?
The fact is, for a lot of these applications, it’s just data. The ceremony of object wrappers doesn’t carry its weight. Just put the data in a hash map (or a list if you don’t need the bits “labeled”) and then process the collection with your iterate, map, and reduce functions. This may sound heretical, but how much Java code could you delete today if you replaced it with a stored procedure?
These alternatives won’t work for all situations, of course. Sometimes polymorphism carries its weight. Unfortunately, it’s too tempting to use objects as if more is always better, like cow bell.
So what would replace objects for supremacy? Well, my point is really that there is no one true way. We’ve led ourselves down the wrong path. Or, to be more precise, we followed a single, very good path, but we didn’t know when to take a different path.
Increasingly, the best, most nimble designs I see use objects with a light touch; shallow hierarchies, small objects that try to obey the Single Responsibility Principle, composition rather than inheritance, etc. Coupled with a liberal use of functional idioms (like iterate, map, and reduce), these designs strike the right balance between the protection of data hiding vs. openness for easy processing. By the way, you can build these designs in almost any of our popular languages. Some languages make this easier than others, of course.
Despite the hype, I think Domain-Specific Languages (DSLs) are also very important and worth mentioning in this context. (Language-Oriented Programming – LOP – generalizes these ideas). It’s true that people drink the DSL Kool-Aid and create a mess. However, when used appropriately, DSLs reduce a program to its essential complexity, while hiding and modularizing the accidental complexity of the implementation. When it becomes easy to write a user story in code, we won’t obsess as much over the details of a BankAccount
as they change from one story to another. We will embrace more flexible data persistence models, too.
Back to OOP and FP, I see the potential for their combination to lead to a rebirth of the old vision of software components, but that’s a topic for another blog post.
Let's Hear it for the Zealots! 88
In his March 1st, 2009 column in SDTimes, Andrew Binstock takes the “Zealots of Agile” to task for claiming that Agile is the one true way. He made the counter-argument that non-agile projects have succeeded, and that agile projects have failed. He implied that the attitudes of the agile zealots are blind to these facts.
What a load of dingoes kidneys!
There is a difference between a zealot, and a religious fanatic. A religious fanatic cannot envision themselves to be wrong. We in the agile community may indeed be zealots, but we know we can be wrong. We know that projects succeeded before agile, and we know that agile projects have failed. We know that there are other schools of thought that are just as valid as ours. Indeed, most of us expect (even hope!) that agile will be superseded by something better one day.
Some of you may remember my keynote address at the 2002 XPUniverse in Chicago. My son, Micah, and I got up in front of the group dressed in Jiu Jistsu garb, and commenced to do a martial arts demonstration.
After the demonstration I made the point that a student of Karate could have given an equally spellbinding, though quite different, show. A student of Tae Kwon Do would amaze us with still other wonderful moves. The morale, of course, was that there are many ways to get the job done. There are many equally valid schools of thought.
However, the knowledge that there are other valid schools of thought does not dilute the zeal a student has for the school of thought he has chosen to study. Do you think that a Jiu Jistsu master isn’t a zealot? You’d better believe he is! To be great, one must have zeal. To rail against zeal is to rail against the desire for greatness. It is zeal that makes you go the extra distance to become great. The Jiu Jitsu master may respect the Karate master, but his goal is to become great at Jiu Jitsu.
So I think it is healthy that agile proponents are vociferous advocates of their school of thought. They are like Karate students who are excited about their new skills, and want to teach those skills to others. If their zeal to excel can infect others with similar zeal, so much the better. We need people in this industry who want to excel.
In his column Andrew derided agile zealots for claiming that Agile was the one true path. I don’t know of any prominent agile advocate who has ever made that claim. Indeed, when Kent Beck and I sat down in 1999 to plan out the XP Immersion training, we were very concerned that we might turn the XP movement into a religion. We did not want to create the impression that XPers were the chosen people and that wisdom would die with us. So we were, in fact, very careful to avoid any hint of the “one true path” argument.
As evidence of this, consider the Snowbird meeting in 2001. That meeting was inclusive, not exclusive. We invited people from all over the spectrum to join us there. DSDM, FDD, Scrum, Crystal, MDA, & RUP were all represented; and the agile manifesto (Yes, Andrew, Manifesto![1]) was the result. That manifesto was a simple statement of values, not a claim to deific knowledge.
Am I an Agile Zealot? Damned right I am! Do I think Agile is the best way to get software done? Damned right I do! I wouldn’t be doing Agile if I didn’t think it was best. Do I think Agile is better than someone else’s school of thought? Let me put it this way: I’m quite happy to go head to head against someone who follows another school of thought. I believe I’d code him right into the ground! Isn’t that what any good student of his craft would think? But I also recognize that one day I’ll lose that competition, and that will be a great day for me because it will give me the opportunity to learn something new and better.
Andrew also mentioned that Kent Beck and I are so obsessed with our “evangelical fervor” that we: “endlessly test and endlessly refine and refactor code to the point that it becomes an all-consuming diversion that does not advance the project”. WTF? I’ve been making monthly releases of FitNesse, and Kent has been cranking out releases of JunitMax; neither of us has been so busy gilding our respective lilies that we are failing to move our respective projects forward. Look around Andrew, we’re shipping code over here brother.
As part of his rant against our “evangelical fervor” Andrew asked this rather snide question: “I broke up that code into a separate class and added seven methods for what benefit, again?” This would appear to be a dig against refactoring – or perhaps excess refactoring. Still the question has a simple answer. We would only break up a module into a class with seven methods if that module was big, ugly, and complex, and by breaking it up we made it simple and easy to understand and modify. It seems to me that leaving the module in a messy state is irresponsible and unprofessional.
Andrew complains that there is very little Agile experience with projects over six million lines of code. I’m not sure where he got that number from, perhaps he’s been watching Steve Austin re-runs. Actually, the majority of my 2008 consulting business has been with very large companies with products well over six million lines of code. Agile-in-the-Large is the name of the game in those companies!
Finally, Andrew says Michael Feathers is a “moderate” Agile proponent. This is the same Michael Feathers who wrote a book defining Legacy code as code without tests – hardly a moderate point of view. Michael is a master, not a moderate.
I think Andrew has fallen into a common trap. He sees the zeal of the agile proponents and mistakes it for religious fanaticism. Instead of respecting that zeal as the outward manifestation of the inward desire to excel, he rails against it as being myopic and exclusive and then views all the statements of the agile proponents through that clouded viewport. Once you have decided that someone is a religious fanatic, it is difficult to accept anything they say.
I think standing against religious fanaticism in the software community is a good thing. I also think that those of us who are zealous must constantly watch that we are not also becoming blind to our own weaknesses and exclusive in our outlook. But I could wish that those who fear religious fanaticism as much as I do would stop confusing it with professional zeal.
So here’s the bottom line. People who excel are, by definition, zealots. People who aren’t zealous, do not excel. So there’s nothing wrong with being a zealot; indeed, zeal is a very positive emotion. The trick to being a zealot is to be mindful that the day will eventually come when some other zealot from a different school of thought will code you into the ground. When that happens, you should thank him for showing you a better way.
1 I kicked off the Snowbird meeting by challenging the group to write a “manifesto”. I chose the word “manifesto” because some years earlier I had read The Object Oriented Database System Manifesto and I thought it was a clever use of the word.
Quality: It's alive! It's ALIVE! 128
Recently James Bach wrote a compelling post entitled Quality is Dead. As much as I’d like to agree, something interesting has just happened that tempts me to believe in a rebirth.
Just when James finally declares the death of quality, along comes the Manifesto for Software Craftsmanship. This simple document that builds upon the four values declared in the Agile Manifesto went live late on Friday evening. Now, at 3PM on Saturday over 600 people have signed it!
What does this mean? I think it means that many of us have seen what James was talking about, and are tired of it. We don’t want our users posting blogs – as James did – crying that they HATE us. We want our users to be astonished by the elegance and simplicity of our creations. We want other software developers to look under the hood and be amazed and impressed at the clarity and simplicity they see.
We want to be proud of our work!
We want our quality back!
And so a few weeks ago some software developers gathered at the offices of 8th Light in Libertyville, Illinois, and began the process of creating this manifesto. There followed a very active and passionate discussion in the Software Craftsmanship email group. The end result was this manifesto.
I encourage all of you to read and sign the manifesto, join the email group, and help bring quality back to life in the software profession.