I’m a sucker for a good one-liner. That one had me thinking, and as I’ve had other conversations about innovation, I keep coming back to that line.

In our Agile practices, we work really hard for a week or two, and then hold a retrospective. The purpose of the retrospective is to find ways to work more effectively for the next two weeks. As we develop better software, we also evolve a better team. We use “tricks” such as tracking our velocity and recording blockages on our ‘waste snake’ to provide data for our decisions, and we use gut feel to evaluate those things that feel like collateral effort to us.

If the practice works, we will see incremental improvement in the team. We will develop ways of avoiding special variations, and we will learn to accept our normal variations. It will make us better at the way we do things now.

XP didn’t come from a series of incremental improvements to the waterfall technology. I wasn’t there when it happened, but it seems that they took on an change in axioms. They didn’t strengthen the contracts between groups, but pulled all the decision-makers onto the same team. They didn’t find more careful ways to examine the code they were changing, but rather decided to lean radically on volumes of tests. They didn’t build practices to improve their anticipatory design, they decided instead not to anticipate at all and simplify their design to allow future change. At the time, this was radical stuff.

I’m sure there have been many other less-successful process mutations, but there is no evolution without mutation.

The man behind the iPod, iPhone, and MacBookPro has had some less-successful product ideas, too. Some exciting high-concept products didn’t make it in the wild. But then some new ideas become category killers.

How do we learn to make the axiomatic changes that lead us to radically better ways to get what we want?

Yes, I know that volumes have been written about TDD and BDD, and that it’s a large topic with many many branches of application, but I didn’t have time for that. I had time for three index cards. I figure that an index card is a token, and it represents a conversation, and that one can always dig deeper later.

They looked more or less like this:

---

Card 1: Uncle Bob’s Three Laws (Object Mentor)

1. Write no production code except to pass a failing test.
2. Write only enough of a test to demonstrate a failure.
3. Write only enough production code to pass the test.

---

Card 2: FIRST Principles (Brett and Tim at Object Mentor)

• Fast: Mind-numbingly fast, as in hundreds or thousands per second.
• Isolated: The test isolates a fault clearly.
• Repeatable: I can run it repeatedly and it will pass or fail the same way each time.
• Self-verifying: The test is unambiguously pass-fail.
• Timely: Produced in lockstep with tiny code changes.

---

Card 3: Flow (using the famous three-node circle diagram) – origin unknown.

• Red: test fails
• Green: test passes
• Refactor: clean code and tests

Sure there is plenty more, but I didn’t know how I could provide significantly less. As is, I’m pretty happy with the exercise. Now I am wondering if I couldn’t produce most of the important information I wish to convey as a series of index cards. Would that be cool or what?

Where to start?

Documents

Why not look at the documentation? <laughs> Just kidding. You know there’s no documentation, and it’s probably not useful.

People and Interactions

You need a partner who is familiar with the code. Having a guide is better than having a map. Not only will you know where to go, but you’ll know ’’how’’ to go and where not to step. Other people are a wonderful resource. Just don’t settle for someone doing the work in front of you… the goal is to learn as much as to do.

No, really, documents!

OTOH, if you have somethng in the way of a summary or architectural overview, that might help. I’d read the abstract and look at the pictures. We’re after big-picture, so read it only, don’t trust it. Documents are seldom accurate, and seldom for long. It might be good preparation for your partner’s visit.

Use The Tests, Luke

Do you have tests? If you are in a test-driven shop, looking at unit tests is a good idea. If they’re written well, they are specifications (by example) of the system. If they’re written poorly, you know where to start your work at least. Your partner could do a great service by helping clean up the code instead of explaining it.

No tests? Uh, oh. No tests and no doc? Now you’re in trouble.

Scan the File Space

Are the classes well-named? Do you know where to begin working? If you can find code by looking at file names then you’ve got the handle you need. Your partner can help improve file naming if you find the code you’ve been handed is in ill-named files.

So you’re in a file, you have a mission. Either you have tests, or you’re ready to write some.

Sense Of Smell

What if the code is still not obvious? You have one more resource before you have to print out the listing and break out the markers to reverse-engineer your way out of code hell. You have your nose and your refactoring editor. You can use refactorings such as ‘rename’, ‘extract method’, and ‘introduce variable’ to clarify an existing method. You can spot duplication and eliminate it. Maybe you can bring it to a point of clarity, and then you will know

Deep Code Spelunking

It’s time to pour a really big cup of tea, get some food, take a couple of preemptive aspirin, and make sure your printer has paper.

Sometimes, you will have to reverse-engineer some of the code you were handed. Your only hope may be to rebuild it from the inside-out.

You might want to either copy the code to a scratch space to do this. Maybe you want to branch the project in version control. You want to be free to dig in an learn this the hard way, and make it better as you emerge from the depths.

You still can/must use refactoring tricks to capture the knowledge you gain, but you may need to take the code off line. And look for new partners.

Imagine my joy as I realized that I could make it 11:30 again, and go enjoy another lunch. Meeting at 3:30? No problem, just turn the hour hand up to 6:00 and go home! I can sleep as long as I want as long as I turn it back to 8:00 when I get to the office. All my work estimates are now “five minutes”, and I complete them every time.

My coworkers have no idea the awesome power that I’ve gained with this one happy accident. They ask “what time is it?” and I say “what time would you like it to be.”

Of course the above is a total fabrication. Pretending it’s 6:00pm when it’s 8:00am isn’t going to do anybody any good at all, and is likely to make a mess of things for me.

But people still try to mandate velocity.

Now I have to be very careful when moving in the drive or sidewalk because of the black ice in some places and thick lumpy stuff in others. I could choose to leave it that way and just walk carefully and try to avoid the area, but I really don’t want it to be like this all winter.

Instead, I bundled up and spent several painful hours trying to clear not only the six to eight inches of new snow, but the ice under it. Clearing the fresh stuff is tough because the rough and lumpy ice is under it. The ice snags the shovel and hurts my elbows and shoulders. I use the metal edge of the shovel to plane off the ice so that it’s not quite so difficult, and the work gets a little easier.

Smooth, planed ice is not a solution. It is actually going to be more slippery and dangerous when the sun hits it and it melts a little. It will then refreeze to an even smoother and slicker sheen. I can’t ignore the problem, and I can’t smooth it over and leave it at that.

I invested in some complicated salt product to lower the melting point of the ice so I can scrape it away. It wasn’t enough, but now there are some safe areas in my driveway. I continue to scrape it and hope that I can get it all squared away so that my wife and kids have safe passage, though it is tedious and unpleasant work.

Yes, it’s my fault. I should have bundled up my sick self or my children and taken care of this when the first snow fell, back when it was easy.

Then it dawned on me that my driveway is a lot like source code. If I always take care of it when it’s relatively easy and not too polluted, it will remain easier to deal with in the longer term. The more unsafe and ugly it is, the more important it is to clean it up. It won’t do to smooth it out a little or to leave it be. It needs to be safe for me and my team.

And I guess that’s my parable for today.

I’m not smart enough to just read it from top to bottom and understand it. Sadly, when I read through and understand what in the heck the thxIniFvt variable is doing, I will forget it by the time I figure out the purpose(es) of pszVbt. I can spend all day, or even a few days to figure out a method, and that’s an admission of feeble-mindedness to be sure. I guess I’m not up to the level of some of the rapid hackers. That’s a limitation I face most days.

I find that I can sometimes understand a method like that only if I just delete all the blank lines and comments first, then reformat to break lines, then inline all methods with seven or more parameters, and then start renaming variables, extracting explanatory variables, and extracting explanatory methods. I may have to break the method into classes even. I guess I’m not one of the smart kids.

I used to be one of the smart kids. I once built a module so complex and fragile that nobody but me could figure out what to do with it. It was all tables and indicators, and stunningly clever. I am so ashamed that I wrote it. It was such a mistake that they eventually disabled it rather than field it in such a state. That was years ago, but so memorable to me. Other programmers said it was like the inside of a swiss watch, all delicate and perfectly balanced, and scary to mess with unless you first knew exactly what each part was doing, and why.

I would like to be faster than I am both mentally and in the sense of quickly producing code. I’d like to be a little less intimidated at the start of a project. .But I would not want those things if it meant building crap that people who are not appreciably more talented than myself would trip over every day. Instead, I sometimes wish I could teach the really fast, smart kids how to dumb down the code for the rest of us morons to read.

The funny thing is that dumbing code to my level doesn’t make it harder for the smart kids to use it, and sometimes allows a compiler to do a better job with it. I guess stupid isn’t so stupid after all.

TDD stands for Test-Driven Development, which is not as clear as TFD (Test-First Development). If the consultant would strive to always say the word “first” in association with testing, most people would more surely grasp the idea. In fact, I’ve begun an experiment in which I will not say the word “test” without the word “first” in close approximation. I’ll let you know how that works out for me.

If the tests are providing nothing more than reassurance on the tail end of a coding phase, then the tests aren’t driving the development. They are like riders instead of drivers. Test-Ridden Development (TRD)[1] would be a better term for such a plan. Even though it is better to have those tail-end tests than to have no automated testing, it misses the point and could not be reasonably be called TDD.

An old mantra for TDD and BDD is “it’s not about testing”. The term BDD was invented largely to get the idea of “testing” out of the way. People tend to associate “test” as a release-preparation activity rather than an active approach to programming. BDD alleviates some of that cognitive dissonance.

In TDD, tests come first. Each unit test is written as it is needed by the programmer. Tests are just-in-time and are active in shaping the code. Acceptance Tests likewise tend to precede programming by some short span of time. [2]

Through months of repetition I have developed the mantra:

TDD isn’t a task. It is not something you do. It is an approach. It is how you write your programs.

I wonder if we shouldn’t resurrect the term Test-First Programming or Test-First Development for simple evocative power. Admittedly there are some who would see that as a phase ordering, but maybe enough people would get the right idea.

Brett Schuchert(with some trivial aid from your humble blogger) has worked up an acronym to help socialize the basic concepts which are somehow being lost in translation to the corporate workplace.

The teaser: Fast, Isolated, Repeatable, Self-validating, and Timely.

As a reader of this blog, you are probably very familiar with all of the terminology and concepts behind TDD. I beg of you, socialize the idea that testing comes first and drives the shape of the code. If we can just get this one simple idea spread into programming dens across our small spheres of influence, then we will have won a very great victory over Test-Ridden Development.

“And there was much rejoicing.”

1 Jeff Langr will refer to this TRD concept as “Test-After-Development”, which he follows with a chuckle and a twinkle, “which is a TAD too late.”

2 Of course, one still needs QC testing as well, however TDD is about driving development, not testing its quality post-facto.

I have always looked at the STL as a way of implementing a class, but never as a replacement for “real” classes with named member values. In some ways, I think that templates have made our C++ code worse.

For instance, I need a two-member structure. Say I am going to store Name and Address. Now, those with C background would immediately think along these lines:

    struct name_and_address { char name[34]; struct address address; };
    struct name_and_address get_name_and_address(void);
    // blah blah blah

    struct name_and_address x = get_name_and_address();
    printf("%s", x.name );
    printf("%s", x.address.line1);
    // ... and so on...
    printf("%05d", x.address.zip);

Got to love those old C programmers. There is a nice, primitive struct, maybe with a nice, primitive struct embedded in it. Look at the clear names of those variables. It is even pretty efficient to copy these things. With a Plain Old Data Object, the compiler knows what you’re doing and can do block copies instead of memberwise copies. That can be very efficient, enough that you can often pass structures around by value instead of by reference. That’s kinda cool. That same efficiency also applies to plain old data classes in C++, but a lot of people don’t know it.

Sadly, a modern C++ developer does what? Yeah, he returns a pair. This is annoying because when I call my function, I have to know that the name is now called “first” and the address is called “second”. But that is only for this pair-returning function. In another pair-returning function, “first” means birthday and “second” is account number. Of course the prototype is not always in view, so I have to have more tiles in my window to check my work as I code.

Pair is named well enough for an arbitrary pair of primitive values, but frankly it makes no sense for x,y coordinates or names and address, or for the rolls of two twenty-sided dice. Wait. For the dice, it works just fine. After all, what would you call them? Dice1 and Dice2? But I digress. The point is that for most two-member structures, a two-member structure with named members is probably a better choice.

Now pair is not alone (oh look, a pun). Even when we are dealing with collections, the interface to a vector, map, or set is unlikely to be the right set of names and semantics for our problem domain objects. Maybe our property should be listed, and not push_back()-ed. Maybe we care about a policy, that:

        policy.covers(theCar)

Maybe we’re not so interested in whether list x will:

        x.find(z.first) != x.end()

The fascination with exposing STL containers (combined with unfortunate naming, as given here for effect) is hurting readability. It also is a very unfortunate form of primitive obsession and a violation of the Dependency Inversion Principle.

The people who work on your code with you should not have to reverse engineer your work in order to add new features or correct bugs. The code should be written so clearly that mistaken assumptions will be more obvious. They should not have to chase down typedefs and keep notes on paper to keep track of your structures. They should especially not have to look for data in obliquely named pairs of pairs (x.second.first means “price”? Or was that x.first.second? Or…).

If you write your code with a pair programming partner, you should get plenty of feedback about the readability of your code. Even if you do not work in pairs, you should read your tests when you get back from lunch or a meeting, and you should see if primitive containers, primitive member functions, and template syntax are obscuring their purpose. If your tests don’t speak clearly, then none of the other users of your class can speak clearly. Get the syntax out of their way by giving them reasonable wrapper functions.

You might think that wrapping your containers will cost you performance by adding dispatch here and there, but you might be surprised to find out that not wrapping it may cost you far more by concealing bugs and obscuring functionality than it ever save you in performance. A better policy is to keep your code obvious and simple, and try a little profiling and optimization where it really matters.

Of course, if one was worried about the cost of one dispatch for a named method v. an exposed container, there is always the option to build an inline member function in the header.

I’m mindful that this is not the time for me to rant about C++ programmers who never profile, and who manage performance by rule of thumb and old wives’ tales. There are more of them than us, my dear reader, and I fear their wrath. Maybe some day later, when I’m feeling brave.

My roots run pretty deep into C++, and I appreciate how hard it is to produce a system. I just want the guys who are now running with the ball right now to realize that it is far more valuable to be clear and obvious than they appreciate, and it is far less expensive to do so than they fear.

I guess the points are:

• Don’t be afraid to write small classes and structs.
• Don’t be afraid to wrap your containers.
• Don’t be afraid to use abstraction.
• Everything does not have to be written in templates.
• Be afraid to write test code that doesn’t tell a story.
• Be afraid to give too much access to your data by exposing containers.

Adjust your values accordingly.

1. Interfaces suddenly seem like a really good idea when you have to start introducing test doubles of various flavors (mocks, etc). You will find yourself creating interfaces in places where you probably would not have created them before. You’ve always known that you shouldn’t depend or derive from concrete classes, and now you are feeling the pain of non-trivial concrete classes. You are forced, more or less, to comply with the Dependency Inversion Principle. Abstraction becomes a way of life.

2. Singletons and static methods no longer seem like a great way to do work (esp in C++) because you can’t easily isolate a module that makes use of them. The exception is when a static method is totally self-contained and testable, and does not use resources like a database, file system, etc. Only the very trivial can be static. Test doubles matter that much. Suddenly substitutability is a primary measure of code goodness.

3. Private makes less sense than it used to. You can’t test anything that’s private. You need to have ways to sense that your tests are working as intended, and you have to be able to test any method that is interesting. That means less private and more accessors. Get used to living in a more public world. If you need to hide something from users, don’t include it in the interface or abstract base class. “Implements”/”public inheritance” is the new “private”.

4. You need to be able to pass a class everything it might need at construction time, so that you can pack it a stubbed logger and a faked database access module. Otherwise, you will not have the isolation you need to write good unit tests. Calling a concrete class constructor inside the body of a method now makes you cringe, because you realize you’ve missed an opportunity for abstraction, and limited the isolation of your method. The fat constructor argument list is a concession to the need for isolation and testing with mocks (and fakes, and stubs).

5. Smaller methods are the norm. It is hard to test a two-hundred line method. It’s far harder than testing a many smaller, equivalent methods. Whatever is easier to test is easier to write. Large is bad. This has always been so, since optimizers love simple functions, but now it’s real to you as well.

6. You hate inheriting code without tests, because you don’t want to reverse-engineer what the other programmer was thinking. In fact, you find yourself looking at tests before you look at code, because you know the tests are better than comments. Testing becomes a kind of “code clarity” mechanism. The profound bit is that code is more clear if it is more testable, even if you have to “pollute” the class to make it testable by moving variables and methods out of the private space or by creating interfaces, or by using fat constructor argument lists. Testability is the new legibility.

7. Hard-to-use class interfaces are now hard for you to use, not just hard for other people. You have to write the tests, so you have to use the class. Usability determines how much you will like your own code, not cleverness.

8. Performance management by old wives tales is dead. You will build for testability, and then use measurement tools to improve performance. This is what the wizened greybeards have been telling us for a lot of years, and with TDD you start listening. You stop avoiding trivial issues like copying PODS and virtual dispatches. You pay attention to more important issues. With optimization, YAGNI applies until it doesn’t, and when it doesn’t you need to know why and where.

9. You care deeply how long it takes to run the tests because you need to be able to run all of the tests all the time, after only a few lines of code are written, and after each refactoring. If it’s taking more than 20 or 30 seconds to run all your unit tests, you start looking for ways to speed it up. You must maintain test performance, because it is key to productivity.

10. Dependency hurts. You can’t afford ‘god classes’ and ‘global hubs’. If test setup becomes a chore, you immediately start reaching for the axe. You have to keep your modules isolated better than ever before. Of course, you should have done this all along, but you never felt it this keenly.

11. “Clever” is dead. Clever is hard to refactor. Clever is hard to isolate, hard to internalize, hard to phrase in tests. One point of “obvious” is worth two hundred points of “clever”.

12. Your IDE is good to the extent that it allows you to do quick write/build/test cycles. You need to do several per minute.

For the most part, TDD forces you to start doing the things you always should have done, though it signals a change in values that should be reflected in your coding standards.

Maybe you should get a highlighter and mark each section of your coding standard or architectural document that encourages any practice that in turn discourages testability. The world has changed, values have shifted, and your style guide needs to change if it is to remain relevant.