I visit a lot of different companies and see a lot of absolutely ghastly code. What I was looking at here wasn’t all that bad. Variable names were not chosen to limit keystrokes and method names appeared to be descriptive. This was good news, as I needed to understand how this was put together before I could offer help with a testing strategy.

As we walked through the code, I noticed that there were classes in the project directory ending in ‘Test’. This took me by surprise. Usually when I’m helping people with legacy code issues, there aren’t any tests. Here, there were tests in place and they actually passed. Very cool, but now my mission wasn’t clear to me as I thought my help was needed getting tests in place around legacy code.

The developers clarified that they wanted help in testing private methods. Ah ha, the plot thickens.

The question of testing private methods comes up frequently whether I’m teaching a class or consulting on a project. My first response is a question. “Is the private method being tested through the public interface to the class?” If that’s the case, then there’s nothing to worry about and I can steer the conversation away from testing private methods to testing behaviors of a class instead of trying to test individual methods. Note that a private method being tested through its public interface would be guaranteed if the class was developed TDD style where the test is written first, followed by one or more public methods to make the test pass, followed by one or more extract method refactorings, which would be the birth of the private methods. This is almost never the case. My client didn’t know how the code was developed, but by inspection they concluded that the parameters of the test were adequately exercising the private method.

It looked like my work was done here. But not so fast.

I have a policy that says that whenever I have code up in an editor, I have to try to leave it just a little bit better than when I found it. Since we had put the testing matter to rest and we still had some time left in the conference room before another meeting started, I suggested that we see if we could make some small improvements to the code we were examining.

As I said earlier, the code wasn’t horrible. The code smell going past my nostrils was Long Method and the cure was Extract Method.

The overall structure of the method we were examining was

    
    if( conditional_1 )
    {
        // do lots of complicated stuff
    }
    else if( conditional_2 )
    {
        // do even more complicated stuff
    }
    else
    {
        // do stuff so complicated nobody understood it
    }
    

where conditional_1 was some horribly convoluted expression involving lots of &&’s, ||’s, and parentheses. Same for condition_2, which also had a few ^’s thrown in for good luck. To understand what the method did, one would have to first understand the details of how the method did it.

I asked the developers if they could come up with a nice, descriptive method name that described what I’m calling condition_1 so that we could do an extract method refactoring and the code would look like:

    
    if( descriptive_name() )
    {
        // do lots of complicated stuff
    }
    // etc
    

Now there were less details to understand when trying to determine what this method did. If we were to stop here and check in the code, we could leave the session feeling good as the code is better than when we started. But we still had time before we had to abandon our conference room so I pressed on.

“Can you summarize what this code does as a descriptive method name,” I asked. The developers pondered a few moments and came up with what they felt was a good name. Excellent. We did the same procedure for the “else if” clause. When we finished that, one of the developers said something along the lines of, “That was the easy part, I have no idea what this [else] remaining code does.” I was going to pat everybody on the back and call it a day because the code had improved tremendously from when we started, but the developers seemed to have a “we can’t stop now” attitude. They studied the code, pondered, cursed, discussed some more, and then one of them said, “This code can never execute!”

I’d like to use the expression, “You could have heard a pin drop,” to describe the silence in the room, but since there were only three of us, the phrase looses its power. As it turns out, now that the if() and else if() conditionals were given descriptive names and people could grok them at a glance, it became obvious that the business rules didn’t permit the final else – the first two conditions were all that could exist and the most complicated code of all was never being called. This was downright comical!

I asked if the tests would still pass if we deleted the code and after a short examination of the tests, the developers weren’t as confident that the test parameters actually hit that area of code. There was a log() statement in that code and one of the developers was going to examine the production logs to see if the code ever executed.

So there you have it, refactor your code and the bad parts just go away!

The issue was simple. Slim tables are, well, tables just like Fit tables are. How should I parse these tables? Fit parses HTML. FitNesse parses wiki text into HTML and delivers it to Fit. Where should Slim fit in all of this?

Keep in mind that Fit must parse HTML since it lives on the far side of the FitNesse/SUT boundary. Fit doesn’t have access to wiki text. Slim tables, on the other hand, live on the near side of the FitNesse/SUT boundary, and so have full access to wiki text, and the built in parsers that parse that text.

So it seemed to me that I had two options.

1. Follow the example of Fit and parse wiki text into HTML, and then have the Slim Tables parse the HTML in order to process the tables.
2. Take advantage of the built in wiki text parser inside of FitNesse and bypass HTML altogether.

I chose the latter of the two because the Parsing system of FitNesse is trivial to use. You just hand it a string of wiki text, and it hands you a nice little parse tree of wiki widgets. All I had to do was walk that parse three and process my tables. Voila!

This worked great! In a matter of hours I was making significant progress on processing Slim decision tables. Instead of worrying about parsing HTML and building my own parse tree, I could focus on the problems of translating tables into Slim directives and then using the return values from Slim to colorize the table.

Generating html was no problem since that’s what FitNesse does anyway. All I had to do was modify the elements of the parse tree and then simply tell the tree to convert itself to HTML. What a dream.

Or so it seemed. Although things started well, progress started to slow before the week was out. The problem was that the FitNesse parser is tuned to the esoteric needs of FitNesse. The parser makes choices that are perfectly fine if your goal is to generate HTML and pass it to Fit, but that aren’t quite so nice when you’re goal is to use the parse tree to process Slim tables. As a simple example, consider the problem of literals.

In FitNesse, any camel case phrase fits the pattern of a wiki word and will be turned into a wiki link. Sometimes, though, you want to use a camel case phrase and you don’t want it converted to a link. In that case you surround the phrase with literal marks as follows: !- FitNesse-!. Anything between literal marks is simply ignored by the parser and passed through to the end.

Indeed, things inside of literals are not even escaped for html! If you put <b>hi</b> into a wiki page, it will escape the text you’ll see <b>hi</b> on the screen instead of a bold “hi”. On the other hand, if you put !- <b>hi</b>-! on a page, then the HTML is left unescaped and a boldfaced “hi” will appear on the screen.

I’m telling you all of this because the devil is in the details—so bear with me a bit longer.

You know how the C and C++ languages have a preprocessor? This preprocessor handles all the #include and #define statements and then hands the resulting text off to the true compiler. Well, the wiki parser works the same way! Literals and !define variables are processed first, by a different pass of the parser. Then all the rest of the wiki widgets are parsed by the true parser. The reason that we had to do this is even more devilishly detailed; so I’ll spare you. Suffice it to say that the reasons we need that preprocessor are similar to the reasons that C and C++ need it.

What does the preprocessor do with a literal? It converts it into a special widget. That widget looks like this: !lit?23? What does that mean? It means replace me with the contents of literal #23. You see, when FitNesse sees !- FitNesse-! it replaces it with !lit?nn? and squirrels FitNesse away in literal slot nn. During the second pass, that !lit?nn? is replaced with the contents of literal slot nn. Simple, right?

OK, now back to SLIM table processing. There I was, in Norway, teaching a class during the day and coding Slim at night, and everything was going just great. And then, during one of my tests, I happened to put a literal into one of the test tables. This is perfectly normal, I didn’t want some camel case phrase turned into a link. But this perfectly normal gesture made a unit test fail for a very strange reason. I got this wonderful message from junit: expected MyPage but was !lit?3?

I knew exactly what this meant. It meant that the value MyPage had been squirreled away by the first pass of the parser. I also knew that I had utterly no reasonable way of getting at it. So I did the only thing any sane programmer would do. I wrote my own preprocessor and used it instead of the standard one. This was “safe” since in the end I simply reconvert all the colorized tables back into wiki text and let the normal parser render it into HTML.

It was a bit of work, but I got it done at one am on a cold Norwegian night. Tests passing, everything great!

Ah, but no. By writing my own preprocessor, I broke the !define variable processing – subtly. And when I found and fixed that I had re-broken literal processing – subtly.

If you were following my tweets at the time you saw me twitter ever more frantically about literals. It was the proverbial ball of Mercury. Every time I put my thumb on it¹ it would squirt away and take some new form.

I fell into a trap. I call it the fudge trap. It goes like this:

forever do {“I can make this work! Just one more little fudge right here!”}

I was in this trap for two months! I was making progress, and getting lots of new features to work, but I was also running into strange little quirks and unexpected bizarre behaviors caused by the fudging I was doing. So I’d fudge a little more and then keep working. But each little fudge added to the next until, eventually, I had a really nasty house of cards (or rather: pile of fudge) ready to topple every time I touched anything else. I started to fear my own code². It was time to stop!

I knew what I had to do. I had to go back to my original architectural decision and make it the other way. There was no escape from this. The FitNesse parser was too coupled to the wiki-ness, and there was no sane way to repurpose it for test table processing.

I dreaded this. It was such a big change. I had built so much code in my house of fudge. All of it would have to be changed or deleted. And, worse, I needed to write an HTML parser.

I was lamenting to Micah about this one day in late November, and he said: “Dad, there are HTML parsers out there you know.”.

Uh…

So I went to google and typed Html Parser. Duh. There they were. Lots and lots of shiny HTML parsers free for the using.

I picked one and started to fiddle with it. It was easy to use.

Now comes the good part. I had not been a complete dolt. Even when I was using the FitNesse parse tree, I ate my own dogfood and wrapped it in an abstract interface. No part of the Slim Table processing code actually knew that it was dealing with a FitNesse parse tree. It simply used the abstract interface to get its work done.

That meant that pulling out the wiki parser and putting in the HTML parser was a matter of re-implementing the abstract interface with the output of the new parser (which happened to be another parse tree!). This took me about a day.

There came a magic moment when I had both the wiki text version of the parser, and the HTML version of the parser working. I could switch back and forth between the two by changing one word in one module. When I got all my tests passing with both parsers, I knew I was done. And then the fun really began!

I deleted ever stitch of that wiki parse tree pile of fudge. I tore it loose and rent it asunder. It was gone, never to darken my door with it’s fudge again.

It took me a day. A day. And the result is 400 fewer lines of code, and a set of Slim tables that actually work the way they are supposed to.

Moral #1: “Fudge tastes good while you are eating it, but it makes you fat, slow, and dumb.”

Moral #2: “Eat the damned dog food. It’ll save your posterior from your own maladroit decisions.

¹ I do not recommend that you actually put your thumb on any Mercury. Never mind that I used to play with liquid Mercury as a child, sloshing it around from hand to hand and endlessly stirring it with my finger. Wiser heads than I have determined that merely being in the same room with liquid Mercury can cause severe brain damage, genetic corruption, and birth defects in your children, grandchildren, and pets.

² Fearing your own code is an indicator that you are headed for ruin. This fear is followed by self-loathing, project-loathing, career-loathing, divorce, infanticide, and finally chicken farming.

When I read those words I was struck with by the wonderful juxtaposition. Here we have a truly recognized master of his craft explaining to us that through all his work, effort, passion, and persistence, he has managed to get to the starting line.

I recently conducted a Coding Dojo at the offices of 8th Light. As you know I typically wear a green band on my wrist. But as I moved to the front of the room to lead the Dojo, I removed the green band and replaced it with a black one to take the role of a sen sei. Once the Dojo was over, I quickly removed the black band and put the green one back on.

Recently I’ve been working a lot on FitNesse. I’ve written more code in the last three months than I’d written in the three years before. And let me tell you: I’m loving it! But the effort has also made me realize how much more I have to learn, and just how badly I do what I do.

Just two days ago I found myself in the bowels of a module trying to move some data that had been calculated in one part of that module to where it could be used in another part. I struggled with the problem for hours. I happened to be at the 8th Light office at the time (I often hang out there to be around other programmers. My own company, Object Mentor is distributed, so I don’t get to hang with the mentors nearly often enough.) One of the guys was in an up mood, and was cracking jokes and telling stories. Though his banter was nothing unusual, nor particularly egregious, I found I could not focus on the problem at hand. I had to move to a quieter part of the office.

First, how did I let the code get to the point where the data I needed was calculated so far away from where it was used? Second, why couldn’t I get this problem into my head in a normal social programming context? I shouldn’t have gotten myself into that fix in the first place, and I shouldn’t have had to sequester myself to get myself out of the fix in the second. I am very disappointed in my performance.

I could go on. I could tell you how I still continually fight the demons that encourage me to rush instead of following a disciplined course. I could tell you about all the tests I don’t write. I could tell you about the constant allure of shortcuts and my imperfection at avoiding them. I could tell you how often I look at that green band on my wrist and shudder at how imperfectly I follow it’s urgings.

There is much I have yet to learn about writing software well. So, although after 56 years of life, and 43 years of programming, I have achieved a modicum of success, and even some glory, Chef Baglio is right. It is from that point that you really start to learn.

Hudson—A Very Quick Demo from unclebob on Vimeo.

I thought the blog was good. Really. No, I did. It’s a pretty good blog. Honestly.

My problem with is is that it points the finger outwards. It’s as though software developers have no responsibility. The blog seems to suggest that projects fail because managers do dumb-ass things like not buying dual monitors, setting deadlines, and requiring documentation.

Reading a blog like Justin’s may make you feel like high-five-ing and doing a little touch-down jig. OK, fine. And, after all, there’s some truth to what Justin has to say. But there’s another side of the coin too. A pretty big side.

Your software will suck if you write it badly. Yes, you should have good tools. Yes, you should work under realistic schedules. Yes, you should have time for social interaction. But these aren’t the things that make software suck. YOU, make your software suck.

Can you write good software with just one monitor? Of course you can. It might not be ideal, but what is?

Can you write good software if the deadlines are unreasonable? Of course you can! The definition of an unreasonable deadline is a deadline you won’t make, so you might as well make the code as good as it can be in the time you’ve got. If we’ve learned anything in the last 50 years it’s that rushing won’t get you to the deadline faster.

Can you write good software if you also have to write documentation? Can you write good software if your machine isn’t top-of-the-line? Can you write good software while standing on your head under water? (er, well, I’ll give you that might be tough, but for all the others:) Of course you can!

Don’t get me wrong. I think short-shrifting on tools, monitors, and schedules is stupid. I think Justin’s points are all valid. But the burden doesn’t fall solely upon management. We also have to do our jobs well.

It’s not so much a “but” as an “aha!”. (The punch line is at the end of this article, so don’t give up in the middle.)

BDD is a variation on TDD. Whereas in TDD we drive the development of a module by “first” stating the requirements as unit tests, in BDD we drive that development by first stating the requirements as, well, requirements. The form of those requirements is fairly rigid, allowing them to be interpreted by a tool that can execute them in a manner that is similar to unit tests.

For example,

  GIVEN an employee named Bob making $12 per hour.
  WHEN Bob works 40 hours in one week;
  THEN Bob will be paid $480 on Friday evening.

The Given/When/Then convention is central to the notion of BDD. It connects the human concept of cause and effect, to the software concept of input/process/output. With enough formality, a tool can be (and has been) written that interprets the intent of the requirement and then drives the system under test (SUT) to ensure that the requirement works as stated.

The argued benefit is that the language you use affects the way you think (See this. and so if you use a language closer to the way humans think about problems, you’ll get better thought processes and therefore better results.

To say this differently, the Given/When/Then convention stimulates better thought processes than the AssertEquals(expected, actual); convention.

But enough of the overview. This isn’t what I wanted to talk about. What struck me the other day was this…

The Given/When/Then syntax of BDD seemed eerily familiar when I first heard about it several years ago. It’s been tickling at the back of my brain since then. Something about that triplet was trying to resonate with something else in my brain.

Then yesterday I realized that Given/When/Then is very similar to If/And/Then; a convention that I have used for the last 20+ years to read state transition tables.

Consider my old standard state transition table: The Subway Turnstile:

Current State	Event	New State	Action
LOCKED	COIN	UNLOCKED	Unlock
LOCKED	PASS	LOCKED	Alarm
UNLOCKED	COIN	UNLOCKED	Thankyou
UNLOCKED	PASS	LOCKED	Lock

I read this as a set of If/And/Then sentences as follows:

• If we are in the LOCKED state, and we get a COIN event, then we go to the UNLOCKED state, and we invoke the Unlock action.
• If we are in the LOCKED state, and we get a PASS event, then we stay in the UNLOCKED state, and we invoke the Alarm action.
• etc.

This strange similarity caused me to realize that GIVEN/WHEN/THEN is simply a state transition, and that BDD is really just a way to describe a finite state machine. Clearly “GIVEN” is the current state of the system to be explored. “WHEN” describes an event or stimulus to that system. “THEN” describes the resulting state of the system. GIVEN/WHEN/THEN is nothing more than a description of a state transition, and the sum of all the GIVEN/WHEN/THEN statement is nothing more than a Finite State Machine.

Perhaps if I rephrase this you might see why I think this is a bit more than a ho-hum.

Some of the brightest minds in our industry, people like Dan North, Dave Astels, David Chelimsky, Aslak Hellesoy, and a host of others, have been pursuing the notion that if we use a better language to describe automated requirements, we will improve the way we think about those requirements, and therefore write better requirements. The better language that they have chosen and used for the last several years uses the Given/When/Then form which, as we have seen, is a description of a finite state machine. And so it all comes back to Turing. It all comes back to marks on a tape. We’ve decided that the best way to describe the requirements of a system is to describe it as a turing machine.

OK, perhaps I overdid the irony there. Clearly we don’t need to resort to marks on a tape. But still there is a grand irony in all this. The massive churning of neurons struggling with this problem over years and decades have reconnected the circle to the thoughts of that brave pioneer from the 1940s.

But enough of irony. Is this useful? I think it may be. You see, one of the great benefits of describing a problem as a Finite State Machine (FSM) is that you can complete the logic of the problem. That is, if you can enumerate the states and the events, then you know that the number of paths through the system is no larger than S * E. Or, rather, there are no more than S*E transitions from one state to another. More importantly, enumerating them is simply a matter of creating a transition for every combination of state and event.

One of the more persistent problems in BDD (and TDD for that matter) is knowing when you are done. That is, how do you know that you have written enough scenarios (tests). Perhaps there is some condition that you have forgotten to explore, some pathway through the system that you have not described.

This problem is precisely the kind of problem that FSMs are very good at resolving. If you can enumerate the states, and events, then you know the number of paths though the system. So if Given/When/Then statements are truly nothing more than state transitios, all we need to do is enumerate the number of GIVENs and the number of WHENs. The number of scenarios will simply be the product of the two.

To my knowledge, (which is clearly inadequate) this is not something we’ve ever tried before at the level of a business requirements document. But even if we have, the BDD mindset may make it easier to apply. Indeed, if we can formally enumerate all the Givens and Whens, then a tool could determine whether our requirements document has executed every path, and could find those paths that we had missed.

So, in conclusion, TDD has led us on an interesting path. TDD was adopted as a way to help us phrase low level requirements and drive the development of software based on those requirements. BDD, a variation of TDD, was created to help us think better about higher level requirements, and drive the development of systems using a language better than unit tests. But BDD is really a variation of Finite State Machine specifications, and FSMs can be shown, mathematically, to be complete. Therefore, we may have a way to conclusively demonstrate that our requirements are complete and consistent. (Apologies to Godel).

In the end, the BDDers may have been right that language improves the way we think about things. Certainly in my silly case, it was the language of BDD that resonated with the language of FSM.