Object Mentor Blog

The 4-contact points of software development

Brett Schuchert — Thu, 19 Aug 2010 22:59:00 -0500

The three laws of TDD are:

Write no production code without a failing test
Write just enough of a test to fail
Write just enough production code to get the test to pass

This list doesn’t include refactoring, which is typically an assumed activity. In fact, some people refer to these rules as “red, green, refactor”. An even older version of this, from the Smalltalk community, is Red, Green, Blue. (Why Blue for refactor? I think someone was thinking RBG for a color space, luckily they didn’t try to use CMYK or LAB!)

In this simple model, there two kinds of code: test & production. There are two kinds of activity: writing & refactoring. Interestingly, at one level it is all code. The thing that distinguishes both sets is intent.

The intent of a test is to demonstrate or maybe specify behavior. The intent of production code is to implement (hopefully) business-relevant functionality.

The intent of writing code is creation. The intent of refactoring code is to change (hopefully improve) its structure without changing its behavior (this is oversimplified but essentially correct).

If you mix those combinations you have the 4-limbs of development:

Writing a test
Writing production code
Refactoring a test
Refactoring production code

An important behavior to practice is doing only one of these at a time. That is, when you are writing tests, don’t also write production code. Sure, you might use tools to stub out missing methods and classes, but the heart of what you are doing is writing a test. Finish that train of thought before focusing on writing production code.

On the other hand, if you are refactoring production code, do just that. Don’t change tests at the same time, try to only do one refactoring at a time, etc.

WHY?

First an analogy that almost always misses since most developers don’t additionally rock climb.

When rock climbing, a good general bit of advice is to only move one contact point at a time. For this discussion, consider your two hands and two feet as your four contact points. Sure, you can use your face or knee, but neither are much fun. So just considering two hands and two feet, that suggests that if, for example, you move your right hand, then leave your left hand and both feet in place.

This gives you stability, a chance to easily recover by simply moving the most recent appendage back in place and, when the inevitable happens, another appendage slips, you have a better chance of not eating rock face. If you move more than one thing at a time, you are in more danger because you’ve taken a risky action and reduced the number of points of contact, or stability.

Will you sometimes move multiple appendages? Sure. But not as a habit. Sometimes you need to take risks. The rock face may not always offer up movement patterns that make applying this recommendation possible. Since you know the environment will occasionally work against you, you need to maintain some slack for the inevitable.

Practicing Test Driven Development is similar. If you change production code and tests at the same time, what happens if a test fails? What is wrong? The production code, the test, both, neither? An even more subtle problem is that tests pass but the test is fragile or heavily implementation-dependent. While not necessarily an immediate threat, it represents design debt that will eventually cause problems. (This also happens frequently when tests are written after the production code as it’s seductively easy to write tests that exercise code, expressing the production’s code implementation but fundamentally hiding the intent.)

Notice, if you had only refactored code, then you know the problem is in one place. When you change both, the problem space actually goes from 1 to 3 (4 if you allow for neither). Furthermore, if you are changing both production and test code at the same time and you get to a point where you’ve entered a bottomless pit, you’ll end up throwing away more work if you choose to restore from the repository.

Are there going to be times when you change both? Sure. Sometimes you may not see a clear path that gives you the option to do only one thing at a given time. Sometimes the tests and code will work against you. Often, you’ll be working in a legacy code base where there are no tests. Given that the environment will occasionally (or frequently) work against you, you need to maintain some slack.

Essentially, be focused on a single goal at any given time: write a test. then get it to pass. clean up production code & keep the tests first unchanging and then passing.

I find that this is a hard thing both to learn and to apply. I frequently jump ahead of myself. Unfortunately I’m “lucky” enough when I do jump ahead that when I fail, I thoroughly fall flat on my face.

This approach is contextual (aren’t they all?). Every time you start working on code, you’ll be faced with these four possibilities. Each time you are, you need to figure out what is the most important thing in the moment, and do that one thing. Once you’ve taken care of the most important thing, you may have just promoted the second most important thing to first place. Even so, reassess. What is the most important thing now? Do that.

Good luck trying to apply this idea to your development work. I’m interesting in hearing about it.

Is it worth killing trees over another C++ Book?

Brett Schuchert — Tue, 17 Aug 2010 00:17:00 -0500

I’ve taught a few C++ courses recently to people primarily moving from C to C++. I know C++ has been around for years and it’s not in vogue like it was 15 years ago. I stopped using it full time in 1997. Even so, there’s been quite a bit of work on the library and language standard. And there are still a lot of places developing new systems with C++. I know some people are still be learning C++ in school.

In my most recent classes, I’ve been teaching students who have recently taken a class on OO A & D based on the work of Craig Larman. The C++ class attempts to follow that class by dovetailing into what it covers. Because of this, I did not use ObjectMentors’ standard C++ and OOD class. Good as it is, it has different starting assumptions. I instead wrote a class, using two problems as the entire basis of all the material I cover. I know, “not build here syndrom.” It pained me to do this, I did my research and I considered retro-fitting the OM course. The discrepancy was too large to consider reuse. And refactoring the existing class to accomodate changes, which I also considered, wasn’t practical. Looking back I made the right decision.

So this course has a few key design elements:

Problem focused
Test oriented (sometimes test-first, other times test-driven, occasionally refactor-oriented)

The problem-focus limits the topic coverage. If something about the language doesn’t come up somewhat naturally (not contrived) in the two projects I use, I don’t cover the material. For example, I don’t mention placement new, I only cover Multiple Inheritance if asked about it. I also try to focus on classes in the standard library. For example, std::array, std::vector, std::map, std::shared_ptr.

Additionally, there’s an early focus on testing. That’s another thing that’s different from how I taught C++ say before 1995 – I really didn’t teach it from 1995 – 2007, so no comments on what I might have done differently in that span of years.

To give you an idea of how early the focus is on test, I only show cout if asked. The first main() calls CommandLineTestRunner::RunAllTests and everything after that runs within a unit test. the last time I taught the class, I demonstrated tests executing with cslim, but still, a test focus.

I have them use unit tests as a way to experiment with the language. In one example, I have them write tests that force a method to become virtual that was not virtual before. In another case, I do the same thing with a virtual destructor. I have them test from raw pointers into shared_ptr and then update their code accordingly.

Because of the test focus, I make certain recommendations that impact overall class design. That means learning C++ with designs supporting testability early on. In C++ this means (among other things):

Dependency Injection (OK, this is not just C++)
Virtual functions and by corollary a virtual destructor
Storing pointers because 1. calling methods through an auto object are not virtually dispatched, and 2. you cannot put references in the standard collections
Use std::shared_ptr to avoid memory leaks, which are detected by the unit testing tool I have them use.

I take the class to a certain level, but I make it clear I’m just scratching the surface. I believe it’s not really possible to learn C++ in a 1-week class. You can get the beginnings of proficiency and be in a good position to continue learning – that’s an assumption I state up front after the students have had their first exercise – about 5 minutes into the start of the class. I try to only go into detail as the students ask questions, but at times I just want to really open up the beast and get into what’s really happening.

To address this, I’ve started novelizing the class I’ve been teaching. I’m following the same outline as the class, but I dive under the surface and at times get to quite a bit of detail that I would not typically get into in a 4.5 day class.

I know this material will augment the class. This will give my students three sources of information:

The class itself, which is exercise-driven
Online videos
A novelization of the class, going into much more detail

The thing I’m wondering is, would the book be worth making more generally available? I’m writing it so that it can stand on its own. There’s a certain advantage to knowing my students have taken the previous OO A & D class I mentioned because it uses a problem that I’ve used on and off since 1992. This allows me to give examples from a problem they have looked at in the past and then at the current problem. I’ve not yet made references to that problem in the book I’m writing. I could, I just have not done so yet – it’s more natural in the second problem and I’m still working on the first problem (and therefor the first half of the book). I’m certain I can make those references without the previous class experience. (It’s the Monopoly problem.)

In any case, I need to do some research. If I do start the publishing process, a key step involves competitive research. Can you recommend any books published for the first time this century (really in the past 7 years) that have any of these characteristics:

Cover a minimal set of C++, enough for decent OO solutions
Have any kind of emphasis on test – at least half the book?
Cover the language strictly through a problem-based approach rather than from a language perspective?
Involve deliberately making mistakes and observing those mistakes to learn how the language works?

Additionally, can you recommend any great C&#43&#43 books? I can list many of the ones I’ve read and enjoyed, but the last time I bought a book for myself on the topic, Amazon did not exist as an online company.

Independent of whether I deal with trying to get this thing published as a printed book, I’m going to finish it because I think it will be useful to my students. I suspect I’ll be teaching this class in the future, so I will find it useful in the future as well. If I don’t attempt publishing it through a major publisher, I’ll put it on my wiki at the very least. Though it’s going to be quite a bit more effort than my typical wiki articles.

But the question I keep coming back to, is this: Is there really a need for this book dead tree version, or it is primarily useful as supplemental material for a class I teach?

I’d like to hear your opinions.

Game Of Life with @lunivore

Uncle Bob — Sun, 15 Aug 2010 20:40:21 -0500

At the #coderetreat in Orlando I spent an hour programming with @lunivore (Liz Keogh). We worked in Clojure on Conway’s game of life. It was quite an experience!

Liz was not very familiar with Clojure, so I felt I had the advantage. Wrong! By the end of the hour she had taken charge and was programming rings around me. It was a lot of fun; if a bit humbling.

Liz came into the session knowing the structure of the algorithm she wanted to implement. She just didn’t know how to implement it in Clojure. When the starting gun was fired, she showed me a picture of a glider (a standard form in life) and said she wanted this to be the acceptance test.

During the session we wrote only a very few unit tests, each very focussed on one particular part of the problem. The size of the steps was considerably larger than I am used to in Java or Ruby; yet it didn’t seem to matter. Writing functions in clojure is easy, and apparently far less error prone.

We finished 10 minutes before the end of the session, and had some time to refactor.

I’ve posted the code below. I’ve cleaned it up a bit (being unable to leave it alone); but the algorithm remains the same; as are the tests.

Nearly 22 years ago

Brett Schuchert — Wed, 11 Aug 2010 22:23:00 -0500

Moved the article and the file. Reduced resolution from 300 dpi to 75 dpi (using a quartz filter on OS X from Jerome Colas

Here’s the story with the updated location: The Moved Blog

Getting Started With cslim in Visual Studio 2010 Using the Command Line Tools

Brett Schuchert — Sun, 08 Aug 2010 22:08:00 -0500

Some of you asked for it. Here’s something to get you started:

http://schuchert.wikispaces.com/cpptraining.UsingCSlimWithVisualStudio2010

These instructions are a work in progress and alpha. However, they do get the basics working.

If you are so inclined, have a look at the NMakefile (you’ll come across it in the instructions) and give me a better way to build it.

I spent probably 8 hours getting a working environment (much yak shaving including a faulty mac book pro DVD drive). I then spent another 8 – 10 hours getting this working. I worked through it about 5 times to minimize the amount of changes I needed to make to the original library source.

I ended up using some link and pre-processor seams to get most of this built. However, most of my time was spent trying to figure out the command line tools and their options.

If you can give me some guidance on improving this, I’d like to hear it. However, this is now a low-burning thread, so some assembly required!

Enjoy,

Rough Notes on using FitNesse with C++

Brett Schuchert — Fri, 30 Jul 2010 22:09:00 -0500

I’ve been working with FitNesse.slim, using the cslim implementation to execute C++ code. I have some rough notes online. These should be enough to get started, though you’ll need to be using G++ 4.4 or later.

In any case, have a look. Give me some feedback if you’d like. I’ll be working on these over the next month or so: http://schuchert.wikispaces.com/cpptraining.ExecutingBinaryOperators

Preprocessor seams and assignment of responsibility

Brett Schuchert — Thu, 22 Jul 2010 10:32:00 -0500

In my previous blog I mentioned adding a single method to the cslim library:

cslim/include/CSlim/SlimListSerializer.h:
void SlimList_Release(char *serializedResults);
cslim/src/CSlim/SlimListSerializer.c:
void SlimList_Release(char *serializedResults)
{
  if(serializedResults)
    free(serializedResults);
}

Now I’m going to explain how I came across this need, what it took to figure it out and then why I picked this solution.

Background

In FitNesse, Query Table Results are a list of list of list:

The outer-most list represents “rows”. It collects all objects found. It has zero or more entries.
The inner-most list represents a single field. It is a key value pair. It is a list of size 2. The first entry is the name of the field (column to FitNesse). The second field is the value. Both are strings.
The middle list represents a single object. It is a collection of fields. It has zero or more entires.

I understand the need for this representation and it takes a little bit to get it built correctly. So much so, I built a simple tool to do it in java.

C++ is no different. In fact, the authors of cslim though the same thing and they created a C Abstract Data Type to help out (and an add-on method to create the correct final form):

SlimList
SlimList_Serializer

They use C because it can be used by both C and C++. I’m using C++ and I wanted to make building query results even easier, so I built a QueryResultAccumulator. The most recent source is in the previous blog and I’ll be putting it on github after I’ve had some more time to work on it.

Here’s the progression to my QueryResultAccumulator class:

Wrote a Query-Table based fixture and followed the example provided with cslim (thank you for that!)
Moved the code from functions into methods on a class
Extracted a class, called SlimListWrapper, which made the fixture code easier to follow.
Went to get takeout and realized that I had named the class incorrectly and that it was really accumulating query results (thus the name). The SlimList was a mechanism, not an intent.
Refactored the class into QueryResultAccumulator (I left the original alone, created a new class, copied code from one to the other and changed it around a bit.

Now it might sound like I didn’t have any tests. In fact, I did. I had my original Acceptance Test in FitNesse. I kept running that, so in a sense I was practicing ATDD.

I was not happy with that, because I was not sure that I had properly handled memory allocation correctly. In fact, I had not. The final result is dynamically allocated and I was not releasing that. So I “fixed” it. (It needs to be released after the return from slim back to FitNesse, so the typical pattern is to lazily delete it, or release it in a “destroy” method called after the execution of a single table.)

I have a memory leak?!

I am simplifying this story a bit. So I’m skipping some intermediate results. Ultimately I wrote the following test to check that you could use a single query result accumulator for multiple results correctly:

TEST(QueryResultAccumulator, CanProduceFinalResultsMultipleTimes) {
    QueryResultAccumulator accumulator;
    accumulator.produceFinalResults();
    accumulator.produceFinalResults();
}

This caused a memory leak of 60 bytes. It was at this point I was up too late and banging my head against a wall. About 3 hours later I figured that out and went to bed. I fixed the problem and sent a patch to authors of cslim in maybe 45 minutes. So I should have gotten more sleep.

Where is that damn thing

In any case, I visually checked the code. I debugged it. I did everything I could initially think of, and I was convinced that my code was correct. (As we’ll see it both was and was not due to a preprocessor seam in cslim.) I got to the point where I even tried different versions of gcc. (I found a parsing error in g++ 4.5 when handling templates, so in desperation and late at night I wasted 5 minutes switching my compiler.) I had the following code in my class:

  if(result)
    free(result);

This was the correct thing, but it was in the wrong location. Again, this was related to a preprocessor seam in cslim.

Eureka!

I looked at the cslim code and confirmed it was doing basic C stuff, nothing surprising. It was at that point that I remembered something important:

cslim depends on CppUTest and uses a different malloc/free

Ah ha! That’s it. So I tired to recompile my C++ code to use the same thing. However, I was not able to do that. CppUTest’s memory tracking implementation does not work with many of the C++ standard classes like <string> and <vector>. So I could not compile my code using the same approach.

I’m glad this happened becuase it made me realize that it was the wrong place anyway. Here’s the logic:

CSlim has a preprocessor seam, comile with or without -Dfree=cpputest_free, -Dmalloc=cpputest_malloc
I’m using a class in cslim to do the allocation, where the policy of allocation is stored
I should not release the memory but instead allow the cslim library to release the memory becaue it has the allocation policy, and therefore the release policy.

About 2 minutes later, I had added a single function to the library:

cslim/include/CSlim/SlimListSerializer.h:
void SlimList_Release(char *serializedResults);
cslim/src/CSlim/SlimListSerializer.c:
void SlimList_Release(char *serializedResults)
{
  if(serializedResults)
    free(serializedResults);
}

I updated my QueryResultAccumulator to use SlimList_Release and my false positive disappeared.

It also turns out that this improved symmetry in the library. To allocate and release entries in an SlimList you use the following functions:

SlimList* SlimList_Create()
void SlimList_Destroy(SlimList*);

Now to serlalize a list and release the memory later you use:

SlimList_Serialize
SlimList_Release

As I write this, I think there’s a better name. I’ll let the authors give it a better name (like SlimList_Release_Serialization_Results). But in any case, if you use a function in the cslim library that allocates something, you use another method in the cslim library to release it.

Since the libray has a preprocessor seam, that symmetry removes a false-positive memory leak.

What took so long?

I had an interesting time with this. Originally I had not released that memory in the class but rather in the unit test. I was working too late and not thinking clearly. I realized that my class needed to manage that.

When I called free in the unit test, it was calling the correct version of free, cpputest_free, becasue it was a unit test using CppUTest. When I moved the code into the class, which has no knowledge of CppUTest (nor can it), the flow of the code was correct, but the compilation (preprocessor symbols) were different and it caused a false positive.

Since I changed the code, I assumed it was a problem with how I changed the code. To me more clear, I though it was a code-flow problem, not a preprocessor seam problem. So I spent a lot of time verifyig my code. Once I determined it was correct, I then moved into debugger mode.

It was not long after that when I finally figured out what was going on.

Conslusions

As with many things in life, this is intuitive once you understand it!-)

That cslim depends on CppUTest might be questionable. However, if I treat cslim as a (mostly) black box, and I don’t know its allocation policy, I should not assume a deallocation policy.

By putting the responsiblity in the correct library level, it fixed the problem and added symmetry to the overall solution.

I also really enjoyed this (after it was done). I’ve come across memory leaks using CppUTest in the past. Often they were my fault. Sometimes they were not. This was interesting because it both was and was not my fault. I originally had written the code incorrectly. When I put he correct steps in my code, I still had wrong code because I put the responsibility in the wrong place. It really was only correct after I moved the actual implementation into the library and then called it from my code that I had finally written it correctly.

Some C++ Fixtures for FitNesse.slim

Brett Schuchert — Thu, 22 Jul 2010 00:32:00 -0500

I continue working on these. I was stuck in the airport for 5 hours. Between that and the actual flight, I managed to create three different test examples against a C++ RpnCalculator. Each example uses a different kind of fixture. I had a request from @lrojas to publish some results on the blog. So this is that, however these are in progress and rough.

I’m still trying different forms to figure out what I like the best.

By the way, that lastValue stuff in the fixtures has to do with the fact that all of the hook methods return a char* but I’m responsible for cleaning up after myself.

A Decision Table

!|ExecuteBinaryOperator    |
|lhs|rhs|operator|expected?|
|3  |4  |-       |-1       |
|5  |6  |*       |30       |

And Its Fixture Code

#include <stdlib.h>
#include <stdio.h>
#include <string>
#include "RpnCalculator.h"
#include "OperationFactory.h"
#include "Fixtures.h"
#include "SlimUtils.h"

struct ExecuteBinaryOperator {
    ExecuteBinaryOperator() {
        lastValue[0] = 0;
    }

    int execute() {
        RpnCalculator calculator(factory);
        calculator.enterNumber(lhs);
        calculator.enterNumber(rhs);
        calculator.executeOperator(op);
        return calculator.getX();
    }

    static ExecuteBinaryOperator* From(void *fixtureStorage) {
        return reinterpret_cast<ExecuteBinaryOperator*>(fixtureStorage);
    }

    OperationFactory factory;
    int lhs;
    int rhs;
    std::string op;
    char lastValue[32];
};

extern "C" {
void* ExecuteBinaryOperator_Create(StatementExecutor* errorHandler, SlimList* args) {
    return new ExecuteBinaryOperator;
}

void ExecuteBinaryOperator_Destroy(void* self) {
    delete ExecuteBinaryOperator::From(self);
}

static char* setLhs(void* fixture, SlimList* args) {
    ExecuteBinaryOperator *self = ExecuteBinaryOperator::From(fixture);
    self->lhs = getFirstInt(args);
    return self->lastValue;
}

static char* setRhs(void* fixture, SlimList* args) {
    ExecuteBinaryOperator *self = ExecuteBinaryOperator::From(fixture);
    self->rhs = getFirstInt(args);
    return self->lastValue;
}

static char* setOperator(void *fixture, SlimList* args) {
    ExecuteBinaryOperator *self = ExecuteBinaryOperator::From(fixture);
    self->op = getFirstString(args);
    return self->lastValue;
}
static char* expected(void* fixture, SlimList* args) {
    ExecuteBinaryOperator *self = ExecuteBinaryOperator::From(fixture);
    int result = self->execute();
    snprintf(self->lastValue, sizeof(self->lastValue), "%d", result);
    return self->lastValue;
}

SLIM_CREATE_FIXTURE(ExecuteBinaryOperator)
    SLIM_FUNCTION(setLhs)
    SLIM_FUNCTION(setRhs)
    SLIM_FUNCTION(setOperator)
    SLIM_FUNCTION(expected)
SLIM_END

}

There’s a bit of duplication. I’ve been experimenting with pointers to member functions and template functions to make it better. I really should be using lambdas, but I’m not there yet. I have them available in some form since I’m using gcc 4.5. I simply compile with the option -sdd=c++0x. Even so, I’m not quite ready to do that.

A Script Table

!|script           |ProgramTheCalculator                   |
|startProgramCalled|primeFactorsOfSum                      |
|addOperation      |sum                                    |
|addOperation      |primeFactors                           |
|saveProgram                                               |
|enter             |4                                      |
|enter             |13                                     |
|enter             |7                                      |
|execute           |primeFactorsOfSum                      |
|check             |stackHas|3|then|2|then|2|then|2|is|true|

And Its Fixture Code

#include <stdlib.h>
#include <stdio.h>
#include <string>
#include "RpnCalculator.h"
#include "OperationFactory.h"
#include "SlimUtils.h"
#include "SlimList.h"
#include "Fixtures.h"

struct ProgramTheCalculator {
    ProgramTheCalculator() : calculator(factory) {
    }

    static ProgramTheCalculator* From(void *fixtureStorage) {
        return reinterpret_cast<ProgramTheCalculator*>(fixtureStorage);
    }

    OperationFactory factory;
    RpnCalculator calculator;
};

extern "C" {

void* ProgramTheCalculator_Create(StatementExecutor* errorHandler, SlimList* args) {
    return new ProgramTheCalculator;
}

void ProgramTheCalculator_Destroy(void *fixture) {
    delete ProgramTheCalculator::From(fixture);
}

static char* startProgramCalled(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    self->calculator.createProgramNamed(getFirstString(args));
    return remove_const("");
}

static char* addOperation(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    self->calculator.addOperation(getFirstString(args));
    return remove_const("");
}

static char* saveProgram(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    self->calculator.saveProgram();
    return remove_const("");
}

static char* enter(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    self->calculator.enterNumber(getFirstInt(args));
    return remove_const("");
}

static char* execute(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    self->calculator.executeOperator(getFirstString(args));
    return remove_const("");
}

static char* stackHasThenThenThenIs(void *fixture, SlimList *args) {
    auto *self = ProgramTheCalculator::From(fixture);
    for(int i = 0; i < 4; ++i) {
            if(self->calculator.getX() != getIntAt(args, i))
                return remove_const("false");
            self->calculator.executeOperator("drop");
    }

      return remove_const("true");
}

SLIM_CREATE_FIXTURE(ProgramTheCalculator)
    SLIM_FUNCTION(startProgramCalled)
    SLIM_FUNCTION(addOperation)
    SLIM_FUNCTION(saveProgram)
    SLIM_FUNCTION(enter)
    SLIM_FUNCTION(execute)
    SLIM_FUNCTION(stackHasThenThenThenIs)
SLIM_END

}

This one is a bit more regular. I am using the updated auto keyword in this code. The fixture is just holding the calculator and its OperationFactory (not my preferred name, but that’s what students wanted to call things like +, -, etc, operations not operators).

The Dreaded Query Table

It’s a bit of a pain to produce query results. So much so, I wrote a simple library in Java to make it easier. I can create a well-formed query result from a single object or a list of objects and even do basic transforms (in names and in paths to data). I started using the jakarta bean utils, but my use was so simple (2 methods), I ripped out that library and just hand-wrote the methods I needed. It was not a case of “not invented here syndrom.” I started by using the library, and I had tests. I didn’t like the size of the library relative to how much I was using it, so I just got rid of it.

Well here I am working C++ and I felt compelled to make it easier work with query results in C++.

First the FitNesse table, then the fixture and finally the support class. I have tests for it as well, I’m not going to show those, however.

!|Query: SingleCharacterNameOperators|
|op                                  |
|+                                   |
|*                                   |
|/                                   |
|!                                   |
|-                                   |

And Its Fixture Code

#include <stdlib.h>
#include <stdio.h>
#include <vector>
#include <string>
#include <memory>
#include "RpnCalculator.h"
#include "OperationFactory.h"
#include "Fixtures.h"
#include "SlimUtils.h"
#include "QueryResultAccumulator.h"

struct SingleCharacterNameOperators {
    OperationFactory factory;
    RpnCalculator calculator;

    SingleCharacterNameOperators() :
        calculator(factory), result(0) {
    }

    ~SingleCharacterNameOperators() {
        delete result;
    }
    static SingleCharacterNameOperators* From(void *fixtureStorage) {
        return reinterpret_cast<SingleCharacterNameOperators*> (fixtureStorage);
    }

    void resetResult(char *newResult) {
        delete result;
        result = newResult;
    }

    void conditionallyAddOperatorNamed(const std::string &name) {
        if (name.size() == 1) {
            accumulator.addFieldNamedWithValue("op", name);
            accumulator.finishCurrentObject();
        }
    }

    void buildResult() {
        v_string names = calculator.allOperatorNames();
        buildResult(names);
    }

    void buildResult(v_string &names) {
        for (v_string::iterator iter = names.begin(); iter != names.end(); ++iter)
            conditionallyAddOperatorNamed(*iter);

        resetResult(accumulator.produceFinalResults());
    }

    QueryResultAccumulator accumulator;
    char *result;
};

extern "C" {
void* SingleCharacterNameOperators_Create(StatementExecutor* errorHandler,
        SlimList* args) {
    return new SingleCharacterNameOperators;
}

void SingleCharacterNameOperators_Destroy(void *fixture) {
    delete SingleCharacterNameOperators::From(fixture);
}

static char* query(void *fixture, SlimList *args) {
    auto *self = SingleCharacterNameOperators::From(fixture);
    self->buildResult();
    return self->result;
}

SLIM_CREATE_FIXTURE(SingleCharacterNameOperators)
    SLIM_FUNCTION(query)SLIM_END
SLIM_END

And the Helper Class

QueryResultAccumulator.h

#pragma once
#ifndef QUERYRESULTACCUMULATOR_H_
#define QUERYRESULTACCUMULATOR_H_

class SlimList;
#include <vector>
#include <string>
class QueryResultAccumulator {
public:
    typedef std::vector<SlimList*> v_SlimList;
    typedef v_SlimList::iterator iterator;

    QueryResultAccumulator();
    virtual ~QueryResultAccumulator();

    void finishCurrentObject();
    void addFieldNamedWithValue(const std::string &name, const std::string &value);
    char *produceFinalResults();

private:
    SlimList* allocate();
    void releaseAll();
    void setInitialConditions();

private:
    v_SlimList created;
    SlimList *list;
    SlimList *currentObject;
    int lastFieldCount;
    int currentFieldCount;
    char *result;

private:
    QueryResultAccumulator(const QueryResultAccumulator&);
    QueryResultAccumulator& operator=(const QueryResultAccumulator&);
};

#endif

I know there are too many fields. The counts help with validating correct usage. I also wrote it so one instance could be re-used and I tried to make sure it was in a “ready to receive fields” state when necessary. In any case, this error checking helped find a defect I introduced while refactoring.

QueryResultAccumulator.cpp

#include "QueryResultAccumulator.h"
#include "DifferentFieldCountsInObjects.h"
#include "InvalidStateException.h"

extern "C" {
#include "SlimList.h"
#include "SlimListSerializer.h"
}

QueryResultAccumulator::QueryResultAccumulator() : result(0) {
    setInitialConditions();
}

QueryResultAccumulator::~QueryResultAccumulator() {
    releaseAll();
    SlimList_Release(result);
}

void QueryResultAccumulator::setInitialConditions() {
    releaseAll();
    list = allocate();
    currentObject = allocate();
    lastFieldCount = -1;
    currentFieldCount = -1;
}

SlimList* QueryResultAccumulator::allocate() {
    SlimList *list = SlimList_Create();
    created.push_back(list);
    return list;
}

void QueryResultAccumulator::releaseAll() {
    for (iterator i = created.begin(); i != created.end(); ++i)
        SlimList_Destroy(*i);
    created.clear();
}

void QueryResultAccumulator::finishCurrentObject() {
    if(lastFieldCount >= 0 && lastFieldCount != currentFieldCount)
        throw DifferentFieldCountsInObjects(lastFieldCount, currentFieldCount);

    SlimList_AddList(list, currentObject);
    currentObject = allocate();

    lastFieldCount = currentFieldCount;
    currentFieldCount = -1;
}

void QueryResultAccumulator::addFieldNamedWithValue(const std::string &name, const std::string &value) {
    SlimList *fieldList = allocate();
    SlimList_AddString(fieldList, name.c_str());
    SlimList_AddString(fieldList, value.c_str());
    SlimList_AddList(currentObject, fieldList);
    ++currentFieldCount;
}

char* QueryResultAccumulator::produceFinalResults() {
    if(currentFieldCount != -1)
        throw InvalidStateException("Current object not written");

    SlimList_Release(result);
    result = SlimList_Serialize(list);
    setInitialConditions();
    return result;
}

Note, this code uses a method I added to the cslim library: SlimListSerializer.h – in include/CSlim

void SlimList_Release(char *serializedResults);

SlimListSerializer.c – in src/CSlim

void SlimList_Release(char *serializedResults)
{
  if(serializedResults)
    free(serializedResults);
}

I needed to add these methods due to a false-positive memory leak indicated when using CppUTest to test this code. That’s another blog.

FitNesse, C++ and cslim, step-by-step instructions

Brett Schuchert — Mon, 19 Jul 2010 23:26:00 -0500

Title says it all: http://schuchert.wikispaces.com/cpptraining.GettingStartedWithFitNesseInCpp

First draft. If you have problems, please let me know.

A Few C plus plus TDD videos

Brett Schuchert — Mon, 12 Jul 2010 06:30:00 -0500

Using CppUTest, gcc 4.4 and the Eclipse CDT.

Rough, as usual.

The Video Album

Might redo first one with increased font size. Considering redoing whole series at 800×600.

Software Calculus - The Missing Abstraction.

Uncle Bob — Mon, 05 Jul 2010 20:26:15 -0500

The problem of infinity plagued mathematicians for millennia. Consider Xeno’s paradox; the one with Achilles and the tortoise. While it was intuitively clear that Achilles would pass the Tortoise quickly, the algebra and logic of the day seemed to suggest that the Tortoise would win every race given a head start. Every time Achilles got to where the tortoise was, the tortoise would have moved on. The ancients had no way to see that an infinite sum could be finite.

Then came Leibnitz and everything changed. Suddenly infinity was tractable. Suddenly you could sum infinite series and write the equations that showed Achilles passing the tortoise. Suddenly a whole range of calculations that had either been impossible or intractable became trivial.

Calculus was a watershed invention for mathematics. It opened up vistas that we have yet to fully plumb. It made possible things like Newtonian mechanics, Maxwell’s equations, special and general relativity and quantum mechanics. It supports the entire framework of our modern world. We need a watershed like that for software.

If you listen to my keynote: Twenty-Five Zeros you’ll hear me go on and on about how even though software has changed a lot in form over the last fifty years, it has changed little in substance. Software is still the organization of sequence, selection, and iteration.

For fifty years we have been inventing new languages, notations, and formulations to manage Sequence, Selection, and Iteration (SSI). Structured Programming is simply a way to organize SSI. Objects are another way to organize SSI. Functional is still another. Indeed, almost all of our software technologies are just different ways of organizing Sequence, Selection, and Iteration.

This is similar to Algebra in the days before calculus. We knew how to solve linear and polynomial equations. We knew how to complete squares and find roots. But in the end it was all just different ways to organize adding. That may sound simplistic, but it’s not. Subtracting is just adding in reverse. Multiplying is just adding repeatedly. Division is just multiplication in reverse. In short, Algebra is an organizational strategy for adding.

Algebra went through many different languages and notations too, just like software has. Think about Roman and Greek numerals. Think how long it took to invent the concept of zero, or the positional exponential notation we use today.

And then one day Newton saw an apple fall, and he changed the way we thought about mathematics. Suddenly it wasn’t about adding anymore. Suddenly it was about infinities and differentials. Mathematical reasoning was raised to a new order of abstraction.

Where is that apple for software (pun intended). Where is the Newton or Leibnitz that will transform everything about the way we think about software. Where is that long-sought new level of abstraction?

For awhile we thought it would be MDA. Bzzzzt, wrong. We thought it would be logic programming like prolog¹. Bzzzt. We thought it would be database scripts and 4GLs. Bzzzt. None of those did it. None of those can do it. They are still just various ways of organizing sequence, selection, and iteration.

Some people have set their sights on quantum computing. While I’ll grant you that computations with bits that can be both states simultaneously is interesting, in the end I think this is just another hardware trick to increase throughput as opposed to a whole new way to think about software.

This software transformation, whatever it is, is coming. It must come; because we simply cannot keep piling complexity upon complexity. We need some new organizing principle that revamps the very foundations of the way we think about software and opens up vistas that will take us into the 22nd century.

¹ Prolog comes closest to being something more than a simple reorganization of sequence, selection, and iteration. At first look logic programming seems very different. In the end, however, an algorithm expressed in prolog can be translated into any of the other languages, demonstrating the eventual equivalence.

C++ Algorithms, Boost and function currying

Brett Schuchert — Sat, 12 Jun 2010 23:41:00 -0500

I’ve been experimenting with C++ using the Eclipse CDT and gcc 4.4. Since I’m a fan of boost, I’ve been using that as well. I finally got into I realistic use of boost::bind.

I converted this:

int Dice::total() const {
  int total = 0;

  for(const_iterator current = dice.begin();
      current != dice.end();
      ++current)
    total += (*current)->faceValue();

  return total;
}

Into this:

int Dice::total() const {
  return std::accumulate(
      dice.begin(),
      dice.end(),
      0,
      bind(std::plus<int>(), _1, bind(&Die::faceValue, _2))
  );
}

To see how to go from the first version to the final version with lots of steps in between: http://schuchert.wikispaces.com/cpptraining.SummingAVector.

This is a first draft. I’ll be cleaning it up over the next few days. If you see typos, or if anything is not clear from the code, please let me know where. Also, if my interpretation of what boost is doing under the covers (there’s not much of that) is wrong, please correct me.

Thanks!

TDD in Clojure

Uncle Bob — Thu, 03 Jun 2010 12:33:00 -0500

OO is a tell-don’t-ask paradigm. Yes, I know people don’t always use it that way, but one of Kay’s original concepts was that objects were like cells in a living creature. The cells in a living creature do not ask any questions. They simply tell each other what to do. Neurons are tellers, not askers. Hormones are tellers not askers. In biological systems, (and in Kay’s original concept for OO) communication was half-duplex.

Clojure is a functional language. Functional languages are ask-dont-tell. Indeed, the whole notion of “tell” is to change the state of the system. In a functional program there is no state to change. So “telling” makes little sense.

When we use TDD to develop a tell-don’t-ask system, we start at the high level and write tests using mocks to make sure we are issuing the correct “tells”. We proceed from the top of the system to the bottom of the system. The last tests we write are for the utilities at the very bottom.

In an ask-don’t-tell system, data starts at the bottom and flows upwards. The operation of each function depends on the data fed to it by the lower level functions. There is no mocking framework. So we write tests that start at the bottom, and we work our way up the the top.

Therein lies the rub.

In a tell-don’t-ask system, the tells at the high level are relatively complex. They branch out into lower subsystems getting simpler, but more numerous as they descend. Testing these tells using mocks is not particularly difficult because we don’t need to depend on the lower level functions being there. The mocks make them irrelevant.

In an ask-don’t-tell system the asks at the low level are simple, but as the data moves upwards it gets grouped and composed into lists, maps, sets, and other complex data structures. At the top the data is in it’s most complex form. Writing tests against that complex data is difficult at best. And there is currently no way to mock out the lower levels¹ so all tests written at the high level depend on all the functions below.

The perception of writing tests from the bottom to the top can be horrific at first. Consider, for example, the Orbit program I just wrote. This program simulates N-body gravitation. Imagine that I am writing tests at the top level. I have three bodies at position Pa, Pb, and Pc. They have masses Ma, Mb, and Mc. They have velocity vectors of Va, Vb, Vc. The test I want to write needs to make sure that new positions Pa’, Pb’, Pc’, and new Velocity vectors Va’, Vb’, and Vc’ are computed correctly. How do I do that?

Should I write a test that looks like this?

test-update {
  Pa = (1,1)
  Ma = 2
  Va = (0,0)

  Pb = (1,2)
  Mb = 3
  Vb = (0,0)

  Pc = (4,5)
  Mc = 4
  Vc = (0,0)

  update-all

  Pa should == (1.096, 4.128)
  Va should == (0.096, 3.128)

  Pb should == (1.1571348402636772, 0.1571348402636774)
  Vb should == (0.15713484026367727, -1.8428651597363226)

  Pc should == (3.834148869802242, 4.818148869802242)
  Vc should == (-0.16585113019775796, -0.18185113019775795)
}

A test like this is awful. It’s loaded with magic numbers, and secret information. It tells me nothing about how the update-all function is working. It only tells me that it generated certain numbers. Are those numbers correct? How would I know?

But wait! I’m working in a functional language. That means that every function I call with certain inputs will always return the same value; no matter how many times I call it. Functions don’t change state! And that means that I can write my tests quite differently.

How does update-all work? Simple, given a list of objects it performs the following operations (written statefully):

  update-all(objects) {
    for each object in objects {
      accumulate-forces(object, objects)
    }
    for each object in objects {
      accelerate(object)
      reposition(object)
    }
  }

This is written in stateful form to make is easier for our non-functional friends to follow. First we accumulate the force of gravity between all the objects. This amounts to evaluating Newton’s F=Gm1m1/r^2 formula for each pair of objects, and adding up the force vectors.

Then, for each object we accelerate that object by applying the force vector to it’s mass, and adding the resultant delta-v vector to it’s velocity vector.

Then, for each object we reposition that object by applying the velocity vector to it’s current position.

Here’s the clojure code for update-all

(defn update-all [os]
  (reposition-all (accelerate-all (calculate-forces-on-all os))))

In this code you can clearly see the bottom-to-top flow of the application. First we calculate forces, then we accelerate, and finally we reposition.

Now, what do these -all functions look like? Here they are:

(defn calculate-forces-on-all [os]
  (map #(accumulate-forces % os) os))

(defn accelerate-all [os]
  (map accelerate os))

(defn reposition-all [os]
  (map reposition os))

If you don’t read clojure, don’t worry. the map function simply creates a new list from an old list by applying a function to each element of the old list. So in the case of reposition-all it simply calls reposition on the list of objects (os) producing a new list of objects that have been repositioned.

From this we can determine that the function of update-all is to call the three functions (accumulate-forces, accelerate, and reposition) on each element of the input list, producing a new list.

Notice how similar that is to a statement we might make about a high level method in an OO program. (It’s got to call these three functions on each element of the list). In an OO language we would mock out the three functions and just make sure they’d been called for each element. The calculations would be bypassed as irrelevant.

Oddly, we can make the same statement in clojure. Here’s the test for update-all

(testing "update-all" 
  (let [
    o1 (make-object ...)
    o2 (make-object ...)
    o3 (make-object ...)
    os [o1 o2 o3]
    us (update-all os)
    ]
    (is (= (nth us 0) (reposition (accelerate (accumulate-forces os o1)
    (is (= (nth us 1) (reposition (accelerate (accumulate-forces os o2)
    (is (= (nth us 2) (reposition (accelerate (accumulate-forces os o3)
    )
  )

If you don’t read clojure don’t worry. All this is saying is that we test the update-all function by calling the appropriate functions for each input object, and then see if the elements in the output list match them.

In an OO program we’d find this dangerous because of side-effects. We couldn’t be sure that the functions could safely be called without changing the state of some object in the system. But in a functional language it doesn’t matter how many times you call a function. So long as you pass in the same data, you will get the same result.

So this test simply checks that the appropriate three functions are getting called on each element of the list. This is exactly the same thing an OO programmer would do with a mock object!

Is TDD necessary in Clojure?

If you follow the code in the Orbit example, you’ll note that I wrote tests for all the computations, but did not write tests for the Swing-Gui. This is typical of the way that I work. I try to test all business rules, but I “fiddle” with the GUI until I like it.

If you look carefully you’ll find that amidst the GUI functions there are some “presentation” functions that could have been tested, but that I neglected to write with TDD[2]. These functions were the worst to get working. I continuously encountered NPEs and Illegal Cast exceptions while trying to get them to work.

My conclusion is that Clojure without TDD is just as much a nightmare as Java or Ruby without TDD.

Summary

In OO we tend to TDD our way from the top to the bottom by using Mocks. In Clojure we tend to TDD our way from the bottom to the top. In either case we can compose our tests in terms of the functions they should call on the lower level objects. In the case of OO we use mocks to tell us if the functions have been called properly. This protects us from side-effects and allows us to decouple our tests from the whole system. In clojure we can rely on the fact that the language is functional, and that no matter how many times you call a function it will return the same value.

¹ Brian Marick is working on something that looks a lot like a mocking framework for clojure. If his ideas pan out, we may be able to TDD from the top to the bottom in Clojure.

² This is an unconscious game we all play with ourselves. When we have a segment of code that we consider to be immune to TDD (like GUI) then we unconsciously move lots of otherwise testable code into that segment. Yes, I heard my green band complain every time I did it; but I ignored it because I was in the GUI. Whoops.

Orbit in Clojure

Uncle Bob — Wed, 02 Jun 2010 17:08:56 -0500

I spent the last two days (in between the usual BS) writing a simple orbital simulator in Clojure using Java interop with Swing. This was a very pleasant experience, and I like the way the code turned out – even the swing code!

You can see the source code here

Those of you who are experienced with Clojure, I’d like your opinion on my use of namespaces and modules and other issues of style.

Those of you who are not experienced with Clojure, should start. You might want to use this application as a tutorial.

And just have fun watching the simulation of the coalescence of an accretion disk around a newly formed star.

A Coverage Metric That Matters

Michael Feathers — Fri, 28 May 2010 05:39:00 -0500

How much test coverage should your code have? 80%? 90%? If you’ve been writing tests from the beginning of your project, you probably have a percentage that hovers around 90%, but what about the typical project? The project which was started years ago, and contains hundreds of thousands of lines of code? Or millions of lines of code? What can we expect from it?

One of the things that I know is that in these code bases, one could spend one’s entire working life writing tests without doing anything else. There’s simply that much untested code. It’s better to write tests for the new code that you write and write tests for existing code you have to change, at the time you have to change it. Over time, you get more coverage, but your coverage percentage isn’t a goal. The goal is to make your changes safely. In a large existing code base, you may never get more than 20% coverage over its lifetime.

Changes occur in clusters in applications. There’s some code that you will simply never change and there’s other areas of code which change quite often. The other day it occurred to me that we could use that fact to arrive at a better metric, one that is a bit less disheartening and also gives us a sense of our true progress.

The metric I’m thinking about is percentage of commits on files which are covered by tests relative to the number of commits on files without tests.

In the beginning, you can expect to have a very low percentage, but as you start to get tests in place for changes that you make, your percentage will rise rapidly. If you write tests for all of your changes, it will continue to rise. At a certain point, you may want to track only a window of commits, say, the commits which have happened only in the last year. When you do this, you can end up very close to 100%.

If you think this through, it might seem a bit dodgy on a couple of fronts. The first is that having tests for code within a file does not mean that that code is completely covered by those tests. But, I often find that the hardest part of getting started with unit testing is getting classes isolated enough from their dependencies to be testable in a harness at all. Another dodgy bit is the fact that once you get some tests in place for a file, all of the commits you’ve ever done for that file count in the percentage. Again, that’s okay for fundamentally the same reason. Once you start getting coverage, you are in a good position with that particular code.

What about the moving window? If you track this metric over, say, the last N months of commits, you’ll progressively lose information about the code that you just aren’t changing. To me, that’s fine. Coverage matters for the code that we are changing.

The metric I’m considering (maybe we can call it ‘change coverage’) gives us information about how tests are really impacting our day to day work. Moreover, it’s likely that it would be a good motivational tool, and really, that’s one of the things that a good metric should be.

Hello World Revisited

Brett Schuchert — Thu, 20 May 2010 10:12:00 -0500

Surprising revelations while taking a TDD approach to writing hello world.

Here it nearly 21 years since I started writing in C++ (and more for C+) and I realize I’ve been blindly writing main functions to actually do something.

This insanity must stop!

What am I talking about? Read it here.

Clojure Prime Factors

Uncle Bob — Sat, 15 May 2010 19:09:59 -0500

Can anyone create a simpler version of prime factors in Clojure?

(ns primeFactors)

(defn of
  ([n]
    (of [] n 2))
  ([factors n candidate]
    (cond
      (= n 1) factors
      (= 0 (rem n candidate)) (recur (conj factors candidate) (quot n candidate) candidate)
      (> candidate (Math/sqrt n)) (conj factors n)
      :else (recur factors n (inc candidate))
      )
    )
  )

TDD is wasting time if...

Brett Schuchert — Sat, 08 May 2010 10:36:00 -0500

You have no design sense.

OK, discuss.

Sufficient Design means Damned Good Design.

Uncle Bob — Wed, 28 Apr 2010 18:02:10 -0500

@JoshuaKerievsky wrote a blog entitled “Sufficient Design”.

Josh makes this point:

‘Yet some programmers argue that the software design quality of every last piece of code ought to be as high as possible. “Produce clean code or you are not a software craftsman!”’

He goes on to say:

“Yet ultimately the craftsmanship advice fails to consider simple economics: If you take the time to craft code of little or moderate value to users, you’re wasting time and money.”

Now this sounds like heresy, and I can imagine that software craftsmanship supporters (like me) are ready to storm the halls of Industrial Logic and string the blighter up by his toenails!

But hold on there craftsmen, don’t get the pitchforks out yet. Look at the scenario that Josh describes. It’s quite revealing.

Josh’s example of “not being a craftsman” is his niggling worry over a function that returns a string but in one derivative case ought to return a void.

Horrors! Horrors! He left a bit of evolved cruft in the design! Revoke his craftsman license and sick the dogs on him!

Ah, but wait. He also says that he spent a half-day trying to refactor this into a better shape but eventually had to give up.

The fool! The nincompoop! The anti-craftsman! A pox on him and all his ilk!

OK, enough hyperbole. It seems to me that Josh was behaving exactly as a craftsman should. He was worrying about exactly the kinds of things a craftsman ought to worry about. He was making the kinds of pragmatic decisions a craftsman ought to make. He was not leaving a huge mess in the system and rushing on to the next feature to implement. Instead he was taking care of his code. The fact that he put so much energy, time, and thought into such a small issue as an inconsistent return type speaks volumes for Josh’s integrity as a software craftsman.

So, as far as Josh Kerievsky is concerned “Sufficient Design” is “Pretty Damned Good Design”.

Look, all our software will have little tiny warts. There’s no such thing as a perfect system. Craftsmen like Josh work hard to keep those warts as small as possible, and to sequester them into parts of the system where they can do the least harm.

So the only aspect of Josh’s post that I disagree with is his contention that the “craftsman” message is one of unrelenting perfection. Craftsmen are first and foremost pragmatists. They seek very high quality code; but they are not so consumed with perfection that they make foolish economic tradeoffs.

Certification: Don't Waste Your Time!

Uncle Bob — Tue, 27 Apr 2010 15:11:00 -0500

As I have said before, there’s nothing particularly wrong with the current mania for certification. If you want to be certified at the cost of a 2-day course, by all means get certified. If you want to certify people for attending your 2-day course, by all means hold the course and hand out the certificates. It’s all good. Make money! Be fruitful and multiply!

But be careful not to waste your time.

How could certification be a waste of time? That depends on your motive.

If you are getting certified in order to impress someone, like a hiring manager, or a recruiter, or your peers, you are wasting your time. Nobody worth impressing is going to find this certification impressive. Indeed, the people you really want to impress are likely to find it a bit mundane.
If you are getting certified in order to get hired, you are wasting your time. Nobody is going to hire you simply because of that “C”, and nobody worth working for is going to require that “C”.
If you have decided to hire only certified people, you are wasting your time. The population of certified people is not richer in talent, skill, or knowledge. Indeed, it may be poorer. Remember, those who have talent don’t need certification as much as those who don’t.

What part of certification is not a waste of time?

The primary benefit you are getting is the instruction; but be careful: There are lots of pretty mediocre instructors out there. Some of the instructors teach pretty good courses, but others are just hoping that all you care about is the certification.
So do a little research and find the best instructors. You may, in fact, find that some of the best instructors and courses do not offer certification. That shouldn’t stop you from considering them.

But wait, aren’t the instructors certified as trainers?

Sure. They paid the money and took the course to become a certified trainer.
But that doesn’t necessarily mean that they:
- are a good instructor.
- know what they are teaching.
- have done what they are teaching.
- are qualified to teach you.

OK, but isn’t there some benefit to the certification itself?

Sure. A nice piece of paper.

Pair Management

Michael Feathers — Mon, 26 Apr 2010 04:52:00 -0500

Every once in a while in life, you get a lucky break. My lucky break was right as I was graduating from college. One of my favorite professors told the class that there were some people interviewing potential hires on campus and they were looking for a programmer. I remember standing dutifully outside his office, in line, waiting for my little mini-interview. When it was my turn, they didn’t see me for a little while – the door stayed closed after they let the last interviewee out. I found out later, that my professor was telling the interviewers that I was the guy that they wanted.

I came in, and showed them some code I’d written outside of schoolwork, and we talked about it. I told them about my passion for programming languages and they said that the job would be at a research department and they needed someone to design and implement a new programming language. I was over the moon. It was my dream job. And, sure enough, it was real. When I was hired, I sat down with my boss and went over ideas for syntax with him, and he pretty much left me alone for a couple of months to design the language, and implement a compiler for it. I was sitting in my first job, with some real responsibility, struggling with YACC, designing a symbol table. It was all very heady.

In retrospect, I was extremely lucky. I not only had more individual trust and responsibility than many developers ever get in their career, I also became part of an extremely good working environment. The guys who hired me, David Lopez de Quintana, and Carlos Perez, also ran the team I was part of. We were a group of about four which eventually grew to about 12. Although, we weren’t working on one big integrated project, we saw ourselves as a team and we had a great cooperative culture. We each had our area of expertise, and most of us were in our 20s, ready to stretch. We tackled all sorts of interesting problems at the intersection of software, mechanics, electronics, optics, physics, chemistry and biology.

About six months into the job, I had an odd experience. Carlos called me to his office and told me he was going to give me my review. I was a bit shocked. The way I understood things, David was actually the guy I worked for, and Carlos headed up part of the team but not my part. Carlos gave me a big smile and said “Yeah, well, you’ve been working for me for the past few months, but we didn’t tell you.”

The review went great and I happily went back to work. I always knew that, on paper we were two different teams, but Dave and Carlos treated us as one big team. We had one big meeting every week or so, and they shared responsibilities and pretty much kept us from having to care about anything outside the work we had to do.

It was a very cool experience. Sometimes, Dave would spend more time outside the team, sort of managing our interface with the rest of the organization. Part of the group wrote software to support research but we had customers outside of our group too. Dave would be off at meetings, and Carlos would handle the day-to-day stuff with our team. Sometimes they reversed the roles. Dave would handle us day to day and Carlos would manage the world outside. The thing is, no one was asking them to allocate responsibilities that way; they just found it a comfortable way of working. It was pair management. Sort of like pair-programming, but at the management level.

The funny thing is that I haven’t seen this much since. Yes, I’ve visited organizations with teams that do something close to this, but it seems like all sorts of forces are aligned against it. The thing which made it work for Dave and Carlos is that they were very good friends. They were hired at the same time in entry-level non-management positions and they rose through the ranks together. They saw each other outside work all the time and joked together at work, making it all a very rich experience. Importantly, their manager never seemed to put them in competition with each other. As I reflect on it more, I think that was the key. Managers are often in competition with each other and that can impede the sort of collaboration and culture that Dave and Carlos built up. They created a wonderful high-performing team. It was one of those once in a lifetime things, and it’s a time that I’ll always remember.

Whenever I think about team health and culture, that first team I joined is my benchmark. I don’t know where the magic came from, but it was there and I think that the culture that Dave and Carlos created with pair management was a very important part of it. In their interactions with each other, they provided a model for us, showing us how to treat each other at work.

The Tricky Bit

Uncle Bob — Fri, 23 Apr 2010 04:28:35 -0500

I once heard a story about the early days of the Concorde. A british MP was on board for a demonstration flight. As the jet went supersonic he disappointedly commented to one of the designers that it didn’t feel any different at all. The designer beamed and said: “Yes, that was the tricky bit.”

I wonder if the MP would have been happier if the plane had begun to vibrate and shake and make terrifying noises.

While at #openvolcano10, Gojko Adzic and Steve Freeman told the story of a company in which one team, among many, had gone agile. Over time that team managed to get into a nice rhythm, passing it’s continuous builds, adding lots of business value, working normal hours, and keeping their customers happy. Meanwhile other teams at this company were facing delays, defects, frustrated customers, and lots of overtime.

The managers at this company looked at the agile team, working like a well-oiled machine, and looked at all the other teams toiling in vain at the bilge pump, and came to the conclusion that the project that the agile team was working on was too simple to keep them busy. That the other teams’ projects were far more difficult. It didn’t even occur to them that the agile team had figured out the tricky bit.

The R.E.A.L.I.T.Y. principles of Agile Software Certification.

Uncle Bob — Wed, 21 Apr 2010 05:29:00 -0500

As you are probably aware the Scrum Alliance is planning to offer a Certified Scrum Developer program. You can read about it here.

Interestingly enough Microsoft, in collaboration with Ken Schwaber, is offering a Professional Scrum Developer program. You can read about that here (look carefully at the url).

The scrum alliance program seems to be about agile skills. The Microsoft program seems to be about Visual Studio. So all is right with the world.

Is there anything wrong with this? Is this somehow immoral or bad? Not at all. This is just people doing what people do: Selling services and making money. I’m all for it. I expect the courses to be of a quality that you’d expect from professionals; and I’m sure that students will learn useful information. Indeed, from time to time, even I have offered CSM courses taught by people I trust and respect.

There are, however, some claims that need to be dealt with. The most important is that these programs shorten the recruiting process.

I’ve been thinking about offering a Certified Recruiter for Agile Professionals course. This fifty-two week course will teach, and reteach, a number of principles of Agile Software Certification. I call this repertoire of principles: R.E.A.L.I.T.Y.

The first principle is the Redaction of Certification Principle (RCP). The principle states:

Certifications generally certify nothing whatever about experience, knowledge, or skill. Generally they certify that the certificate holder has attended (or at least paid to attend) a course. Perhaps they took (and maybe even passed) an exam based on that course.

Based on this principle, any recruiter taking my Certified Recruiter for Agile Professionals course will not be allowed to complete the course or receive their certification until they have taken the following oath.

As a Cerftified Recruiter for Agile Professionals I solemnly swear before Knuth and all here present:

That before I read a resume, I will find every use of the word “certified” on that document and redact it with whiteout.

That if a candidate uses the word “certified” in an interview, I will ask the candidate to repeat him- or herself without using that word.

That I will pay no attention whatever to any implications of that word, nor will that word in any way influence my opinions regarding that candidate.

This oath is an exemplar of the Certification Nullification Principle (CNP), which is another of the R.E.A.L.I.T.Y. principles of common sense and moderately competent thinking. The principle states:

The word “certified”, when used in the context of Agile Software Development, is a filler word that has no bearing on anything salient or interesting about individuals, skills, knowledge, or anything else. It is a marketing word similar to “new”, “improved”, “natural” and “organic”. It can safely be removed from all documents and discussions without changing their meaning.

This principle leads to the well known Certification Uncertainty Principle (CUP), yet another of the R.E.A.L.I.T.Y. principles. This principle states:

Any acronym that includes the letter ‘C’ standing for the word “Certification” can be safely changed into a similar acronym that eliminates the letter ‘C’ and puts a ’?’ at the end. This transformation of the acronym vastly improves the meaning of the orginal.

That last sentence probably requires some justification. After all, extraordinary claims require extraordinary substantiation. So lets try a few experiments:

Statement                    Ac'nm   CUP  Resulting Statement 
---------------------------- -----   ---- ---------------------
Certified Scrum Master        CSM    SM?  Scrum Master?
Certified Scrum Developer     CSD    SD?  Scrum Developer?
Certified Scrum Trainer       CST    ST?  Scrum Trainer?
Certified Scrum Professional  CSP    SP?  Scrum Professional?
Certified Scrum Product Owner CSPO   SPO? Scrum Product Owner?
Certified Scrum Coach         CSC    SC?  Scrum Coach?

As we can see, the meanings of the statements are indeed clarified. In my course the Certified Recruiter for Agile Professionals is taught that the question mark is the most significant aspect of each of the resultant statements.

In recent weeks we have uncovered a potentially profound new principle which we may find necessary to include in the R.E.A.L.I.T.Y. repertoire. This is the so-called Scrum Exclusion Principle (SEP) which tentatively states:

Wherever the word SCRUM appears in any statement, or is represented within any acronym, it can be safely excluded without changing any meaning.

The following table show just how profound the effects of this principle are:

Statement                    Ac'nm   CUP  SEP   Resulting Statement 
---------------------------- -----   ---- ----  ---------------------
Certified Scrum Master        CSM    SM?  M?    Master?
Certified Scrum Developer     CSD    SD?  D?    Developer?
Certified Scrum Trainer       CST    ST?  T?    Trainer?
Certified Scrum Professional  CSP    SP?  P?    Professional?
Certified Scrum Product Owner CSPO   SPO? PO?   Product Owner?
Certified Scrum Coach         CSC    SC?  C?    Coach?

We are not yet willing to say that these transformation are a reliable feature of nature. We can only say that, at this point, there are very troubling implications that support it.

Our final advice to Certified Recruiters for Agile Professionals is based on one final principle: The Smoke and Mirrors principle, (SAM) which states:

Well… You know what it states.

The Mayans are Coming!

Uncle Bob — Sat, 17 Apr 2010 11:22:59 -0500

This one is just a quickie; too long for a tweet.

I gave a lightning keynote (actually a key-rant) today at #accu2010. It was about a bet between John Lakos and Kevlin Henney. Kevlin “won” the bet last night by showing that he could get a C++ program to behave in a way that John said couldn’t be done. My rant was about how both of them were wrong (not technically, but morally) and so they both owed me the sum of the bet. (I’m always happy to insert myself into any available revenue stream.)

The problem they had coded, and for which I presented a far superior solution in Clojure, was that old work-horse of prob/stat: “What’s the probability that out of N people in a room two will have the same birthday?”

Now, just to show you how strange James Bach is (and why his strangeness is in some bizarre way actually valuable); once my talk was done, James got up and said “What if some of the people in the room are ancient Mayans who use a completely different kind of calendar making it impossible to compare their birthdays?”

I’ve long said that testers were deviant souls with twisted minds that delight in reducing helpless software systems into piles of smoking slag. I believe James may be the exemplar.

Parts 5 and 6 of the 4-part series

Brett Schuchert — Fri, 16 Apr 2010 20:57:00 -0500

The title says it all. I was bothered by a few things in the Shunting Yard Algorithm (actually many more things), but I felt compelled to fix two of those things.

So if you have a look at the album, you’ll notice 2 more videos:

Video 5 of 4: Remove the need for spaces between tokens.
Video 6 of 4: Remove duplication of operators in algorithm and tokenizer.

Hope these are interesting or at least entertaining.

Sapient Testing: The "Professionalism" meme.

Uncle Bob — Thu, 15 Apr 2010 06:18:00 -0500

James Bach gave a stirring keynote today at ACCU 2010. He described a vision of testing that our industry sorely needs. Towit: Testing requires sapience.

Testing, according to Bach, is not about assuring conformance to requirements; rather it is about understanding the requirements. Even that’s not quite right. It is not sufficient to simply understand and verify the requirements. A good tester uses the behavior of the system and the descriptions in the requirements, (and face-to-face interaction with the authors of both) to understand the motivation behind the system. Ultimately it is the tester’s job to divine the system that the customer imagined; and then to illuminate those parts of the system that are not consistent with that imagination.

It seems to me that James is attempting to define “professionalism” as it applies to testing. A professional tester does not blindly follow a test plan. A professional tester does not simply write test plans that reflect the stated requirements. Rather a professional tester takes responsibility for interpreting the requirements with intelligence. He tests, not only the system, but also (and more importantly) the assumptions of the programmers, and specifiers.

I like this view. I like it a lot. I like the fact that testers are seeking professionalism in the same way that developer are. I like the fact that testing is becoming a craft, and that people like James are passionate about that craft. There may yet be hope for our industry!

There has been a long standing frission between James’ view of testing and the Agile emphasis on TDD and automated tests. Agilists have been very focussed on creating suites of automated tests, and exposing the insanity (and inanity) of huge manual testing suites. This focus can be (and has been) misinterpreted as an anti-tester bias.

It seems to me that professional testers are completely compatible with agile development. No, that’s wrong. I think professional testers are utterly essential to agile development. I don’t want testers who rotely execute brain-dead manual test plans. I want testers using their brains! I want testers to be partners in the effort to create world-class, high-quality software. As a professional developer I want – I need – professional testers helping me find my blind spots, illuminating the naivete of my assumptions, and partnering with me to satisfy the needs of our customers.

C# TDD Videos - You asked for them

Brett Schuchert — Wed, 14 Apr 2010 23:29:00 -0500

Several people asked for them, so here is a series of 4 videos. The first series on the RPN calculator in Java was a bit rough, these are even rougher.

Even so, hope you find them valuable.

Shunting Yard Algorithm in C# Video Album

Comments and feedback welcome.

Open Spaces at conferences, what's your take?

Brett Schuchert — Wed, 07 Apr 2010 10:39:00 -0500

I’m the host for an open space this year at Agile Testing Days 2010, Berlin (October 4th – 7th, the open space being the 7th). I’ve attended a few open spaces and I have some ideas on what the role of host might entail.

But, I know I’m better at collecting requirements than sourcing them, so what have you experienced that worked. What has not worked. Any advice you want to give me?

Please help me better understand what I can do to facilitate a successful open space.

What questions should I be asking?

Do you think having a few things scheduled up front in one room, and several unscheduled rooms left to be determined October 4th – 6th, through a a mix if you will, is an OK thing to do, or should what happens be left completely open?

Or, leave everything completely open until the 7th, then start with the “traditional” introductions and let the agenda form then an there?

I’m aware of a few models of open spaces. What I really want to know is, what works for you.

20% more bugs? Or 20% less features?

Uncle Bob — Tue, 06 Apr 2010 22:03:23 -0500

People often make the argument that time to market is more important that quality. I’m not sure just what they mean by that. Do they mean that it’s ok if 20% of the features don’t work so long as they deliver quickly? If so, that’s just stupid. Why not develop 20% fewer features, and develop them well. It seems to me that choosing which 20% you are not going to develop and then choosing to develop the other 80% to a high standard of quality is a better management decision than telling the developers to work sloppily.

First Pass Completed: Rough Draft TDD Demonstration Videos

Brett Schuchert — Mon, 05 Apr 2010 01:15:00 -0500

As promised, I’ve made a complete sweep through a series of videos. You can find all of them: here.

These videos include several warts and false starts. Depending on the interest and feedback, I’ll redo these at some point in the future.

Question: Should I repeat this series in C#, or some other language? Some people have expressed interest in this. It’s probably 15 hours of work in C#, so I’d need to know it was worth the effort. What’s your opinion on that?

The videos are meant to be watched in order. The link above is to an album that is in the correct order. Here are the direct links (also in the intended order):

I’ve already received several comments both here on the blog as well as with the videos. I’ll keep track of those comments and incorporate the ones that fit for me.

Each video has a link on its page to download it. However, to download a video, you will have to create an account and log in. So here are the links, these won’t work without first creating an account (I’ll update original blog with these as well):

Some Rough Draft TDD Demonstration Videos

Brett Schuchert — Tue, 30 Mar 2010 21:01:00 -0500

I’m doing a series of videos on TDD. The ultimate result will be a much more polished version with embedded slides, and such. But as a part of the development process, I’m creating scratch videos.

Much of what you see in these videos will be in the final versions, but those are far in the future relative to this work.

Hope you find them interesting.

Comments welcome.

Here is what is already available:

Next up:

Removing violation of Open/Closed principle
Removing duplication in operations with a combination of the Strategy pattern and the Template Method pattern
Adding new operators after the removal of duplication.
Reducing coupling by using the Abstract Factory pattern, Dependency Inversion and Dependency Injection
Adding a few more operations
Allowing the creation of complex “programs” or “macros” by using the Composite pattern – and avoiding Liskov Substitution Principle inherent in the GoF version of the pattern
Driving the calculator via FitNesse + Slim

Anyway, that’s the plan. I’ll try to add each of these videos over the next few weeks.

Injecting Those Dependencies

Brett Schuchert — Mon, 22 Mar 2010 16:00:00 -0500

Last week I was teaching a class with a good group of C++ developers in Chicago. They were up on C++, current standardization efforts, virtual machines, performance analysis and tuning. At one point I mentioned a metric for virtual method dispatch on the JVM and one of the students used my numbers to work backwards to determine the number of instructions on different processors.

I was teaching our Working Effectively with Legacy Code class. Michael Feathers usually teaches this class, but he was busy (probably working with legacy code somewhere, which, ironically, is why he wrote the book – to solve the problem once and for all and to stop working with legacy code).

In an early project we have some problem code (the class was in Java, but we discussed C++ quite a bit):

    public void addEvent(Event event) {
        event.added();
        events.add(event);
        mailService.sendMail("jacques@spg1.com", "Event Notification", event
                .toString());
        display.showEvent(event);
    }

The problem with this code, however, comes from what happens in the constructor:

    public Scheduler(String owner, SchedulerDisplay display) {
        this.owner = owner;

        mailService = MailService.getInstance();
        this.display = new SchedulerDisplay();
    }

The display object is associated with a real device (OK, not in our simulation, but you get the point). We need to fix that dependency.

So we traditionally talk about a few ways to address that:

Create an interface and then have a concrete and test-double implementation
Create a test subclass

Those are a few typical ways to handle this problem, but given that the students also worked in C++ and were very concerned with performance (and dynamic binding), so we discussed more alternatives: 1. Link seam – build the system using the original SchedulerDispaly header file but link in a different version of the class. 2. Template parameter (compiler seam) – introduce a template parameter and compile in the dependency rather than link it in.

#pragma once

template<class D> class Scheduler
{
public:
  Scheduler(D &display);
  ~Scheduler();
  void handleEvent();

private:
  D &display;
};

So here are 4 ways to remove the dependency (working bottom up in a sense):

Link seam: link in a different version of SchedulerDisplay
Compiler seam: use a template parameter
Dynamic seam: create a test subclass that removes external dependency
Dynamic seam: extract an interface from existing concrete class, and make Scheduler depend on the new interface.

For C++, the first option might be a good option if it allows a very quick build/test cycle. Imagine using a class that brings in several unwanted dependencies. By making a test version replacement and linking to it, you might be able to get up and running quickly. On the other hand, you have to have a custom build target, which is really outside of the language.

The second option is good for C++ and maybe C#, but not Java. Java’s implementation of generics is so bad, that it requires an interface for this particular case, so you end up with the final option anyway. Using templates puts the seam into the language rather than the build system. It will affect clients because either:

They will have to provide a template parameter
Or, by providing a default value for the template parameter, clients will be aware of a class they were not previously aware of. Still, a viable option, but it will increase build time at least a little.

The third option may not work in any language without changing the original problem code. Imagine a no argument constructor that performs some static initialization. Without changing the constructor, you cannot guarantee that this approach will work. In any case, the problem as presented requires a change since the constructor was doing the initialization of the display; it did not allow for dependency injection.

The final option requires a bit more change, but is pretty flexible if you can afford the virtual method overhead. In Java this isn’t much of an issue (for a number of reasons, mostly related to effective JIT optimization). The cost is a bit more of a problem in C++ because introducing a first virtual method is a big deal. Personally, I don’t work on embedded systems, so I can generally ignore that problem. But this is not something to simply ignore, context does matter.

So which option is the best?

None of the above.

As a developer, you should probably be aware of all of these options (or maybe not the template option of you’re strictly a Java developer). Then, when you are faced with a legacy coding problem, you’ll have more tools from which to choose.

Developer Certification WTF?

Uncle Bob — Sun, 07 Mar 2010 12:16:43 -0600

Watch the following video. It will convince you that we have to do something about the horrible state of software engineering.

Bad Code from unclebob on Vimeo.

How could any team of disciplined professionals have produced a wretched mess like that? Clearly they were ignorant of good practices. Clearly they were inexperienced novices. Clearly their priorities and values were all wrong. If only they had been taught good coding practices, and good development skills. If only we could have gotten to them before they made such a horrible mess.

What we need to do is create a certification program that provides developers with the knowledge and skills that they need. This program will involve a course that teaches good development practice, and a certification that they are now knowledgeable developers. They can use that certification to prove to their employers and their fellow professionals that they are worthy of being considered true and clean software developers.

That’s what we need. Right?....

Eagle’s Entrails! Deer Droppings! and Elephant Phlatulence!

I’m sorry, am I being vague? Is my opinion not clear enough? All right then, allow me to elaborate.

What Problem does Certification solve?

Certainly there are certification organizations who have the problem of dwindling revenues. A new developer certification program would likely solve their problem.

And just as certainly there are tin-men who need hearts, lions who need courage, scarecrows who need brains, and developers who need self-esteem. A new developer certification program might just help those lackluster developers.

But who else is served by a developer certification program? Employers?

The Trials of Hercules.

Consider Doctors and Lawyers. When a medical student earns an M.D., or a law student passes the Bar, that’s an achievement. The certification, in those cases is deeply meaningful because the certification is not easily won. To get it you have to spend many years, and many tens of thousands of dollars. What’s more, you have to acquire in-depth knowledge and at least a modicum of provable skill. You have to do something significant.

If we had a developer certification like that, then employers might find it useful. But I know of no developer certification program (except for one that I’ll mention later) that offers proof that the developer has accomplished anything of significance. Most certification programs prove little more than that the “developer” paid to attend a 3-5 day course. As an employer, I’m not particularly interested in the ability of a developer to pay to attend a 3-5 day course. As an employer I want to know what the developer can do.

Michaelangelo’s Apprentice.

If you were hiring a guitarist for your band, you might be very impressed with someone who toured with Clapton. You don’t get to tour with Clapton unless you are talented. But more than just talented, you have to show up every day, you have to work well with others, you have to be willing to get the job done under pressure. Clapton doesn’t work with slouches!

A signed letter of recommendation from Clapton is a certification of significance! Not just anyone can get one of those. That letter will impress potential employers forever after.

As an employer, I want to hire people who have succeeded at working with others in the past. The more I trust those others, the better! No developer certification program that I know of (except for one that I’ll mention later) provides that kind of credible reference.

Over-promise and Under-deliver.

The certification programs that I know of (except for one that I’ll mention later) make implicit promises that they can’t hope to begin to deliver. The implied promise is that the certificant has been shown to have some significant skill, ability, or knowledge. The reality is that you have to look at the word “significant” through the wrong end of some very powerful binoculars before it matches the truth. Indeed, there are certification programs that show absolutely nothing about the certificant except that they bought a seat in a class.

Notice that I said that the promises were implicit. That’s the whole key to making lightweight certifications profitable. The goal is to imply that the benefits of certification are so huge that people will pay to get the certification. Often that implication can be as simple and subtle as the choice of words used to name the certification.

Consider, for example, the title of “Certified Development Chief” imprinted on the certificate of someone who just took a “Development Chief” course. Who wouldn’t want to be a Certified Development Chief? And if you aren’t a Certified Development Chief, then what kind of Development Chief are you?

A course, a card, and a Bridge to sell.

What significant accomplishment can a developer make by paying to attend (or even actually attending) an 3-5 day course? How credible a recommendation about any of the 20-odd students could an instructor of such a course make? I can answer both questions in two words, with apologies to SNL. “Jack” and “Cheese”. The accomplishment is miniscule, and the recommendation is meaningless.

So as an employer of developers I think you can take your course and card and show it to some other sucker. It don’t impress me much. Go away and come back later when you’ve accomplished the trials of Hercules and apprenticed under Michaelangelo. And make sure you bring letters of reference. NEXT!

The one I said I’d mention later.

I’ve got an idea for a certification program that just might work. Now, bear with me because this is a little complicated. It’s an idea so revolutionary, so different, so incredibly new that I’m getting giddy just thinking about it.

What if we asked young graduates to actually get jobs as “interns” for awhile. What if we gave out responsibility to them in small incremental measures. What if they gradually learned more and more things while working for us. What if we guided them to ever greater accomplishments. What if we slowly gave them trials of Hercules to follows, and had them work with Michaelangelos.

Of course some of our people might leave and go to greener pastures. We’ll need to replace them. Also, our company might succeed and grow, so we’ll need to hire more good developers. So what if we…

OK, now hold on to your hat because this is where it gets really tricky. You might want to get up and walk around a bit, get a diet coke, eat a twinkie, or something to get your brain working.

Ready? OK, here we go…

So what if we… interviewed ... candidate developers? What if we asked them what accomplishments they’d achieved in previous employment. What if we asked them who they used to work with. And… ok, this is even wilder ... what if we called those references and checked them out?

Are you still with me?

What if we….

Naaaaahhhhh. Dumb idea.

Honestly, I think I’ve wasted your time. It’s a stupid idea. Never mind. We should all just go get certified instead.

One way to configure FitNesse for team Development

Brett Schuchert — Fri, 19 Feb 2010 14:12:00 -0600

Here are some notes (and instructions) on a way to configure FitNesse for use in a team environment: Page Hierarchy For Team Development.

Comments and suggestions welcome.

Updated ConTest Instructions

Brett Schuchert — Thu, 18 Feb 2010 12:12:00 -0600

Just updated the instructions for using ConTest – a tool to assist in testing concurrent applications for Java. Check them out here: ConTest Instructions

DotNet Development Using Parallels Is No Longer Painful

Bob Koss — Tue, 16 Feb 2010 15:16:00 -0600

Because of the travel involved with being an Object Mentor, my main computer is my laptop, a MacBookPro. I have to be able to work wherever I happen to be, whether I’m at home, at Object Mentor Galactic Headquarters, or in a plane, train, or automobile. We’re allowed to buy whatever we want, and we all have independently settled on Macs, running OS X. I’ve developed on lots of different platforms over the years, but I find that OS X is the most productive development environment that I’ve ever used.

Besides the travel, we have to be jacks of all trades. One week I’m programming in Java, the next week I’m teaching a design class, and the next I’m developing in .NET. I’m able to do it all on my Mac, even the .NET work, through the magic of virtualization. I choose to run Parallels so that I’m able to run Windows as a process under OS X. Windows crashing isn’t a problem anymore, in that it doesn’t bring my entire computer down. One process just dies (or I’m forced to kill it) and I just start it up again, and all the other running processes just keep right on working.

I’m a backup fanatic. Besides having bootable backups produced using SuperDuper, I just love the way TimeMachine is built into OS X, allowing me to recover versions of any file on my computer going back months. And it’s all automatic. TimeMachine just runs in the background, keeping me safe.

But the thing about Parallels + TimeMachine that I just hate, is that Parallels stores all of Windows as one big file. A big one. Huge! So whenever I’m working in Parallels, the one big file is being changed and is going to be backed up again the next time TimeMachine runs. Needless to say this takes up a lot of space on my TimeMachine drive, but it’s also a damn nuisance. About the time that TimeMachine finishes its backup, it’s time to start all over again just because of the file size. Yet I dare not back up. I’m in the camp that believes it’s not a question of if your computer will fail, but when.

Because Parallels creates a share with my home directory on the OS X side, I thought about just keeping my source code there. That actually works, until I try to build or do anything with the code other than edit it, and then I get a security violation of some sort.

While I’m developing in Windows, I do frequent commits to git, my new favorite source code control system. It’s easy to go back to a previous version of any file, something that I can’t say about ctrl-z in Visual Studio.

The light finally lit up over my bald head. Git is a distributed version control system. Distributed doesn’t have to be any farther away than the OS X side of my file system.

I set up a git repository in OS X for every project that I work on. Then I can push/pull from/to it, just like I would do if I was working on a team project hosted on github. Why don’t I just get a private github account? This is easier and I don’t have to be connected to the net to do push/pulls.

Here are the details in case anyone wants to try it.

In a Terminal window in OS X:


            MBP:~ bob$ cd Archive/git
        MBP:git bob$ mkdir PresenterAsStateMachine.git
        MBP:git bob$ cd PresenterAsStateMachine.git
        MBP:PresenterAsStateMachine.git bob$ 

        MBP:PresenterAsStateMachine.git bob$ git --bare init
        Initialized empty Git repository in /Users/bob/Archive/git/PresenterAsStateMachine.git/
        MBP:PresenterAsStateMachine.git bob$

then, in Windows, using the bash window that comes with git:


            $ git init
        Initialized empty Git repository in c:/Documents and Settings/Administrator/My Documents/Visual Studio 2008/Projects/PresenterAsStateMachine/.git/

        $ git add .

        $ git commit -a -m "Initial commit. Ready to add state machine." 
        [master (root-commit) f844369] Initial commit.  Ready to add state machine.
         16 files changed, 795 insertions(+), 0 deletions(-)
         create mode 100644 .gitignore
         create mode 100644 PresenterAsStateMachine.sln
         create mode 100644 PresenterAsStateMachine.suo
         create mode 100644 PresenterAsStateMachine/Form1.Designer.cs
         create mode 100644 PresenterAsStateMachine/Form1.cs
         create mode 100644 PresenterAsStateMachine/Form1.resx
         create mode 100644 PresenterAsStateMachine/LoginPresenter.cs
         create mode 100644 PresenterAsStateMachine/LoginPresenterTest.cs
         create mode 100644 PresenterAsStateMachine/LoginView.cs
         create mode 100644 PresenterAsStateMachine/PresenterAsStateMachine.csproj
         create mode 100644 PresenterAsStateMachine/Program.cs
         create mode 100644 PresenterAsStateMachine/Properties/AssemblyInfo.cs
         create mode 100644 PresenterAsStateMachine/Properties/Resources.Designer.cs
         create mode 100644 PresenterAsStateMachine/Properties/Resources.resx
         create mode 100644 PresenterAsStateMachine/Properties/Settings.Designer.cs
         create mode 100644 PresenterAsStateMachine/Properties/Settings.settings

Now I set up the git repository that I had previously created in OS X as the remote:


            $ git remote add origin //.psf/Home/Archive/git/PresenterAsStateMachine.git/

        $ git push origin master
        Counting objects: 20, done.
        Compressing objects: 100% (19/19), done.
        Writing objects: 100% (20/20), 12.74 KiB, done.
        Total 20 (delta 1), reused 0 (delta 0)
        Unpacking objects: 100% (20/20), done.
        To //.psf/Home/Archive/git/PresenterAsStateMachine.git/
         * [new branch]      master -> master

And now push and pull to my heart’s content, as well as add and commit to my local repository.

Now I configure TimeMachine to not back up the Parallels virtual machine directory. It still gets backed up daily by SuperDuper, but they only files that I want to be able to recover versions of is the source code that I’m working on – and this technique serves me well for that.

Getting Started in Objective-C with XCode

Brett Schuchert — Thu, 11 Feb 2010 23:07:00 -0600

As I mentioned in a previous previous blog, I wrote up some notes on getting started practicing TDD using XCode and Objective-C. I’m intending to write a bit more, so the top level of the overall tutorial path starts here: A First Objective-C Project

If you just want to see the details of getting started: XCodeProjectSetup

The biggest change from the previous blog is an embedded video demonstration. If you’re just interested in the video: Getting started

Want to seem more? Something in particular? Let me know.

Comments welcome.

Is TDD Language Neutral?

Brett Schuchert — Tue, 09 Feb 2010 09:38:00 -0600

On another blog, a commenter wrote (I’m leaving names out, though you can see the original blog if you look):

I don’t think that it makes sense to say “teach a TDD class in several languages simultaneously”. TDD is a language-independent technique.

What follows is a slightly changed version of my response. What I’m wondering is this: Do you think the practice of TDD is fundamentally impacted by your programming language?

Read on for my response. You might want to add a comment before reading my response to avoid “group think.”

Based on the original comment, I started with: It seems to me that you’ve not written professionally in many different languages. Or if you have, you’ve not effectively used those different programming languages.

Now let me be a bit more clear. Java and C++ are different languages, to be sure. However, they are both class-based, statically typed languages without lambdas (well C++ and Java will both be getting them, but they don’t have them now).

Contrast Java and C++ with Smaltalk and Self. Smalltalk is class based and supports inheritance but it is dynamically typed. Self is object-oriented, but it does not have classes. In fact, it technically does not have inheritance, but it fully supports OOP. How? Delegation. JavaScript is closer to Self than it is to Smalltalk. But JavaScript, Self and Smalltalk are all closer than C++, Java, C# and Objective-C.

How about the Open/Closed principle? How I interpret that design principle depends on the context (language) in which work. In raw Java, what I think needs to be closed is different than say raw Ruby. If I add Aspect Oriented Programming into the mix (yes, I’ve actually written and deployed systems that used Aspect J – in a limited fashion), then that opens things up a bit, but still not as much as in Ruby.

In Smalltalk, the closest analog to static methods are class methods. However, a class method in Smalltalk is actually an instance method on a class object, which is a subclass of the Class class. (Yes, that’s what I meant.)

Why is this important? When I’m working with legacy code in C++/Java/C#/VB.Net – languages where static methods are never dynamically bound, I avoid using static methods and, in fact, I redesign using what Michel Feathers calls “Introducing an Instance Delegator”.

In Smalltalk, this technique is irrelevant and unnecessary.

In languages like JavaScript, Self, Smaltalk, Ruby, etc., interfaces are not really necessary. Sure you can use them, but they are more for documentation.

Consider templates in C++ versus generics in Java. In Java, generics experience type erasure so you often need to use interfaces with generics – or at least tell the compiler a type it can assume will always be used with a generic. In C++, the most you really need to do is let the complier know that a template parameter is a typename as opposed to a primitive. When the code is compiled, any implied requirements of the type parameter will either be satisfied or not. (Woe be it to you if they are not!)

Why? In Java, generics are syntactic sugar. C++ generates a unique class for each template occurrence. C# is much closer to C++ than Java in this respect. Java got it wrong (in my opinion) – though what they did add did improve the language a bit.

How about moving away from OO languages? Scala is a mixed language, as is F#. Haskel is not. Lisp as well. How you break a problem into parts is (or at least should be) different in these languages. I don’t mean different in syntax, to be sure it will be. I mean how you break a problem into its parts is fundamentally different in pure functional languages than OO languages.

Or consider COBOL. I’ve programmed in it. I can thank COBOL for my touch-typing skills. COBOL is yet a different kind of language. Yes, syntactically, but that’s not the interesting difference. COBOL is a record-oriented processing language. It is well suited to batch processing. AWK is similar in this respect as is SNOBOL. (I’ve used AWK quite a bit, and I’ve used a variant of SNOBOL called SPITBOL as well.)

Let’s continue with a building analogy. If all I have to build is bamboo then I have available to me certain techniques for building a house. If I have steel, I have different alternatives. The design of my house will be directly impacted by the media (language) available.

Additionally, if I’m building on clay versus bedrock, I will also make fundamentally different design issues. When I lived in Texas I had a floating slab foundation, which were common for several reasons: cost, temperature zone, available building materials. Texas was the first place I lived that did not in general have basements. The other option, pier and beam, was no longer practiced in general due to cost.

I’m originally from the Mid-West (Iowa). In Iowa, buildings are built such that their foundation is below the frost line. The frost line was something like 30” (I’m remembering something from 20 – 30 years ago). So all the houses had basements. Or if not basements, half-basements.

The same kinds of things can be said about programming languages as well as their ecosystems as well as the political environment. I’ve already made the language point above. But what about the political environment?

I have worked at places where open source was forbidden. So this rules out Hibernate and Spring. Taking away these tools does not stop me from building a system, but it does have an impact on gross assignment of responsibility, which is an important component of software architecture.

Now, back down to a TDD class. When I teach TDD, I am not teaching just TDD. Furthermore, I am not just talking about TDD. I want students to apply the practice. So I start with a problem – a requirements statement. That does not initially vary with the language. In practice it eventually does for several reasons:

How far the class can get with the problem is impacted by by their skills as well as the programming language. C++ and vi are slower to work in that Java and Eclipse.
What you can reasonably do is impacted by both the language as its environment. For example, it is easy to look at a DLL and dynamically find all concrete implementations of an interface in any .Net language. It’s harder to do that in Java, though possible. In C++ it is platform specific.
There are more…

The students do need to learn the mechanics:

How to write a test using a particular unit testing framework
How to write production code
The shortcut keys of the environment
The refactoring support in the IDE

But what are the students testing? Units? Sure, let’s go there.

Language affects the units. You are testing units. Different kinds of units, different kinds of responsibilities. Different kinds of responsibilities to be verified, different unit tests.

Things are not so cleanly separated as you would like to imagine.

This very idea can be seen in natural languages. Different languages are better/worse at expressing things. The language you use directly affects how you interpret the world. This actually impacts the physical wiring in the brain. Really, it does.

So I do not agree.

And I challenge you to write the same problem in two fundamentally different languages (e.g. Java and Lisp) using TDD. Then have those two different solutions reviewed by experts in each of those languages.

I claim that if you’ve done an effective job of using those languages, your solutions will be different AND the unit tests you created to get to those solutions will be fundamentally different (not just mechanically different).

If, on the other hands, the unit tests are the same (other than the mechanics), then at least one group of people will say your solution is a bad use of their particular language. You’ll either be programming Java in Lisp or Lisp in Java.

Rudimentary TDD with XCode and Objective-C

Brett Schuchert — Mon, 08 Feb 2010 18:24:00 -0600

We have a potential client that would like us to teach a TDD class in several languages simultaneously:

Java
JavaScript
C++
Objective-C

In turns out that for what we teach in our TDD class, Java, C++ and Objective-C can be taught at the same time fairly easily. JavaScript, however, not so much. It is a very different (great, but different) beast.

In fact I could add C# to the mix as well, but I’d have to leave out LINQ and Lambdas.

In any case, to teach a class like that, I prefer to have the students do a bit of preliminary work so that they know how to set up their environments. When there’s just one language, I do this on the fly. But when there’s 3, I don’t want 2/3rds of the class waiting while 1/3rd learns how to create a basic project supporting TDD.

The last time I used Objective-C was in 1991, so I needed to figure out how to do this in the environment it is most likely to be used. This turned out to be a bit more work than I expected. I found it all using the good old google-machine, but in pieces. Since there really wasn’t a great (in my opinion) one-stop place that clearly laid out all of the steps, I created one.

This is in a early state. If you have comments, I’d appreciate them. I’ll probably add more. (If you’d like to see more, let me know that as well.) At this point, the basics are pretty much there, so I don’t actually need to do anything else. However, I have taken this just a bit further: This is the top-level of the tutorial

Again, comments and suggestions welcome.

CppUTest Recent Experiences

Brett Schuchert — Thu, 04 Feb 2010 13:42:00 -0600

Background

Mid last year I ported several exercises from Java to C++. At that time, I used CppUTest 1.x and Boost 1.38. Finally, half a year later, it was time to actually brush the dust off those examples and make sure they still work.

They didn’t. Bit rot. Or user error. Not sure which.

Bit rot: bits decay to the point where things start failing. Compiled programs do have a half-life.

User Error: maybe things were not checked in as clean as I remember. Though I suspect they were, I don’t really have any evidence to prove it, so I have to leave that option available.

To add to the mix, I decided to upgrade to CppUTest 2.x and to the latest version of the Boost library (1.41). I think that broke many several things. But the fixes were simple, once I figured out what I needed to do.

The Fixes

What follows are the three things I needed to do to get CppUTest 2.0, Boost and those exercises playing nicely together.

Header File Include Order

First, I used to have the header file for the CppUTest Test Harness, included first. It seems logical, but it caused all sorts of problems with CppUTest 2. That header file, includes a file, that ultimately includes something that uses macros to redefine new and delete. This is done so the testing framework can do simple memory tracking, which lets you know if your unit tests contains memory leaks.

I like this feature. Sure, it’s simple and light-weight, but it coves a lot of ground for a little hassle. The hassle? Include that header file last, instead of first. Problem Solved. Well at least the code compiles without hundreds of errors.

Boost Shared Pointer

Rather than hold pointers directly, I used the boost shared pointer class for a light-weight way to manage memory allocation. This is something I would do on a real project as well.

Somehow, the updated memory tracking in CppUTest 2.0 found something I had missed when using CppUTest 1.0.

I need to be able to control the date, so I have a simple date factory. By default, the date factory, when asked for the current date, returns the current date. Several unit tests want to simulate different dates. E.g., check out a book on one day, return it 14 days later. To do that, I manipulate the date factory (a form of dependency injection). This works fine, but by default the date factory is allocated using new.

When I replaced the existing date factory, I was not resetting it after the test. It turns out that this did not break anything because I was “lucky”. (Actually unlucky, I like things to fail fast.) CppUTest caught this in the form of not deallocating memory correctly:

I want to replace behavior
To do so I used polymorphism
Polymorphism in C++ requires virtual methods (please don’t correct me by suggesting that overloading is polymorphism, that is an opinion with which I strongly disagree)
Methods are only virtually dispatched via references or pointers
References cannot be changed, so I must use a pointer if I want a substitutable factory, which I wanted
Pointers suggest dynamic memory allocation

To fix this, I updated the setup method to store the original date factory in an attribute and then I updated the teardown method to restore the original date factory from that attribute. That I missed this suggests that my test suite is not adequate. I did not fix this problem for no good reason other than I was porting existing tests, so I left it as is. For the context it will not cause a problem. Pragmatic or lazy? You decide.

One Time Allocation

Here is a simple utility that uses Boost dates and regex:

ptime DateUtil::dateFromString(const string &dateString) {
  boost::regex e("^(\\d{1,2})/(\\d{1,2})/(\\d{4})$");
  string replace("\\3/\\1/\\2");
  string isoDate = boost::regex_replace(dateString, e, replace, boost::match_default | boost::format_sed);
  return ptime(date(from_string(isoDate)));
}

Now this is somewhat simplistic code. So be it, it serves the purposes of the exercise. I can think of ways to fix this, but there’s an underling issue that exists if you use the regex library from Boost.

When you use the library, it allocates (in this example) 10 blocks of memory. If you read the documentation (I did), it’s making space for its internal state machine for regex evaluation. This is done once and then kept around.

So what’s the problem? Well, when I run my tests, the first test that happens to exercise this block of code reports some memory allocation issues:

c:\projects\cppppp\dependencyinversionprinciple\dependencyinversionprinciple\pat
rongatewaytest.cpp:34: error: Failure in TEST(PatronGateway, AddAFew)
        Memory leak(s) found.
Leak size: 1120 Allocated at: <unknown> and line: 0. Type: "new" Content: "
?"
Leak size: 16 Allocated at: <unknown> and line: 0. Type: "new" Content: ?a"
Leak size: 20 Allocated at: <unknown> and line: 0. Type: "new" Content: "êà4"
Leak size: 52 Allocated at: <unknown> and line: 0. Type: "new" Content: ä4"
Leak size: 4096 Allocated at: <unknown> and line: 0. Type: "new" Content: ""
Leak size: 52 Allocated at: <unknown> and line: 0. Type: "new" Content: "êâ4"
Leak size: 20 Allocated at: <unknown> and line: 0. Type: "new" Content: ~4"
Leak size: 32 Allocated at: <unknown> and line: 0. Type: "new" Content: "?à4"
Leak size: 32 Allocated at: <unknown> and line: 0. Type: "new" Content: "h~4"
Leak size: 80 Allocated at: <unknown> and line: 0. Type: "new" Content: "êä4"
Total number of leaks:  10

This is a false positive. This is a one-time allocation and a side-effect of C++ memory allocation and static initialization.

There is a way to “fix” this. You use a command line option, -r, to tell the command line test runner to run the tests twice. If the allocation problem happens the first time but not the second time, then the tests are “OK”.

I didn’t want to do this.

The tests do take some time to run (30 seconds maybe, but still that doubles the time)
The output is ugly
It’s off topic for what the exercise is trying to accomplish

I tried a few different options but ultimately I went with simply calling that method before using the command line test runner. So I changed my main from:

#include <CppUTest/CommandLineTestRunner.h>

int main(int argc, char **argv) {
  return CommandLineTestRunner::RunAllTests(argc, argv);
}

To this:

#include "DateUtil.h"
#include <CppUTest/CommandLineTestRunner.h>

/** ************************************************************
    The boost regex library allocates several blocks of memory
    for its internal state machine. That memory is listed as a 
    memory leak in the first test that happens to use code that
    uses the boost regext library. To avoid having to run the
    tests twice using the -r option, we instead simply force
    this one-time allocation before starting test execution.
    *********************************************************** **/

void forceBoostRegexOneTimeAllocation() {
  DateUtil::dateFromString("1/1/1980");
}

int main(int argc, char **argv) {
  forceBoostRegexOneTimeAllocation();
  return CommandLineTestRunner::RunAllTests(argc, argv);
}

Since this one-time allocation happens before any of the tests run, it is no longer reported as a problem by CppUTest.

Before I introduced this “fix”, I spent quite a bit of time to verify that each of the 10 allocations were done by one of the three lines dealing with regex code in my DateUtil class. I used a conditional breakpoint and looked at the stack trace. (I know, using the debugger is considered a code smell, but not all smells are bad.)

Conclusion

I still like CppUTest. I’ve used a few C++ unit testing tools but there are several I have not tried. I don’t have enough face-time with C++ for this to be an issue. I am not terribly comfortable with the order of includes sensitivity. I’m not sure if that would scale.

I do appreciate the assistance with memory checking, though dealing with false positives can be a bit of a hassle. There was another technique, that of expressing the number of allocations. But in this case, that simply deferred the reporting of memory leaks to after test execution. In any case, I do like this. I’m not sure how well it would scale so it leaves me a bit uneasy.

If you happen to be using these tools, hope this helps. If not, and you are using C++, what can you say about your experiences with using this or other unit testing tools?

Software on the Cheap

Uncle Bob — Mon, 01 Feb 2010 16:17:00 -0600

When it comes to software, you get what you pay for.

Have you ever stopped to wonder how much a line of code costs? It ought to be easy to figure out.

In the last 14 months, I have written about 20KSLOC in FitNesse. Of course that was part time. My real job is running Object Mentor, consulting, teaching, mentoring, writing, and a whole load of other things. Programming takes up perhaps 15% of my time.

On the other hand most programmers have lots of other things to do. They go to meetings, and then they go to more meetings. When they are done with those meetings, they go to meetings. And then there are the meetings to go to. Oh yeah, and then there’s all the fiddling around with time accounting tools, and horrific source code control systems that perform like a salamander crawling through frozen mud.

So, maybe 15% isn’t such a bad ratio.

The loaded rate (Salary plus everything else) for a typical programmer is on the order of $200K. (I know that sounds like a lot, but you can look it up.) So $200K / (20KSLOC / 14mo * 12mo) = $11.66/SLOC.

Let’s look at one of those lines: StringBuffer nameBuffer = new StringBuffer(); Does that look like $11.66 to you? Would you pay that much for it? Well, don’t answer yet, because for each StringBuffer line you buy, you get import java.lang.StringBuffer; absolutely free!

Some factories pay their employees a “piece rate”. Would you accept $11.66 per line of code instead of a salary? Of course it couldn’t just be any old line of code. It’d have to be tested!

Hey, I bet all programmers would do TDD if we paid them a piece rate!

Down to business.

The point of that silly analysis was to demonstrate that software is expensive. Even the dumbest little app will likely require more than 1,000 lines of code; and that means it could cost $12K to write!

Imagine that you aren’t a programmer, but you have a clever idea for a new website that’ll make you a zillion dollars. You’ve storyboarded it all out. You’ve worked out all the details. Now all you need is some high-school kid to zip out the code for you. Right? Hell, you could pay him minimum wage! The little twerp would be happy to get it!

That tragic comedy is altogether too common. Too many people have borrowed money against their father’s retirement account to fund a terrible implementation of a good idea. Appalled at how much the reputable firms charge per hour ($100 or more) they go looking for a cheap solution.

“After all, this software is simple.” Or so the reasoning goes. “It’s not like we’re trying to send a rocket to the moon or anything. And, besides, those expensive guys were just out to cheat us. Software just isn’t that hard to write.” Uh huh.

So the poor schmuck finds some freshman in college, or maybe a bored housewife who read a book on HTML last year and created a cute website to show off her kittens. Have these programmers heard about TDD? Have they heard about Design Patterns? Principles? How about source code control?

Clearly they haven’t. They’re going to sling a bunch of horrific code together, without any tests, versioning, or control. The project will start well, with exciting initial results. But then it will slowly grind to a halt, while the cash continues out the door unabated.

In the end the website isn’t going to get built (and the poor schmuck’s father won’t be retiring as soon as he thought). It will be a disaster that will either be terminated, or will require double or triple the investment to get right.

The Bottom Line.

The bottom line is that, when it comes to software, you get what you pay for. If you want good software done well, then you are going to pay for it, and it will probably cost you $12/line or more. And, believe me, that’s the cheapest way to get your software done.

If you go out hunting for the cheap solution, then you’re going to end up paying more, and losing time. Software is one of those things that costs a fortune to write well, and double that to write poorly. If you go for cheap, you’re going to pay double; and maybe even triple.

Makefiles, what are you doing with them?

Brett Schuchert — Wed, 27 Jan 2010 09:02:00 -0600

Half a year ago, I took some exercises written in Java and ported them to C++. At that time I punted on writing makefiles and decided to reply on an IDE (Visual Studio). That worked fine but recently we almost had a customer that was going to take this class and, bless their heart, use the command line.

Turns out they are going to use visual studio, so I can shelve this, but I figured I’d get feedback on what I have created for a makefile. Since I have not written a makefile for 12ish years, I figured things have changed. They have, but I suspect they have even more.

What follows are my environment assumptions and the makefile. I’d appreciate any suggestions.

FWIW, I wrote this makefile because it turns out fully building all files takes longer than just building what has changed. I know this seems obvious, but it is not always the case with small projects like this. There is some point where the project becomes big enough to warrant dependency-based building. The exercise meets that criteria. Sometimes the time taken to evaluate build changes is enough overhead that it can be safely removed in lieu of a full build. This is a consideration when creating custom build files for legacy-based refactoring efforts.

Environment

Generic Unix
zsh – though any shell will do really
g++
gnu make
This source base will still be used in Visual Studio but not this makefile
CppUTest used for unit testing tool
Provide default location for CppUTest, which can be overridden by environment variable
Boost library installed in /usr/local/include and /usr/local/lib (reasonable assumption or should I go ahead and allow this to be configured easily in the makefile – what’s your opinion?)
Project is small enough that all headers and source files are in the same directory (should I build the .o’s and put them in another directory to hide them?)

Current Makefile

ifdef CPP_U_TEST
    CppUTest_base = $(CPP_U_TEST)
else
    CppUTest_base = $(wildcard ~/src/cpputest)
endif

LIB_DIRS = -L$(CppUTest_base)/lib
INC_DIRS = -I$(CppUTest_base)/include
LIBS = -lCppUTest -lboost_date_time -lboost_regex -lboost_thread
CFLAGS = -g $(INC_DIRS) 
LDFLAGS = $(LIB_DIRS) $(LIBS)
CC = g++
SRCS := $(wildcard *.cpp)
OBJS = $(SRCS:.cpp=.o)
OUT = run_all_tests
DEP_DIR = dependencies

all: $(OBJS)
    $(CC) $(CFLAGS) $(LDFLAGS) -o $(OUT) $(OBJS)

clean:
    @rm -rf $(DEP_DIR) $(OBJS) $(OUT)

-include $(SRCS:.cpp=$(dep_dir)/.d)

.cpp.o:
    @if [ ! -d $(DEP_DIR) ] ; then mkdir $(DEP_DIR) ; fi
    $(CC) -c $(CFLAGS) -o $@ $< 
    @$(CC) -MM $(CFLAGS) $*.cpp > $(DEP_DIR)/$*.d

Mocking Mocking and Testing Outcomes.

Uncle Bob — Sat, 23 Jan 2010 11:32:00 -0600

The number of mocking frameworks has proliferated in recent years. This pleases me because it is a symptom that testing in general, and TDD in particular, have become prevalent enough to support a rich panoply of third-party products.

On the other hand, all frameworks carry a disease with them that I call The Mount Everest Syndrome: “I use it because it’s there.” The more mocking frameworks that appear, the more I see them enthusiastically used. Yet the prolific use of mocking frameworks is a rather serious design smell…

Lately I have seen several books and articles that present TDD through the lens of a mocking framework. If you were a newbie to TDD, these writings might give you the idea that TDD was defined by the use of mocking tools, rather than by the disciplines of TDD.

So when should use use a mocking framework? The answer is the same for any other framework. You use a framework only when that framework will give you a significant advantage.

Why so austere? Why shouldn’t you use frameworks “just because they are there”? Because frameworks always come with a cost. They must be learned by the author, and by all the readers. They become part of the configuration and have to be maintained. They must be tracked from version to version. But perhaps the most significant reason is that once you have a hammer, everything starts to look like a nail. The framework will put you into a constraining mindset that prevents you from seeing other, better solutions.

Consider, for example, this lovely bit of code that I’ve been reviewing recently. It uses the Moq framework to initialize a test double:


var vehicleMock = Mocks.Create<IClientVehicle>()
 .WithPersistentKey()
 .WithLogicalKey().WithLogicalName()
 .WithRentalSessionManager(rsm =>
    {
      var rs = Mocks.Create<IRentalSession>();
      rsm.Setup(o => o.GetCurrentSession()).Returns(rs.Object);
      rsm.Setup(o =>
       o.GetLogicalKeyOfSessionMember(It.IsAny<string>(),
        It.IsAny<int>())).Returns("Rental");
    })
 .AddVehicleMember<IRoadFactory>()
 .AddVehicleMember<IRoadItemFactory>(rf => rf.Setup(t => 
    t.CreateItems(It.IsAny<IRoad>())).Returns(pac))
 .AddVehicleMember<ILegacyCorporateRental>()
 .AddVehicleMember<IRentalStation>(
    m => m.Setup(k => k.Facility.FacilityID).Returns(0))
 .AddVehicleMember<IRoadManager>(m=>
    m.Setup(k=>k.GetRoundedBalanceDue(25,It.IsAny<IRoad>())).Returns(25));

Some of you might think I’m setting up a straw-man. I’m not. I realize that bad code can be written in any language or framework, and that you can’t blame the language or framework for bad code.

The point I am making is that code like this was the way that all unit tests in this application were written. The team was new to TDD, and they got hold of a tool, and perhaps read a book or article, and decided that TDD was done by using a mocking tool. This team is not the first team I’ve seen who have fallen into this trap. In fact, I think that the TDD industry as a whole has fallen into this trap to one degree or another.

Now don’t get me wrong. I like mocking tools. I use them in Ruby, Java, and .Net. I think they provide a convenient way to make test-doubles in situations where more direct means are difficult.

For example, I recently wrote the following unit test in FitNesse using the Mockito framework.


  @Before
  public void setUp() {
    manager = mock(GSSManager.class);
    properties = new Properties();
  }

  @Test
  public void credentialsShouldBeNullIfNoServiceName() throws Exception {
    NegotiateAuthenticator authenticator = 
      new NegotiateAuthenticator(manager, properties);
    assertNull(authenticator.getServerCredentials());
    verify(manager, never()).createName(
      anyString(), (Oid) anyObject(), (Oid) anyObject());
  }

The first line in the setUp function is lovely. It’s kind of hard to get prettier than that. Anybody reading it understands that manager will be a mock of the GSSManager class.

It’s not too hard to understand the test itself. Apparently we are happy to have the manager be a dummy object with the constraint that createName is never called by NegotiateAuthenticator. The anyString() and anyObject() calls are pretty self explanatory.

On the other hand, I wish I could have said this:

assertTrue(manager.createNameWasNotCalled());

That statement does not require my poor readers to understand anything about Mockito. Of course it does require me to hand-roll a manager mock. Would that be hard? Let’s try.

First I need to create a dummy.


  private class MockGSSManager extends GSSManager {
    public Oid[] getMechs() {
      return new Oid[0];
    }

    public Oid[] getNamesForMech(Oid oid) throws GSSException {
      return new Oid[0];
    }

    public Oid[] getMechsForName(Oid oid) {
      return new Oid[0];
    }

    public GSSName createName(String s, Oid oid) throws GSSException {
      return null;
    }

    public GSSName createName(byte[] bytes, Oid oid) throws GSSException {
      return null;
    }

    public GSSName createName(String s, Oid oid, Oid oid1) throws GSSException {
      return null;
    }

    public GSSName createName(byte[] bytes, Oid oid, Oid oid1) throws GSSException {
      return null;
    }

    public GSSCredential createCredential(int i) throws GSSException {
      return null;
    }

    public GSSCredential createCredential(GSSName gssName, int i, Oid oid, int i1) throws GSSException {
      return null;
    }

    public GSSCredential createCredential(GSSName gssName, int i, Oid[] oids, int i1) throws GSSException {
      return null;
    }

    public GSSContext createContext(GSSName gssName, Oid oid, GSSCredential gssCredential, int i) throws GSSException {
      return null;
    }

    public GSSContext createContext(GSSCredential gssCredential) throws GSSException {
      return null;
    }

    public GSSContext createContext(byte[] bytes) throws GSSException {
      return null;
    }

    public void addProviderAtFront(Provider provider, Oid oid) throws GSSException {
    }

    public void addProviderAtEnd(Provider provider, Oid oid) throws GSSException {
    }
  }

“Oh, ick!” you say. Yes, I agree it’s a lot of code. On the other hand, it took me just a single keystroke on my IDE to generate all those dummy methods. (In IntelliJ it was simply command-I to implement all unimplemented methods.) So it wasn’t particularly hard. And, of course, I can put this code somewhere where nobody had to look at it unless they want to. It has the advantage that anybody who knows Java can understand it, and can look right at the methods to see what they are returning. No “special” knowledge of the mocking framework is necessary.

Next, let’s’ make a test double that does precisely what this test needs.


  private class GSSManagerSpy extends MockGSSManager {
    public boolean createNameWasCalled;

    public GSSName createName(String s, Oid oid) throws GSSException {
      createNameWasCalled = true;
      return null;
    }
  }

Well, that just wasn’t that hard. It’s really easy to understand too. Now, let’s rewrite the test.


  @Test
  public void credentialsShouldBeNullIfNoServiceNameWithHandRolledMocks() throws Exception {
    NegotiateAuthenticator authenticator = new NegotiateAuthenticator(managerSpy, properties);
    assertNull(authenticator.getServerCredentials());
    assertFalse(managerSpy.createNameWasCalled);
  }

Well, that test is just a load easier to read than verify(manager, never()).createName(anyString(), (Oid) anyObject(), (Oid) anyObject());.

“But Uncle Bob!” I hear you say. “That scenario is too simple. What if there were lots of dependencies and things…” I’m glad you asked that question, because the very next test is just such a situation.


  @Test
  public void credentialsShouldBeNonNullIfServiceNamePresent() throws Exception {
    properties.setProperty("NegotiateAuthenticator.serviceName", "service");
    properties.setProperty("NegotiateAuthenticator.serviceNameType", "1.1");
    properties.setProperty("NegotiateAuthenticator.mechanism", "1.2");
    GSSName gssName = mock(GSSName.class);
    GSSCredential gssCredential = mock(GSSCredential.class);
    when(manager.createName(anyString(), (Oid) anyObject(), (Oid) anyObject())).thenReturn(gssName);
    when(manager.createCredential((GSSName) anyObject(), anyInt(), (Oid) anyObject(), anyInt())).thenReturn(gssCredential);
    NegotiateAuthenticator authenticator = new NegotiateAuthenticator(manager, properties);
    Oid serviceNameType = authenticator.getServiceNameType();
    Oid mechanism = authenticator.getMechanism();
    verify(manager).createName("service", serviceNameType, mechanism);
    assertEquals("1.1", serviceNameType.toString());
    assertEquals("1.2", mechanism.toString());
    verify(manager).createCredential(gssName, GSSCredential.INDEFINITE_LIFETIME, mechanism, GSSCredential.ACCEPT_ONLY);
    assertEquals(gssCredential, authenticator.getServerCredentials());
  }

Now I’ve got three test doubles that interact with each other; and I am verifying that the code under test is manipulating them all correctly. I could create hand-rolled test doubles for this; but the wiring between them would be scattered in the various test-double derivatives. I’d also have to write a significant number of accessors to get the values of the arguments to createName and createCredential. In short, the hand-rolled test-double code would be harder to understand than the Mockito code. The Mockito code puts the whole story in one simple test method rather than scattering it hither and yon in a plethora of little derivatives.

What’s more, since it’s clear that I should use a mocking framework for this test, I think I should be consistent and use if for all the tests in this file. So the hand-rolled MockGSSManager and ManagerSpy are history.

“But Uncle Bob, aren’t we always going to have dependencies like that? So aren’t we always going to have to use a mocking framework?”

That, my dear reader, is the real point of this blog. The answer to that salient questions is a profound: “No!“

Why did I have to use Mockito for these tests? Because the number of objects in play was large. The module under test (NegotiateAuthenticator) used GSSName, GSSCredential, and GSSManager. In other words the coupling between the module under test and the test itself was high. (I see lightbulbs above some of your heads.) That’s right, boys and girls, we don’t want coupling to be high!

It is the high coupling between modules and tests that creates the need for a mocking framework. This high coupling is also the cause of the dreaded “Fragile Test” problem. How many tests break when you change a module? If the number is high, then the coupling between your modules and tests in high. Therefore, I conclude that those systems that make prolific use of mocking frameworks are likely to suffer from fragile tests.

Of the 277 unit test files in FitNesse, only 11 use Mockito. The reason for small number is two-fold. First, we test outcomes more often than we test mechanisms. That means we test how a small group of classes behaves, rather than testing the dance of method calls between those classes. The second reason is that our test doubles have no middle class. They are either very simple stubs and spies or they are moderately complex fakes.

Testing outcomes is a traditional decoupling technique. The test doesn’t care how the end result is calculated, so long as the end result is correct. There may be a dance of several method calls between a few different objects; but the test is oblivious since it only checks the answer. Therefore the tests are not strongly coupled to the solution and are not fragile.

Keeping middle-class test doubles (i.e. Mocks) to a minimum is another way of decoupling. Mocks, by their very nature, are coupled to mechanisms instead of outcomes. Mocks, or the setup code that builds them, have deep knowledge of the inner workings of several different classes. That knowledge is the very definition of high-coupling.

What is a “moderately complex fake” and why does it help to reduce coupling? One example within FitNesse is MockSocket. (The name of this class is historical. Nowadays it should be called FakeSocket.) This class derives from Socket and implements all its methods either to remember what was sent to the socket, or to allow a user to read some canned data. This is a “fake” because it simulates the behavior of a socket. It is not a mock because it has no coupling to any mechanisms. You don’t ask it whether it succeeded or failed, you ask it to send or recieve a string. This allows our unit tests to test outcomes rather than mechanisms.

The moral of this story is that the point at which you start to really need a mocking framework is the very point at which the coupling between your tests and code is getting too high. There are times when you can’t avoid this coupling, and those are the times when mocking frameworks really pay off. However, you should strive to keep the coupling between your code and tests low enough that you don’t need to use the mocking framework very often.

You do this by testing outcomes instead of mechanisms.

Dependency Injection Inversion

Uncle Bob — Sun, 17 Jan 2010 12:42:00 -0600

Dependency Injection is all the rage. There are several frameworks that will help you inject dependencies into your system. Some use XML (God help us) to specify those dependencies. Others use simple statements in code. In either case, the goal of these frameworks is to help you create instances without having to resort to new or Factories.

I think these frameworks are great tools. But I also think you should carefully restrict how and where you use them.

Consider, for example, this simple example using Google’s Guice framework.


 public class BillingApplication {
   public static void main(String[] args) {
    Injector injector = Guice.createInjector(new BillingModule());
    BillingService billingService = injector.getInstance(BillingService.class);
    billingService.processCharge(2034, "Bob");
  }
 }

My goal is to create an instance of BillingService. To do this, I first get an Injector from Guice. Then I use the injector to get an instance of my BillingService class. What’s so great about this? Well, take a look at the constructor of the BillingService class.


class BillingService {
  private CreditCardProcessor processor;
  private TransactionLog transactionLog;

  @Inject
  BillingService(CreditCardProcessor processor, TransactionLog transactionLog) {
    this.processor = processor;
    this.transactionLog = transactionLog;
  }

  public void processCharge(int amount, String id) {
    boolean approval = processor.approve(amount, id);
    transactionLog.log(
      String.format("Transaction by %s for %d %s",
      id, amount, approvalCode(approval)));
  }

  private String approvalCode(boolean approval) {
    return approval?"approved":"denied";
  }
}

Oh ho! The BillingService constructor requires two arguments! A CreditCardProcessor and a TransactionLog. How was the main program able to create an instance of BillingService without those two arguments? That’s the magic of Guice (and of all Dependency Injection frameworks). Guice knows that the BillingService needs those two arguments, and it knows how to create them. Did you see that funky @Inject attribute above the constructor? That’s how it got connected into Guice.

And here’s the magic module that tells Guice how to create the arguments for the BillingService


public class BillingModule extends AbstractModule {
  protected void configure() {
    bind(TransactionLog.class).to(DatabaseTransactionLog.class);
    bind(CreditCardProcessor.class).to(MyCreditCardProcessor.class);
  }
}

Clever these Google-folk! The two bind functions tell Guice that whenever we need an instance of a TransactionLog it should use an instance of DatabaseTransactionLog. Whenever it needs a CreditCardProcessor it should use an instance of MyCreditCardProcessor.

Isn’t that cool! Now you don’t have to build factories. You don’t have to use new. You just tell Guice how to map interfaces to implementations, and which constructors to inject those implementations in to, and then call Injector.getInstance(SomeClass.class); and voila! You have your instance automatically constructed for you. Cool.

Well, yes it’s cool. On the other hand, consider this code:


public class BillingApplicationNoGuice {
  public static void main(String[] args) {
    CreditCardProcessor cp = new MyCreditCardProcessor();
    TransactionLog tl = new DatabaseTransactionLog();
    BillingService bs = new BillingService(cp, tl);
    bs.processCharge(9000, "Bob");
  }
}

Why is this worse? It seems to me it’s better.

But Uncle Bob, you’ve violated DIP by creating concrete instances!

True, but you have to mention concrete instances somewhere. main seems like a perfectly good place for that. Indeed, it seems better than hiding the concrete references in BillingModule.

I don’t want a bunch of secret modules with bind calls scattered all around my code. I don’t want to have to hunt for the particular bind call for the Zapple interface when I’m looking at some module. I want to know where all the instances are created.

But Uncle Bob, You’d know where they are because this is a Guice application.

I don’t want to write a Guice application. Guice is a framework, and I don’t want framework code smeared all through my application. I want to keep frameworks nicely decoupled and at arms-length from the main body of my code. I don’t want to have @Inject attributes everywhere and bind calls hidden under rocks.

But Uncle Bob, What if I want to get an instance of BillingService from deep in the bowels of my application? With Guice I can just say injector.getInstance(BillingService.class);.

True, but I don’t want to have createInstance calls scattered all through my code. I don’t want Guice to be poured all over my app. I want my app to be clean, not soaked in Guice.

But Uncle Bob, That means I have to use new or factories, or pass globals around.

You think the injector is not a global? You think BillingService.class is not a global? There will always be globals to deal with. You can’t write systems without them. You just need to manage them nicely.

And, no, I don’t have to use new everywhere, and I don’t need factories. I can do something as simple as:


public class BillingApplicationNoGuice {
  public static void main(String[] args) {
    CreditCardProcessor cp = new MyCreditCardProcessor();
    TransactionLog tl = new DatabaseTransactionLog();
    BillingService.instance = new BillingService(cp, tl);

    // Deep in the bowels of my system.
    BillingService.instance.processCharge(9000, "Bob");
  }
}

But Uncle Bob, what if you want to create many instances of BillingService rather than just that one singleton?

Then I’d use a factory, like so:


public class BillingApplication {
   public static void main(String[] args) {
    Injector injector = Guice.createInjector(new BillingModule());
    BillingService.factory = new BillingServiceFactory(injector);

    // Deep in the bowels of my code.
    BillingService billingService = BillingService.factory.make();
    billingService.processCharge(2034, "Bob");
  }
}

But Uncle Bob, I thought the whole idea was to avoid factories!

Hardly. After all, Guice is just a big factory. But you didn’t let me finish. Did you notice that I passed the Guice injector into the factory? Here’s the factory implementation.


public class BillingServiceFactory extends AbstractModule {
  private Injector injector;

  public BillingServiceFactory(Injector injector) {
    this.injector = injector;
  }

  protected void configure() {
    bind(TransactionLog.class).to(DatabaseTransactionLog.class);
    bind(CreditCardProcessor.class).to(MyCreditCardProcessor.class);
  }

  public BillingService make() {
    return injector.getInstance(BillingService.class);
  }
}

I like this because now all the Guice is in one well understood place. I don’t have Guice all over my application. Rather, I’ve got factories that contain the Guice. Guicey factories that keep the Guice from being smeared all through my application.

What’s more, if I wanted to replace Guice with some other DI framework, I know exactly what classes would need to change, and how to change them. So I’ve kept Guice uncoupled from my application.

Indeed, using this form allows me to defer using Guice until I think it’s necessary. I can just build the factories the good old GOF way until the need to externalize dependencies emerges.

But Uncle Bob, don’t you think Dependency Injection is a good thing?

Of course I do. Dependency Injection is just a special case of Dependency Inversion. I think Dependency Inversion is so important that I want to invert the dependencies on Guice! I don’t want lots of concrete Guice dependencies scattered through my code.

BTW, did you notice that I was using Dependency Injection even when I wasn’t using Guice at all? This is nice and simple manual dependency injection. Here’s that code again in case you don’t want to look back:



public class BillingApplicationNoGuice {
  public static void main(String[] args) {
    CreditCardProcessor cp = new MyCreditCardProcessor();
    TransactionLog tl = new DatabaseTransactionLog();
    BillingService bs = new BillingService(cp, tl);
    bs.processCharge(9000, "Bob");
  }
}

Dependency Injection doesn’t require a framework; it just requires that you invert your dependencies and then construct and pass your arguments to deeper layers. Consider, for example, that the following test works just fine in all the cases above. It does not rely on Guice, it only relies on the fact that dependencies were inverted and can be injected into BillingService



public class BillingServiceTest {
  private LogSpy log;

  @Before
  public void setup() {
    log = new LogSpy();
  }

  @Test
  public void approval() throws Exception {
    BillingService bs = new BillingService(new Approver(), log);
    bs.processCharge(9000, "Bob");
    assertEquals("Transaction by Bob for 9000 approved", log.getLogged());
  }

  @Test
  public void denial() throws Exception {
    BillingService bs = new BillingService(new Denier(), log);
    bs.processCharge(9000, "Bob");
    assertEquals("Transaction by Bob for 9000 denied", log.getLogged());    
  }
}

class Approver implements CreditCardProcessor {
  public boolean approve(int amount, String id) {
    return true;
  }
}

class Denier implements CreditCardProcessor {
  public boolean approve(int amount, String id) {
    return false;
  }
}

class LogSpy implements TransactionLog {
  private String logged;

  public void log(String s) {
    logged = s;
  }

  public String getLogged() {
    return logged;
  }
}

Also notice that I rolled my own Test Doubles (we used to call them mocks, but we’re not allowed to anymore.) It would have been tragic to use a mocking framework for such a simple set of tests.

Most of the time the best kind of Dependency Injection to use, is the manual kind. Externalized dependency injection of the kind that Guice provides is appropriate for those classes that you know will be extension points for your system.

But for classes that aren’t obvious extension points, you will simply know the concrete type you need, and can create it at a relatively high level and inject it down as an interface to the lower levels. If, one day, you find that you need to externalize that dependency, it’ll be easy because you’ve already inverted and injected it.

UI Test Automation Tools are Snake Oil

Michael Feathers — Mon, 04 Jan 2010 05:35:00 -0600

It happens over and over again. I visit a team and I ask about their testing situation. We talk about unit tests, exploratory testing, the works. Then, I ask about automated end-to-end testing and they point at a machine in the corner. That poor machine has an installation of some highly-priced per seat testing tool (or an open source one, it doesn’t matter), and the chair in front of it is empty. We walk over, sweep the dust away from the keyboard, and load up the tool. Then, we glance through their set of test scripts and try to run them. The system falls over a couple of times and then they give me that sheepish grin and say “we tried.” I say, “don’t worry, everyone does.”

It’s a very familiar ‘rabbit hole’ in the industry. It’s sort of like the old days when you’d find a couple of classes generated by a CASE tool in every code base you visited if you looked hard enough. People started with the merry notion that they were going to round-trip code with some CASE tool and they learned (like most lucky teams do) that it just doesn’t pay for itself, it’s not worth the time or the frustration. UI Test Automation tools are in the same category. Personally, I think that in this day and age selling them is irresponsible. Developing them open-source? Well, let your conscience be your guide, but really, even though people can use them responsibly, they hardly ever do because these tools are sold with a dream, a very seductive and dangerous one.

The Dream

Janet comes into work in the morning and she sits down at her super-duper testing console. She presses a button and the testing system springs to life. The application comes up all at once across ten monitors. Cursors move, selections are made (silently) and tests run against the user interface magically, as if some eager set of ghost elves took control, mischievously burrowing through the nooks and crannies of the application, running scripts to completion, and making little notes whenever there is a failure. Janet sits back in her chair, waiting for the elves to report back to her. She stirs her coffee gently.

The Reality

Janet hasn’t gone home yet. It’s 2AM and she has to report completion of all her test cases at a meeting in the morning. She thinks she’s past the last configuration issue but she’s not sure. For the last hour, she’s been trying to make sure that a particular button is pressed at step 14 of her script, but quirky latency on the server is preventing it from happening consistently. Sadly, she has to run the script from the beginning each time. Oh, and five hours ago she discovered UI changes which invalidated 30% of the regression tests. Most of the changes were easy but she still has 12 cases to go and her 9AM meeting looms ahead of her.

This gap between the dream and the reality is not a matter of flawed execution, it’s endemic. Here’s the scoop.

UI Test Automation Tools are Brittle

You might not think this is fair but it is, really. I haven’t seen one of these tools yet which isn’t susceptible to missed events or timing madness. It just happens. The fact of the matter is, it is hard to sit on the outside of an application an instrument it. It’s a very technology sensitive problem. You need to hook into either the OS or the browser or both. Neither are ever really built from the ground up for that sort of access.

UI Based Testing Is Not the Solution That Vendors Imply It Is

This is the big issue, the one which really hurts the industry. The fact of the matter is that UI based testing should be used for UIs: that’s it. You should not be testing your full application end-to-end through a UI Testing tool. First of all, that sort of testing couples some of the most important tests in your system to one of the most volatile parts of it. It’s easy for us to see business logic as volatile, but really, the UI is the thing which twists and ripples in the winds of change. When customers want new features, often those features involve new workflows. When usability experts discover better ways of models of interaction, an agile business seizes upon them and makes the changes—if they can. You’d be surprised at the number of applications which continue to sport out of day user interfaces simply because the development organization is terrified of throwing away all of their regression tests which (by the way) go through the UI. Even if you’re not a consultant like me, visiting teams and seeing their development processes, you can see hints from the outside. Think of every website or shrink-wrap application which has “the same old workflow” and a UI that has become more cluttered over the years. Often it’s because of that lock-in.

UI Based Testing Takes More Staff and Time Than You Expect

This, really, is the most common failure case. It’s the case which explains the dust on the testing box’s keyboard. Someone, usually disconnected from the development organization, decides that “hey, we need to solve the testing problem. We have too many people doing manual testing. It’s taking forever.” So, they do their research, find a vendor with with a good licensing model and a good pitch and then they push it on the development organization. They are, of course, looking to reduce staff so when they realize that translating all of those manual tests to the tool is very labor-intensive, they are taken aback. But, of course, it is just a temporary cost, right? But, then it takes far longer than they expect. Remember Janet’s story? It’s really hard to catch up with a UI-Based testing tool. It’s hard to even stay in place with one. Typically it takes a number of people to do UI-Based automated testing for a development team in sync with an iteration and worse, they’ll always lag behind a bit because you can’t really write UI-based tests ahead of time the way you can with FIT and other beneath-the-UI testing tools. From what I’ve seen UI-based testing, done diligently, takes the effort of about one tester for every two to three developers. That’s what it seems to cost amortized across all of the maintenance of UI-induced test breakage. Oh, and by the way, if think you are going to save labor using record and playback? Nope, you aren’t. It doesn’t work.

Solutions

The fact of the matter is, you can use these tools effectively, but in a very narrow space. It’s nice to be able to test the UI—by itself. However, this sort of thing requires an architectural change.

In general, UIs are too volatile for end-to-end testing. Teams that do it well, typically develop a small task-focused scripting layer and build tests on top if it so that the actual tests don’t touch the UI directly. But, if they happen across that technique, they are lucky. Still, it isn’t an ideal solution. You really want to be below the UI working against an API which exposes the business logic. And, because of that nearly mystical synergy between testability and good design, that API layer is often useful for many things other than testing.

Conclusion/Challenge

I recognize that I’ve been rather vicious in the this blog. If you develop these tools for a living, you might not think it’s fair. But consider this. If you don’t think I’m being fair, take a look at how your tools are marketed. In particular, show me where the product literature discourages end-to-end testing through the tool. Otherwise, well, you know, you are probably developing snake-oil.

The Polyglot Tester

Uncle Bob — Sat, 19 Dec 2009 11:22:00 -0600

Behavior Driven Development, and it’s emphasis on the Given / When / Then structure of specification has been well accepted by many parts of the software industry. Tools such as JBehave, Cucumber, GivWenZen, have taken a prominent role, and rightly so. After all, it’s hard to argue with the elegance of simple statements such as:

Given that I am a user named Bob with password xyzzy
When I log in with username Bob and password xyzzy
Then I should see “Welcome Bob, you have logged in”.

Yes, it’s hard to argue with it, but argue I shall…

The BDD style is very pretty. Certainly business people can easily read and write it. Moreover, the BDD style provides a conceptual framework within which automated test specification and be efficiently composed. Better still, the BDD tools provide a powerful parsing mechanism that conveniently translates the natural GWT statements into function calls to be executed as tests.

The problem is that the BDD style is simply not appropriate for all, or even most kinds of tests. Why? Because it’s wordy. Consider:

Given a Juke box that shows 0 credits
When I deposit .25
Then the Juke Box shows 1 credit.

Given a Juke box that shows 0 credits.
When I deposit 1.00
Then the Juke Box shows 5 credits.

Given a Juke Box that shows 0 credits.
When I deposit 5.00
Then the Juke Box shows 30 credits.

How is this better than:

Jukebox Credit Table

Deposit	Credits
25	1
1.00	5
5.00	30

Which of the two is the more elegant?
Which of the two is easier to read?
If you were looking for a specification error, in which of the two would you be more likely to find it?

I think the answers are rather obvious.

There is value in brevity.
There is elegance in sparseness.
Elaborate wordy structures are not always the best approach.

OK, I can hear the complaints starting to bubble up already; so hold on. I’m not arguing against BDD. I really like the GWT style. And Yes, I realize that many of the BDD testing tools allow you to compress your tests into tables so that you can avoid the wordiness.

My issue is not with the tools. My issue is with the idea that BDD is the only true way, and that all tests should be expressed in GWT format forever and ever amen.

Is this a strawman? Unfortunately not. In my travels over the last several years, I have seen this attitude showing up over and over again. Someone will write a suite of complex tests using GWT style, without realizing that they could shorten it by a factor of 10 or so by recomposing it into a simple table.

Actually my issue is with the tools. Cucumber, for example, is written around Given, When, Then. If you use Cucumber, or JBehave, or any of the other BDD tools, you think in GWT and find it hard to even express something as simple as the table above.

Consider, for example, the following strange test. It measures how well the juke box selects random songs based upon their popularity. More popular songs should be played more often than less popular songs. You might specify it this way in GWT style.

Given a Juke box with 1 credit
When the user presses “You Decide”
Then the juke box will randomly choose a song based on its ranking.

Now this is a perfectly good statement of intent, and I’d certainly want it to turn green if I ran the test. However, if this turns green, what have we really learned? The test result does not really tell us whether or not the selection algorithm is accurate. The GWT statements above are really just hand-waving statements that loudly say Trust Me without necessarily verifying anything.

Now look at the following tables.

Repeat you decide 10000 times and count results

Average credits should ~=1.3

Song	Times played
Stairway to Heaven	880 < _ < 1120
In-a-godda-da-vida	1880 < _ < 2120
Viva La Vida	2880 < _ < 3120
Incense and Peppermint	880 < _ < 1120
Comfortably Numb	2880 < _ < 3120

Perhaps you don’t like the magic numbers. There are ways to deal with that, but they are beyond the scope of this blog. So let’s forget about the magic-ness of the numbers for the moment. When you run this test you see something like this:

Repeat you decide 10000 times and count results

Average credits should 1.295 ~=1.3

Song	Times played
Stairway to Heaven	880 < 998 < 1120
In-a-godda-da-vida	1880 < 2007 < 2120
Viva La Vida	2880 < 2950 < 3120
Incense and Peppermint	880 < 1019 < 1120
Comfortably Numb	2880 < 3026 < 3120

It’s hard to argue that the green-ness is hand-waving. Nothing here says: Trust Me. You can see that the selection algorithm is random, and is weighted properly.

Now I’m certain that we could construct a set of GWT statements that captures the above semantics perfectly; but why would we? The simple tables express the intent in a format that is simpler and more accessible than GWT.

Conclusion

As programmers we have already learned that we must be polyglots. If you want to write a rails system you’d better know ruby, haml, css, erb, xml, etc. If you want to write a J2EE system, you’d better know Java, JSP, HTML, XML, CSS, etc.

Testers also need to be polyglots. Writing in a single style such as GWT is not going to cut it. GWT has it’s place, but it’s not sufficient. Other testing languages are also useful. Therefore testing tools need to be polyglot tools.

Name that refactoring: 2 - Version 2

Brett Schuchert — Wed, 16 Dec 2009 02:14:00 -0600

A few updates applied to the second name that refactoring. Note that I’m using a star to represent a problem dependency. It is the “star” of the refactoring. I’m looking for a better image. I could go with a database icon, but the principle is more general than that. The cloud was confusing. So if you have an idea, please let me know what it is!

Again, thanks for taking a look and giving me the feedback.

Step 0: Problematic Dependency

Step 1: Capture dependency in a single class

Step 2: Raise the abstraction to a domain-level interface

Step 3: Introduce test doubles as needed

Name that refactoring: 1 - Version 2

Brett Schuchert — Wed, 16 Dec 2009 01:43:00 -0600

Here is an update to the first name that refactoring based on feedback. How about this?

Step 0: Method with embedded dependency

Step 1:Extract method to move dependency to another method

Step 2: Subclass and override method with new code that does not have original dependency

So what do you think? Better or worse?

What about using the star in other image? In general, the star could represent the dependency we’re trying to get rid of.

Name that refactoring: 2

Brett Schuchert — Sun, 13 Dec 2009 22:51:00 -0600

Ok, so I’m getting good feedback on the first picture. Now for a series of pictures.

How do you interpret this series?

Step 0

Step 1

Step 2

Step 3

Name that refactoring: 1

Brett Schuchert — Sun, 13 Dec 2009 19:35:00 -0600

I’m working on writing up a few handouts I can use in a TDD class (and a few other places). There are a few drawings I keep doing and I’m trying to replicate them so I can refer to something.

Problem is, with many such drawings, the observation of the creation is as important as the end product. I’d like your feedback/recommendations (or links to better pictures). I’m curious about at least two things.

What does this picture suggest to you or what can you draw from it, if anything?
Do you think this would be better served as a series of pictures showing the build-up?

Thanks for your feedback.

Object Mentor Blog

The 4-contact points of software development

Is it worth killing trees over another C&#43;&#43; Book?

Game Of Life with @lunivore

Nearly 22 years ago

Getting Started With cslim in Visual Studio 2010 Using the Command Line Tools

Rough Notes on using FitNesse with C++

Preprocessor seams and assignment of responsibility

Background

I have a memory leak?!

Where is that damn thing

Eureka!

What took so long?

Conslusions

Some C++ Fixtures for FitNesse.slim

A Decision Table

And Its Fixture Code

A Script Table

And Its Fixture Code

The Dreaded Query Table

And Its Fixture Code

And the Helper Class

FitNesse, C++ and cslim, step-by-step instructions

A Few C plus plus TDD videos

Software Calculus - The Missing Abstraction.

C++ Algorithms, Boost and function currying

TDD in Clojure

Is TDD necessary in Clojure?

Summary

Orbit in Clojure

A Coverage Metric That Matters

Hello World Revisited

Clojure Prime Factors

TDD is wasting time if...

Sufficient Design means Damned Good Design.

Certification: Don't Waste Your Time!

Pair Management

The Tricky Bit

The R.E.A.L.I.T.Y. principles of Agile Software Certification.

The Mayans are Coming!

Parts 5 and 6 of the 4-part series

Sapient Testing: The "Professionalism" meme.

C# TDD Videos - You asked for them

Open Spaces at conferences, what's your take?

20% more bugs? Or 20% less features?

First Pass Completed: Rough Draft TDD Demonstration Videos

Some Rough Draft TDD Demonstration Videos

Injecting Those Dependencies

Developer Certification WTF?

Eagle’s Entrails! Deer Droppings! and Elephant Phlatulence!

What Problem does Certification solve?

The Trials of Hercules.

Michaelangelo’s Apprentice.

Over-promise and Under-deliver.

A course, a card, and a Bridge to sell.

The one I said I’d mention later.

One way to configure FitNesse for team Development

Updated ConTest Instructions

DotNet Development Using Parallels Is No Longer Painful

Getting Started in Objective-C with XCode

Is TDD Language Neutral?

Rudimentary TDD with XCode and Objective-C

CppUTest Recent Experiences

Background

The Fixes

Header File Include Order

Boost Shared Pointer

One Time Allocation

Conclusion

Software on the Cheap

Down to business.

The Bottom Line.

Makefiles, what are you doing with them?

Mocking Mocking and Testing Outcomes.

Dependency Injection Inversion

UI Test Automation Tools are Snake Oil

The Dream

The Reality

UI Test Automation Tools are Brittle

UI Based Testing Is Not the Solution That Vendors Imply It Is

Is it worth killing trees over another C++ Book?