Testing is a slippery subject, and it’s reasonably hard to talk about for one simple reason: the nomenclature is chaotic. Years ago, I went to a summit with some testing gurus. I was one of the lone developers there and I asked about the taxonomy of testing. Cem Kaner, Bret Pettichord, Brian Marick, and James Bach went through it for us on a flipchart, and it was a nightmare. You can name tests after their scope: (unit, component, system), their place in the development process (smoke, integration, acceptance, regression), their focus (performance, functional) their visibility (white box, black box), the role of the people writing them (developer, customer).. The list goes on. There are far more than I can remember.

Why is it so confusing? There are are a couple of reasons. One is that different communities have developed different nomenclature over time. But, let’s face it, that’s true in most fields. The thing which makes testing nomenclature worse is that the tests themselves aren’t all that different, or at least, they are often not different enough for us to for us to distinguish them without being told. Yes, we can tell the difference between a unit test and an acceptance test in most systems, but really there is no force which prevents tests of different types from bleeding through into each other. Often the “type” of a test is more like an attribute: “here I have a blackbox smoke test, written by a developer for component integration.” In the end, all we have are tests and each of them can serve purposes beyond the purpose we originally intended.

Earlier today, I read a blog by Stephen Walther: TDD Tests are not Unit Tests. In it, he draws some distinctions between various types of testing. It’s great that he wrote it because it’s nice for us to have mental categories for these things, but we have to remember is that they really are just categories. We get to choose how distinct they will be. When I write code, most of my TDD tests end up being the same as my unit tests. I find value in forcing that overlap, and in general, I think overlapping test purposes are great to the degree that the purposes don’t conflict. You get more for less that way.

I don’t see any remedy for the muddle of test types. We will continue to make up terms to distinguish tests. We’ll just have to remember that the types are labels, not bins.

1. A value is the instantaneous state of an object. – In OO languages, we have objects. In FP languages, we throw out the object and instead manipulate the values it would take on over time.
2. Algebraic data types are classes. – Every case in an ADT is a state that an “object” can be in.

   
   data Tree = Empty 
             | Leaf Int 
             | Node Tree Tree
   

   When we write functions over ADTs, we are obliged to cover all of the cases. So, for instance, if we define depth for Empty, we have to define depth for the Leaf and Node cases as well. When we do, we can evaluate depth t for any tree value and have a well-defined result.
3. The functions which we define over an ADT can be considered its public interface. – There’s a school of thought which says that encapsulation doesn’t matter in an functional programming language because values are immutable and corruption can’t happen. Nothing could be further from the truth. If we add or remove a case from an ADT all of the functions which pattern match against it are impacted. While we don’t need to have an encapsulation boundary as tight as we might have in an OO language – it pays to be conscious of how far ADTs travel in a program. Encapsulation is the act of forming a boundary by transforming an ADT into some other form of data.

Each of these statements is a lie, an artful simplification, but they are a convenient and not entirely false way of thinking about functional programming until it becomes second-nature.

Functional programming is in the air. It’s nearly unavoidable. Even if you haven’t heard people talk about it or haven’t read a blog about it, you’ve probably seen its influence in your project. People are taking ideas that they learned in functional programming and applying them in straight object-oriented code. In Java, where there are no closures, today you are much more likely to encounter someone’s hand-rolled fold or map abstraction, and even if they haven’t gone that far, you are likely to find attempts to replace mutable data with immutable data. It’s part of the learning process, and there are some interesting patterns which occur along the way.

OO programmers often think in terms of entities. They imagine objects with identity and changeable local state; messages sent to objects alter their local state and trigger further message sends. Classic OO is intrinsically time-oriented. How do we pull this into a functional world? One thing that we can do is choose to see an entity as a series of values over time. Here’s one way to do it. If we have an entity like this:

public class EventDay {
    private LocalDate date;

    ...

    public void advance() {
        do {
            date = RegionalEventCalendar.nextValidDate(date)
        } while (!fitsLocalCalendar(date));
    }

    ...
}

public class EventDay {
    private final LocalDate date;

    ...

    public EventDay next() {
        return new EventDay(prospectiveDay(date));
    }

    ...
}

LocalDate next  = date.nextDay().nextWeek().fridayAfter();

I spent most of yesterday afternoon working on a paper I’m co-writing. It was one of those days when the writing came easy. I was moving from topic to topic, but then I realized that I was reaching too far backward – I was explaining things which I shouldn’t have had to explain to the audience I was trying to reach.

There are times when I sit down to work on some piece of code and I see a bit of duplication. I remove it. Then I see a bit more, and a bit more. When I’m done with all of my extractions, I look back at the code and notice that it’s increased in size about 20 or 30 percent. If I’m working with someone else, it’s often a shock for them. After all, we are used to thinking that code gets smaller when we remove duplication. Often it does, but not always.

private void adjustPaths() {
  pathNew = pathNew.removeFirstSegments(1).removeFileExtension()
                  .addFileExtension("class");
  pathOld = pathOld.removeFirstSegments(1).removeFileExtension()
                  .addFileExtension("class");
}

private void adjustPaths() {
  pathNew = adjustPath(pathNew);
  pathOld = adjustPath(pathOld);
}

private IPath adjustPath(IPath path) {
  return path.removeFirstSegments(1).removeFileExtension()
               .addFileExtension("class");
}

/* cruft */ private IPath adjustPath(IPath path) { 
/* meat  */    return path.removeFirstSegments(1).removeFileExtension()
/* meat  */                .addFileExtension("class");
/* cruft */ } 

It’s been fun watching the reactions to new features in C# 4.0. Some people love them. Others wonder, legitimately, where it is all going to end. The argument for feature addition is simple: over time we find better ways of doing things and our languages, as tools, should allow us that flexibility. The only downside to this strategy is that you end up with sprawling, complex languages over time – you never get to revisit the foundations.

Fortunately, however, people design new languages all the time and some of them do eventually enter the mainstream. We get a chance to start over and address foundational problems. And, that’s nice because we can do better than Java and C# for mainstream development and I don’t think there is any way to mutate either language into a better foundation.

Before I launch into the wish list, however, I want to set the context.

When I say “mainstream language” I am talking about languages which are in the Java/C#/VB market space – languages which are light on rocket science, seen as suitable for large-scale development, and don’t scare people. So, I’m not going to suggest dynamic typing or (on the other side of the coin) tight Hindley-Milner type systems and lazy evaluation. I love those approaches and I’m happy (in particular) that Ruby is gaining widespread acceptance, but I’m not going to fight that fight here. In the immediate future, for whatever reason, there will be development shops which feel much more comfortable with traditional static typing – the kind found in Java, C#, and VB. Given that, the question becomes: what can we do to make that sort of language better?

Here’s my list:

1. Immutability by Default – Over the past few years, a rather broad consensus has emerged around the idea that code is easier to understand and maintain when it has less mutable state. This isn’t a new idea; it’s been around for as long as functional programming, but our recent concerns with concurrency and our move toward multi-core computing just underscore the state problem. A mainstream language should, at the very least, make mutable data something special. References should be immutable by default and mutable state should be marked by a special keyword so that its use leaps out at you. It’s too late for the current crop of languages to make such a pervasive change, but the next mainstream language could.
2. Support for Components – In large-scale development, teams have to manage usage and dependency across an organization. The notions of public, protected, and private are too coarse as protection mechanisms, and we really need scopes larger than class, assembly, and package. In the end, this sort of protection is a social issue, so we should have mechanisms which make use very easy within a team (3-10 people working together) and somewhat more manageable between teams. It’s odd that language-based support for this work stopped with Ada and the Modula family of languages. Java’s recent move toward support for modules seems to be an exception.
3. Support for Scoped and Explicit Metaprogramming – In the past, language designers avoided adding meta-programming support to their languages because they were scared it would be abused. However, we’ve learned that without meta-programming support, people create yet another distinctive type of mess. If there is a middle ground for mainstream languages it probably involves scoping the use of metaprogramming and making it far more detectable. If, for instance, all of the code which modifies a given component had to be registered with some easily locatable component-specific construct, maintenance would be much easier.
4. Support for Testing – This one is only a matter of time, I think. In the last 10 years we’ve seen an explosion of mocking tools and testing frameworks. It’s not clear to me that we’ve reached any sort of consensus yet, but I suspect that at the very least we could add constructs to a language which make mocking and stubbing much easier. It’s also about time that languages attempt to solve the problems that dependency injection addresses.
5. Imposed I/O Separation – This is the controversial one. The more I work with Haskell, the more I notice that there is a beneficial discipline that comes from factoring your program so that its points of contact with the outside world can not be mixed with the pieces doing the work. When you first start to work that way, it feels like a straitjacket, but I think the benefit is apparent to anyone who has had to go on a spelunking expedition in an application to find and mock parts of the system which touch the outside world. Is that discipline too much for a mainstream language? I hope not.

So, that’s my list. There is no “grand language planning board” which decides these things. We will move forward chaotically like ever other industry, but I do hope that some of these features make it into the next mainstream programming language.

The week after next, I’ll be visiting Russia to do some training and some University lectures.

The lectures will be at Ural State University, Ekaterinburg. Each starts at 18:30:

Monday, Dec 8th Recovering a Code Base

Wednesday, Dec 10th Error Handling as a First Class Consideration in Design

Thursday, Dec 11th Design Sense – Reaching Consensus on Excellence

I’ll also be speaking to the Agile Russia group in Moscow Friday, Dec 12th.

The internet tells me it might be colder than Miami. I will probably bring a jacket.

When you’re working in a batch of legacy C code, it’s often hard to build and test just a part of it. You may want to build some set of functions, but then you discover that they call other functions, and those functions call yet other functions. When link dependencies are very bad, you may discover that you’re linking in the entire system.

Fortunately, there is a way out of this. Pick a set of files that appear to be a “component” and attempt to build them into an executable along with a simple main function.

You will get link errors.

Create a file called <componentname>_stubs.c and then go through each of the link errors. Look up the full declaration of the variable or function the error describes. Create a stub for each of them in the stub file.

Here’s an example of a function stub:

int irc_send_ppr(struct ppr *pprn, int nSize)
{
}

The only problem with this is that you have to place something in the stub function. For functions which return values you need to, at the very least, provide a return value. But what should it be?

int irc_send_ppr(struct ppr *pprn, int nSize)
{
  /* is negative -1 okay?   If it isn’t, how will we know? */
  return -1;
}

There is alternative ++:

int irc_send_ppr(struct ppr *pprn, int nSize)
{
  assert(!”irc_send_ppr boom!!”);
}

This is an exploding link stub. When you have it in place, you will know when a function you’re testing calls a stub. When you see the boom you go to the code and replace the assert with a better stub implementation.

When you use link stubs, you have to be able to build your component two different ways. Production builds link to the rest of the code. Test builds link to the stubs and a testing main. There’s no need to stub out all of the external functions that a component uses. Many low level functions can remain direct calls, but stubbing out calls to other high level components can give you a decent amount of leverage as you try to get an area under test.

Fortunately, for any given component you only have to go through the massive grunt work once. Once you do, you can reap the reward of easier testing in that component forever.

++ Note: If you are using C99, you can genericize this code by using the __func__ predefined macro. In any function, __func__ expands to the name of the enclosing function.

Take a look at this code:

public static int findPositionInInterval(int [] content, int start, int end) {
  for(int n = 0; n < content.length; n++) {
    int current = transform(content[n]);
    if (start <= current && current <= end) {
      return n;
    }
  }
  return -1;
}

How does it look? Great? Poor? Is it code with a glaring problem?

Hold that thought for a second and then take a look at this code:

public static int findPositionInInterval(final int [] content, final int start, final int end) {
  for(int n = 0; n < content.length; n++) {
    final int current = transform(content[n]);
    if (start <= current && current <= end) {
      return n;
    }
  }
  return -1;
}

I like it better.

The fact of the matter is: mutable state hurts. Unless you’ve done some functional programming you might not realize that it hurts, but it does. It may feel okay to walk barefoot, but if some gives you a comfy pair of shoes you quickly learn that, well, your feet were irritated by pebbles and twigs and there was a better way of walking around, you just didn’t know about it. Fixing state and making it immutable is just like that. When you become used to that style, you notice that it’s easier to reason about your code.

In my mind’s eye, the first snippet is a like a box containing loose parts. If you shake it, it rattles. Parts that don’t need to move are fixed in place by final. In code as brief as this, it doesn’t smell mightily, but it smells nonetheless.

So mutable data is a smell and we can fix it. However, there’s a problem. I’d argue that the second snippet is just a little noisier than the first one. Java forces us to do something special to make data immutable. C# and C++ are the same way: mutable is the default and immutability requires special keywords. There are some languages which make immutability the default, or least make don’t make you pay the price of an extra token to make something immutable. Haskell, OCaml, Scala, and F# all fit into this category. In the older languages, however, we’ll continue to have a lot of rattling data.

Listen for it.

The other day, I was browsing some code with a team and we came across a cluster of static methods. I looked at them and asked whether they were used someplace else without an object reference. I could’ve done a find references in the IDE, but one of the team members had the answer: “No, they’re just used here.”

“Okay, so why are they static?”, I asked.

“Well, the IDE pointed out that they don’t refer to any instance data, so I made them static.”

Technically, there’s nothing wrong with that. I do the same thing sometimes. I make a method static to document its independence of instance data. But, this scenario highlights something important about static and a few other keywords: their uses can be seen as statements of fact or statements of intention. Static can be read as “Hey, this function doesn’t use any instance data, and you should know that.” Or, it can be read as “I am making this static so that it can be used easily anyplace without an instance.” When you’re trying to read a pile of unfamiliar code, it’s nice to know whether you can count on one meaning or the other.

I don’t think I’ve ever heard anyone articulate this directly, but there’s a tendency in a lot of best practice literature to close the gap between fact and intention. Joshua Bloch’s advice to use make fields final whenever you can in Java is a good example. The traditional advice to make methods private whenever they are not used outside of a class is another.

Part of me feels that closing the gap is a good practice but the fact is, there are holes in it. If you are developing a library or a framework, you do have to write code that may not reflect the facts of your code, but rather will reflect the facts of your code and the code that people write to use it or extend it. Beyond that, it might be overly conservative to reduce visibility of things that aren’t used beyond a particular scope. For instance, imagine a method that is used only by other methods of a class. We understand the invariant of the class and we see that the method could be public. We could make it private now, but that means that anyone who wants to make it public later would have to do some re-analysis to arrive at the answer that we have right now.

The Fact/Intention Gap is a very real thing. Whether we know it or not, we confront it every time we try to understand unfamiliar code. I think there’s only one way to solve it, and that’s to try to separate fact from intention in our languages and tooling. Imagine what it would be like if your IDE gave you a visual indicator for all methods which didn’t use instance data. If it did, you could use static on methods only when you want to indicate that the intention is to use them without an instance.

It seems that IDE developers are moving in this direction. I wonder if any language designers will follow suit.