Open Spaces at conferences, what's your take? 44

Posted by Brett Schuchert Wed, 07 Apr 2010 15:39:00 GMT

I’m the host for an open space this year at Agile Testing Days 2010, Berlin (October 4th – 7th, the open space being the 7th). I’ve attended a few open spaces and I have some ideas on what the role of host might entail.

But, I know I’m better at collecting requirements than sourcing them, so what have you experienced that worked. What has not worked. Any advice you want to give me?

Please help me better understand what I can do to facilitate a successful open space.

What questions should I be asking?

Do you think having a few things scheduled up front in one room, and several unscheduled rooms left to be determined October 4th – 6th, through a a mix if you will, is an OK thing to do, or should what happens be left completely open?

Or, leave everything completely open until the 7th, then start with the “traditional” introductions and let the agenda form then an there?

I’m aware of a few models of open spaces. What I really want to know is, what works for you.

Updated ConTest Instructions 80

Posted by Brett Schuchert Thu, 18 Feb 2010 18:12:00 GMT

Just updated the instructions for using ConTest – a tool to assist in testing concurrent applications for Java. Check them out here: ConTest Instructions

Hiding global methods, a C++ example 34

Posted by Brett Schuchert Fri, 12 Jun 2009 15:13:00 GMT

Background

This is a continuation of this discussion. A few of the postings by GBGames have been swallowed so the context is somewhat lost. What follows is an excerpt from an email he sent me and some example C++ code that demonstrates making global methods “testable” in a sense. Or rather, making your code not use global methods during test.

The Offending Method

Here is a method that GBGames has under test:
void SDLHardwareLayer::initializeHardware()
{
    const SDL_version * sdlVersion = SDL_Linked_Version();
    if (sdlVersion->minor < 2 && sdlVersion->major < 2)
    {
        //Error message
    }
    else if (SDL_Init(SDL_INIT_VIDEO |
            SDL_INIT_AUDIO |
            SDL_INIT_NOPARACHUTE) < 0)
    {
        //Error message
    }
    else
    {
        m_screen = SDL_SetVideoMode(m_x, m_y, m_bitDepth, SDL_DOUBLEBUF);
        if (NULL != m_screen)
        {
            SDL_WM_SetCaption(m_title.c_str(), 0);
        }
    }
}

This is how this code is organized:

So What’s Wrong?

One problem he notes is that the test code driving this production code causes windows to pop up during testing. Is this a problem? If it works, then it may be OK. However, you can remove that and also make test-ability a bit easier. You’ll be testing interactions and responses rather than directly using the library through its global functions.

Global Functions Considered Harmful

First off, this code as written directly uses global functions. If you leave this code unchanged, then the only option you have to “fix” the problem of windows popping up is a link-seam (see Working Effective with Legacy Code, page 233-234. In a nutshell, a link seam will link different versions of a library, one for testing, one for actual execution. So you’ll need some “make fu” to get the link seam working in a manner that easily allows building for testing versus building for execution. Since we’re using C++ and we own this code, I chose a different route:
  • Change this code to use an object that uses the library
  • That object implements an interface (a base class with all pure virtual methods)
  • Change this code so that it acquires this object through a factory (I could have simply used constructor injection, but I’ve already written it using a configurable factory [really singleton] – but it would have been simpler if I had injected the interface)
  • Created a test double that can be used when writing new test doubles making code maintenance a bit easier (mentioned in the other blog posting).
  • Write test-method specific test doubles

Changing Code to Use Factory + Object w/virtual Methods

Rather than directly calling the various global SDL_* methods, e.g.:
const SDL_version * sdlVersion = SDL_Linked_Version();
I instead use a factory to get an instance and call
const SDL_version * sdlVersion = SdlLayerFactory::getInstance()->SDL_Linked_Version();

Base Interface

To make this work, I created a base interface representing all of the methods needed by the SDLHardwareLayer::initializeHardware:
#pragma once

struct SDL_version;

class SdlLayerInstanceDelegator {
public:
  virtual ~SdlLayerInstanceDelegator(void) = 0;

  virtual const SDL_version *SDL_Linked_Version() = 0;
  virtual int SDL_Init(int bitMask) = 0;
  virtual void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode) = 0;
  virtual void SDL_WM_SetCaption(const char *title, int someIntValue) = 0;

protected:
  SdlLayerInstanceDelegator();

private:
  SdlLayerInstanceDelegator(const SdlLayerInstanceDelegator &rhs);
  SdlLayerInstanceDelegator &operator = (const SdlLayerInstanceDelegator &);
};

Real Calls Global Methods

Getting rid of the direct call to the global method makes this work. However, you still need to call those global methods somewhere. That’s where the “real” version of the interface comes in. The real version of this class simply calls the global methods (code only, not showing header file right now):
const SDL_version *RealSdlLayerInstanceDelegator::SDL_Linked_Version() {
  return ::SDL_Linked_Version();
}

int RealSdlLayerInstanceDelegator::SDL_Init(int bitMask) {
  return ::SDL_Init(bitMask);
}

void *RealSdlLayerInstanceDelegator::SDL_SetVideoMode(int x, int y, int bitDepth, int mode) {
  return ::SDL_SetVideoMode(x, y, bitDepth, mode);
}

void RealSdlLayerInstanceDelegator::SDL_WM_SetCaption(const char *title, int someIntValue) {
  ::SDL_WM_SetCaption(title, someIntValue);
}

The Factory: Inversion of Control

The original code directly used global methods. We need to invert this relationship so that rather than depending directly on those methods, it depends on an interface. Then if it happens that it uses a “real” version, the underlying code will call the actual library. If, however, it is instead using a test double, it will not. It will do whatever the test double dictates.

We cannot leave the decision of which object to talk to in the SdlHardwareLayer class. Instead, I’ve used a configurable factory (singleton). A test can configure the factory, call the method under test, and then reset the factory back after the test.

The SdlLayerFactory allows such use(again, code only, no header file):
SdlLayerInstanceDelegator *SdlLayerFactory::instance = 0;

SdlLayerInstanceDelegator *SdlLayerFactory::getInstance() {
  return instance;
}

SdlLayerInstanceDelegator *SdlLayerFactory::replaceInstance(SdlLayerInstanceDelegator *replacement) {
  SdlLayerInstanceDelegator *original = instance;
  instance = replacement;
  return original;
}

Handling Growing Interfaces

One problem with “interfaces” in any language is that as you add methods to them, their implementations need to be updated. Since the SdlLayerInstanceDelegator class is not complete yet, this will case problems with test doubles. There are three ways to handle this problem: * Deal with it, as you add new methods to the interface, update all existing classes * Use a mocking library – I have not used any for C++, but if this were Java I’d use Mockito and if this were .Net I’d use Moq. * Create a base test double that implements all of the methods. All test doubles derive from that and only implement the methods needed for test.

The last option does not fix the problem, it controls it. When new methods are added, you update one test double class and all other classes still work.

This may not sound like much of an issue, but in general you’ll have many small test doubles rather than a few large test doubles (or at least that’s the standard recommendation). Why? Because a small, focused test double is less likely to be broken. It also better expressed your intention.

Here is that test double (header file excluded again): TestDoubleSdlLayerInstanceDelegator.h

#pragma once
#include "SdlLayerInstanceDelegator.h"

class TestDoubleSdlLayerInstanceDelegator: public SdlLayerInstanceDelegator {
public:
  TestDoubleSdlLayerInstanceDelegator();
  virtual ~TestDoubleSdlLayerInstanceDelegator();

  const SDL_version *SDL_Linked_Version();
  int SDL_Init(int bitMask);
  void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode);
  void SDL_WM_SetCaption(const char *title, int someIntValue);
};

TestDoubleSdlLayerInstanceDelegator.cpp

#include "TestDoubleSdlLayerInstanceDelegator.h"

TestDoubleSdlLayerInstanceDelegator::TestDoubleSdlLayerInstanceDelegator(){}

TestDoubleSdlLayerInstanceDelegator::~TestDoubleSdlLayerInstanceDelegator(){}

const SDL_version *TestDoubleSdlLayerInstanceDelegator::SDL_Linked_Version() {
  return 0;
}

int TestDoubleSdlLayerInstanceDelegator::SDL_Init(int bitMask) {
  return 0;
}

void *TestDoubleSdlLayerInstanceDelegator::SDL_SetVideoMode(int x, int y, int bitDepth, int mode) {
  return 0;
}

void TestDoubleSdlLayerInstanceDelegator::SDL_WM_SetCaption(const char *title, int someIntValue){}

A Test

Finally, we need some tests and some test-specific test-doubles:
#include <CppUTest/TestHarness.h>

#include "sdl.h"
#include "SdlLayerFactory.h"
#include "SdlHardwareLayer.h"
#include "TestDoubleSdlLayerInstanceDelegator.h"

TEST_GROUP(SdlHardwareLayerTest) {
  virtual void setup() {
    original = SdlLayerFactory::getInstance();
    layer = new SdlHardwareLayer;
  }

  virtual void teardown() {
    if(SdlLayerFactory::getInstance() != original)
      delete SdlLayerFactory::getInstance();

    SdlLayerFactory::replaceInstance(original);
    delete layer;
  }

  SdlLayerInstanceDelegator *original;
  SdlHardwareLayer *layer;
};

struct MajorMinorSdlhardwareLayer : public TestDoubleSdlLayerInstanceDelegator {
  struct SDL_version v;

  MajorMinorSdlhardwareLayer() {
    v.major = 0;
    v.minor = 0;
  }

  const SDL_version *SDL_Linked_Version() {
    return &v;
  }
};

TEST(SdlHardwareLayerTest, ItGeneratsErrorWithLowMajorMinorVersionNumber) {
  SdlLayerFactory::replaceInstance(new MajorMinorSdlhardwareLayer);
  try {
    layer->initializeHardware();
    FAIL("Should have thrown int value");
  } catch (int value) {
    LONGS_EQUAL(1, value);
  }
}

struct InitFailingSdlHardwareLayer : public MajorMinorSdlhardwareLayer {
  InitFailingSdlHardwareLayer () {
    v.major = 3;
    v.minor = 3;
  }

  int SDL_Init(int bitMask) {
    return -1;
  }
};

TEST(SdlHardwareLayerTest, ItGeneratesErrorWhenInitReturnsLessThan0) {
  SdlLayerFactory::replaceInstance(new InitFailingSdlHardwareLayer);
  try {
    layer->initializeHardware();
    FAIL("Should have thrown int value");
  } catch (int value) {
    LONGS_EQUAL(2, value);
  }
}

Notice that there are two test doubles. Also notice that I create them as full (implicitly) inlined classes. Virtual methods and inline methods do not interact well. C++ will make a copy of the method bodies in every .o and increase link times. However, these test classes are only used in one place, so this really does not cause any problems.

In lieu of a mocking library, this is how I’d really write this code.

The Final Big Picture

Here’s an image of the completed product:

Summary

So is this overkill? Is using the actual SDL library causing problems? Is it OK for windows to appear during test? I’m neutral on that subject. The question I want to know is: are the tests working?

  • Do they add value or are they busy work.
  • Do they allow me to make small, incremental steps towards a complete, working implementation
  • Do they run fast enough?
  • Will they run in any environment? (Can I run on a headless system?)
  • Can I run multiple tests at the same time? (Not generally an issue, but it can be.)

If the SDL library is not causing a problem, then this might be overkill. However, on a long-lived project, this little amount of extra work will pay big dividends.

Oh, how long did this really take? Around 1.5 hours. But that included:
  • Starting a Windows XP VM
  • Creating a new projec
  • Setting up the link paths and include paths
  • Creating the initial file and the necessary support to get it to compile
  • Writing the additional classes

In fact, the actual work was probably < 30 minutes total. So while this might look big, in fact it is not. It’s nearly an idiom, so it’s mostly a matter of putting the hooks in place. So is full isolation from SDL worth 30 minutes?

Yes!

All your source files are belong to us

If this looks like a complete example, it is. I have this building and running in Visual Studio 2008. I took GBGame’s original method and got it to compile and link with a minimal set of additional source files. Then I made the structural changes to support test. And here’s all of the source code, file by file:

sdl.h

#pragma once

extern "C" {
  const int SDL_INIT_VIDEO = 1;
  const int SDL_INIT_AUDIO = 2;
  const int SDL_INIT_NOPARACHUTE = 4;
  const int SDL_DOUBLEBUF = 8;

  struct SDL_version {
    int major;
    int minor;
  };

  const SDL_version *SDL_Linked_Version();
  int SDL_Init(int bitMask);
  void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode);
  void SDL_WM_SetCaption(const char *title, int someIntValue);
};

sdl_implementation.cpp

#include "sdl.h"

const SDL_version *SDL_Linked_Version() {
  return 0;
}

int SDL_Init(int bitMask) {
  return  - 1;
}

void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode) {
  return 0;
}

void SDL_WM_SetCaption(const char *title, int someIntValue){}

SdlHardwareLayer.h
#pragma once

#include <string>

class SdlHardwareLayer {
public:
  SdlHardwareLayer();
  virtual ~SdlHardwareLayer();
  void initializeHardware();

private:
  int m_x;
  int m_y;
  int m_bitDepth;
  std::string m_title;
  void *m_screen;
};

SdlHardwarelayer.cpp

#include "SdlHardwareLayer.h"

#include "sdl.h"
#include "SdlLayerFactory.h"
#include "SdlLayerInstanceDelegator.h"

SdlHardwareLayer::SdlHardwareLayer() {
}

SdlHardwareLayer::~SdlHardwareLayer() {
}

void SdlHardwareLayer::initializeHardware() {
    const SDL_version * sdlVersion = SdlLayerFactory::getInstance()->SDL_Linked_Version();
    if (sdlVersion->minor < 2 && sdlVersion->major < 2) {
      throw 1;
    }
    else if (SdlLayerFactory::getInstance()->SDL_Init(SDL_INIT_VIDEO |
            SDL_INIT_AUDIO |
            SDL_INIT_NOPARACHUTE) < 0) {
              throw 2;
    }
    else {
        m_screen = SDL_SetVideoMode(m_x, m_y, m_bitDepth, SDL_DOUBLEBUF);
        if (0 != m_screen) {
            SDL_WM_SetCaption(m_title.c_str(), 0);
        }
    }
}

SdlLayerFactory.h
#pragma once

class SdlLayerInstanceDelegator;

class SdlLayerFactory
{
public:
  static SdlLayerInstanceDelegator *getInstance();
  static SdlLayerInstanceDelegator *replaceInstance(SdlLayerInstanceDelegator *replacement);

private:
  static SdlLayerInstanceDelegator *instance;

  SdlLayerFactory();
  ~SdlLayerFactory();
};

SdlLayerFactory.cpp

#include "SdlLayerFactory.h"

SdlLayerInstanceDelegator *SdlLayerFactory::instance = 0;

SdlLayerFactory::SdlLayerFactory(){}

SdlLayerFactory::~SdlLayerFactory(){}

SdlLayerInstanceDelegator *SdlLayerFactory::getInstance() {
  return instance;
}

SdlLayerInstanceDelegator *SdlLayerFactory::replaceInstance(SdlLayerInstanceDelegator *replacement) {
  SdlLayerInstanceDelegator *original = instance;
  instance = replacement;
  return original;
}

SdlLayerInstanceDelegator.h
#pragma once

struct SDL_version;

class SdlLayerInstanceDelegator {
public:
  virtual ~SdlLayerInstanceDelegator(void) = 0;

  virtual const SDL_version *SDL_Linked_Version() = 0;
  virtual int SDL_Init(int bitMask) = 0;
  virtual void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode) = 0;
  virtual void SDL_WM_SetCaption(const char *title, int someIntValue) = 0;

protected:
  SdlLayerInstanceDelegator();

private:
  SdlLayerInstanceDelegator(const SdlLayerInstanceDelegator &rhs);
  SdlLayerInstanceDelegator &operator = (const SdlLayerInstanceDelegator &);
};

SdlLayerInstanceDelegator.cpp

#include "SdlLayerInstanceDelegator.h"

SdlLayerInstanceDelegator::SdlLayerInstanceDelegator(){}

SdlLayerInstanceDelegator::~SdlLayerInstanceDelegator(){}

TestDoubleSdlLayerInstanceDelegator.h
#pragma once
#include "SdlLayerInstanceDelegator.h"

class TestDoubleSdlLayerInstanceDelegator: public SdlLayerInstanceDelegator {
public:
  TestDoubleSdlLayerInstanceDelegator();
  virtual ~TestDoubleSdlLayerInstanceDelegator();

  const SDL_version *SDL_Linked_Version();
  int SDL_Init(int bitMask);
  void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode);
  void SDL_WM_SetCaption(const char *title, int someIntValue);
};

TestDoubleSdlLayerInstanceDelegator.cpp

#include "TestDoubleSdlLayerInstanceDelegator.h"

TestDoubleSdlLayerInstanceDelegator::TestDoubleSdlLayerInstanceDelegator(){}

TestDoubleSdlLayerInstanceDelegator::~TestDoubleSdlLayerInstanceDelegator(){}

const SDL_version *TestDoubleSdlLayerInstanceDelegator::SDL_Linked_Version() {
  return 0;
}

int TestDoubleSdlLayerInstanceDelegator::SDL_Init(int bitMask) {
  return 0;
}

void *TestDoubleSdlLayerInstanceDelegator::SDL_SetVideoMode(int x, int y, int bitDepth, int mode) {
  return 0;
}

void TestDoubleSdlLayerInstanceDelegator::SDL_WM_SetCaption(const char *title, int someIntValue){}

RealSdlLayerInstanceDelegator.h
#pragma once
#include "SdlLayerInstanceDelegator.h"

class RealSdlLayerInstanceDelegator : public SdlLayerInstanceDelegator
{
public:
  RealSdlLayerInstanceDelegator();
  virtual ~RealSdlLayerInstanceDelegator();

  const SDL_version *SDL_Linked_Version();
  int SDL_Init(int bitMask);
  void *SDL_SetVideoMode(int x, int y, int bitDepth, int mode);
  void SDL_WM_SetCaption(const char *title, int someIntValue);
};

RealSdlLayerInstanceDelegator.cpp

#include "RealSdlLayerInstanceDelegator.h"
#include "sdl.h"

RealSdlLayerInstanceDelegator::RealSdlLayerInstanceDelegator(){}

RealSdlLayerInstanceDelegator::~RealSdlLayerInstanceDelegator(){}

const SDL_version *RealSdlLayerInstanceDelegator::SDL_Linked_Version() {
  return ::SDL_Linked_Version();
}

int RealSdlLayerInstanceDelegator::SDL_Init(int bitMask) {
  return ::SDL_Init(bitMask);
}

void *RealSdlLayerInstanceDelegator::SDL_SetVideoMode(int x, int y, int bitDepth, int mode) {
  return ::SDL_SetVideoMode(x, y, bitDepth, mode);
}

void RealSdlLayerInstanceDelegator::SDL_WM_SetCaption(const char *title, int someIntValue) {
  ::SDL_WM_SetCaption(title, someIntValue);
}

SdlHardwareLayerTest.h
#include <CppUTest/TestHarness.h>

#include "sdl.h"
#include "SdlLayerFactory.h"
#include "SdlHardwareLayer.h"
#include "TestDoubleSdlLayerInstanceDelegator.h"

TEST_GROUP(SdlHardwareLayerTest) {
  virtual void setup() {
    original = SdlLayerFactory::getInstance();
    layer = new SdlHardwareLayer;
  }

  virtual void teardown() {
    if(SdlLayerFactory::getInstance() != original)
      delete SdlLayerFactory::getInstance();

    SdlLayerFactory::replaceInstance(original);
    delete layer;
  }

  SdlLayerInstanceDelegator *original;
  SdlHardwareLayer *layer;
};

struct MajorMinorSdlhardwareLayer : public TestDoubleSdlLayerInstanceDelegator {
  struct SDL_version v;

  MajorMinorSdlhardwareLayer() {
    v.major = 0;
    v.minor = 0;
  }

  const SDL_version *SDL_Linked_Version() {
    return &v;
  }
};

TEST(SdlHardwareLayerTest, ItGeneratsErrorWithLowMajorMinorVersionNumber) {
  SdlLayerFactory::replaceInstance(new MajorMinorSdlhardwareLayer);
  try {
    layer->initializeHardware();
    FAIL("Should have thrown int value");
  } catch (int value) {
    LONGS_EQUAL(1, value);
  }
}

struct InitFailingSdlHardwareLayer : public MajorMinorSdlhardwareLayer {
  InitFailingSdlHardwareLayer () {
    v.major = 3;
    v.minor = 3;
  }

  int SDL_Init(int bitMask) {
    return -1;
  }
};

TEST(SdlHardwareLayerTest, ItGeneratesErrorWhenInitReturnsLessThan0) {
  SdlLayerFactory::replaceInstance(new InitFailingSdlHardwareLayer);
  try {
    layer->initializeHardware();
    FAIL("Should have thrown int value");
  } catch (int value) {
    LONGS_EQUAL(2, value);
  }
}

RunAllTests.cpp
#include <CppUTest/CommandLineTestRunner.h>

int main(int ac, char **av) {
  return CommandLineTestRunner::RunAllTests(ac, av);
}

Rich Hickey on Testing 177

Posted by Dean Wampler Fri, 05 Jun 2009 17:40:00 GMT

It was an interesting week at JavaOne, with lots of talks and hallway discussions about new languages on the JVM. One of those languages is Clojure.

Rich Hickey, the creator of Clojure, gave a talk at the Bay Area Clojure User Group Wednesday evening. During the Q&A part, he said that he’s not big on writing tests, although he always runs the tests that other people have written before he commits changes.

Of course, there are many people, including us Object Mentors, who consider TDD to be an essential part of professional software development. Obviously not everyone agrees. James Coplien has been another critic of this view.

We should never accept any dogma. Why is TDD considered important? What does it purport to do? TDD provides two important benefits.

  • Driving the design.
  • Building a suite of automated regression tests.

So, if you can satisfy both requirements without TDD, then technically you don’t need it. In Rich’s case, he said he spends a lot of time thinking about what he’s going to do before he does it. In this way, he satisfies the first requirement, driving the design. I had a spirited discussion with some Thoughtworkers afterwards and Ola Bini said what a lot of us think, “I do that thinking by writing tests.” I’ll freely admit that when I am really experimenting with ideas, I might just write code, but once I know how to proceed, I return to the beginning and test drive the “production” code.

Rich also made an off-hand comment that if he screws something up, he’s got thousands of users who will let him know! That ex post facto testing, along with the Rich’s own devotion to doing high-quality work, does a good job of handling regressions.

But Rich mentioned something else that is also very important. In a functional language, where values are immutability and mutable state is handled in specific, principled ways, regressions don’t happen nearly as often. Clojure has one of the most deeply thought out approaches for handling state, which is the genius of Clojure.

I asked Rich how long he worked on Clojure before releasing it to the world. He spent about 2 1/2 years, much of that time working exclusively on Clojure (and eating through his savings). When he finally “announced” it, his “marketing” consisted of one email to some friends in the Common Lisp community. The rest was viral, a testament to the justified excitement Clojure has generated.

For me, I’ll probably always do my design thinking through tests, especially when I’m writing code in imperative languages, like Java and Ruby. I’ll continue to encourage my clients to use TDD, because I find that TDD is the most productive way to achieve high quality. I want the safety net of a good test suite. I’m also writing more and more of my code in a functional style, with minimal side effects and mutable data. You should, too.

FitNesse Tutorials 194

Posted by Brett Schuchert Fri, 03 Apr 2009 00:35:00 GMT

Here is another tutorial for FitNesse: http://schuchert.wikispaces.com/FitNesse.Tutorials.2.

This tutorial and the first now all fit together and form one ongoing example. If you work though the first three tutorials at: http://schuchert.wikispaces.com/FitNesse.Tutorials, you’ll have practiced:
  • Using Decision Tables
  • Using Query Tables
  • Refactoring within FitNesse
  • Using SetUp and TearDown pages
  • Understanding inheritance of SetUp and TearDown pages
  • Basic test organization under a suite
  • Switching into unit testing from acceptance testing

There’s more to go, but that’s a good start to get you cracking at the fundamentals of FitNesse.

As a bonus, there’s a demonstration of some code in Java that produces query results in a snap. The source code is on github: http://github.com/schuchert/queryresultbuilder/tree/master.

Here is one such example taken from that tutorial:
   public List<Object> query() {
      List<Program> programs = CreateSeasonPassFor.getSeasonPassManager()
            .toDoListContentsFor(programId);

      QueryResultBuilder builder = new QueryResultBuilder(Program.class);
      builder.register("timeSlot", new TimeSlotPropertyHandler());
      QueryResult result = builder.build(programs);
      return result.render();
   }

Hope this is useful!

!define TEST_SYSTEM {fit:A} part II 26

Posted by Brett Schuchert Wed, 04 Mar 2009 10:07:00 GMT

In 11/2008 I wrote the first part of this article but I really did not give the background on why I originally asked Bob to add this feature.

So why does FitNesse need to be able to run different parts of suites in different VM’s?

Imagine you are testing services registered on an ESB. That ESB allows deployment of services as EJB’s. There’s something important about that fact. By default, different EJB’s run with their own class loader. You can dig into the EJB spec if you’d like, but there’s roughly a hierarchy of 5 class loaders by the time you get to an executing EJB (can be more, could be less).

Why is this important? If two EJB’s make reference to the “same” JAR file, then each could get loaded by the same or a different class loader. If the JAR file is installed high in the class-loader hierarchy, then they will be loaded by the same class loader instance and therefore will be the same classes.

However, let’s assume that each EJB represents ones service. Furthermore, each EJB makes a reference to some JAR file directly, that is, it is not visible to a hierarchically higher class loader. Then that same JAR file will get loaded twice, once by each class loader (this really happens), and those exact same classes will be considered different.

That’s right, the exact same JAR file loaded by 2 different class loaders are treated as unrelated classes.

In reality, each deployed EJB is “complete” in that in includes the JAR’s it needs to execute. So the “same” jar – same in terms of file contents – exists twice in the system and is loaded by two class loaders.

Now, it gets a little worse. Those two EJB’s work off a “common” data model. Common in that the underlying business objects are common across a set of applications. However, the common data model is a moving target (it’s grown as needed). To manage complexity, not all services are forced to update to the latest version of the common data model at the same time (it’s not really a manageable solution – generally).

So now we have 2 (really several) EJB’s that refer to logically the same JAR but different versions. Luckily, this is no problem because each EJB has its own class loader, so it is OK for 2 EJB’s to refer to different versions of the same logical data model.

How does any of this have anything to do with FitNesse? When FitNesse executes a suite, by default all tests are executed in the same VM with one class loader. So if you have a suite of tests that test different services, each of which might be using a different version of the common data model, then the first service executed loads the common data model, which really isn’t exactly common. This causes tests to fail that would not fail if each text executed with its own class loader.

Rather than deal with writing custom class loaders, Bob instead make it possible to name a VM (see the title for the syntax). By doing this, Bob make it possible to effect a similar environment to that experienced by EJB’s by default.

FitNesse.Slim table table example and migration to a DSL 69

Posted by Brett Schuchert Mon, 23 Feb 2009 12:03:00 GMT

I’m working on some notes related to an upcoming presentation. You can see an example of using the Slim table table here: Table Table Example

It’s a work in process. If you see something that doesn’t quite make sense, please let me know.

Using SliM in FitNess with .Net 74

Posted by Brett Schuchert Sun, 22 Feb 2009 21:10:00 GMT

If you’d like a quick introduction to using SliM with .Net, Mike Stockdale is working on an implementation and here’s a quick writeup of what you need to do to get it working: Using Slim.Net in FitNesse

TDD on Three Index Cards 103

Posted by tottinger Fri, 07 Mar 2008 01:39:00 GMT

I had the opportunity to talk to a fellow who missed part of a class on TDD. I told him that I could give him a 15-minute overview, and give him all the essentials of TDD on three index cards.

Yes, I know that volumes have been written about TDD and BDD, and that it’s a large topic with many many branches of application, but I didn’t have time for that. I had time for three index cards. I figure that an index card is a token, and it represents a conversation, and that one can always dig deeper later.

They looked more or less like this:


Card 1: Uncle Bob’s Three Laws (Object Mentor)
  1. Write no production code except to pass a failing test.
  2. Write only enough of a test to demonstrate a failure.
  3. Write only enough production code to pass the test.

Card 2: FIRST Principles (Brett and Tim at Object Mentor)
  • Fast: Mind-numbingly fast, as in hundreds or thousands per second.
  • Isolated: The test isolates a fault clearly.
  • Repeatable: I can run it repeatedly and it will pass or fail the same way each time.
  • Self-verifying: The test is unambiguously pass-fail.
  • Timely: Produced in lockstep with tiny code changes.

Card 3: Flow (using the famous three-node circle diagram) – origin unknown.
  • Red: test fails
  • Green: test passes
  • Refactor: clean code and tests

Sure there is plenty more, but I didn’t know how I could provide significantly less. As is, I’m pretty happy with the exercise. Now I am wondering if I couldn’t produce most of the important information I wish to convey as a series of index cards. Would that be cool or what?

Unit Testing C and C++ ... with Ruby and RSpec! 110

Posted by Dean Wampler Tue, 05 Feb 2008 04:08:00 GMT

If you’re writing C/C++ code, it’s natural to write your unit tests in the same language (or use C++ for your C test code). All the well-known unit testing tools take this approach.

I think we can agree that neither language offers the best developer productivity among all the language choices out there. Most of us use either language because of perceived performance requirements, institutional and industry tradition, etc.

There’s growing interest, however, in mixing languages, tools, and paradigms to get the best tool for a particular job. <shameless-plug>I’m giving a talk March 7th at SD West on this very topic, called Polyglot and Poly-Paradigm Programming </shameless-plug>.

So, why not use a more productive language for your C or C++ unit tests? You have more freedom in your development chores than what’s required for production. Why not use Ruby’s RSpec, a Behavior-Driven Development tool for acceptance and unit testing? Or, you could use Ruby’s version of JUnit, called Test::Unit. The hard part is integrating Ruby and C/C++. If you’ve been looking for an excuse to bring Ruby (or Tcl or Python or Java or…) into your C/C++ environment, starting with development tasks is usually the path of least resistance.

I did some experimenting over the last few days to integrate RSpec using SWIG (Simplified Wrapper and Interface Generator), a tool for bridging libraries written in C and C++ to other languages, like Ruby. The Ruby section of the SWIG manual was very helpful.

My Proof-of-Concept Code

Here is a zip file of my experiment: rspec_for_cpp.zip

This is far from a complete and working solution, but I think it shows promise. See the Current Limitations section below for details.

Unzip the file into a directory. I’ll assume you named it rspec_for_cpp. You need to have gmake, gcc, SWIG and Ruby installed, along with the RSpec “gem”. Right now, it only builds on OS X and Linux (at least the configurations on my machines running those OS’s – see the discussion below). To run the build, use the following commands:

    
        $ cd rspec_for_cpp/cpp
        $ make 
    

You should see it finish with the lines

    
        ( cd ../spec; spec *_spec.rb )
        .........

        Finished in 0.0***** seconds

        9 examples, 0 failures
    

Congratulations, you’ve just tested some C and C++ code with RSpec! (Or, if you didn’t succeed, see the notes in the Makefile and the discussion below.)

The Details

I’ll briefly walk you through the files in the zip and the key steps required to make it all work.

cexample.h

Here is a simple C header file.

    
        /* cexample.h */
        #ifndef CEXAMPLE_H
        #define CEXAMPLE_H
        #ifdef __cplusplus
         extern "C" {
        #endif
        char* returnString(char* input);
        double returnDouble(int input);
        void  doNothing();

        #ifdef __cplusplus
         }
        #endif
        #endif
    

Of course, in a pure C shop, you won’t need the #ifdef __cplusplus stuff. I found this was essential in my experiment when I mixed C and C++, as you might expect.

cpp/cexample.c

Here is the corresponding C source file.

    
        /* cexample.h */

        char* returnString(char* input) {
            return input;
        }

        double returnDouble(int input) {
            return (double) input;
        }

        void  doNothing() {}
    

cpp/CppExample.h

Here is a C++ header file.

    
        #ifndef CPPEXAMPLE_H
        #define CPPEXAMPLE_H

        #include <string>

        class CppExample 
        {
        public:
            CppExample();
            CppExample(const CppExample& foo);
            CppExample(const char* title, int flag);
            virtual ~CppExample();

            const char* title() const;
            void        title(const char* title);
            int         flag() const;
            void        flag(int value);

            static int countOfCppExamples();
        private:
            std::string _title;
            int         _flag;
        };

        #endif
    

cpp/CppExample.cpp

Here is the corresponding C++ source file.

    
        #include "CppExample.h" 

        CppExample::CppExample() : _title("") {}
        CppExample::CppExample(const CppExample& foo): _title(foo._title) {}
        CppExample::CppExample(const char* title, int flag) : _title(title), _flag(flag) {}
        CppExample::~CppExample() {}

        const char* CppExample::title() const { return _title.c_str(); }
        void        CppExample::title(const char* name) { _title = name; }

        int  CppExample::flag() const { return _flag; }
        void CppExample::flag(int value) { _flag = value; }

        int CppExample::countOfCppExamples() { return 1; }
    

cpp/example.i

Typically in SWIG, you specify a .i file to the swig command to define the module that wraps the classes and global functions, which classes and functions to expose to the target language (usually all in our case), and other assorted customization options, which are discussed in the SWIG manual. I’ll show the swig command in a minute. For now, note that I’m going to generate an example_wrap.cpp file that will function as the bridge between the languages.

Here’s my example.i, where I named the module example.

    
        %module example
        %{
            #include "cexample.h" 
            #include "CppExample.h"    
        %}
        %include "cexample.h" 
        %include "CppExample.h" 
    

It looks odd to have header files appear twice. The code inside the %{...%} (with a ’#’ before each include) are standard C and C++ statements, etc. that will be inserted verbatim into the generated “wrapper” file, example_wrap.cpp, so that file will compile when it references anything declared in the header files. The second case, with a ’%’ before the include statements1, tells SWIG to make all the declarations in those header files available to the target language. (You can be more selective, if you prefer…)

Following Ruby conventions, the Ruby plugin for SWIG automatically names the module with an upper case first letter (Example), but you use require 'example' to include it, as we’ll see shortly.

Building

See the cpp/Makefile for the gory details. In a nutshell, you run the swig command like this.

    
        swig -c++ -ruby -Wall -o example_wrap.cpp example.i
    

Next, you create a dynamically-linked library, as appropriate for your platform, so the Ruby interpreter can load the module dynamically when required. The Makefile can do this for Linux and OS X platforms. See the Ruby section of the SWIG manual for Windows specifics.

If you test-drive your code, which tends to drive you towards minimally-coupled “modules”, then you can keep your libraries and build times small, which will make the build and test cycle very fast!

spec/cexample_spec.rb and spec/cppexample_spec.rb

Finally, here are the RSpec files that exercise the C and C++ code. (Disclaimer: these aren’t the best spec files I’ve ever written. For one thing, they don’t exercise all the CppExample methods! So sue me… :)

    
        require File.dirname(__FILE__) + '/spec_helper'
        require 'example'

        describe "Example (C functions)" do
          it "should be a constant on Module" do
            Module.constants.should include('Example')
          end
          it "should have the methods defined in the C header file" do
            Example.methods.should include('returnString')
            Example.methods.should include('returnDouble')
            Example.methods.should include('doNothing')
          end
        end

        describe Example, ".returnString" do
          it "should return the input char * string as a Ruby string unchanged" do
            Example.returnString("bar!").should == "bar!" 
          end  
        end

        describe Example, ".returnDouble" do
          it "should return the input integer as a double" do
            Example.returnDouble(10).should == 10.0
          end
        end

        describe Example, ".doNothing" do
          it "should exist, but do nothing" do
            lambda { Example.doNothing }.should_not raise_error
          end
        end
    

and

    
    require File.dirname(__FILE__) + '/spec_helper'
    require 'example'

    describe Example::CppExample do
      it "should be a constant on module Example" do
        Example.constants.should include('CppExample')
      end
    end

    describe Example::CppExample, ".new" do
      it "should create a new object of type CppExample" do
        example = Example::CppExample.new("example1", 1)
        example.title.should == "example1" 
        example.flag.should  == 1
      end
    end

    describe Example::CppExample, "#title(value)" do
      it "should set the title" do
        example = Example::CppExample.new("example1", 1)
        example.title("title2")
        example.title.should == "title2" 
      end
    end

    describe Example::CppExample, "#flag(value)" do
      it "should set the flag" do
        example = Example::CppExample.new("example1", 1)
        example.flag(2)
        example.flag.should == 2
      end
    end
    

If you love RSpec like I do, this is a very compelling thing to see!

And now for the small print:

Current Limitations

As I said, this is just an experiment at this point. Volunteers to make this battle-ready would be most welcome!

General

The Example Makefile File

It Must Be Hand Edited for Each New or Renamed Source File

You’ve probably already solved this problem for your own make files. Just merge in the example Makefile to pick up the SWIG- and RSpec-related targets and rules.

It Only Knows How to Build Shared Libraries for Mac OS X and Linux (and Not Very Well)

Some definitions are probably unique to my OS X and Linux machines. Windows is not supported at all. However, this is also easy rectify. Start with the notes in the Makefile itself.

The module.i File Must Be Hand Edited for Each File Change

Since the format is simple, a make task could fill a template file with the changed list of source files during the build.

Better Automation

It should be straightforward to provide scripts, IDE/Editor shortcuts, etc. that automate some of the tasks of adding new methods and classes to your C and C++ code when you introduce them first in your “spec” files. (The true TDD way, of course.)

Specific Issues for C Code Testing

I don’t know of any other C-specific issues, so unit testing with Ruby is most viable today for C code. Although I haven’t experimented extensively, C functions and variables are easily mapped by SWIG to a Ruby module. The Ruby section of the SWIG manual discusses this mapping in some detail.

Specific Issues for C++ Code Testing

More work will be required to make this viable. It’s important to note that SWIG cannot handle all C++ constructs (although there are workarounds for most issues, if you’re committed to this approach…). For example, namespaces, nested classes, some template and some method overloading scenarios are not supported. The SWIG manual has details.

Also, during my experiment, SWIG didn’t seem to map const std::string& objects in C++ method signatures to Ruby strings, as I would have expected (char* worked fine).

Is It a Viable Approach?

Once the General issues listed above are handled, I think this approach would work very well for C code. For C++ code, there are more issues that need to be addressed, and programmers who are committed to this strategy will need to tolerate some issues (or just use C++-language tools for some scenarios).

Conclusions: Making It Development-Team Ready

I’d like to see this approach pushed to its logical limit. I think it has the potential to really improve the productivity of C and C++ developers and the quality of their test coverage, by leveraging the productivity and power of dynamically-typed languages like Ruby. If you prefer, you could use Tcl, Python, even Java instead.

License

This code is complete open and free to use. Of course, use it at your own risk; I offer it without warranty, etc., etc. When I polish it to the point of making it an “official” project, I will probably release under the Apache license.

1 I spent a lot of time debugging problems because I had a ’#’ where I should have had a ’%’! Caveat emptor!

Older posts: 1 2