Today we picked one method in particular and attempted to work our way through it. This method was a classic legacy method (no unit tests). It also had a switch on type and then it also did one or more things based on a set of options. All of this was in one method.

If you read Fowler’s Refactoring Book, it mentions a combination of encapsulating the type code followed by replacing switch with polymorphism for the first problem in this method (the switch). We were able to skip encapsulating the type code since we wanted to keep the external API unchanged (legacy code).

So we first created a base strategy for the switch and then several empty derived classes, one for each of the enums. This is a safe legacy refactoring because it only involved adding new code.

Next, we created a factory to create the correct strategy based on the type code and added that to the existing method (we also added a few virtual methods). Again, a safe refactoring since it only involved adding effectively unused code (we did create the factory using nearly strict TDD). Finally, we delegated from the original method to the strategy returned from the factory. Safe again, since we had tested the factory.

So far, so good. But next, we wanted to push the method down to each of the subclasses and remove the parts of the logic that did not apply to each given type. We did a spike refactoring to see what that’d be like and it was at least a little dicey. We finally decided to get the original method under test so that as we refactored, we had the safety net necessary to refactor with confidence.

We started by simply calling the method with null’s and 0 values. We worked our way through the method, adding hand-rolled test doubles until we came across our first static class.

• Added an instance of the class as a static member (essentially creating a singleton).
• Added a property setter and getter (making it an overridable singleton).
• We then copied the body of the static method into an instance method, which we made virtual.
• We then delegated the static method to the virtual method.
• Then, in the unit test, we set the singleton to a hand-coded test double in the setup and reset the singleton in the tear down method.

We had to do this several times and on the third time (I think it was the third time), the hand-rolled test double would have had to implement several (17ish) methods and it became clear that we were ready to use a mocking framework. They are using Moq so we started using Moq to accomplish the remainder of the mocking.

After some time, we managed to get a test that essentially sent a tracer bullet through one path of the method we wanted to get under test. When the test turned green there was much rejoicing.

• This method currently makes calls to the service layers and those calls depend on both an enumeration (replaced with a shallow and wide hierarchy of strategies) and options (to be replaced with a composition of strategies).
• It also changes some values in an underling domain object.

So that’s what we needed to characterize.

• Begin as we already had, get through the method with a tracer.
• Determine the paths we want to get under test.
• For each path:
  • Create a test using strict mocks (which fail as soon as an unexpected method is called)
  • Use a Setup to document this call as expected – this is essentially one of the assertions for the characterization test.
  • Continue until we have all the Setups required to get through the test.
  • Add any final assertions based on state-based checks and call VerifyAll on the Moq-based mock object.

This would be a way we could work through the method and characterize it before we start refactoring it in earnest.

This might sound like a lot of work and it certainly is no cake walk, but all of this work was done by one of the attendees and as a group they certainly have the expertise to do this work. And in reality, it did not take too long. As they practice and get some of the preliminary work finished, this will be much easier.

• Spent time on one project practicing TDD and refactoring to patterns (they implemented 5 of the GoF patterns).
• Spent time practicing some of Fowler’s refactorings and Feather’s legacy refactorings.
• Spent a day practicing TDD using mocks for everything but the unit under test. At the end they had a test class, one production class and several interfaces.

In retrospect, the work they did in the first three days was nearly exactly what they needed to practice for the final day of effort. When we started tackling their legacy code, they had already practiced everything they used in getting the method under test.

So overall great week with a fun group of guys in Canada.

Background

When I was working through this example, I was a little exception happy. I was trying to learn how to make certain things happen with the moq. I’ve simply repeated my experiments here, when I use this later on, I’ll approach this problem a bit differently.

Example 1: Logging in successfully

00: [Test]
01: public void WhenUserFoundAndPasswordMatchesUserShouldBeLoggedIn()
02: {
03:   var userMock = new Mock<IUser>();
04:   userMock.Setup(user => user.PasswordMatches(It.IsAny<string>())).Returns(true);
05:
06:   var daoMock = new Mock<IUserDao>();
07:   daoMock.Setup(dao => dao.Get(It.IsAny<string>())).Returns(userMock.Object);
08:
09:   var service = new UserService(daoMock.Object);
10:   service.Login("Brett", "password");
11:
12:   userMock.VerifySet(user => user.LoggedIn = true);
13: }

Notice that rather than setting expectations, line 12 verifies only what I want to verify. It is possible to call Setup more times and then use “VerifyAll” to confirm all the Setup’s were actually used. However, that makes the test more coupled to the underlying implementation. All I care about is that as a result of this configuration of objects, a user was LoggedIn.

By default, mocks are not strict. That is, code may call any methods it wishes on a give mock object. So the default behavior is more like a stub than a mock object. If you call “VerifyAll” on the userMock or the daoMock, then all of the descriptions in the various Setup method invocations need to happen.

Example 2: User not found? Throw an exception

00: [Test]
01: [ExpectedException(typeof(UserDoesNotExistException))]
02: public void WhenUserNotFoundShouldThrowException()
03: {
04:   var daoMock = new Mock<IUserDao>();
05:   daoMock.Setup(dao => dao.Get(It.IsAny<string>())).Returns((IUser) null);
06:
07:   var service = new UserService(daoMock.Object);
08:   service.Login("Brett", "password");
09: }

Notice that this example includes another non-strict mock. You can call it as often as you wish and there’s no checking on the methods called. However, this stub should force the implementation of the Login method to throw a UserDoesNotExist exception.

Example 3: Throw an Exception if the password does not match

00: [Test]
01: [ExpectedException(typeof(PasswordDoesNotMatchException))]
02: public void WhenLoginAttemptDoesNotMatchPasswordThrowsException()
03: {
04:   var userMock = new Mock<IUser>();
05:   userMock.Setup(user => user.PasswordMatches(It.IsAny<string>())).Returns(false);
06:
07:   var daoMock = new Mock<IUserDao>();
08:   daoMock.Setup(dao => dao.Get(It.IsAny<string>())).Returns(userMock.Object);
09:
10:   var service = new UserService(daoMock.Object);
11:   service.Login("", "");
12: }

Eample 4: Three failed login attempts should cause user account to be revoked.

00: [Test]
01: public void WhenLoginAttemptThreeTimesInARowWithInvalidPasswordUserRevoked()
02: {
03:   var userMock = new Mock<IUser>();
04:   userMock.Setup(user => user.PasswordMatches(It.IsAny<string>())).Returns(false);
05: 
06:   var daoMock = new Mock<IUserDao>();
07:   daoMock.Setup(dao => dao.Get(It.IsAny<string>())).Returns(userMock.Object);
08:
09:   var service = new UserService(daoMock.Object);
10:   for (int i = 0; i < 3; ++i)
11:     AttempLoginIgnoringException(service, "");
12:   
13:   userMock.VerifySet(user => user.Revoked = true);
14: }

Example 5: Failing 2x with one account and then switching to another account and failing again does not revoke account.

00: [Test]
01: public void LoginWrongPassword2XandThenSwitchAccountsWithWrongPassowrdNothingRevoked()
02: {
03:   var userMock1 = new Mock<IUser>();
04:   userMock1.Setup(user => user.PasswordMatches(It.IsAny<string>())).Returns(false);
05:
06:   var userMock2 = new Mock<IUser>(MockBehavior.Strict);
07:   userMock2.Setup(user => user.PasswordMatches(It.IsAny<string>())).Returns(false);
08: 
09:   var daoMock = new Mock<IUserDao>();
10:   daoMock.Setup(dao => dao.Get("user1")).Returns(userMock1.Object);
11:   daoMock.Setup(dao => dao.Get("user2")).Returns(userMock2.Object);
12:
13:   var service = new UserService(daoMock.Object);
14:
15:   AttempLoginIgnoringException(service, "user1");
16:   AttempLoginIgnoringException(service, "user1");
17:   AttempLoginIgnoringException(service, "user2");
18:
19:   userMock2.VerifyAll();
20: }

Conclusion

• A book may have a receipt (this is a poor design, that’s part of the exercise).
• A receipt may have a book.

Both sides of the relationship are 0..1. After creating a receipt, I end up with a circular reference between Receipt and Book.

In the existing Java and C# implementations, there was no cleanup code in the test teardown to handle what happens when the receipt goes away. This was not a problem since C# and Java garbage collection algorithms easily handle this situation.

Shared pointers, however, do not handle this at all. They are good, sure, but not as good as a generation-scavenging garbage collector (or whatever algorithms are used these days – I know the JVM for 1.6 sometimes uses the stack for dynamic allocation based on JIT, so it’s much more sophisticated than a simple generation-scavenger, right?)

boost::shared_ptr<Receipt> r = ...;
CHECK(xxx, yyy);
r.setCopy(boost::shared_ptr<Book>());

(I did not use these types like this. When I use templates, especially those in a namespace, I use typedefs and I even, gasp, use Hungarian-esque notation.)

That would work, though it is ugly. Also, it is error prone and will either require violating DRY or making an automatic variable a field.

I could have removed the back reference from the Receipt to the book. That’s OK, but is a redesign of a system deliberately written with problems (part of the assignment).

Maybe I could explicitly “return” the book, which could remove the receipt and the back-reference. That would make the test teardown a bit more complex (and sort of upgrade the test from a unit test to something closer to an integration test), but it makes some sense. The test validate borrowing a book, so to clean up, return the book.

Instead of any of these options, I decided to use a boost::weak_ptr on the Receipt side. (This is the “technology to the rescue solution”, thus my question, is this an OK approach.)

I did this since the lifetime of a book object is much longer than its receipt (you return your library books, right?). Also, the Receipt only exists on a book. But the book could exist indefinitely without a Receipt.

This fixed the problem right away. I got a clean run using CppUTest. All tests passed and no memory leaks.

aReceipt->getBook()->getIsbn();

(Yes, violating Law of Demeter, get over it, the alternative would make a bloated API on the Book class.)

aReceipt->getBook().lock()->getIsbn();

The lock() method promotes a weak_ptr to a shared_ptr for the life of the expression. In this case, it’s a temporary in that line of code.

boost::shared_ptr<Book> getBook() {
    return book.lock();
}

On the one hand, anybody using the getBook() method is paying the price of the promotion. However, the weak_ptr doesn’t allow access to its payload without the promotion so it’s really required to be of any value. Or at least that’s my take on it.

Do you have different opinions?

Please keep in mind, this is example code we use in class to give students practice naming things like methods and variables and also practice cleaning up code by extracting methods and such.

Even so, what practice do you use, if any, when using shared_ptr? Do you use weak_ptr?

Thanks in advance for your comments. I’ll be reading and responding as they come up.

• It expresses the design intent
• It’s for performance
• The code as written cannot be safely overridden.

(Before I go any further, I am not arguing against the use of sealed or final. I will say that in my practice, it has to be justified to be used rather than the other way around.)

Back to the reasons. I generally don’t buy these reasons. Each may be perfectly valid but none of them stand on their own; each requires further discussion.

• It expresses design intent – you don’t know how to design
• It’s for performance – do you have any idea how a VM works?
• The code as written cannot be safely overridden – So the code is so badly written that nobody understands it?

Again, let me be clear, there are good reasons to seal things. My point is that while there are good reasons, most of the time I’ve seen the habit is a form of cargo culting; monkey see, monkey do.

This observation sat around for some time until I read some stuff in a book called CLR via C# (good reading if you want to better understand the CLR, and if you’re developing code for a virtual machine, I think it’s a good idea to understand the virtual machine at least a little).

People are sealing as much as they are because virtual table binding is done way to early.

The thing I read in the book I mentioned above relates to when virtual methods are bound. This applies to 2.0 version of the CLR, but I don’t think there’s any change for the latest version of the CLR and this seems like a pretty “hard” design decision. This book mentions that binding of virtual methods is performed at compile time.

I’m not making a distinction between compile time and link time for this discussion, but I’ll be a little more precise. The generated code stored in an assembly “knows” which virtual function it is invoking because in the generated byte code is an index into a virtual table.

How is this a problem?

I write a base class and ship it. You write a derived class (OK, maybe you use delegation, good for you, but the problem still remains). Your coupling to my class involves using a method with the virtual keyword (Java programmers, you don’t have to do this, regular methods [not static, not a constructor, ...] have the potential for dynamic dispatch unless you declare them final). At compile time, your object module (which eventually resides in an assembly that will get loaded and executed some time later) knows that it wants to call the 3rd virtual method in the v-table.

So far, so good.

Next, I add a new virtual method to my class and ship a new assembly.

Question. To use this new assembly, do you need to recompile?

Yes.

What?

Yes. What happens if my newly added method is the first virtual method? Then your compiled code that is happily living in a peaceful assembly somewhere has a stale reference. It thinks it wants to run the 3rd method in the virtual table but now it really should be using the 4th virtual method.

This is similar to a caching issue or premature optimization. The calling code has a compile-time generated index into the v-table. Why is this? Is it for performance? Not a good argument, this pre-calculation is irrelevant with modern JIT compiling techniques. I think the reason it is done this way is that C++ does things this way. C++ needs to because of its execution model (there’s not virtual machine between the code and its execution).

No big deal, you think, I’ll just recompile my code.

That will fix the problem. Or will it?

Say you’re stuck using an old version of some assembly that uses an old version of another assembly that you also use. Do you recompile both your code and that other assembly? You’ll have to but you might forget. Or you might not be able to.

.Net handles this by having an excellent dependency system to track which versions of which assemblies work with with other versions of assemblies. They need this to address problems introduced by the decision to perform such early binding.

Simply put, this is an unfortunate decision that I believe was more based on precedence or history than on sound design decisions.

This is not what happens in the Java Virtual Machine (JVM).

In the JVM, the binding is done later—much later. It is possible that my class got loaded and JIT’ed so that all of the methods that could be virtually invoked (not final) are in fact not virtually invoked. So far, no subclasses have been loaded, so there’s been no need. After my system has been running for a few weeks, a new class (a subclass) gets loaded and invalidates the JIT’d version of my class. No problem, the system will dump the old version and JIT it again, on the fly.

Even if there is a need for virtual dispatch, it might be that most of the time I use one version of method for a particularly heavily used subclass. The JVM might inline the common case and virtually dispatch the others.

Bottom line: what’s being done with JVM’s is indistinguishable from black magic for most of us.

I can use new JAR files and not have to worry about recompiling to remove stale v-table references. The method binding will just work. (This does not remove all problems with changing versions of JAR’s but it gets rid of one persnickety one.)

But it goes deeper. As I mentioned above, I’ve noticed several Microsoft shops that use a lot of sealed classes or avoid using virtual methods because “they perform poorly”.

OK, let’s talk about method invocation for just a second (or make that pico second). On my machine, I timed virtual method dispatch. I should check my numbers, but what I got was that virtual method dispatch was taking around 450 pico-seconds. So you can invoke roughly 2.2 Billion methods per second.

• Virtual method dispatch is not that expensive
• The JVM can address caching issues with code analysis and inlining
• You only pay the price if your design requires it
• The people writing the JVM are better and micro optimization than certainly me, and I’d say most people

• Testability
• Maintainability
• Flexibility
• Extensibility

Look up the percentage of time spent “building” a system versus “maintaining” a system. The percentage has been on the rise, and it’s 80 – 95% typically. 80% of the cost of development is spent maintaining code. (I believe a lot of this percentage has to do with the definition of “done” – but that’s a whole other discussion).

You might think that locking things down make them more stable and that stability leads to maintainability.

That’s not really the case.

Things change. Requirements change.

Locking down a class to keep it from breaking is like signing off on requirements to keep them from changing. Neither thing is really a good idea.

I CAN lock down my classes in a sense. I can use tests to describe my assumptions so the class is locked down so long as we all agree that we’ll keep the tests passing. That is a fine level of granularity.

But I digress.

Back to the whole sealed thing. I was wondering why so many places that use Microsoft stuff seal their classes and do not use virtual methods (in my experience, it’s about 50%). I then read that thing about the binding of virtual methods at compile time. And then about 2 weeks later it hit me.

If you use virtual methods and you have a messed up build system, then you’ll get strange behavior. Sealing things makes those surprising behaviors go away, so seal stuff.

It makes sense. This is something that C++ was notorious about. You hoped you’d get a segmentation violation but often the wrong method could get called and the system continued to run…

Unfortunately, the “solution” – sealing – is attacking a symptom, not the actual problem. This is where following the 5-why’s of Toyota would have been a good idea.

• Because they will be more stable. More safe/it expresses my design. Why?
• Because my design says that this should not change. Why?
• Because when things change, sometimes “bad things happen.” Why?
• I don’t know…

And right there is where you know the solution is a bad idea.

In reality I suspect the habit perpetuates because once something appears to work, it is cut and pasted ad nauseam.

Should you never seal things?

Never say never.

Or better yet, never say “never, never, never.”

• Never: Don’t do it because you may not know what you are doing. (Once you know the rules, you can do it.)
• Never, Never: You know what you’re doing most of the time, you are doing a bad thing but you really do know better.
• Never, Never, Never: DON’T DO IT!

There are few things that are Never, Never, Never. Dereferencing NULL comes to mind (but then maybe I waned a core dump if I get to this point in the code).

I’d place using sealed/final in the Never, Never camp if I’m using tests to describe the semantics of my classes. OR, if I’m writing concurrent code, using final on fields provides certain guarantees…

However, let’s say that I’ve been told not to use tests (I’ve seen it at more than one place). Then that “never, never” becomes a “sure do it to cover my ass”. Someone can’t subclass and mess things up. Someone can’t forget to compile their code when mine changes and cause their code to break…

Now is this a reason to avoid using Microsoft products? No. I really do like C# (for a statically-typed language that is). And I’m convinced that Java is a better language because of the work done by and in C#. And let’s face it, having more VM’s is a good thing. It improves competition. It raises the level at which we can expect to work.

But to quote Tim Booth/James in a rather trite way, “there’s a chain of consequence within, without.” That one thing, binding to the v-table so early, has caused a chain of events that leads to a platform that seems a little bit more sluggish to develop in to me. (I expect flames for this.)

I think this design decision is a reflection of a mind-set that permeates the development environment. For example, if I have a large solution with several projects, it sure seems that dependency checking in the build environment takes a long time. When I watch Developer Studio build things, it looks like make is running under the covers and performing a bunch of file checks to see what has changed. Where’s the incremental compilation?

I’ve seen very good work in both .Net and Java development efforts. It just seems that the frequency of unnecessary, basic kinds of environmental design debt occur more frequently in a .NET development effort than in similar Java development efforts.

Maybe it’s a sampling effect. To paraphrase Weinberg, “there’s always a sample, be aware of it, you can’t remove it.” I’m a consultant, I go places where they need consultants. I never, never go places that don’t need consultants, so my sample set is biased (I have one experience, maybe two, where I went and the people did not need consultants because they were doing fine – or maybe they didn’t need technically-oriented consultants).

• Is using sealed by default a good design choice?
• Is the binding of methods early a good thing?
  • Was the decision by design or because of history?
• Has/will the CLR change?

I did some experimenting over the last few days to integrate RSpec using SWIG (Simplified Wrapper and Interface Generator), a tool for bridging libraries written in C and C++ to other languages, like Ruby. The Ruby section of the SWIG manual was very helpful.

My Proof-of-Concept Code

Here is a zip file of my experiment: rspec_for_cpp.zip

This is far from a complete and working solution, but I think it shows promise. See the Current Limitations section below for details.

Unzip the file into a directory. I’ll assume you named it rspec_for_cpp. You need to have gmake, gcc, SWIG and Ruby installed, along with the RSpec “gem”. Right now, it only builds on OS X and Linux (at least the configurations on my machines running those OS’s – see the discussion below). To run the build, use the following commands:

    
        $ cd rspec_for_cpp/cpp
        $ make 
    

You should see it finish with the lines

    
        ( cd ../spec; spec *_spec.rb )
        .........

        Finished in 0.0***** seconds

        9 examples, 0 failures
    

Congratulations, you’ve just tested some C and C++ code with RSpec! (Or, if you didn’t succeed, see the notes in the Makefile and the discussion below.)

The Details

I’ll briefly walk you through the files in the zip and the key steps required to make it all work.

cexample.h

Here is a simple C header file.

    
        /* cexample.h */
        #ifndef CEXAMPLE_H
        #define CEXAMPLE_H
        #ifdef __cplusplus
         extern "C" {
        #endif
        char* returnString(char* input);
        double returnDouble(int input);
        void  doNothing();

        #ifdef __cplusplus
         }
        #endif
        #endif
    

Of course, in a pure C shop, you won’t need the #ifdef __cplusplus stuff. I found this was essential in my experiment when I mixed C and C++, as you might expect.

cpp/cexample.c

Here is the corresponding C source file.

    
        /* cexample.h */

        char* returnString(char* input) {
            return input;
        }

        double returnDouble(int input) {
            return (double) input;
        }

        void  doNothing() {}
    

cpp/CppExample.h

Here is a C++ header file.

    
        #ifndef CPPEXAMPLE_H
        #define CPPEXAMPLE_H

        #include <string>

        class CppExample 
        {
        public:
            CppExample();
            CppExample(const CppExample& foo);
            CppExample(const char* title, int flag);
            virtual ~CppExample();

            const char* title() const;
            void        title(const char* title);
            int         flag() const;
            void        flag(int value);

            static int countOfCppExamples();
        private:
            std::string _title;
            int         _flag;
        };

        #endif
    

cpp/CppExample.cpp

Here is the corresponding C++ source file.

    
        #include "CppExample.h" 

        CppExample::CppExample() : _title("") {}
        CppExample::CppExample(const CppExample& foo): _title(foo._title) {}
        CppExample::CppExample(const char* title, int flag) : _title(title), _flag(flag) {}
        CppExample::~CppExample() {}

        const char* CppExample::title() const { return _title.c_str(); }
        void        CppExample::title(const char* name) { _title = name; }

        int  CppExample::flag() const { return _flag; }
        void CppExample::flag(int value) { _flag = value; }

        int CppExample::countOfCppExamples() { return 1; }
    

cpp/example.i

Typically in SWIG, you specify a .i file to the swig command to define the module that wraps the classes and global functions, which classes and functions to expose to the target language (usually all in our case), and other assorted customization options, which are discussed in the SWIG manual. I’ll show the swig command in a minute. For now, note that I’m going to generate an example_wrap.cpp file that will function as the bridge between the languages.

Here’s my example.i, where I named the module example.

    
        %module example
        %{
            #include "cexample.h" 
            #include "CppExample.h"    
        %}
        %include "cexample.h" 
        %include "CppExample.h" 
    

It looks odd to have header files appear twice. The code inside the %{...%} (with a ’#’ before each include) are standard C and C++ statements, etc. that will be inserted verbatim into the generated “wrapper” file, example_wrap.cpp, so that file will compile when it references anything declared in the header files. The second case, with a ’%’ before the include statements¹, tells SWIG to make all the declarations in those header files available to the target language. (You can be more selective, if you prefer…)

Following Ruby conventions, the Ruby plugin for SWIG automatically names the module with an upper case first letter (Example), but you use require 'example' to include it, as we’ll see shortly.

Building

See the cpp/Makefile for the gory details. In a nutshell, you run the swig command like this.

    
        swig -c++ -ruby -Wall -o example_wrap.cpp example.i
    

Next, you create a dynamically-linked library, as appropriate for your platform, so the Ruby interpreter can load the module dynamically when required. The Makefile can do this for Linux and OS X platforms. See the Ruby section of the SWIG manual for Windows specifics.

If you test-drive your code, which tends to drive you towards minimally-coupled “modules”, then you can keep your libraries and build times small, which will make the build and test cycle very fast!

spec/cexample_spec.rb and spec/cppexample_spec.rb

Finally, here are the RSpec files that exercise the C and C++ code. (Disclaimer: these aren’t the best spec files I’ve ever written. For one thing, they don’t exercise all the CppExample methods! So sue me… :)

    
        require File.dirname(__FILE__) + '/spec_helper'
        require 'example'

        describe "Example (C functions)" do
          it "should be a constant on Module" do
            Module.constants.should include('Example')
          end
          it "should have the methods defined in the C header file" do
            Example.methods.should include('returnString')
            Example.methods.should include('returnDouble')
            Example.methods.should include('doNothing')
          end
        end

        describe Example, ".returnString" do
          it "should return the input char * string as a Ruby string unchanged" do
            Example.returnString("bar!").should == "bar!" 
          end  
        end

        describe Example, ".returnDouble" do
          it "should return the input integer as a double" do
            Example.returnDouble(10).should == 10.0
          end
        end

        describe Example, ".doNothing" do
          it "should exist, but do nothing" do
            lambda { Example.doNothing }.should_not raise_error
          end
        end
    

and

    
    require File.dirname(__FILE__) + '/spec_helper'
    require 'example'

    describe Example::CppExample do
      it "should be a constant on module Example" do
        Example.constants.should include('CppExample')
      end
    end

    describe Example::CppExample, ".new" do
      it "should create a new object of type CppExample" do
        example = Example::CppExample.new("example1", 1)
        example.title.should == "example1" 
        example.flag.should  == 1
      end
    end

    describe Example::CppExample, "#title(value)" do
      it "should set the title" do
        example = Example::CppExample.new("example1", 1)
        example.title("title2")
        example.title.should == "title2" 
      end
    end

    describe Example::CppExample, "#flag(value)" do
      it "should set the flag" do
        example = Example::CppExample.new("example1", 1)
        example.flag(2)
        example.flag.should == 2
      end
    end
    

If you love RSpec like I do, this is a very compelling thing to see!

And now for the small print:

Current Limitations

As I said, this is just an experiment at this point. Volunteers to make this battle-ready would be most welcome!

General

The Example Makefile File

It Must Be Hand Edited for Each New or Renamed Source File

You’ve probably already solved this problem for your own make files. Just merge in the example Makefile to pick up the SWIG- and RSpec-related targets and rules.

It Only Knows How to Build Shared Libraries for Mac OS X and Linux (and Not Very Well)

Some definitions are probably unique to my OS X and Linux machines. Windows is not supported at all. However, this is also easy rectify. Start with the notes in the Makefile itself.

The module.i File Must Be Hand Edited for Each File Change

Since the format is simple, a make task could fill a template file with the changed list of source files during the build.

Better Automation

It should be straightforward to provide scripts, IDE/Editor shortcuts, etc. that automate some of the tasks of adding new methods and classes to your C and C++ code when you introduce them first in your “spec” files. (The true TDD way, of course.)

Specific Issues for C Code Testing

I don’t know of any other C-specific issues, so unit testing with Ruby is most viable today for C code. Although I haven’t experimented extensively, C functions and variables are easily mapped by SWIG to a Ruby module. The Ruby section of the SWIG manual discusses this mapping in some detail.

Specific Issues for C++ Code Testing

More work will be required to make this viable. It’s important to note that SWIG cannot handle all C++ constructs (although there are workarounds for most issues, if you’re committed to this approach…). For example, namespaces, nested classes, some template and some method overloading scenarios are not supported. The SWIG manual has details.

Also, during my experiment, SWIG didn’t seem to map const std::string& objects in C++ method signatures to Ruby strings, as I would have expected (char* worked fine).

Is It a Viable Approach?

Once the General issues listed above are handled, I think this approach would work very well for C code. For C++ code, there are more issues that need to be addressed, and programmers who are committed to this strategy will need to tolerate some issues (or just use C++-language tools for some scenarios).

Conclusions: Making It Development-Team Ready

I’d like to see this approach pushed to its logical limit. I think it has the potential to really improve the productivity of C and C++ developers and the quality of their test coverage, by leveraging the productivity and power of dynamically-typed languages like Ruby. If you prefer, you could use Tcl, Python, even Java instead.

License

This code is complete open and free to use. Of course, use it at your own risk; I offer it without warranty, etc., etc. When I polish it to the point of making it an “official” project, I will probably release under the Apache license.

¹ I spent a lot of time debugging problems because I had a ’#’ where I should have had a ’%’! Caveat emptor!

If you are developing web applications, you are already doing this, of course. Your web tier probably uses several “languages”, e.g., HTML, JavaScript, JSP/ASP, CSS, Java, etc.

However, most people use only one language for the business/mid tier. I think you should consider using several; a high-productivity language environment for most of your work, with the occasional critical functionality implemented in C or C++ to optimize performance, but only after actually measuring where the bottlenecks are located.

This approach is much too rare, but it has historical precedents. One of the most successful and long-lived software projects of all time is Emacs. It consists of a core C-based runtime with most of the functionality implemented in Emacs lisp “components”. The relative ease of extending Emacs using lisp has resulted in a rich assortment of support tools for various operating systems, languages, build tools, etc. Even modern IDEs and and other graphical editors have not completely displaced Emacs.

Java has embraced the mixed language philosophy somewhat reluctantly. JNI is the official and most commonly-used API for invoking “native” code, but it is somewhat hard to use and few people actually use it. In contrast, for example, the Ruby world has always embraced this approach. Ruby has an easy to use API for invoking native C code and good alternatives exist for invoking code in other languages. As a result, many of the 3rd-party Ruby libraries (or gems) contain both Ruby and native C code. The latter is built on the fly when you install the gem. Hence, there are many high-performance Ruby applications. This is not a contradiction in terms, because the performance-critical sections run natively, even though interpreted Ruby is relatively slow.

Of course, you have to be judicious in how you use mixed-language programming. Crossing the language boundary is often somewhat heavyweight, so you should avoid doing such invocations inside tight loops, for example.

So, I think the solution to the dilemma of needing high performance sometimes and high productivity the rest of the time is to pick the right tools for each circumstance and make them interoperate. Even constrained embedded devices like cell phones would be easier to implement if most of the code were written in a language like Ruby, Python, Smalltalk, or Java and performance-critical components were written in C or C++.

If I were starting such a greenfield project, I would assume that time-to-money is the top priority and write most of my code in Ruby (my personal current favorite), using TDD of course. I would profile it constantly, as part of the nightly or continuous-integration build. When bottlenecks emerge, I would first determine if a refactoring is sufficient to fix them and if not, I would rewrite the critical sections in C. If the project were for an embedded device, I would also watch the resource usage carefully.

For my embedded device, I would test from the beginning whether or not the overhead of the interpreter/VM and the overall performance are acceptable. I would also be sure that I have adequate tool support for the inevitable remote debugging and diagnostics I’ll have to do. If I made the wrong tool choices after all, I would know early on, when it’s still relatively painless to retool.

If you’re an IT or web-site developer, you have fewer performance limitations and more options. You might decide to make the cross-language boundary a cross-process boundary, e.g., by communicating through some sort of lightweight web services. This is one way to leverage legacy C/C++ code while developing new functionality in a more productive language.

Based on my observations last month, as well as previously experience, I’ve come to the conclusion that TDD in C++ is about an order of magnitude slower than TDD in Java. Mostly, this is due to poor or non-existent tool support for automated refactorings, no error detection as you type, and the requirement to compile and link an executable test.

So, here is my list of impediments that I encountered last month. I’ll mostly use Java as the comparison language, but the arguments are more or less the same for C# and the popular dynamic languages, like Ruby, Python, and Smalltalk. Note that the dynamic languages tend to have less complete tool support, but they make up for it in other ways (off-topic for this blog).

Getting Started

There is more setup effort involved in configuring your build environment to use your chosen unit testing framework (e.g., CppUnit) and to create small executables, one each for a single or a few tests. Creating many small tests, rather than one big test (e.g., a variant of the actual application). This is important to minimize the TDD cycle.

Fortunately, this setup is a one-time “charge”. The harder part, if you have legacy code, is refactoring it to break hard dependencies so you can write unit tests. This is true for legacy code in any language, of course.

Complex Syntax

C++ has a very complex syntax. This makes it hard to parse, limiting the capabilities of automated tools and slowing build times (more below).

The syntax also makes it harder to program in the language and not just for novices. Even for experts, the visual noise of pointer and reference syntax obscures the story the code is trying to tell. That is, C++ code is inherently less clean than code in most other languages in widespread use.

Also, the need for the developer to remember whether each variable is a pointer, a reference, or a “value”, and how to manage its life-cycle, requires mental effort that could be applied to the logic of the code instead.

Obsolete Tool Support

No editor or IDE supports non-trivial, automated refactorings. (Some do simple refactorings like “rename”.) This means you have to resort to tedious, slow, and error-prone manual refactorings. Extract Method is made worse by the fact that you usually have to edit two files, an implementation and a header file.

There are no widely-used tools that provide on-the-fly parsing and error indications. This alone increases the time between typing an error and learning about it by an order of magnitude. Since a build is usually required, you tend to type a lot between builds, thereby learning about many errors at once. Working through them takes time. (There may be some commercial tools with limited support for on-the-fly parsing, but they are not widely used.)

Similarly, none of the common development tools support incremental loading of object code that could be used for faster unit testing and hence a faster TDD cycle. Most teams just build executables. Even when they structure the build process to generate small, focused executables for unit tests, the TDD cycle times remain much longer than for Java.

Finally, while there is at least one mocking framework available for C++, it is much harder to use than comparable frameworks in newer languages.

Manual Memory Management

We all know that manual memory management leads to time spent finding and fixing memory errors and leaks. Avoiding these problems in the first place also consumes a lot of thought and design effort. In Java, you just spend far less time thinking about “who owns this object and is therefore responsible for managing its life-cycle”.

Dependency Management

Intelligent handling of include directives is entirely up to the developer. We have all used the following “guard” idiom:

    #ifndef MY_CLASS_H
    #define MY_CLASS_H
    ...
    #endif

Unfortunately, this isn’t good enough. The file will still get opened and read in its entirety every time it is included. You could also put the guard directives around the include statement:

    #ifndef MY_CLASS_H
    #include "myclass.h"
    #endif

This is tedious and few people do it, but it does avoid the wasted file I/O.

Finally, too few people simply declare a required class with no body:

    class MyClass;

This is sufficient when one header references another class as a pointer or reference. In our experience with clients, we have often seen build times improve significantly when teams cleaned up their header file usage and dependencies, in general. Still, why is all this necessary in the 21st century?

This problem is made worse by the unfortunate inclusion of private and protected declarations in the same header file included by clients of the class. This creates phantom dependencies from the clients to class details that they can’t access directly.

Other Debugging Issues

Limited or non-existent context information when an exception is thrown makes the origin of the exception harder to find. To fill the gap, you tend to spend more time adding this information manually through logging statements in catch blocks, etc.

The std::exception class doesn’t appear to have a std::string or const char* argument in a constructor for a message. You could just throw a string, but that precludes using an exception class with a meaningful name.

Compiler error messages are hard to read and often misleading. In part this is due to the complexity of the syntax and the parsing problem mentioned previously. Errors involving template usage are particular hard to debug.

Reflection and Metaprogramming

Many of the productivity gains from using dynamic languages and (to a lesser extent) Java and C# are due to their reflection and metaprogramming facilities. C++ relies more on template metaprogramming, rather than APIs or other built-in language features that are easier to use and more full-featured. Preprocessor hacks are also used frequently. Better reflection and metaprogramming support would permit more robust proxy or aspect solutions to be used. (However, to be fair, sometimes a preprocessor hack has the virtue of being “the simplest thing that could possibly work.”)

Library Issues

Speaking of std::string and char*, it is hard to avoid writing two versions of methods, one which takes const std::string& arguments and one which takes const char* arguments. It doesn’t matter that one method can usually delegate to the other one; this is wasted effort.

Discussion

So, C++ makes it hard for me to work the way that I want to work today, which is test-driven, creating clean code that works. That’s why I rarely choose it for a project.

However, to be fair, there are legitimate reasons for almost all of the perceived “deficiencies” listed above. C++ emphasizes performance and backwards-compatibility with C over all other considerations. However, they come at the expense of other interests, like effective TDD.

It is a good thing that we have languages that were designed with performance as the top design goal, because there are circumstances where performance is the number one requirement. However, most teams that use C++ as their primary language are making an optimal choice for, say, 10% of their code, but which is suboptimal the other 90%. Your numbers will vary; I picked 10% vs. 90% based on the fact that performance bottlenecks are usually localized and they should be found by actual measurements, not guesses!

Workarounds

If it’s true that TDD is an order of magnitude slower for C++ then what do we do? No doubt really good C++ developers have optimized their processes as best as they can, but in the end, you will just have to live with longer TDD cycles. Instead of write just enough test to fail, make it pass, refactor, it will be more like write a complete test, write the implementation, build it, fix the compilation errors, run it, fix the logic errors to make the test pass, and then refactor.

A Real Resolution?

You could consider switching to the D language, which is link compatible with C and appears to avoid many of the problems described above.

There is another way out of the dilemma of needing optimal performance some of the time and optimal productivity the rest of the time; use more than one language. I’ll discuss this idea in my next blog.