Object Mentor Blog: Unit Testing C and C++ ... with Ruby and RSpec!

Unit Testing C and C++ ... with Ruby and RSpec!

Dean Wampler — Mon, 04 Feb 2008 22:08:00 +0000

If you’re writing C/C++ code, it’s natural to write your unit tests in the same language (or use C++ for your C test code). All the well-known unit testing tools take this approach.

I think we can agree that neither language offers the best developer productivity among all the language choices out there. Most of us use either language because of perceived performance requirements, institutional and industry tradition, etc.

There’s growing interest, however, in mixing languages, tools, and paradigms to get the best tool for a particular job. <shameless-plug>I’m giving a talk March 7^th at SD West on this very topic, called Polyglot and Poly-Paradigm Programming </shameless-plug>.

So, why not use a more productive language for your C or C++ unit tests? You have more freedom in your development chores than what’s required for production. Why not use Ruby’s RSpec, a Behavior-Driven Development tool for acceptance and unit testing? Or, you could use Ruby’s version of JUnit, called Test::Unit. The hard part is integrating Ruby and C/C++. If you’ve been looking for an excuse to bring Ruby (or Tcl or Python or Java or…) into your C/C++ environment, starting with development tasks is usually the path of least resistance.

I did some experimenting over the last few days to integrate RSpec using SWIG (Simplified Wrapper and Interface Generator), a tool for bridging libraries written in C and C++ to other languages, like Ruby. The Ruby section of the SWIG manual was very helpful.

My Proof-of-Concept Code

Here is a zip file of my experiment: rspec_for_cpp.zip

This is far from a complete and working solution, but I think it shows promise. See the Current Limitations section below for details.

Unzip the file into a directory. I’ll assume you named it rspec_for_cpp. You need to have gmake, gcc, SWIG and Ruby installed, along with the RSpec “gem”. Right now, it only builds on OS X and Linux (at least the configurations on my machines running those OS’s – see the discussion below). To run the build, use the following commands:

    
        $ cd rspec_for_cpp/cpp
        $ make

You should see it finish with the lines

    
        ( cd ../spec; spec *_spec.rb )
        .........

        Finished in 0.0***** seconds

        9 examples, 0 failures

Congratulations, you’ve just tested some C and C++ code with RSpec! (Or, if you didn’t succeed, see the notes in the Makefile and the discussion below.)

The Details

I’ll briefly walk you through the files in the zip and the key steps required to make it all work.

`cexample.h`

Here is a simple C header file.

    
        /* cexample.h */
        #ifndef CEXAMPLE_H
        #define CEXAMPLE_H
        #ifdef __cplusplus
         extern "C" {
        #endif
        char* returnString(char* input);
        double returnDouble(int input);
        void  doNothing();

        #ifdef __cplusplus
         }
        #endif
        #endif

Of course, in a pure C shop, you won’t need the #ifdef __cplusplus stuff. I found this was essential in my experiment when I mixed C and C++, as you might expect.

`cpp/cexample.c`

Here is the corresponding C source file.

    
        /* cexample.h */

        char* returnString(char* input) {
            return input;
        }

        double returnDouble(int input) {
            return (double) input;
        }

        void  doNothing() {}

`cpp/CppExample.h`

Here is a C++ header file.

    
        #ifndef CPPEXAMPLE_H
        #define CPPEXAMPLE_H

        #include <string>

        class CppExample 
        {
        public:
            CppExample();
            CppExample(const CppExample& foo);
            CppExample(const char* title, int flag);
            virtual ~CppExample();

            const char* title() const;
            void        title(const char* title);
            int         flag() const;
            void        flag(int value);

            static int countOfCppExamples();
        private:
            std::string _title;
            int         _flag;
        };

        #endif

`cpp/CppExample.cpp`

Here is the corresponding C++ source file.

    
        #include "CppExample.h" 

        CppExample::CppExample() : _title("") {}
        CppExample::CppExample(const CppExample& foo): _title(foo._title) {}
        CppExample::CppExample(const char* title, int flag) : _title(title), _flag(flag) {}
        CppExample::~CppExample() {}

        const char* CppExample::title() const { return _title.c_str(); }
        void        CppExample::title(const char* name) { _title = name; }

        int  CppExample::flag() const { return _flag; }
        void CppExample::flag(int value) { _flag = value; }

        int CppExample::countOfCppExamples() { return 1; }

`cpp/example.i`

Typically in SWIG, you specify a .i file to the swig command to define the module that wraps the classes and global functions, which classes and functions to expose to the target language (usually all in our case), and other assorted customization options, which are discussed in the SWIG manual. I’ll show the swig command in a minute. For now, note that I’m going to generate an example_wrap.cpp file that will function as the bridge between the languages.

Here’s my example.i, where I named the module example.

    
        %module example
        %{
            #include "cexample.h" 
            #include "CppExample.h"    
        %}
        %include "cexample.h" 
        %include "CppExample.h"

It looks odd to have header files appear twice. The code inside the %{...%} (with a ’#’ before each include) are standard C and C++ statements, etc. that will be inserted verbatim into the generated “wrapper” file, example_wrap.cpp, so that file will compile when it references anything declared in the header files. The second case, with a ’%’ before the include statements¹, tells SWIG to make all the declarations in those header files available to the target language. (You can be more selective, if you prefer…)

Following Ruby conventions, the Ruby plugin for SWIG automatically names the module with an upper case first letter (Example), but you use require 'example' to include it, as we’ll see shortly.

Building

See the cpp/Makefile for the gory details. In a nutshell, you run the swig command like this.

    
        swig -c++ -ruby -Wall -o example_wrap.cpp example.i

Next, you create a dynamically-linked library, as appropriate for your platform, so the Ruby interpreter can load the module dynamically when required. The Makefile can do this for Linux and OS X platforms. See the Ruby section of the SWIG manual for Windows specifics.

If you test-drive your code, which tends to drive you towards minimally-coupled “modules”, then you can keep your libraries and build times small, which will make the build and test cycle very fast!

`spec/cexample_spec.rb` and `spec/cppexample_spec.rb`

Finally, here are the RSpec files that exercise the C and C++ code. (Disclaimer: these aren’t the best spec files I’ve ever written. For one thing, they don’t exercise all the CppExample methods! So sue me… :)

    
        require File.dirname(__FILE__) + '/spec_helper'
        require 'example'

        describe "Example (C functions)" do
          it "should be a constant on Module" do
            Module.constants.should include('Example')
          end
          it "should have the methods defined in the C header file" do
            Example.methods.should include('returnString')
            Example.methods.should include('returnDouble')
            Example.methods.should include('doNothing')
          end
        end

        describe Example, ".returnString" do
          it "should return the input char * string as a Ruby string unchanged" do
            Example.returnString("bar!").should == "bar!" 
          end  
        end

        describe Example, ".returnDouble" do
          it "should return the input integer as a double" do
            Example.returnDouble(10).should == 10.0
          end
        end

        describe Example, ".doNothing" do
          it "should exist, but do nothing" do
            lambda { Example.doNothing }.should_not raise_error
          end
        end

and

    
    require File.dirname(__FILE__) + '/spec_helper'
    require 'example'

    describe Example::CppExample do
      it "should be a constant on module Example" do
        Example.constants.should include('CppExample')
      end
    end

    describe Example::CppExample, ".new" do
      it "should create a new object of type CppExample" do
        example = Example::CppExample.new("example1", 1)
        example.title.should == "example1" 
        example.flag.should  == 1
      end
    end

    describe Example::CppExample, "#title(value)" do
      it "should set the title" do
        example = Example::CppExample.new("example1", 1)
        example.title("title2")
        example.title.should == "title2" 
      end
    end

    describe Example::CppExample, "#flag(value)" do
      it "should set the flag" do
        example = Example::CppExample.new("example1", 1)
        example.flag(2)
        example.flag.should == 2
      end
    end

If you love RSpec like I do, this is a very compelling thing to see!

And now for the small print:

Current Limitations

As I said, this is just an experiment at this point. Volunteers to make this battle-ready would be most welcome!

General

The Example `Makefile` File

It Must Be Hand Edited for Each New or Renamed Source File

You’ve probably already solved this problem for your own make files. Just merge in the example Makefile to pick up the SWIG- and RSpec-related targets and rules.

It Only Knows How to Build Shared Libraries for Mac OS X and Linux (and Not Very Well)

Some definitions are probably unique to my OS X and Linux machines. Windows is not supported at all. However, this is also easy rectify. Start with the notes in the Makefile itself.

The `module.i` File Must Be Hand Edited for Each File Change

Since the format is simple, a make task could fill a template file with the changed list of source files during the build.

Better Automation

It should be straightforward to provide scripts, IDE/Editor shortcuts, etc. that automate some of the tasks of adding new methods and classes to your C and C++ code when you introduce them first in your “spec” files. (The true TDD way, of course.)

Specific Issues for C Code Testing

I don’t know of any other C-specific issues, so unit testing with Ruby is most viable today for C code. Although I haven’t experimented extensively, C functions and variables are easily mapped by SWIG to a Ruby module. The Ruby section of the SWIG manual discusses this mapping in some detail.

Specific Issues for C++ Code Testing

More work will be required to make this viable. It’s important to note that SWIG cannot handle all C++ constructs (although there are workarounds for most issues, if you’re committed to this approach…). For example, namespaces, nested classes, some template and some method overloading scenarios are not supported. The SWIG manual has details.

Also, during my experiment, SWIG didn’t seem to map const std::string& objects in C++ method signatures to Ruby strings, as I would have expected (char* worked fine).

Is It a Viable Approach?

Once the General issues listed above are handled, I think this approach would work very well for C code. For C++ code, there are more issues that need to be addressed, and programmers who are committed to this strategy will need to tolerate some issues (or just use C++-language tools for some scenarios).

Conclusions: Making It Development-Team Ready

I’d like to see this approach pushed to its logical limit. I think it has the potential to really improve the productivity of C and C++ developers and the quality of their test coverage, by leveraging the productivity and power of dynamically-typed languages like Ruby. If you prefer, you could use Tcl, Python, even Java instead.

License

This code is complete open and free to use. Of course, use it at your own risk; I offer it without warranty, etc., etc. When I polish it to the point of making it an “official” project, I will probably release under the Apache license.

¹ I spent a lot of time debugging problems because I had a ’#’ where I should have had a ’%’! Caveat emptor!

"Unit Testing C and C++ ... with Ruby and RSpec!" by cyguy

Sun, 15 Jun 2008 07:54:33 +0000

Maybe also have a look at http://metasm.cr0.org or http://cil.sourceforge.net !

"Unit Testing C and C++ ... with Ruby and RSpec!" by Nolan Eakins

Sun, 27 Apr 2008 00:42:31 +0000

I’m just about ready to try doing this same thing. The only problem I see is that SWIG may introduce bugs. Not sure if that should be a big worry. I am finding Rake to be helpful to build things.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Ben Mabey

Fri, 08 Feb 2008 04:21:52 +0000

Dean, Nice write up. I made a similar post to my blog back in September: http://www.benmabey.com/2007/09/09/bdd_your_c/

One difference in what I did was that I used mkmf to generate my Makefile. I was only playing around with it a little but with what I tried it seemed to work well. So by using that you can avoid having to edit the Makefile by hand.

-Ben

"Unit Testing C and C++ ... with Ruby and RSpec!" by Dean Wampler

Wed, 06 Feb 2008 18:03:10 +0000

@Bheeshmar, thanks for the useful information about SWIG and C++ strings. Your experience rewriting your AT interpreter is a great example of why I think combining languages and tools is worth the trouble; a 100 line program in any language will be far easier to maintain and evolve over time than a 5000 line program!

@Eric, it will be interesting to watch the Ruby CLR integration projects, like IronRuby. Microsoft is certainly working hard these days on integrating different languages.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Eric

Wed, 06 Feb 2008 10:08:56 +0000

Nice article Dean.

My first use of Ruby on a real project was using it to automate testing of a DCOM object. Didn’t require the amount of work shown here because of the Win32OLE libraries.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Carlo

Wed, 06 Feb 2008 02:13:52 +0000

Oh! What a good experiment. Itâ€™s seems interesting and not complicated to use, Iâ€™ll try one day since itâ€™s free. Of course, Iâ€™ll take my own risk.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Bheeshmar

Tue, 05 Feb 2008 17:38:41 +0000

I actually first started using Ruby after reading this article by Phil Tomson on using SWIG and Ruby together: http://www.ddj.com/web-development/184401747

It reduced the acceptance test interpreter I had written in C++ for my QA team from 5000 lines of C++ to about 100 lines of Ruby in about 3 days.

BTW, to get std::string et al in SWIG, you have to include in the .i file ‘%include std_string.i’ inside the wrapped module.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Allen Rohner

Tue, 05 Feb 2008 09:56:36 +0000

@yachris2112: Maybe I’m missing something, but the nice thing about SWIG is that it automatically generates the ruby interface for you. Using the ruby native interface would require something to parse your C/C++ code and generate the ruby interfaces.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Dean Wampler

Tue, 05 Feb 2008 08:53:44 +0000

@Ola, I really like what you did with JtestR (and your role in JRuby, for that matter…). Too bad C and C++ aren’t as accomodating.

@yachris2112, I will look at the native interface. I grabbed SWIG for the experiment because I’ve used it before and I wasn’t sure if the Ruby native interface handles C++ very well.

"Unit Testing C and C++ ... with Ruby and RSpec!" by yachris2112

Tue, 05 Feb 2008 07:34:25 +0000

Cool - nice to see multi-language work - use the best tool, and all that.

You might want to look at the “native” Ruby C interface, rather than using SWIG. I was doing some neural net stuff a while back, and using the FANN (Fast Artificial Neural Network) library, and it was nearly trivial to get it working with Ruby. I’ve done TCL and Perl bindings back and forth with C, and Ruby’s was much cleaner and simpler.

However, it’s been long enough ago that I don’t remember it well enough to swear that it’s simpler than SWIG.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Tim

Tue, 05 Feb 2008 07:23:56 +0000

There are lots of ways to improve on coding. I guess with the enumerations above you’ve taken coding a step further. I should experiment with C more becuase I’ve only gotten used to C++.

"Unit Testing C and C++ ... with Ruby and RSpec!" by Ola Bini

Tue, 05 Feb 2008 05:44:56 +0000

If you want to use the same approach for testing Java code, you can use JtestR, that includes both RSpec, dust and Test::Unit.

Object Mentor Blog: Unit Testing C and C++ ... with Ruby and RSpec!

Unit Testing C and C++ ... with Ruby and RSpec!

My Proof-of-Concept Code

The Details

cexample.h

cpp/cexample.c

cpp/CppExample.h

cpp/CppExample.cpp

cpp/example.i

Building

spec/cexample_spec.rb and spec/cppexample_spec.rb

Current Limitations

General

The Example Makefile File

The module.i File Must Be Hand Edited for Each File Change

Better Automation

Specific Issues for C Code Testing

Specific Issues for C++ Code Testing

Is It a Viable Approach?

Conclusions: Making It Development-Team Ready

License

"Unit Testing C and C++ ... with Ruby and RSpec!" by cyguy

"Unit Testing C and C++ ... with Ruby and RSpec!" by Nolan Eakins

"Unit Testing C and C++ ... with Ruby and RSpec!" by Ben Mabey

"Unit Testing C and C++ ... with Ruby and RSpec!" by Dean Wampler

"Unit Testing C and C++ ... with Ruby and RSpec!" by Eric

"Unit Testing C and C++ ... with Ruby and RSpec!" by Carlo

"Unit Testing C and C++ ... with Ruby and RSpec!" by Bheeshmar

"Unit Testing C and C++ ... with Ruby and RSpec!" by Allen Rohner

"Unit Testing C and C++ ... with Ruby and RSpec!" by Dean Wampler

"Unit Testing C and C++ ... with Ruby and RSpec!" by yachris2112

"Unit Testing C and C++ ... with Ruby and RSpec!" by Tim

"Unit Testing C and C++ ... with Ruby and RSpec!" by Ola Bini

`cexample.h`

`cpp/cexample.c`

`cpp/CppExample.h`

`cpp/CppExample.cpp`

`cpp/example.i`

`spec/cexample_spec.rb` and `spec/cppexample_spec.rb`

The Example `Makefile` File

The `module.i` File Must Be Hand Edited for Each File Change