Observations on TDD in C++ (long) 56
I spent all of June mentoring teams on TDD in C++ with some Java. While C++ was my language of choice through most of the 90’s, I think far too many teams are using it today when there are better options for their particular needs.
During the month, I took notes on all the ways that C++ development is less productive than development in languages like Java, particular if you try to practice TDD. I’m not trying to start a language flame war. There are times when C++ is the appropriate tool, as we’ll see.
Most of the points below have been discussed before, but it is useful to list them in one place and to highlight a few particular observations.
Based on my observations last month, as well as previously experience, I’ve come to the conclusion that TDD in C++ is about an order of magnitude slower than TDD in Java. Mostly, this is due to poor or non-existent tool support for automated refactorings, no error detection as you type, and the requirement to compile and link an executable test.
So, here is my list of impediments that I encountered last month. I’ll mostly use Java as the comparison language, but the arguments are more or less the same for C# and the popular dynamic languages, like Ruby, Python, and Smalltalk. Note that the dynamic languages tend to have less complete tool support, but they make up for it in other ways (off-topic for this blog).
Getting Started
There is more setup effort involved in configuring your build environment to use your chosen unit testing framework (e.g., CppUnit) and to create small executables, one each for a single or a few tests. Creating many small tests, rather than one big test (e.g., a variant of the actual application). This is important to minimize the TDD cycle.
Fortunately, this setup is a one-time “charge”. The harder part, if you have legacy code, is refactoring it to break hard dependencies so you can write unit tests. This is true for legacy code in any language, of course.
Complex Syntax
C++ has a very complex syntax. This makes it hard to parse, limiting the capabilities of automated tools and slowing build times (more below).
The syntax also makes it harder to program in the language and not just for novices. Even for experts, the visual noise of pointer and reference syntax obscures the story the code is trying to tell. That is, C++ code is inherently less clean than code in most other languages in widespread use.
Also, the need for the developer to remember whether each variable is a pointer, a reference, or a “value”, and how to manage its life-cycle, requires mental effort that could be applied to the logic of the code instead.
Obsolete Tool Support
No editor or IDE supports non-trivial, automated refactorings. (Some do simple refactorings like “rename”.) This means you have to resort to tedious, slow, and error-prone manual refactorings. Extract Method is made worse by the fact that you usually have to edit two files, an implementation and a header file.
There are no widely-used tools that provide on-the-fly parsing and error indications. This alone increases the time between typing an error and learning about it by an order of magnitude. Since a build is usually required, you tend to type a lot between builds, thereby learning about many errors at once. Working through them takes time. (There may be some commercial tools with limited support for on-the-fly parsing, but they are not widely used.)
Similarly, none of the common development tools support incremental loading of object code that could be used for faster unit testing and hence a faster TDD cycle. Most teams just build executables. Even when they structure the build process to generate small, focused executables for unit tests, the TDD cycle times remain much longer than for Java.
Finally, while there is at least one mocking framework available for C++, it is much harder to use than comparable frameworks in newer languages.
Manual Memory Management
We all know that manual memory management leads to time spent finding and fixing memory errors and leaks. Avoiding these problems in the first place also consumes a lot of thought and design effort. In Java, you just spend far less time thinking about “who owns this object and is therefore responsible for managing its life-cycle”.
Dependency Management
Intelligent handling of include directives is entirely up to the developer. We have all used the following “guard” idiom:
#ifndef MY_CLASS_H
#define MY_CLASS_H
...
#endifUnfortunately, this isn’t good enough. The file will still get opened and read in its entirety every time it is included. You could also put the guard directives around the include statement:
#ifndef MY_CLASS_H
#include "myclass.h"
#endifThis is tedious and few people do it, but it does avoid the wasted file I/O.
Finally, too few people simply declare a required class with no body:
class MyClass;This is sufficient when one header references another class as a pointer or reference. In our experience with clients, we have often seen build times improve significantly when teams cleaned up their header file usage and dependencies, in general. Still, why is all this necessary in the 21st century?
This problem is made worse by the unfortunate inclusion of private and protected declarations in the same header file included by clients of the class. This creates phantom dependencies from the clients to class details that they can’t access directly.
Other Debugging Issues
Limited or non-existent context information when an exception is thrown makes the origin of the exception harder to find. To fill the gap, you tend to spend more time adding this information manually through logging statements in catch blocks, etc.
The std::exception class doesn’t appear to have a std::string or const char* argument in a constructor for a message. You could just throw a string, but that precludes using an exception class with a meaningful name.
Compiler error messages are hard to read and often misleading. In part this is due to the complexity of the syntax and the parsing problem mentioned previously. Errors involving template usage are particular hard to debug.
Reflection and Metaprogramming
Many of the productivity gains from using dynamic languages and (to a lesser extent) Java and C# are due to their reflection and metaprogramming facilities. C++ relies more on template metaprogramming, rather than APIs or other built-in language features that are easier to use and more full-featured. Preprocessor hacks are also used frequently. Better reflection and metaprogramming support would permit more robust proxy or aspect solutions to be used. (However, to be fair, sometimes a preprocessor hack has the virtue of being “the simplest thing that could possibly work.”)
Library Issues
Speaking of std::string and char*, it is hard to avoid writing two versions of methods, one which takes const std::string& arguments and one which takes const char* arguments. It doesn’t matter that one method can usually delegate to the other one; this is wasted effort.
Discussion
So, C++ makes it hard for me to work the way that I want to work today, which is test-driven, creating clean code that works. That’s why I rarely choose it for a project.
However, to be fair, there are legitimate reasons for almost all of the perceived “deficiencies” listed above. C++ emphasizes performance and backwards-compatibility with C over all other considerations. However, they come at the expense of other interests, like effective TDD.
It is a good thing that we have languages that were designed with performance as the top design goal, because there are circumstances where performance is the number one requirement. However, most teams that use C++ as their primary language are making an optimal choice for, say, 10% of their code, but which is suboptimal the other 90%. Your numbers will vary; I picked 10% vs. 90% based on the fact that performance bottlenecks are usually localized and they should be found by actual measurements, not guesses!
Workarounds
If it’s true that TDD is an order of magnitude slower for C++ then what do we do? No doubt really good C++ developers have optimized their processes as best as they can, but in the end, you will just have to live with longer TDD cycles. Instead of write just enough test to fail, make it pass, refactor, it will be more like write a complete test, write the implementation, build it, fix the compilation errors, run it, fix the logic errors to make the test pass, and then refactor.
A Real Resolution?
You could consider switching to the D language, which is link compatible with C and appears to avoid many of the problems described above.
There is another way out of the dilemma of needing optimal performance some of the time and optimal productivity the rest of the time; use more than one language. I’ll discuss this idea in my next blog.
Trackbacks
Use the following link to trackback from your own site:
http://blog.objectmentor.com/articles/trackback/8785
Good analysis. I have one remark regarding IDEs. Try latest SlickEdit. It has some re factorings that works pretty well (even extract class). However it works ok only if you can setup g++ for your project. I use sun compilers so it’s not a solution for me :(
I agree that C++ is poorly suited for TDD but I don’t agree with all your points. To me, interfaces are the thing I miss the most. Not having interfaces makes dependency injection difficult. You can do it with abstract base classes but then you have to make your functions virtual which is not in the spirit of the language. Or you can do it with templates but this introduces quite a bit of complexity. Regarding manual memory management, I think this is more of a religious debate. When I code C#, I find that my dependencies tend to form a non-directed graph whereas my C++ code forms a tree structure. It’s a little more rigid but in my experience it scales a little bit better (I more or less the same feeling about static typing compared to dynamic typing). I am aware that a lot of people disagree with me on this. As for include guards, modern compilers take care of that for you. No need for those any more. And I never, ever write a method that accepts char* instead of string..
Kristian, What problem do you find with using abstract base classes with pure virtual methods? For me it’s an interface and whever it’s in the spirit of the language or not is a religious debate :)
Przemyslaw, I wasn’t aware that SlickEdit did this for g++. Some of the developers I worked with used SlickEdit and probably didn’t even know this feature existed. I certainly didn’t.
Concerning your particular problem, could you create a development build (e.g., for your unit tests) that uses g++ instead of the Sun compilers? That way you could have your refactoring when you need it.
Kristian, thanks for the observation about memory management. I can see how attention to it can lead to a better object graph. It’s an interesting tradeoff; a little more manual effort vs. a slightly less-well-structured graph, in many cases.
Also, it seems that most (?) C++ developers I talk to aren’t aware of how compilers handle includes these days. I didn’t know either (except for some nonstandard pragma directives), so I’ve got some homework to do,
Finally, I’m glad you can avoid char*. A lot of C++ and Java code I see uses too many primitive types when real objects would be better. It seems that this often forces you to follow suit and support both string and char*, in this particular case.
Dean, idea with doing separate build using g++ is good but somebody long time ago allow the code in system i take care of to rot(“broken window…”) and now code is just not compiling with g+. I now that using g+ -Wall -pedantic can help me find at least few issues and improve code quality but it’s gonna be a long journey :) I need to reserve a week for that and finally make it happen.
Dean,
While std::exception does not provide a constructor with a string arg, the what() function is virtual so you could create your own exception classes that define the string or take one in the constructor and override what to use it.
I believe that is what all the exceptions in do.
Obsolete Tool Support
It is possible to implement it: http://www.wholetomato.com/products/featureChoice.asp
Manual Memory Management
Reflection and Metaprogramming
Library Issues
Use boost: http://www.boost.org/
If you need a gc: http://www.hpl.hp.com/personal/Hans_Boehm/gc/
Dependency Management
This is not an easy one, but it’s possible to live with it.
http://ccache.samba.org/ can help to reduce compilation times. It is possible to use precompiled headers (which may or may not work).
C++ can be used in several ways. It is mostly used as C + objects, but the possibilities are really wide, it depends on the resources and the project how far you should go.
On the other end, just do not use C++ if there are more effective ways to solve the problem. If you need the speed, create an extension/module for your favorite language.
Przemyslaw, I think that the performance overhead that virtualized functions cause is “paying for something that I don’t need” which is almost officially against the spirit of the language :) I know that premature optimization is evil and all, but I think that when doing TDD, I use dependency injection so much that I end up making a lot of methods virtual. It may be silly, but I often use templates instead.. (But, I use C++ for game development so I am more allergic to these issues than most :)
> The file will still get opened and read in its entirety every time it is included.
This is not true of any modern compilers. At my previous job I worked with John Lakos, who recommends “redundant include guards” in his 1996 book, but a colleague re-ran his tests and found that only one compiler in use at the firm actually re-opened the files, and that compiler had been put out to pasture by the vendor. Newer versions of that compiler, and all others in use at the firm, recognise the include guards for what they are and will cache a list of headers that have been seen and don’t need to be re-opened.
GCC has done this for about 10 years, since at least version 2.95, here are the docs for the latest release: http://gcc.gnu.org/onlinedocs/gcc-4.2.0/cpp/Once_002dOnly-Headers.html
EDG documents this feature in their compiler front-end. I’m pretty sure Sun’s compiler has had a similar optimisation since Forte 6, maybe earlier.
I think you’re missing a fair bit. For example:
I also think you’re on the wrong track making a new .exe for each test. They should be grouped together, to reduce your overhead.
Memory management is just part of C+, but something that if done well (and using smart_pointer from Boost, for example), doesn’t need to be that painful. This has little to do with TDD, although you could (and probably should) have some memory leak tests. I will say, in my years of C/C+ development, finding memory leaks (and I used to work with consoles, where no leaks were allowed) rarely cost a significant amount of time, although tools & techniques helped considerably.
I’ll admit C++ can be annoying, and without reflection (and with header files) TDD is a lot more work. But it’s not an order of magnitude.
OOPs. comments ate my angle-bracketed reference to stdexcept. I had tried to say:
I believe that is what all the exceptions in “stdexcept” do.
> C++ emphasizes performance and backwards-compatibility with C over all other considerations.
I think if you asked the ISO committee members you’d get a number of very different answers. I would say backwards-compatibility with C++ is at least as important!
I agree with Ben above, except that #pragma once should be avoided, it’s non-portable, and most compilers do the same thing automatically anyway.
OBricker’s right too, you should use the derived types in stdexcept (or your own types derived from std::exception) rather than throwing std::exception directly. Throw a derived type, catch by reference-to-base type – there’s an obvious analogy to dealing with polymorphic types through base class interfaces.
Why would you overload on const std::string& and const char* ? You can pass a const char* to the string version directly. There might be cases where you want to avoid the temporary std::string that this creates, but they will probably be the uncommon case … I certainly think it’s easy to avoid writing those overloads.
Complaints about tool support are certainly valid. The complex grammar is due to the C ancestry and it does mean tools that need to understand the grammar need a full-blown compiler front-end. GDB’s simple parser just can’t handle complex template instantiations. Eclipse/CDT does include some limited refactoring tools, and work is ongoing c.f. http://ifs.hsr.ch/downloads/Flyer_Cerp_eng.pdf
Ben, You and several of the other posters pointed me to some tools that I didn’t know about, for which I’m grateful.
Just one other comment about memory management. I heard recently of some anecdotal evidence that overuse of reference-counting smart pointers can actually be a bottleneck, because of all the “counting” overhead. I’m not saying these tools are “evil” or anything like that ;), just that no solution is without consequences and we should always profile our applications!
I agree with almost all of your points. However, regarding Dependency Management, external include guards aren’t necessary. g++ remembers if a header has internal include guards and skips the extraneous file I/O when they encounter it a second time [Llopis] . I know not every C++ project is built with g++, but for many teams the external include guards aren’t necessary.
I’ve used Ref++ for refactoring support in Visual Studio 2003. It works quite well, even on C only code, for the refactorings it supports. I have mostly used rename, extract method, change signature, and introduce variable.
I’ve done some work making it easier to TDD in vim. I have a set of macros and abbreviations that really shorten the time to get code running.
I absolutely do not have separate exes for my .cpp files. I like them grouped up. I also do not create any header files for my tests. Header files are for the benefit of someone who wants to include them, and there is no such person in CPPUnit testing. I write all tests inline in the class declaration, so there is only the registration line and the function. I then have a hot-key map to register my test functions (and to remove them utterly).
I will probably create a macro for renaming test functions. It won’t be as nice as a rename refactoring, but it would be helpful. As is, I use ”*” to highlight the name and jump to the next usage, CWnewname to change the first, then * to jump to the next usage and dot (.) to change it also. It usually takes me all of two seconds, and I do it enough to matter.
If you use the CompilerOutputter and build a single executable, your VIM editor becomes an IDE. Not Eclipse, maybe, nor even quite EMACS, but more than usable and even quite handy.
I also incorporated valgrind into my builds so that memory management problems are much much easier to spot and fix.
It’s not so hard to set up compared with configuring a Java project especially. But I would still rather be working in a more modern dynamic language like python. It’s hard to compete with zero setup, though.
+1 on always deriving an exception class from the std::exception. I don’t like throwing the languages’ exceptions for my problems.
I haven’t used any good C++ refactoring tools, sadly.
@Ben: I agree with Dean that TDD in C++ is probably an order of magnitude slower than in Java.
I agree with you that Visual Assist is quite good at refactorings, although it is far, far from as powerful as Eclipse. Also, Eclipse is extremely powerful when it comes to generating code, something that saves a lot of time.
However, what’s really powerful IMO when TDDing in Java is jMock. In C++ you have to manually create every mock class. With jMock, that’s done with a single line of code. That really saves time, and removes a lot of duplication.
With this said, I still love (and hate) C++ as a language :)
I work in a Company were most of the code is writen in C++, we use agile and have no trouble using TDD. I’m wondering what we do different.
Ricardo, do you mind sharing what tools do you use on unix platforms? We do TDD as well but it’s kinda slow as we do everything by hand (with a great help from VIM).
In addition to the other tools mentoined amop is also a great tool when practising TDD with C++.
thank you for sharing the post ,it is really good PDF to BMP Converter is the
excellent combination of super high conversion speed and perfect output quality. At the same time, PDF to BMP Converter supports
various output formats like: JPEG, TIFF, TGA, RLE, EMF, WMF and so on.
Thanks for this great post, i like this article very informative.
Awesome post! My windows at home broke last night, just had to get http://www.zh-bamboo.com them fixed – i feel your pain!
TDD in C++ with some Java. While C++ was my language of choice through m
Indonesian Teak Furniture: Indoor Teak Furniture, Teak Garden Furniture, Teak Table, Teak Chairs
has just told him that hand-washing is too expensive, and he should stop doing it.”
Thanks for sharing such a good article, I like your point of view, this matter sorted out your thoughtful blogger exchange is my honor, I wish I could, and you become good friends, for more a more in-depth exchanges! Delaware Mortgage Modification
You bring out some really good points in your post. viagra
over and over again, I like to come to here, thanks.
Intertech Machinery Inc. provides the most precise Plastic Injection Mold and Rubber Molds from Taiwan. With applying excellent unscrewing device in molds, Intertech is also very professional for making flip top Cap Molds in the world.
to a busy signal.”While breitling bentley swiss replica signal.”While I was thinking yesterday that I Mens Rings I would reschedule, now I am just rolex yacht master replica just thinking: Give
Better reflection and metaprogramming support would permit more robust proxy or aspect solutions to be used.
We are the professional jacket manufacturer, jacket supplier, jacket factory, welcome you to custom jacket.
Women Replica Sunglass at cheap discount price Inspired ,MEN designer Sunglasses
hey man,i am a first time visitor here and like your blog.
To be, or not to be- that is a question.Whether ipad bag tis nobler in the mind to suffer The slings and Game Controllers arrows of outrageous fortune Or to take arms against a sea of troubles, And USB Gadgets by opposing end them.
Tremendous work. I am getting benefit from your post. So i have to share thank you. You can go long way by your skill.
Our company is engaged in the professional manufacturer of damper, air cylinder, oil cylinder, and hydraulic station. The company has many year’s production experience and strong technical power.
i can not say anything good article…
Gerçek ki?ilerle sohbet ederek okey oyunu oyna ve internette online oyun oynaman?n zevkini ç?kar.
I visited this page first time and found it Very Good Job of acknowledgment and a marvelous source of info…......Thanks Admin!
I visited this page first time and found it Very Good Job of acknowledgment and a marvelous source of info…......Thanks Admin!
Have the christian louboutin patent leather pumps is a happy thing. Here have the most complete kinds of christian louboutin leather platform pumps.
Online UK costume and fashion jewellery shop with, g
Beats by dr dre studio with look after talk in white. extra attributes on Monster Beats By Dr. Dre Pro Headphones Black a specific tri-fold design and design and carrying circumstance which make for compact and uncomplicated safe-keeping when not in use. Beats by dr dre solo .
you really have avery nice blog,it’s the first time to be here but it won’t be the last untill then keep blogging.goodluck!
? ?? ?? ? ? ?? ?? ? ??? ?? ?? ?? ? . ?? ?? ???? ?? ? ??? ???? ??? ??? ???? ?? ?? . ? ?? ?? ??? ??? ?? ?? ??? ??? .
Canada Goose Outlet is Marmot 8000M Parka. The Marmot 8000M Parka is really a waterproof, breathable jacket with 800 fill canada goose jacket feathers. It truly is design and light colored shell is produced for trendy, but uncomplicated, protection from cold temperatures. Reinforced shoulders, elbows and adjustable waist and hem make the Marmot a perfect alternate for skiing and other outdoor sports that want fairly a bit of arm motion. The 8000M Parka weighs three lbs., comes in bonfire and black colours and might be stuffed and stored like a sleeping bag to your convenience.This is one of well-know and prime down jacket brands.Hope our friends like its!Like canada goose womens and Canada Goose Expedition Parka.There are wholesale canada goose.
Slewing bearing called slewing ring bearings, is a comprehensive load to bear a large bearing, can bear large axial, radial load and overturning moment.
nice to see this site is very good information. thanks for sharing.
The cause of justice is the cause of humanity. Its advocates should overflow with universal good will. We should love this cause, for it conduces to the general happiness of mankind.
Christian Louboutin Rolando Hidden-Platform Pumps Golden is a fashion statement – that’s sexy, it makes you look longer highlight, and it highlights the curves in the woman body and makes the body look more elegant and thinner without any diet.
?Brand: Christian Louboutin ?Material: Golden leather ?Specialty: Signature red sole ?Color: Golden ?Heel height: Approximately 130mm/ 5.2 inches high and a concealed 20mm/ 1 inch platform ?Condition: Brand New in box with dust bags & Original Box
Fashion, delicate, luxurious Christian louboutins shoes on sale, one of its series is Christian Louboutin Rolando Pumps, is urbanism collocation. This Christian louboutins shoes design makes people new and refreshing. Red soles shoes is personality, your charm will be wonderful performance.
thank you man.
NFL,MLB,NBA,NHL,SOCCER Sunglasses
thank you sow mach
well. you guys really give us the sample of C++ programing skill. So. why not try this method and do a better code. next time. have another try.
nice to see this site is very good information. thanks for sharing.
??? ???