Object Mentor Blog: A Coverage Metric That Matters

A Coverage Metric That Matters

Michael Feathers — Fri, 28 May 2010 05:39:00 -0500

How much test coverage should your code have? 80%? 90%? If you’ve been writing tests from the beginning of your project, you probably have a percentage that hovers around 90%, but what about the typical project? The project which was started years ago, and contains hundreds of thousands of lines of code? Or millions of lines of code? What can we expect from it?

One of the things that I know is that in these code bases, one could spend one’s entire working life writing tests without doing anything else. There’s simply that much untested code. It’s better to write tests for the new code that you write and write tests for existing code you have to change, at the time you have to change it. Over time, you get more coverage, but your coverage percentage isn’t a goal. The goal is to make your changes safely. In a large existing code base, you may never get more than 20% coverage over its lifetime.

Changes occur in clusters in applications. There’s some code that you will simply never change and there’s other areas of code which change quite often. The other day it occurred to me that we could use that fact to arrive at a better metric, one that is a bit less disheartening and also gives us a sense of our true progress.

The metric I’m thinking about is percentage of commits on files which are covered by tests relative to the number of commits on files without tests.

In the beginning, you can expect to have a very low percentage, but as you start to get tests in place for changes that you make, your percentage will rise rapidly. If you write tests for all of your changes, it will continue to rise. At a certain point, you may want to track only a window of commits, say, the commits which have happened only in the last year. When you do this, you can end up very close to 100%.

If you think this through, it might seem a bit dodgy on a couple of fronts. The first is that having tests for code within a file does not mean that that code is completely covered by those tests. But, I often find that the hardest part of getting started with unit testing is getting classes isolated enough from their dependencies to be testable in a harness at all. Another dodgy bit is the fact that once you get some tests in place for a file, all of the commits you’ve ever done for that file count in the percentage. Again, that’s okay for fundamentally the same reason. Once you start getting coverage, you are in a good position with that particular code.

What about the moving window? If you track this metric over, say, the last N months of commits, you’ll progressively lose information about the code that you just aren’t changing. To me, that’s fine. Coverage matters for the code that we are changing.

The metric I’m considering (maybe we can call it ‘change coverage’) gives us information about how tests are really impacting our day to day work. Moreover, it’s likely that it would be a good motivational tool, and really, that’s one of the things that a good metric should be.

"A Coverage Metric That Matters" by Übersetzungsbüro Karlsruhe Französisch

Mon, 30 Jan 2012 05:52:18 -0600

It is alway the hardest part to get started with e test.

metric differences should not be that great

"A Coverage Metric That Matters" by online docs

Tue, 17 Jan 2012 06:29:21 -0600

a awesome artical….i like it.

"A Coverage Metric That Matters" by Polo Ralph Lauren Pas Cher

Thu, 12 Jan 2012 20:04:31 -0600

Thanks for this beautiful website. I have enjoyed reading through a few of the articles.

"A Coverage Metric That Matters" by Moncler Women Jackets

Mon, 02 Jan 2012 02:52:51 -0600

If you are well-built or rather have a heavy Cheap Moncler Jackets then you should wear full sleeve t-shirts and wearing Moncler Down Jackets as they tend to make you look warmer.

"A Coverage Metric That Matters" by Cheap authentic NFL jerseys

Tue, 20 Dec 2011 02:33:17 -0600

Cheap authentic NFL jerseys is the best resource for the best deals on everything Internet makert. You simply look eBay and definitely you will find something for you. Can the auction stay there for some time than expected, but nevertheless authentic NFL jerseys is the best venue for the buying of the Wholesale Cheap authentic NFL jerseys. Have patient and you will be rewarded. In addition you can spend this extra time, close while controlling the functions of the Jersey, so that Authentic NFL Jerseys after the deal you don’t like a looser. Moreover feel some auctions on eBay; you should not primarily to invitations to tender.

"A Coverage Metric That Matters" by Drug Addiction Intervention

Thu, 15 Dec 2011 03:48:11 -0600

Hello everyone, please quickly visit this site and know about this great post. It is really a good post. I like this.

Drug Addiction Intervention

"A Coverage Metric That Matters" by Opioid Addiction

Thu, 01 Dec 2011 23:31:06 -0600

So many many thanks for this article. I know lot of things about this post. Its so popular. I like this.

Opioid Addiction

"A Coverage Metric That Matters" by T-Classic

Thu, 17 Nov 2011 20:12:51 -0600

Great post, amuse address added about this, and I like it. I absolutely adore account your blog accepted distributed: a acceptable commodity cat-and-mouse for you! Greate post,please address added about this,and I like it,I absolutely adore account you blog accepted distributed: a acceptable commodity cat-and-mouse for you!

"A Coverage Metric That Matters" by A Coverage Metric That Matters

Mon, 14 Nov 2011 05:10:45 -0600

cheers for the amazing content on your website, I am very interested in this article and you have really helped me. I have just told a few of my friends about this on FaceBook and they love your content just as much as I do.

Dexedrine Addiction

"A Coverage Metric That Matters" by fake oakleys

Wed, 09 Nov 2011 01:01:20 -0600

fake oakleys have already not summer special weapon. Arrived qiu dong, sunglass is still can let you joke put cool. In the colorful and various kinds of sunglasses “visor”, wild city “da” fake oakleys were not roasted meats and sun, burning heat skin, still through the wild city streets, and a series of fashion to move the fake oakleys flush everywhere the scenery line.

"A Coverage Metric That Matters" by christian louboutin

Mon, 31 Oct 2011 07:59:12 -0500

Great post, please write more about this, and I like it. I really enjoy reading your blog popular distributed: a good article waiting for you! Greate post,please write more about this,and I like it,I really enjoy reading you blog popular distributed: a good article waiting for you!

"A Coverage Metric That Matters" by http://www.discountsgolfshop.com

Sat, 29 Oct 2011 00:33:58 -0500

That’s new from Ping Golf this summer? The company has a suite of new golf clubs on the market – let’s call it their Summer 2011 new product release. The Ping G20 has been launched its newest fairway wood. Golf Clubs For Sale With the launch of Left Handed TaylorMade R11 Driver such an exciting new fairway wood, here’s the review of Ping G20 fairway wood.Left Handed TaylorMade R11 Fairway Wood Mizuno MP-59 Irons Titleist VG3 Irons Left Handed Callaway RAZR X HL Irons

"A Coverage Metric That Matters" by http://www.discountsgolfshop.com

Sat, 29 Oct 2011 00:29:50 -0500

"A Coverage Metric That Matters" by Property Marbella

Fri, 28 Oct 2011 13:55:03 -0500

Good site that has all the valid information and that will also give the use full information to all.

"A Coverage Metric That Matters" by Pandora Jewelry

Thu, 27 Oct 2011 20:12:40 -0500

Whether pandora wholesale you have been unable to find a stylish and beautiful shape of the necklace? Then small series with a look at the beauty of these big fake collar necklace has a lovely pandora charm retro small lapel T units for two consecutive quarters as the most popular fashion point. This winter, a small lapel is not limited to fashion, it is more to necklaces pandora necklace discount Pandora Jewelry of the field, the most tide “false collar necklace this year, major fashion brand designers start of the article on the collar, 2011 Milan Fashion Week on, a dazzling white pointed collar Pandora Bracelets collective debut. Chanel’s little black dress with small round white mix is has become a classic, shot in the street this year is everywhere now This stock has fallen a false collar style accessories Pandora Earrings. than the fabric is too low silver pandora chains round neck, collar necklace of fake luxury metal may dazzling land and more. false collar means is a fabric made into a good texture a collar and this collar no pandora necklaces silver bracelets sleeves, no breast, no back, just only a collar, so call it false lead. the most beautiful moment of a woman’s life is put on the holy wedding, into the wedding hall, the love will be forever a Beads Pandora when worn on the finger.

"A Coverage Metric That Matters" by rascel ds

Thu, 27 Oct 2011 00:14:32 -0500

Thanks for this article. Its really a good topic. I like this side. Give me some information about this side.It will help me a lot.Thanks.

flood damage

"A Coverage Metric That Matters" by rascel ds

Wed, 26 Oct 2011 17:19:34 -0500

Hey you done a wonderful job. good writing style i got lots of information from them. thanks for sharing this nice post. keep it up and give us more information. nice to visit here. i will be coming soon.

roofing calculator

"A Coverage Metric That Matters" by http://www.hotgolfsale.com/

Wed, 19 Oct 2011 03:05:08 -0500

But the closer I was getting, as long as I was in the top 10, it was Titleist CB 712 Irons great confidence and very positive for me to continue to work in the right direction.” The beauties are always popular in any occupation, any countrly and anywhere. The world are more beautiful for their exsistence. see more hot female….

ping anser Ping Rapture V2 Irons Callaway RAZR X Hybrid

"A Coverage Metric That Matters" by ysbearing/yisong@1stbearing.com

Wed, 19 Oct 2011 01:02:44 -0500

Slewing ring is also called slewing bearing, some people called: rotary support, swing support. English Name: slewing bearing or slewing ring bearing or turn table bearing, slewing ring in the real industrial applications is very wide.

"A Coverage Metric That Matters" by afasdf

Sun, 16 Oct 2011 12:51:48 -0500

I’ve also been thinking the similar thing personally lately. Delighted to see somebody on the same wavelength! Nice article.

Meth Abuse

"A Coverage Metric That Matters" by Well

Sun, 16 Oct 2011 01:13:58 -0500

I really salute you webmaster for having this wonderful theme and for sharing your ideas. Hop that you will continue writing more articles than this. Have a great day.

LSD Abuse

"A Coverage Metric That Matters" by Christian

Mon, 10 Oct 2011 22:31:30 -0500

as a0s+dfsd+sd sss

"A Coverage Metric That Matters" by moncler

Mon, 10 Oct 2011 22:27:29 -0500

asdf sdf+gd8+d8+ddd

"A Coverage Metric That Matters" by Louboutins

Mon, 10 Oct 2011 22:23:00 -0500

asd as0d+9fas8+sss

"A Coverage Metric That Matters" by bakiretr

Mon, 10 Oct 2011 21:00:38 -0500

Thank you so much for this tutorial. I really learned alot from it.