Unit Tests Coverage: Less Is More 19

Posted by tottinger Mon, 07 May 2007 13:06:00 GMT

In TDD a unit test has to be very small to isolate failures. This does funny things to code coverage as a metric. Each test should have a very small area of effect, and so each unit test should have a negligible effect on the overall code coverage statistic. Bear with me here, see where I’m missing out.

Say you have an existing (legacy) system with no coverage at all. Zero percent. If you start doing TDD today, the overall coverage percentage should barely change at all. If only the new code is test-driven, then the old code is not gaining coverage except where tests are necessary to ensure that the new code is being called. The low coverage per test is a good thing because it shows that the unit tests have good isolation. In such a situation, code coverage is really telling you the ratio of new code to old code. Again, this is so obvious and logical to me that I must be missing some cool subtleties.

If all the code was test-driven from the beginning, you should have a very high coverage number, and writing a new unit test before you add code should not impact that number. You only write enough code to pass the test, so again you aren’t getting much in the way of uncovered code. A lowering of the ratio might indicate a problem with test-to-production-code ratio. This seems pretty simple and logical, so I’m sure I’m missing some interesting corner cases.

OTOH, system tests and integration tests paths through many components at once, and should have a more significant effect on coverage in a previously-untested system, though their job is to prove function points, not to raise the metrics. These non-unit tests are the thing that boost your coverage of old code. That is also a good effect, because it is the goal of the system test to ensure that the parts work together.

I’m not saying that code coverage should be low, only that as we move incrementally, the unit tests we write should individually have negligible effect on our overall code coverage numbers… a thought that intrigues me.

Comments

Leave a response

  1. Avatar
    Lindsay 3 months later:

    Of course in a Test Driven Context, the first stage in fixing a bug is writing a test to expose it, so you should end up with some coverage of old code.

  2. Avatar
    Tim 3 months later:

    Of course. But even then, your test shouldn’t change the total coverage percentage by much. It matters more than it shows, which is my problem with coverage in general.

    You can show progress by bumping up the coverage number, but your tests don’t have to be any good. But if you write good tests where they’re needed (as given by Lindsay) then you have the real goal covered, but the prosthetic goal is largely untouched.

    Coverage grows very slowly in legacy code.

  3. Avatar
    liding rode over 3 years later:

    OK, I got it. Thanks for your sharing. That is very interesting Smile I love reading and I am always searching for informative information like this.

  4. Avatar
    best blu ray converter over 3 years later:

    Blu Ray Converter can easily convert blu ray dvd/common dvd/DVD IFO file/DVD folder/DVD ISO file to most common Video, HD Video, Flash Video, iPod, iPhone, iPad, Apple TV, PSP/PS3, Creative Zen, BlackBerry, Zune, Xbox, Mobile Devices, Archos, Common Audio, Supported Portable Devices.

    Editing funcionts like trim and crop DVD/ Blu-Ray clips, customize profile list and different watermarks?

  5. Avatar
    iPod to iTunes Transfer over 4 years later:

    iPod to iTunes Transfer, transfer iPod to iTunes library, is an all-function transfer for iPod users that supports all versions of iPod and other iPhone devices. It can perform like a transfer, a converter, a ringmaker.

  6. Avatar
    replica watch rado over 4 years later:

    iPod to iTunes Transfer, transfer iPod to iTunes library, is an all-function transfer for iPod users that supports all versions of iPod and other iPhone devices. It can perform like a transfer, a converter, a ringmaker.

  7. Avatar
    Criminal Records over 4 years later:

    These non-unit tests are the thing that boost your coverage of old code. That is also a good effect, because it is the goal of the system test to ensure that the parts work together.

  8. Avatar
    Tenant Screening over 4 years later:

    These non-unit tests are the thing that boost your coverage of old code. That is also a good effect, because it is the goal of the system test to ensure that the parts work together.

  9. Avatar
    Gaylene Mortell over 4 years later:

    Intimately, the article is in reality the freshest on that valuable topic. I match in with your conclusions and also definitely will eagerly look forward to your coming updates. Simply saying thanks cannot just be adequate, for the extraordinary lucidity in your writing. I can right away grab your rss feed to stay abreast of any updates. De lightful work and also much success in your business endeavors!

  10. Avatar
    okey oyunu oyna over 4 years later:

    nice..

    Okey oynamak hiƧ bu kadar zevkli olmadi. Online ve 3 boyutlu okey oyunu oyna ve turnuvalara sende katil.

  11. Avatar
    christian louboutin shoes on sale over 4 years later:

    Have the christian louboutin patent leather pumps is a happy thing. Here have the most complete kinds of christian louboutin leather platform pumps.

  12. Avatar
    bagsupplyer over 4 years later:

    It is nice of you to post this.I will pay more attention on it. Fashion brand Men G-star Short jeans from China for wholesale at on line store

  13. Avatar
    jeremy scott shoes over 4 years later:

    Company manufactures shape wears by adding finest element to make the products able to wear in all circumstances. This is all to achieve trust and faith of customers on the company’s products, which lead to successfully make highest market share. And company has done all this in a very well manner.

  14. Avatar
    north face sale over 4 years later:

    v

  15. Avatar
    backup iPhone sms over 5 years later:

    Well. Though I am not a good application developer. And I need do more hard work to improve myself. When I come to here. I know that I have come to the right place to learn something I need. Thanks for your good advice. And I will do the practice as possible as I can. Thanks.

  16. Avatar
    Tisa snapbacks over 5 years later:

    Wholesale snapback hat and sports activities interconnection is not the end. Baseball hat does not contract for the majority. This could be the simple fact that everyone feels relaxing when wearing this Last Kings Snapback.

  17. Avatar
    snapback hats over 5 years later:

    Strap back hats are not difficult to find when one wants to purchase one. The consumer is able to purchase snapback hats cheap at any department store, sports team venue, concerts, or at an embroidery store. These hats are customizable, so the consumer can pick out any color, style, as well as having their own art embroidered on the front or back of the hat. The snapback hats wholesale price range is around ten to thirty dollars, and that depends on the type of store that the customer purchases it from. Usually at sports events, the hats will cost more than if one purchases it at a department store. These hats are a great all around hat to wear since they keep sun out of the face as well.

  18. Avatar
    discount north face jackets over 5 years later:

    The discount north face are also one of the most successful products. They are appreciated all over the world for their quality and designs. discount north face jacketsare available for both men and women; moreover there is another category for kids. These Jackets are available in many colors, styles and designs.

  19. Avatar
    birkenstock sandals for women over 5 years later:

    I want style and comfort in doing my existence, specially in the the summer season, this also is just what Birkenstock sandals supply.Ultimately, I select birkenstock sandals for women repeatedly since they will be highly affordable when compared to various other premium quality companies that you could obtain. I will be simply no Jennifer Aniston, thus I do not want to pay on each ladies I recieve.

Comments