Randy Johnson (The Big Unit)
A common evil that often rides along with unit testing is the unit testing metric. These metrics come in many flavors, including percent code coverage, number of tests and percentage of tests passing. Like any statistic, there is the potential for misuse and misinterpretation, which is the point of a post on Javalobby titled “Effective Code Coverage (and Metrics in General)“.
Management mandates to have X% of code coverage.
… Upper management, without a clue about the health of the codebase, mandates that developers should reach X% of code coverage. Period. … In many cases, when the codebase was not designed with testability in mind, I’ve seen developers writing a lot of test cases, without any assertions! All the tests pass, and the code coverage goal is met.
This seems like a pretty easy problem to solve. Review the unit tests. Ensure that there are actual assertions (i.e. tests) going on inside each test method. Problem solved.
Unit tests are code. Just as code should be reviewed, both by peers and architects, so should unit tests.
I have never seen this happen. However, if I ever do encounter it, here’s what I would do (in this specific order):
- Re-examine the company’s culture of mandating unit testing, and determine what it was that would cause otherwise reputable developers (you don’t hire anything but, right?) to go to such lengths to play the system.
- If you can’t find anything wrong with the company’s culture, look harder. A good, honest developer will not do this unless pushed to the limit. Are there abstract and unrealistic deadlines being set? Do they have the tools they need to write the tests? How about training? The company, from the top executive down, must embrace and support developer-driven testing.
What I have seen, however, are projects with a lack of unit tests. This is almost always due to technical leadership failing (architects and team leads) to emphasize the importance of user testing, or the aforementioned lack of executive sponsorship.
Tags: metrics, unit testing