Question

What does code coverage actually tell you?

Vault Verified
Curated Intelligence
Definitive Source
Answer

Which lines were executed while the tests ran — and nothing at all about whether those lines were checked, or whether the behaviour is correct. It is a measure of what was not tested, which is useful, and is routinely misread as a measure of quality, which it is not.

The kinds, which differ substantially:

Line coverage — was this line run. The weakest and most quoted.

Branch coverage — was each side of each conditional taken. Considerably more informative, and a common trap is that a function can reach 100% line coverage while an entire else path was never exercised.

Condition and path coverage, finer still and rarely practical at scale.

Mutation testing, which is the honest measure: it deliberately alters your code — flipping conditions, changing operators — and checks whether any test fails. If a mutant survives, the line was executed but nothing was actually verifying it. This is slow and tells you more than every other metric combined.

Why the number misleads:

A test with no assertions gives full coverage. Calling the code counts; checking the result does not.

Coverage says nothing about inputs. Running a function once with a happy-path value covers it completely while every boundary, empty case and error path goes untested.

Trivial code inflates it — getters, generated code, configuration — so a high number can be dominated by code nobody was worried about.

What happens when it becomes a target. A mandated threshold reliably produces tests written to reach it: assertion-free tests, tests of trivial accessors, and exclusion annotations. The measure stops measuring anything — which is Goodhart's law operating exactly as described.

How to use it well:

As a gap-finder, reading the uncovered report rather than the percentage.

Watch the trend rather than the absolute.

Require coverage on changed lines in review, which is far more useful than a global threshold.

Care most about coverage of complex, risky code, and be relaxed about the rest.

Related Questions