Coverage

Coverage measures which lines were executed while the tests ran. That is all it measures, and the gap between that and “which behaviour was verified” is where most of the confusion in this subject lives.

A suite that calls every method and asserts nothing reaches 100% coverage. This is not a hypothetical — it is what a coverage target produces once the target becomes the thing being optimised.

What it is good for

Finding what is not tested at all. This is the honest use, and it is valuable. Sort by uncovered lines, look at the top of the list, and ask whether each one matters. A payment path at 0% is a finding. Generated DTOs at 0% are not.

Watching the direction on changed code. “This pull request adds 200 lines and covers 12 of them” is a useful review comment. The absolute number for the whole repository is not.

What it is not good for

A target. The moment a number gates a merge, it becomes the objective and stops being a measurement. Teams reach 80% by testing getters, by asserting nothing, and by excluding the hard packages — all of which raise the number and none of which raise confidence. The number goes up; the escaped defects do not go down.

Comparing teams or services. A parser and a CRUD controller have different meaningful ceilings. Comparing them ranks the domain, not the discipline.

Evidence of quality. High coverage with escaping defects is one of the most common configurations there is, and it is the one that produces the most confident wrong decisions.

Branch, not line

If a coverage number is going to be quoted, quote branch coverage.

if (customer.isGold() && order.total() > 100) applyDiscount();

One test with a gold customer and a 120-euro order gives 100% line coverage. It leaves three branch combinations unexercised, including the two that most likely contain the bug.

Mutation testing: the honest measure

Mutation testing changes the production code deliberately — flips a comparison, removes a call, alters a constant — and re-runs the tests. If the tests still pass, that mutant survived, and the tests do not actually verify that behaviour.

It measures what the coverage number is usually assumed to measure: not what ran, but what would have been caught.

The output is directly actionable in a way a percentage never is. `Changed

to >= on line 47: survived` means there is no test for the boundary at that condition. That is a specific missing test, not a number to improve.

The costs are real. It is slow — every mutant means a test run, so it is a nightly or weekly job on a changed subset rather than a merge gate. And some survivors are equivalent mutants: changes that do not alter behaviour, which cannot be killed and have to be suppressed by hand.

Where it pays. On the code where correctness matters most — pricing, authorisation, state machines, anything with money or safety in it. Running it across an entire repository produces a report nobody reads; running it on the discount engine produces four missing tests that someone writes that afternoon.

Better signals than either

Both of the above measure the tests. These measure the outcome, which is what anyone actually cares about:

Escaped defect level. For each defect that reached production, at which level should it have been caught? The distribution tells you which part of the strategy is not working. See quality-driven development.

Change failure rate and lead time. What fraction of changes cause a degradation, and how long from commit to production. These move when quality moves, and they are hard to game because gaming them requires actually shipping worse or slower.

Mean time to diagnose a failing test. Rarely measured, and highly diagnostic: if a red build takes an hour to attribute, the suite has a reporting problem or a determinism one, and both cost more than the coverage percentage ever will.

The practical policy

• Measure branch coverage; report it per pull request on changed lines.
• No repository-wide target. Review the uncovered list instead.
• Mutation-test the critical modules, nightly or weekly, and treat surviving mutants as a backlog of missing tests.
• Never exclude a package to make a number look better. If something genuinely does not need testing — generated code, DTOs — exclude it with a comment saying why, and expect that comment to be read.