Skip to content

The strategy

A test strategy is not a list of tools. It is a set of decisions about where confidence comes from, written down so that the decisions can be argued with before a release rather than after an incident.

This is that page. Everything else in the documentation expands one of its rows.

A strategy is complete when a new team member can answer all six of these without asking anyone:

  1. What are we testing, and what are we deliberately not testing?
  2. At which level does each property get checked, and only once?
  3. Where does each level run?
  4. With what tool, and why that one?
  5. Who answers when it fails?
  6. When do we stop, and what does “enough” mean?

Question 1 is the one most strategies skip, and it is the only one that makes the others finite. See risk-based testing.

Level Answers Where it runs Owner Blocks
Unit Is this logic correct, and is this design usable? In-process, no I/O The author The merge
Component Does this module do its job behind its own boundary? In-process, doubles at the edges The author The merge
Integration Do we talk to that database, broker or API correctly? Ephemeral containers The author The merge
Contract Will the other side still understand us? In-process against a contract Both sides The merge, on both sides
End-to-end Does the critical journey work through the real stack? Deployed candidate The team The release
Exploratory What did none of us think to write a test for? Deployed candidate Whoever takes the charter Nothing — it informs
Performance Does it hold at the load we promised? Production-like, isolated The service team The release, against SLOs
Resilience What happens when a dependency misbehaves? Production-like, faults injected The service team The release, for tier-1
Security What can someone do that they should not? Pipeline, plus scoped tests The team, with security The merge, on known CVEs
Accessibility Can everyone actually use it? Component and journey level The author, with design The merge, on automatable rules
In production Is it still true now, with real traffic? Production On-call The rollout

Read down the Owner column: the word “QA” does not appear. That is shift-left accountability made concrete, and it is the row-by-row commitment the rest of the strategy depends on.

Each property is checked at the cheapest level that can check it honestly, and at exactly one level.

The failure mode this rule exists to prevent is not under-testing. It is a validation rule verified in a unit test, again in an integration test, again in an end-to-end test and again by a manual script — four tests that fail together, take four times as long to update, and give one signal between them.

“Honestly” is doing the work in that sentence. A unit test with the database mocked cannot honestly check that a query returns the right rows; moving it down a level does not make it cheaper, it makes it wrong.

A campaign that cannot say when it is finished never is. Every campaign states both.

Entry — what has to be true before it starts. A release regression campaign that begins before the build is deployable spends its first day on the environment.

Exit — what “done” means, agreed before the run. Not “all tests pass”: almost no real campaign ends with a clean sheet. Something like no open critical or major defects on the release scope; every known-failing test triaged with an owner; the risk register signed by the release manager.

Naming this is the strategy’s most useful paragraph, and the one most often left out. The current list:

  • Third-party behaviour we do not control. We test our integration with a payment provider, not the provider. Their outage is a resilience question, not a functional one.
  • Generated code, at the level it was generated from. An OpenAPI client is tested by the contract, not by unit tests over its serialisation.
  • Framework guarantees. No test asserts that the ORM persists a field or that the router routes. These fail loudly the first time anything else runs.
  • Exhaustive browser and device matrices. Two engines and three viewports on the critical journeys; the rest is a risk decision reviewed per release, not a standing commitment.
  • Load beyond the stated SLO. We test the promise, plus a headroom factor. Testing to destruction is a separate, occasional exercise with its own purpose.

Every item is a decision that can be revisited. What it must not be is an accident.

Twice a year, and after any incident that a test could have caught. The post-incident question is never “who missed it” — it is at which level should this have been caught, and why was there no test there? The answer is usually that the level was wrong, not that the person was careless. See quality-driven development on reading a defect as feedback about the process.