Tooling

Tooling is the shortest-lived part of a test strategy and the part most often mistaken for it. The principles survive a change of tools; the typologies survive a change of language; this section will be wrong first.

So each page here states what the tool is for and what would have to change for a different one to be right, rather than presenting a ranking.

Where each typology is run

Typology	Tooling
Unit, component	Unit and component
Integration, contract	API and contract
End-to-end, accessibility	Browser and mobile
Performance, resilience	Performance
Security	Security
Exploratory	Almost none, on purpose — a notes template and a recorder
Every campaign, before and after the run	Management and reporting

The last row is the odd one: Xray and Allure sit either side of a campaign rather than inside it. Neither executes anything — the tests stay plain code in the repository, and these two define the scope and report the outcome.

How a tool gets chosen here

Five criteria, in this order. The order is the argument: a tool that scores perfectly on the last two and badly on the first is the one a team abandons in eighteen months.

1. Does the team already know it? A familiar adequate tool beats an unfamiliar excellent one, because the cost of a testing tool is dominated by the tests nobody writes while learning it.

2. Does it run in the pipeline unattended? Anything requiring a GUI, a licence server or a person is a bottleneck with a brand name.

3. Are the tests plain code in the same repository? Versioned with the code they test, reviewed in the same pull request, refactored with the same tools. This single criterion eliminates most record-and-playback products, and it is the reason it sits above capability.

4. Does it fail informatively? See reporting. The quality of the failure artefact is worth more than a feature list.

5. Is it still maintained, and can we leave? Ask what leaving costs on the day you arrive. A proprietary format that cannot be exported is a decision made once, permanently.

What is deliberately not on the list

Record-and-playback tools. They generate selectors bound to the DOM’s current shape, produce artefacts nobody can review, and fail criterion 3 outright. They demonstrate well and they do not survive the second refactoring.

Anything that requires a licensed workstation to run a test. It cannot gate a merge, so it cannot be part of the loop that shift-left accountability depends on.

Separate “automation frameworks” owned by a separate team. The tests belong in the repository with the code, owned by the people who change it. A framework in its own repository, maintained by its own team, recreates the handover this strategy exists to remove.

AI-generated assertions over existing behaviour. A tool that writes tests by observing what the code currently does produces a change detector at scale — it will agree with every bug that is already there. The MCP section is about generating tests from acceptance criteria, which is a different input and the only one worth automating from.