Risk analysis

Status: described, not built.

What it would do

Take a change set — a pull request, a release diff — and produce a ranked risk table in the shape risk-based testing asks for: each changed area with a likelihood, an impact and the evidence behind both.

The inputs are things most organisations already have and never join:

Change frequency and coupling, from version control. Which files change most, which change together, and which have many authors. Frequently-changed complex code is where defects live, and this is mechanical to compute.

Defect history. Which modules previously produced defects, and how severe. The best single predictor available, and it usually sits in an issue tracker that nothing else reads.

Incident history. Which areas caused production incidents. Small sample, enormous signal.

Test coverage of the changed lines, from the existing report. Not as a score — as one input into likelihood.

Blast radius, from the architecture. Who calls this, and what happens downstream. The component catalogue in arch-hub is where that lives.

The output is the table from risk-based testing, pre-filled — including a proposed confidence from column drawn from the strategy.

Where its boundary is

It does not decide what gets tested. It produces evidence for the conversation. The decision — including the deliberate decision to test nothing — belongs to a human who will answer for it, and moving that decision to a tool would remove exactly the accountability that shift-left is about.

It does not assign impact on its own. Likelihood is computable from history. Impact is a business judgement — money, legal exposure, safety, reputation — and the tool can only propose it from previous classifications of the same area, with the reasoning shown.

It shows its evidence, always. “High likelihood” is useless; “changed in 14 of the last 20 releases, 3 defects in the last year, 62% branch coverage on the changed lines” is auditable, and a human can disagree with it specifically.

It does not learn from its own output. A tool trained on what a team chose to test would reinforce the team’s existing blind spots, which are precisely what this is meant to surface.

What it would be good at

Two things people are demonstrably bad at.

Noticing the quiet area. Attention goes to what changed loudly. The module that quietly changed in every release for six months and has no coverage is invisible in a review and obvious in the data.

Consistency. A human risk assessment on a Friday afternoon before a release is not the same assessment as one on a Tuesday morning. A mechanical baseline does not have moods, and the human’s job becomes correcting a starting point rather than producing one from nothing.

Why it might not be worth building

The data is usually poor. Defect history that does not name modules, incidents that were never written up, coverage reports that exclude the interesting packages. The tool would be an accurate reading of bad inputs, which is more dangerous than no reading.

A ranking can be treated as an answer. A table with numbers has an authority a conversation does not, and a team may stop arguing with it. Preserving the argument is the entire point of risk-based testing, and a tool that ends the argument makes the practice worse.

Most of the value is in the joining, not the model. A SQL query over version control and the issue tracker gets a large share of this, and it is a week of work rather than a product.

That last point is the strongest objection on the page, and it is also the cheapest first step: build the query, see whether anyone uses the table, and only then ask what a model would add.