Why a performance DSL

The observation: performance test scripts decay faster than any other test artefact a team owns.

The hypothesis: because the thing worth reviewing — the workload model — is buried inside the thing nobody wants to read, which is HTTP mechanics, correlation logic, authentication flows and data feeders.

This page argues that the observation is real and the hypothesis is plausible. The survey checks whether existing tools already solve it, and the sketch proposes something concrete.

The observation

Symptoms, in the order they usually appear:

• The script was written once, by one person, often a contractor.
• It is in a language or format nobody else on the team uses daily.
• It is reviewed by nobody, because nobody can review it meaningfully.
• It breaks when an endpoint changes, and the fastest fix is to comment out the failing scenario.
• Two releases later, the “performance test” exercises three of the eleven journeys it was written for, and nobody knows which three.
• The results are still reported as if they covered the system.

The last line is what makes this a quality problem rather than a maintenance annoyance. A degraded functional suite goes red; a degraded performance suite goes green over a smaller scope, silently.

Why it decays faster than other tests

Four reasons, and they compound.

It is touched rarely. A unit test is read every time its code changes. A performance script is opened twice a year, so its drift is discovered all at once.

Nobody owns it. It usually lives outside the service repository, and the rule from tooling — tests are plain code in the same repository — is most often broken exactly here.

It cannot be reviewed. This is the load-bearing reason. A reviewer wants to answer one question: is this the right workload? Answering it requires reading past several hundred lines of correlation and header manipulation to reconstruct a model that could have been stated in ten.

Its failures are ambiguous. A functional test fails or passes. A performance run produces numbers that require interpretation, so a scenario that silently stopped exercising anything looks like a scenario that got faster.

The two things in every script

Read any performance script and the content divides cleanly.

The model — what is being simulated. Which journeys, in what proportion, at what arrival rate, with what think time, against what data distribution, and what thresholds decide pass or fail. This is the part with domain content. It is usually ten to thirty lines’ worth of information, and it is what a product owner, an SRE and a test manager all need to check.

The mechanics — how the simulation is executed. Building requests, extracting a token from a response and putting it in the next header, parameterising from a CSV, handling redirects, pacing loops, checking status codes.

The mechanics are necessary and uninteresting. They are also the majority of the lines, and they are what makes the model unreadable.

What a separation would buy

Reviewability. A model file short enough to read in a pull request, in terms a non-specialist recognises: 70% browse, 20% search, 10% checkout; 200 concurrent users; think time 3–8 seconds; p95 under 300 ms.

Durability. When an endpoint changes, the mechanics change and the model does not. Today a mechanical change requires editing the file the model lives in, which is how models get accidentally altered.

Portability. The same model compiled to k6 for the pipeline and to Gatling for a large distributed run, without rewriting the intent. Also an exit path from any of these tools, which is criterion 5 in how a tool gets chosen.

Generation. A model is a small structured artefact, so it can be derived: from production access logs, giving the real traffic mix rather than a guessed one; from an OpenAPI contract, giving the endpoints; from the SLOs, giving the thresholds. That is what the MCP performance DSL server sketches.

What would make this a bad idea

Stated up front, because a research page that only lists benefits is advocacy.

Another layer to learn. Teams already know k6 or Gatling. A DSL adds an indirection, and when the generated script misbehaves, someone has to debug through it.

The 80/20 trap. Every DSL over a general-purpose tool eventually meets the scenario it cannot express, and the answer is either an escape hatch — which reintroduces the mechanics it was hiding — or “you cannot test that”.

It may be a convention problem. Perhaps a documented file structure that separates model from mechanics, with no new tooling, captures most of the value. That is outcome 2 in the section overview, and it is cheaper than everything else on this page.

Maintenance. A language is a product. It needs versioning, documentation, error messages and someone to own it — for a team that runs performance tests twice a release.

The honest test for whether this is worth building: would a product owner read the model file and correct it? If yes, the separation earns its cost. If the model file also ends up read only by the person who wrote it, this is a refactoring with a grand name.