Every QA practice ever built rests on one assumption: run the same input twice, get the same output twice. Break that and most of the toolbox stops working — exact-match assertions, golden files, snapshot tests, the lot.
That's the position teams are in the moment an LLM lands in the product. The feature works. It just never answers the same way twice.
Stop asserting the words. Assert the contract.
The output varies. The obligations don't. Almost every AI feature has a set of properties that must hold on every run regardless of phrasing:
- Task completion — did it actually do the thing that was asked?
- Grounding — is every factual claim traceable to the context it was given, or did it invent one?
- Format — valid JSON, required fields present, schema respected.
- Refusal boundaries — did it decline what it should decline, and only that?
- Tool use — did it call the right tool, with sane arguments, and handle a failure?
- Safety and leakage — no system prompt, no other user's data, no credentials.
Those are testable on non-deterministic output because they're properties of a response, not a specific string.
Who grades it
Some of those checks are mechanical — schema validation, a regex for a leaked key, a check that a tool was called. Run those first; they're cheap and they never disagree with themselves.
The rest need a judge model, and a judge model brings its own two problems. It is itself non-deterministic, and it is itself a target — content in the response can try to talk to it.
Drift is the real failure mode
AI features rarely fail loudly. They degrade. A prompt gets tweaked, a model version rolls forward, a retrieval index goes stale — and quality slides a few percent a week until someone notices in a support ticket.
The defence is a baseline: a fixed scenario set, scored on every run, tracked over time. What matters isn't the absolute score, it's the delta. A five-point drop against last week's baseline is a signal; a score of 0.83 in isolation is a number.
Scenarios have to keep coming
A hand-written scenario set decays. It reflects what you feared at the time you wrote it, and real users are more creative than that. Feeding production traffic back into the scenario set — the odd phrasings, the multi-turn detours, the prompts that tried to jailbreak it — is what keeps the eval honest a quarter later.
Test the properties that must always hold, watch the delta rather than the score, and keep the scenario set growing from real traffic.
QAEverest's AI Agent & LLM testing runs black-box evaluation, red-team scenarios and drift baselines against your agent — no access to weights or prompts required.