"AI writes your test cases" is an easy promise to make and a hard one to live with. Most QA leads have already seen the failure mode: a tool generates hundreds of test cases from a user story in under a minute, the team spends the next two days reading them, keeps a handful, and quietly stops using the tool. Time saved: negative.
So it's worth being precise about which part of test design gets faster — and what deliberately doesn't.
A note on the missing percentage
Every tool in this category, ours included, is tempted to put a number on this. You'll see figures quoted with a confidence that nobody has earned.
We're not going to give you one, for a simple reason: the honest answer depends entirely on your stories, your conventions, your domain and your review bar. A team with well-written acceptance criteria and a light review process sees something dramatic. A team working from three-line tickets in a regulated domain sees something real but far more modest. A single percentage flattens that difference into marketing.
What we can describe precisely is where the time goes and which part of it disappears. That's more useful anyway, because it tells you whether your team is the kind that will benefit — and there's a way to measure it yourself at the end of this piece.
What test design actually consists of
Pull the task apart and it isn't one activity. It's five, and they cost very different amounts.
| Step | What it is | Who's good at it |
|---|---|---|
| Read and interpret the story | Understanding what's being built and what "done" means | Human, mostly |
| Enumerate the paths | Happy path, alternates, negatives, boundaries, roles, states | Machine, genuinely |
| Write each case out | Preconditions, steps, test data, expected results | Machine, overwhelmingly |
| Apply house conventions | Naming, granularity, format, linkage to requirements | Machine, once told |
| Judge what matters | Which risks are real here, what to cut, what's missing | Human, non-negotiable |
In most teams, the middle three consume the overwhelming majority of the clock. They are enumeration and transcription — mechanical work performed by expensive people. That's the part that collapses.
The first step gets easier, because the tool forces the ambiguities in a story to the surface early. The last step doesn't get faster at all, and shouldn't.
That's the honest shape of the claim: a reduction in authoring time, not in thinking time. Anyone quoting you a figure that quietly includes the thinking is selling you something.
The fear is not that AI is bad at this. It's that it's confidently mediocre.
The real objection from experienced QA people isn't "the machine can't write a test case." It plainly can. The objection is subtler and better:
- It will produce volume that looks like coverage but isn't.
- It won't know that this particular payment flow has a regulatory edge case nobody wrote down.
- It won't follow our naming conventions, so everything needs rewriting anyway.
- Reviewing generated output will cost more than writing it ourselves.
- And when a test is wrong, nobody will be able to say why it exists.
Every one of those is a control problem, not an intelligence problem. Which is why "keeping your QA team in control" isn't the reassuring clause at the end of a marketing sentence — it's the part that determines whether the saving is real or imaginary.
What "in control" has to mean, concretely
Control is not a checkbox. It's a set of specific properties in the workflow. Ours look like this.
Nothing enters a suite without a human saying so. Generation produces a draft. A person reviews it, edits it, deletes what's noise, adds what's missing, and then promotes it. The gate is the default, not an optional strict mode. If a tool auto-commits generated cases into your regression pack, you don't have a QA process, you have a content farm.
Everything is editable text, not an opaque artefact. A generated case is a normal test case. You change the steps, the data, the expected result. There's no regeneration penalty, no "the AI owns this file now."
The refinement step comes before generation. A vague story produces vague tests — that's true of humans too. Refining the story into explicit acceptance criteria first, with the team correcting it, is where most of the quality comes from. It also surfaces the ambiguity while it's still cheap to fix, which is a benefit that has nothing to do with test cases at all.
Your conventions are configuration, not a wish. Granularity, naming, format, how much detail a step carries — these are set once and applied to every generation, so review is about content, not reformatting. The review that kills adoption is the one where a reviewer retypes every title.
It reads your context, not just the prompt. Generation grounded in your own requirements, existing suites and prior test cases produces something that looks like it came from your team. Generation grounded in nothing produces something that looks like it came from the internet.
Everything is traceable. Each case links back to the requirement it came from. That gives you two things reviewers actually want: the ability to ask "why does this test exist?" and get an answer, and a coverage view that shows which requirements have no tests at all — the gap that manual authoring is worst at spotting.
Permissions and audit are real. Who can generate, who can approve into a suite, who can delete, and a record of what happened. In a regulated environment this is the difference between a pilot and a rollout.
The workflow, end to end
- Story in — from Jira, a GitHub branch, a Figma file, a document, or typed straight in.
- Refine — ambiguity surfaced, acceptance criteria made explicit, team corrects it.
- Generate — cases across the paths, in your format, linked to requirements.
- Review — the QA engineer does the part only they can do: cut, sharpen, add the domain edge cases the story never mentioned.
- Promote — approved cases become a suite; automation and execution follow from there.
The shape of the engineer's day inverts. Most of it currently goes on transcription, and a thin slice on judgement. Afterwards, the transcription is largely handled and nearly all of the day is judgement. That's the actual claim, and it's a better one than raw speed.
What this does not fix
Worth saying plainly, because the limits are where trust is earned:
- Exploratory testing is untouched. Nothing here replaces a skilled person poking at a build with intent. Generated cases cover the specified; exploration finds the unspecified.
- Undocumented domain knowledge stays undocumented. If the reason a flow is risky lives only in a senior tester's head, the tool won't know it until someone writes it down. Review is where that knowledge enters.
- Test data strategy is still yours. Realistic, compliant, environment-appropriate data is a design problem, not a generation problem.
- Review time is not zero. It's real, it's the point, and it should be budgeted. Any saving worth talking about is the one left after review — everything else is a demo.
How to measure it in your own team
Since we won't hand you a number, here's how to produce your own. It's a cheap experiment.
- Take three stories of ordinary complexity from your last sprint.
- Have someone author test cases the usual way. Record wall-clock time, case count, and reviewer time.
- Run the same three stories through the generate-and-review path. Record the same three figures — including review.
- Compare on three axes: time per approved test case, requirements covered, and, a sprint later, defects those cases actually caught.
So the question I'd put to any QA lead reading this: if the transcription went away tomorrow, what would your team do with the attention it freed?
The teams that get the most out of this don't use it to write the same tests faster. They use it to finally cover the flows they've been knowingly skipping for two years.