Two days before a release, a question lands in the QA lead's inbox. It comes from a customer's compliance team, and it looks simple:
"For requirement PAY-112 — refunds over the limit need a second approval — which tests verify it, and when did they last pass?"
The QA lead — call them Meera — knows the refund flow well. They also know what answering properly involves. The suite has two thousand tests. The requirements live in Jira. The results live in CI. None of the three knows the others exist.
So Meera does what everyone does. Search test titles for "refund". Open each match and read the steps. Cross-check the last run in CI history. Copy it all into a spreadsheet.
The answer goes out on Friday afternoon, and it's more interesting than the question. Three tests mention refunds. One is really about partial refunds. One hasn't run since March. And the second approval — the actual requirement — isn't verified by anything at all.
Nobody did anything wrong here. The tests were written carefully. The requirement was written clearly. They were simply never connected, and connecting them by hand took most of a week.
The data exists. It was never joined.
This is the part worth sitting with. Every piece of the answer already existed:
- The requirement was in Jira, with an ID and acceptance criteria.
- The tests were in the suite, with steps and a business priority.
- The results were in CI, with a date and a status.
What was missing is the join between them. Three systems, three kinds of identifier, and no row anywhere that says "this test verifies that requirement".
Requirements traceability is an old idea — regulated industries have kept traceability matrices for decades. The usual form is a spreadsheet, maintained by hand, and the usual problem is that it's accurate on the day it's finished and slightly less accurate every day after that.
The five questions a matrix is for
It helps to be precise about what a traceability matrix is actually meant to answer, because each question needs a different join.
| The question | Who asks it | What has to be joined |
|---|---|---|
| Which tests verify this requirement? | Auditors, customers, the release call | Requirement to test |
| Did they pass on the build we're shipping? | The release call | Test to latest result |
| Which requirements have no tests at all? | The QA lead | Requirements with zero links |
| If we delete this test, what loses coverage? | Whoever is cleaning up the suite | Test back to requirement |
| Which tests does this change affect? | The pull request | Changed code to requirement to test |
The fourth one is quietly expensive. When nobody can say what a test protects, nobody feels safe deleting it — so suites only ever grow, and the maintenance bill grows with them.
Why hand-made links don't survive
A traceability link maintained by hand is a promise that someone will update a spreadsheet every time a test or a story changes. That promise gets broken in the busiest week of every quarter, which is exactly when the matrix is needed.
The fix is to stop treating the link as a separate task. A link should be a side effect of work the team is already doing. Generate test cases from a story, and the link from that story to those tests exists. Build an automation suite from a story, and the link to the suite exists — along with the real pass or fail status of every test in it, once it runs.
Nobody has to remember anything. The matrix stays current because it's built from the same actions that create the tests.
Coverage is not a percentage of tests
"We have 80% coverage" is one of the least informative sentences in software. 80% of what?
Code coverage measures lines that were executed. It says nothing about whether any requirement was verified. A test can execute the refund code path and assert nothing about the second approval.
Requirement coverage asks a different question: of the things the product is supposed to do, how many have at least one test attached? And two refinements make that number honest:
- Covered and passing, not just covered. A requirement with one linked test that last ran in March shows up as covered. It isn't verified on the build you're shipping.
- Weighted by business risk. An uncovered low-risk requirement is a backlog item. An uncovered critical one is the headline. Grouping requirements into Critical, High, Medium and Low — and counting the high-risk ones with no tests at all — turns a percentage into a to-do list in priority order.
Filling old gaps without inventing links
New work creates links automatically. The harder problem is the backlog — two years of tests written before anyone was linking anything.
AI can help here, carefully. For each uncovered requirement, find the covered requirements that are most similar to it, and propose their tests as candidate links. A requirement about refund approvals probably shares tests with a requirement about refund limits.
The important word is propose. Every suggested link is accepted or rejected by a person, because a wrong link is worse than a missing one. A missing link shows up as a gap, and someone investigates it. A wrong link makes the gap look covered, and nobody ever looks again.
A missing link is a visible gap. A wrong link is an invisible one.
From the matrix to the pull request
Once requirements are joined to tests, and tests are joined to their latest results, one more join unlocks something new: joining changed code to requirements.
Take a pull request's changed files. Map each file to the requirements it implements. Pull the tests linked to those requirements. Rank them by business risk, and take the smallest set that covers most of the risk the change touches.
We walked through what that looks like from a developer's afternoon in an earlier post, "It's a two-line fix. So why is CI running 2,000 tests?". What matters here is that none of it works without the matrix underneath. Test selection is only as good as the requirement-to-test links it reads.
And the same analysis produces two findings that are worth more than the time saved:
- Changed code with no tests. The pull request edits the refund approval logic. The matrix knows that requirement has zero linked tests. That isn't a failing test — nothing will go red — and that's exactly why it needs saying out loud before the merge.
- Files the matrix has never heard of. Changed files that map to no requirement at all are blind spots. Usually it's a shared utility or a config file. Sometimes it's a whole module nobody ever wrote requirements for, and that's worth finding on a Tuesday rather than during an incident.
Where this gets things wrong
Any honest version of this needs the limits stated plainly.
Mapping files to requirements is a heuristic. Matching file paths against requirement text works well when code and requirements use the same words. It struggles with shared utilities, renamed modules and configuration changes. A missed mapping means a missed test, so the full suite still belongs on merge and on a nightly run.
Story-level coverage is not criterion-level proof. Linking a test to a story says the story is exercised. Whether each individual acceptance criterion is checked is harder — the best available answer is an estimate from matching criteria against test steps, and it should be read as one.
Links are only as good as the requirements. A vague story produces vague matches, both for the file mapping and for the AI suggestions. Traceability doesn't fix requirement quality. It does make poor requirements much easier to spot.
Time estimates are estimates. "About 7 minutes instead of 41" is a count of tests multiplied by an assumption, unless per-test durations are recorded. Good for "minutes, not hours". Not good for a promise.
Try this before you buy anything
You can measure your own traceability gap in an afternoon, with no tooling at all:
- Pick your ten most critical requirements. The ones a customer, auditor or incident would ask about first.
- For each one, time how long it takes to name the tests that verify it and the date they last passed. Be strict: the tests have to actually check the requirement, not just mention it.
- Count how many took more than five minutes, and how many turned out to have no real test at all.
The first count is what a traceability question costs your team today. The second is your high-risk uncovered number, measured by hand.
"Which requirement does this test cover?" should be a lookup, not a project. The data to answer it is already sitting in three systems. Joining them is what turns a week of spreadsheet archaeology into a question the QA lead answers before the call has finished — and turns every pull request into a question the matrix can answer too.
