Every conversation about moving to a test automation platform reaches the same question, usually about twenty minutes in: what happens to the 900 tests we already have?
It's the right question. Those 900 specs are four years of accumulated knowledge about how the product actually behaves — that checkout needs the cookie banner dismissed first, that the reports page takes eleven seconds on staging and always has, that there are three separate ways a discount code can be invalid and only two of them show a message. Nobody wants to type that in again.
So the platform says: we'll import them. And it does. Eleven minutes later there are 900 test cases in the new system, everyone agrees the migration risk was overstated, and the meeting moves on.
Then someone — let's call them Meera, who has been maintaining that suite since 2022 — opens one.
Step 1: loginPage.signIn(user)
Step 2: cartPage.addItem("SKU-4471")
Step 3: checkoutPage.completePurchase()
Expected: Order confirmation is displayed
Meera knows exactly what those three lines do. That's the problem. Nobody else in the company does. No manual tester can pick that up and run it. And an engine that resolves locators from plain language has just been handed three method names it has no way to resolve, because the thing they refer to lives in a file that didn't come along.
What arrived in the new system isn't a test case. It's a directory listing of the old one.
Moving a file is not a migration
There are two honest reasons to import an existing suite, and neither of them is "so the tests exist in two places".
The first is that you want the platform's engine to run them — which means the steps have to be in the platform's native form, not in yours. If the import produces a script the platform stores and shells out to, you haven't migrated anything. You've added a second place to look when it breaks.
The second reason is the one people underestimate: everything else the platform does is keyed to test cases, not scripts. Self-healing locators only heal a step the engine resolved in the first place. Requirements traceability needs a test case to link to a requirement. Smart test selection needs to know what a test covers in order to decide whether a diff touches it. A blob of imported Playwright is opaque to all three.
So the output of an import has to be a real test case — the same shape as one written by hand in the platform, indistinguishable from it afterwards. Which means somebody has to turn code into behaviour. That's the whole job, and it has exactly one hard part.
Page objects are the hard part
The page object is the best idea in UI automation. It is also precisely what makes a suite unreadable to anything that isn't the suite.
loginPage.signIn(user) is not a step. It's three steps living in another file:
| What the spec says | What actually happens |
|---|---|
| loginPage.signIn(user) | Navigate to /login · Enter the email in the Email field · Enter the password in the Password field · Click the Sign in button |
| cartPage.addItem("SKU-4471") | Search for "SKU-4471" · Click the first result · Click Add to cart · Verify the cart badge reads 1 |
Four years of good engineering has pushed every concrete detail — every URL, every field label, every wait — one file away from the test that depends on it. Any importer that reads only the spec file is reading the least informative file in the repository.
So don't read only the spec file. Follow its imports, pull in the page objects, fixtures, custom commands and helpers it actually uses, and hand those to the model alongside the test. Then flatten: wherever the extractor can see a helper's body, the call is replaced by the concrete actions it performs.
The rule that makes it useful is a negative one. No class or method name survives into a step. Not "call signIn on the login page", not "use the LoginPage helper" — those are the same failure wearing a sentence. A step reads Navigate to "/login", or it isn't done. The test you get back is one a manual tester can execute by hand on a laptop, which is a decent test of whether the flattening worked at all.
Now the part nobody demos
Sometimes the helper can't be followed.
It happens more than you'd think, and for boring reasons: the helper lives outside the folder you pointed the import at. It's inherited from a base class two levels up. It's built at runtime from a string. The spec is enormous and the tail of it didn't fit in one pass. Or the call is real and visible but its behaviour depends on data the importer doesn't have.
At that moment an extractor has two options, and this is the entire quality difference between one importer and another.
Option one: write a plausible step. The method is called completePurchase, so emit "Complete the purchase" and move on. It reads fine. The import comes back 100% complete. Every test case looks equally trustworthy in the list.
That last sentence is the whole cost. A test case that reads perfectly and executes wrongly is worse than one that's obviously incomplete, because it consumes the reviewer's trust rather than their attention. And there is no way to find them later — nothing marks the difference between the 830 cases where the tool actually knew and the 70 where it was writing prose from a method name.
Option two: write down what you couldn't follow. The call goes into an unresolved list on that test case, verbatim, exactly as it appeared. The confidence score on that case drops. The review screen says so, in the specific: Could not follow: checkoutPage.completePurchase() — the steps for these were inferred from the call name. Review before automating.
We took option two, and the deciding argument was not accuracy. It's that option one produces a number nobody can act on, and option two produces a worklist.
An import you can't trust costs more than no import at all — you now have 900 test cases and no idea which ones to read.
Confidence has to be taken, not asked for
There's a trap in the middle of this that's worth naming, because it isn't specific to test migration.
If you ask a model to rate its own output, it will tell you 0.9. Ask it about work it did badly and it will tell you 0.85. Self-reported confidence rises with fluency, not with correctness, and fluency is exactly what a model has when it's writing a plausible step from a method name.
So the extractor asks for a confidence score, and then doesn't trust it. The importer clamps it in code, from facts it knows independently of what the model claimed:
| Situation the importer can see for itself | What it does to the score |
|---|---|
| The case has unresolved helper calls | Capped, and lowered further for each one |
| The source file was too large and got truncated | Capped at 0.6 |
| The case came back with no steps at all | Capped at 0.3 |
None of those require the model's cooperation. They're conditions the pipeline observed while doing the work, applied afterwards to a number the model was too generous with. It is a small amount of code and it's the difference between a confidence column that means something and one that's decoration.
What review actually looks like
The reviewer's job is not to read 900 test cases. It's to find the ones worth reading, which is why the review screen leads with counts rather than content: how many cases, how many with unresolved helpers, how many under 60% confidence.
From there it's three controls and a checkbox:
- Show only low-confidence / unresolved — collapses 900 cases down to the ones with a question attached, usually a small fraction.
- Reject low-confidence — one click rejects everything below 60%, so the confident majority can go through and the rest waits for a human.
- Edit in place — the title, steps, expected results, preconditions and test data are all editable before anything is created, because the fastest fix for an inferred step is usually a reviewer who knows the answer typing it.
A rejected case simply doesn't get created. Nothing is hidden and nothing is silently dropped; the counts at the top always add up to what you scanned.
The unglamorous half that has to work
Flattening is the interesting problem. These are the ones that decide whether the import is usable on Monday:
- Data-driven tests stay one test case. A parametrize block, a cy.each, a Scenario Outline or a TestNG DataProvider with fifty rows becomes one test case with fifty rows of test data attached — not fifty near-identical test cases that will be maintained fifty times. The steps reference the columns.
- Setup becomes preconditions. beforeEach, fixtures, Background, @BeforeMethod: none of those are test cases. They're the state each affected case needs, so they land as preconditions on the cases themselves.
- Skipped tests come across, tagged skipped. A suite's skipped tests are information about the product. Dropping them silently rewrites history.
- Grouping blocks are grouping blocks. A describe or context that only groups tests contributes to purpose and tags. It does not become a test case of its own.
- Your folder tree becomes the folder tree. The repo's directory structure is mirrored as nested folders, one suite per file. Nobody has to re-learn where anything is.
- BDD stays BDD if you want it to. A Cucumber or Robot suite can land as Gherkin scenarios, or be converted to traditional steps — your call at review time, not the importer's.
And the run-order details, which matter more than they sound: preflight is free and shows you the frameworks, the folders and an estimated test count before anything is spent; billing is per test case as results land, so a job you cancel halfway has only paid for what it produced.
Where this gets it wrong
Any honest version of this article needs this section.
Flattening throws away your abstraction, on purpose. That's the trade, stated plainly: you had one page object that 200 specs shared, and now you have 200 independent test cases. Locator drift is handled — the engine resolves elements at run time, so a renamed button doesn't break anything. A genuine flow change is different: it used to be one edit in signIn, and now it's a change across the cases that use it. Suite prerequisite steps take some of that weight back, but not all of it. If your suite's main value is that one edit fixes 200 tests, be deliberate about what you're moving.
Re-importing doesn't merge. Within one import job, files already turned into suites are left alone. But running a fresh import of the same repository later creates a new set of suites rather than diffing against the old one. Incremental re-import is not built.
Assertions inside helpers land as steps. If your page object asserts as well as acts, flattening turns that assertion into a "Verify ..." step in the middle of the case rather than a clean expected result. It's correct, and it's untidy.
The confidence score is triage, not measurement. Sixty percent is a threshold we chose because it usefully separates two piles in review. It is not a probability that the test case is right.
And we haven't measured this on your repository. The only end-to-end runs so far are our own and a public sample project. We could publish an extraction-accuracy figure computed on a repository we chose, and it would be worth nothing. When there's a number from real customer suites, we'll publish that instead.
Three questions for any tool that says it imports your tests
Ours included. All three are answerable in a demo, with your own repository, in about ten minutes:
- Show me a test case that came from a page-object call. If a step contains a class name or a method name, the tool moved your code — it didn't read it. Ask a manual tester in the room to execute the case out loud.
- What did you do with the helper you couldn't find? Every real repository has some. If the import came back with everything at high confidence and nothing flagged, that's not a good sign about the repository; it's an answer about the tool.
- What do I own afterwards? Test cases the platform's engine executes, heals and traces — or scripts in a new wrapper, with the maintenance problem intact and one more place to look when something breaks.
The 900 tests were never the asset. What the team knew when they wrote them was. An import is worth doing when it moves the knowledge — and it's worth trusting when it's willing to tell you the parts it couldn't read.
