What it actually does, in plain terms
This is not a code-diff test runner. It does not read your git history and map changed files to tests. It works from your requirements: you give it a requirements or business document, and it produces the test scenarios that document implies, a coverage matrix over the features and test types it found, and a computed confidence for each scenario. From there it helps you decide which scenarios are worth running now.
The smallest suite that still covers the requirement
Given the generated scenarios and a time budget, the platform runs a set-cover selection: it repeatedly picks the scenario that adds the most new coverage per unit of time until the budget is spent. This is a greedy algorithm with a known optimality bound, so the result is a defensible minimal suite, not an arbitrary shortlist. You get back the chosen scenarios, what each one covers, and what was left out.
Why this test
Every scenario can be explained. The "why this test" view shows the risk factors behind it, an estimate of how many mutations it would catch, the confidence score, where it sits on the L1 to L5 test ladder, and a one-line human summary. Suite quality itself is scored by mutation kill rate against a target, so "is this suite any good" has a number, not an opinion.
What changed since last time
Re-upload a revised requirements document and the platform diffs it against the last stored version of the same document, at the model level: which requirement nodes were added, removed or changed. Only the delta needs new scenarios, so a requirement edit does not force you to regenerate everything. This is a document-revision diff, not a source-code diff.
Where the AI fits, and where it does not
Scenario generation and scoring are deterministic by default. An optional local model can enrich them, and it runs on the platform, so nothing is sent to an external AI provider. When the local model is unavailable, the endpoints return the deterministic result rather than a fabricated one. Smart test selection is part of the AI tools bundle, so access is decided by your plan.
What we do not claim
There is no source-code impact analysis today: the platform does not key a run to a commit and select the tests that commit can break. If that is what you need, say so on a demo call and we will tell you plainly whether and when it is planned, rather than imply it exists. We also publish no pipeline-time reduction figure, because it depends entirely on your suite; measure it on yours.
Questions people ask about this
- Does it pick tests from a git commit?
- No. It works from a requirements document, not from source changes. The "what changed" view diffs a re-uploaded document against its previous version, not a code diff.
- Is the AI required?
- No. Generation and scoring are deterministic; a local model can enrich them and degrades gracefully to the deterministic output when it is off or busy. No external AI provider is called.
Source: /documentation/smart-test-selection, updated 2026-09-18. Answers are checked against the shipped product on that date.
Found something out of date? The people who build the module answer the contact form on the home page, and corrections to this guide are dated above. Tell us.