Short, plain definitions of the terms used across the platform, its documentation and the comparisons. Each one links to the guide that goes deeper. If a definition is wrong or missing, tell us.
Apdex (Application Performance Index) turns response times into one score between 0 and 1. You choose a target time T; responses within T count as satisfied, responses up to 4T as tolerating, and slower ones as frustrated.
The score is (satisfied + tolerating / 2) / total. It is easier to put on a dashboard than a percentile, and it hides the tail, so read it next to P95 and P99.
Load-test runs report Apdex in real time alongside P50, P95, P99 and P99.9.
A contract test checks that an API still answers in the shape its consumers depend on: the fields, their types, the status codes and the error format.
It catches the breaking change a unit test cannot see, because the break is between two systems rather than inside one.
In the API studio, schema assertions check the shape of a response against the OpenAPI or GraphQL schema the studio reads, and a failed assertion fails the run.
A confirmed finding is a security result the scanner could prove: it carries the request that triggered the behaviour, the response that shows it, and a proof of concept a reviewer can replay.
An observation is a pattern that looks risky but was not proven, such as a missing header or a version string. Observations are reported, labelled as such, and never fail a quality gate on their own.
The distinction is what makes a security report usable by an engineer who was not in the room.
Core Web Vitals are the three page-experience measures Google grades a page on: Largest Contentful Paint (loading, good at or under 2.5 seconds), Interaction to Next Paint (responsiveness, good at or under 200 milliseconds) and Cumulative Layout Shift (visual stability, good at or under 0.1).
They are graded on real visitors at the 75th percentile over 28 days, so a lab score is for debugging, not for grading.
The website analysis module measures the lab values from one real page load, and names what it could not measure.
DAST tests a running application from the outside, the way an attacker would: it sends requests and inspects responses, without reading the source code.
It finds what only exists at runtime, such as an injection that works through a particular parameter, or an authorisation check that is missing on one route.
Our scanner is a DAST engine with one rule most scanners lack: active tests run only against targets whose ownership you have proven.
Element drift is a change in where or how an element renders, such as a button moving, shrinking or changing colour, without a change in the pixels around it that a screenshot comparison would catch.
It is measured on the DOM: position, size and computed style of named elements between a baseline and a later version.
The visual regression module watches drift on a second tab next to the pixel comparison.
An end-to-end test drives the application through a real browser the way a person would: open a page, fill a form, submit it, check the result.
It is the slowest and most realistic kind of automated test, so teams keep a small set of them for the journeys that matter and cover the rest with API and unit tests.
A recorded flow is an end-to-end test; the same recording also feeds the load profile, the security scan and the accessibility audit.
An evidence bundle is the package attached to a confirmed security finding: the exact request and response, the proof of concept, the scan scope, and an integrity badge that shows the bundle was not edited after the scan.
It lets a reviewer, an auditor or a vendor reproduce the finding without re-running the scan.
A flaky test passes and fails on the same code without any change in between. The usual causes are timing (the test asks before the page is ready), shared state between tests, order dependence, and an environment that differs from run to run.
Flakiness is expensive because it teaches a team to ignore red builds. Re-running until green hides the cause; the fix is to find it.
A locator that stops matching is one cause, and it is the one self-healing with human review addresses.
Heal review is the queue where a proposed locator repair waits for a person to approve or reject it.
When a recorded step cannot find its element, the platform tries more than fifteen fallback strategies and proposes the best match. Nothing changes in the test until someone accepts the proposal; approvals teach the element repository so the same fix is not proposed twice.
Heals cannot be auto-approved, by design.
A load test applies the traffic you expect and measures how the system responds. A stress test raises load until something breaks, to find the limit. A soak test holds a steady load for hours to find leaks and slow degradation.
All three report the same numbers: throughput, error rate and latency percentiles.
Here a load profile comes from a recorded flow or a plain-language description of the traffic you expect, and runs within your plan’s virtual-user cap.
A locator is the rule a test uses to find an element: by role and accessible name, by a test id, by label, by CSS selector or by XPath.
Stable locators describe what the user sees (a button named Save); brittle ones describe where the element happens to sit in the markup (the third div in the second section).
The recorder captures every locator available for each element and orders them by stability, which is what the fallback strategies draw on.
A mock server answers API requests with responses you define, so a test can run without the real dependency being available, fast, or willing to return the error you want to test.
It is how a front-end test checks the behaviour on a 500 or a timeout without breaking production to get one.
The API studio includes a mock server that serves responses from the same collections you test.
The OWASP API Security Top 10 is a list of the ten most common classes of API vulnerability, maintained by the Open Worldwide Application Security Project. The current edition is 2023.
It leads with object-level and function-level authorisation failures (one user reading another user’s record by changing an id), which traditional web scanners rarely find.
The scanner covers it alongside the OWASP Top 10, and reports each result as confirmed or observed.
The OWASP Top 10 is the most widely used list of web application security risks, updated every few years. The 2025 edition adds software supply chain failures and the mishandling of exceptional conditions, and folds server-side request forgery into broken access control.
It is a vocabulary for talking about risk, not a complete test plan; a scan that reports in its terms is easier for a security team to read.
P95 latency is the response time that 95 percent of requests beat; P99 is the time 99 percent beat. They describe the slow tail that an average hides.
If the average is 200 milliseconds and P99 is 4 seconds, one request in a hundred is waiting 4 seconds, and that is the one the customer remembers.
Load-test results report P50, P95, P99 and P99.9 together, in real time.
Pass^k is the probability that a system passes a test on every one of k independent trials. Pass@k, the more common number, is the probability it passes at least once.
Pass@k is a capability ceiling: it rises as you retry. Pass^k is a reliability floor: it falls as you retry, and the floor is what a customer actually gets.
The LLM assurance suite reports Pass^k floors separately for attack traffic and for benign traffic, with confidence intervals, rather than one blended accuracy.
Prompt injection is text in a language model’s input that tries to override the instructions it was given: directly, in a user message, or indirectly, inside a document or web page the model was asked to read.
It is the top risk in the OWASP Top 10 for LLM applications because retrieved text and instructions arrive in the same channel. The defence is to treat every retrieved string as data, never as authority, and to authorise tool calls on their own merits.
The LLM testing surface probes for both forms and reports what the gateway blocked, with the evidence.
A quality gate is a rule in the delivery pipeline that stops a release when a condition is met: a confirmed security finding above a severity, a failed regression step, an accessibility finding at a level you chose, or a load-test regression.
A gate is only useful if its inputs are trustworthy, which is why observations and unproven results never trigger one here.
A regression test re-checks behaviour that used to work, so a change elsewhere cannot break it unnoticed.
The value of a regression suite is in its signal: it must be fast enough to run on every change and stable enough that a failure means something.
Every recording replays as a regression test; smart test selection picks the subset worth running for a given change.
A self-healing test repairs its own locators when the page changes, instead of failing on the first renamed id.
Done badly, self-healing hides real defects by silently accepting any element that looks similar. Done well, it proposes a repair with its reasoning and lets a person decide.
Here a proposal is never applied automatically; it waits in heal review.
Smart test selection picks the smallest set of tests that covers what a change or a requirement touches, within a time budget, instead of running everything every time.
Each pick carries a written rationale and a coverage-confidence score, so a reviewer can see why a test was included or left out.
We publish no pipeline-time percentage for it; the saving depends on your suite, so measure it on your own pipeline.
Synthetic test data is generated, realistic and fictitious: names, addresses, identifiers and amounts that look right, validate correctly, and belong to nobody.
It lets a staging environment run without a copy of production personal data, which is the simplest way to keep test environments out of scope for a privacy regulator.
The data factory generates it from a topic, a schema or a parent-child relationship, seeded so a run is reproducible, with checksum-valid national and financial identifiers.
A test recorder captures a browser session as a replayable test: each click and input, the element it targeted, the locators available for it, the network calls it caused, and any assertion you add.
The recording is the shared input for the rest of the platform: the same session becomes a load profile, a security scan scope and an accessibility audit.
Recordings export as Playwright, Selenium or Robot Framework code, so nothing recorded here is locked in.
A virtual user is one simulated client in a load test, running the flow on its own timeline. Ten thousand virtual users means ten thousand concurrent sessions, not ten thousand requests.
Per-plan caps on virtual users per run are enforced by the platform, not printed as marketing: 1,000 on Solo Pro, 10,000 on Team and 30,000 on Enterprise. Load testing is not included on the Free plan.
Visual regression testing compares a screenshot of a page against a baseline and reports what changed: a percentage, a pass or fail against the threshold you set, and an image that highlights the pixels that moved.
It catches what functional tests ignore, such as a layout that collapsed or a button that is now invisible. It also flags intended changes, so the baseline is updated on purpose.
A VPAT (Voluntary Product Accessibility Template) is the form; an ACR (Accessibility Conformance Report) is the completed document that states, criterion by criterion, how a product conforms to WCAG, Section 508 or EN 301 549.
Procurement teams ask for it before a purchase. One written from a template says little; one generated from a real audit says what was tested and what was not.
The accessibility module generates a VPAT from an actual scan, with criteria a machine cannot judge marked for manual review rather than auto-passed.
WCAG 2.2 (Web Content Accessibility Guidelines) is the W3C standard that defines what an accessible web page must do, at three levels: A, AA and AAA. Most laws and contracts require level AA.
The 2.2 edition added criteria for focus appearance, dragging alternatives, minimum target size, consistent help and accessible authentication. The European Accessibility Act and EN 301 549 reference it.
Audits here run in a real browser with axe-core against 2.0, 2.1 and 2.2 at levels A and AA.
We use one optional analytics tool, run on our own servers, to see which pages help people. It stays off unless you turn it on. Storage needed to sign you in, keep the site secure and remember this choice is always on. Cookie Policy