Tools compared · updated 2026-11-10 · method stated · no tool ranked first
Test automation tools compared, by the job they do
20 tools that come up in the same evaluation, compared by the job each is built for rather than by a rank. Every cell about another vendor comes from that vendor’s own pages on the date shown; Dokimore is our product and is listed with the same fields as the others, in alphabetical order, never first.
The short answer, by team
You only test APIs
An API client is enough. Postman is the category reference; Bruno, Hoppscotch and Insomnia are open-source clients; Dokimore’s API studio imports all four formats and adds gRPC and SOAP, but earns its place only when the same team also needs UI, load, security or accessibility evidence.
Playwright or Selenium. Both are free, both are code-first, and a platform adds cost without adding much for a team that already runs its own harness. Dokimore exports to both, which is the exit path if you try it.
Look at what each tool does with a repair: apply it, show it, or hold it for approval. Katalon heals during the run and keeps the new locator only after someone approves it; TestMu AI and BrowserStack ship automatic healing; Dokimore rescues the step during the replay and shows each rescue in the run detail. Decide first whether a repair must be approved before it sticks.
You need real mobile devices or hundreds of browser combinations
A device cloud: BrowserStack or TestMu AI. Dokimore has no real mobile devices and one execution region; it can export generated code that then runs on a cloud grid.
A release needs load, security and accessibility evidence, not only UI tests
A unified platform (Katalon, Dokimore) or a stack of single-purpose tools (Apache JMeter or k6, ZAP or Burp Suite, axe). The platform trade is one bill and one recorded flow against the best tool per job.
k6, Gatling, Locust and Artillery model load as code; Dokimore keeps the thread-group model behind a UI, and a profile set up by hand on its load screen exports as a k6 script. None of them loads a .jmx file, so plan a rewrite per plan, not a conversion.
Modelling almost any load profile with thread groups, timers, assertions and samplers, at no licence cost, by engineers who think in its terms; extensible with plugins.
Free, open source (Apache License 2.0).
.jmx XML plans edited in a desktop GUI; percentiles and Apdex in an HTML dashboard report generated after a run rather than live; load plans kept separate from UI test suites.
Visual AI testing platform (Eyes SDKs, Autonomous)
Visual checkpoints added to an existing code-based test framework and rendered across browsers on its grid, plus a no-code Autonomous product; contrast checks against WCAG 1.4.3, 1.4.6 and 1.4.11.
Tiered subscription (Starter, Professional, Enterprise) with unlimited users; Starter is sized in checkpoints, Professional and Enterprise in active pages and components; the pricing page lists a free trial and no permanent free plan.
Focused on visual and functional UI testing, with API testing and a contrast-based accessibility check also documented; we found no load-testing or security-scanning feature described. Vendor-hosted by default, with a dedicated-cloud or on-premises option on Enterprise.
Load testing CLI (open source, MPL-2.0 core) with Artillery Cloud
Load tests as YAML or JavaScript in a repository, browser-level load through a built-in Playwright engine, and serverless distributed runs from your own AWS account (Lambda or Fargate); running on Azure Container Instances beyond evaluation requires an Artillery subscription.
CLI source on GitHub: most code under MPL-2.0, Azure-specific modules under the Business Source License; Artillery Cloud metered by recorded test reports, with a Free plan; enterprise add-ons listed with a starting monthly price.
Its docs describe it as designed for testing backend systems, and its welcome page also lists Playwright end-to-end testing and synthetic monitoring; browser load runs on Chromium only; its documentation index lists no accessibility testing, and its CLI reference no HAR, Postman or OpenAPI import command.
Running tests you already own across thousands of real browsers and mobile devices; it also ships visual review (Percy), accessibility testing with real screen readers and, generally available since 14 October 2026, load testing from JMeter, k6, Gatling or Locust scripts and Playwright, Selenium, WebdriverIO or Nightwatch tests.
Live by number of users and Automate by parallel tests; other products priced separately, with some bundles; single sign-on in an Enterprise add-on.
We found no application security scanning in its product list or pricing; load, accessibility and functional testing are separate products rather than one recorded flow.
Web application security testing toolkit and DAST scanner (PortSwigger)
Manual penetration testing with the Proxy, Repeater and Intruder toolkit, and crawl-and-audit scanning of web applications and APIs from an OpenAPI, WSDL, Postman or GraphQL definition, on the desktop in Professional or at enterprise scale in DAST.
Community Edition free; Professional as a per-user subscription; DAST on a tailored quote.
Built for web application security testing; its documentation describes no UI test automation, load, accessibility or visual testing. Each DAST finding carries the request and response evidence its scanner captured; the desktop editions are local installs.
Open-source test runner (MIT) with Cypress Cloud (SaaS)
JavaScript and TypeScript end-to-end and component tests that run inside the browser, with Cypress Studio recording, API calls through cy.request, and Cloud features for replay, analytics and UI coverage.
The App is free and MIT-licensed; Cypress Cloud has a free Starter plan and paid Team, Business and quote-only Enterprise plans metered by test results.
Its docs say it is not the optimal tool for load or performance testing, and it has no security scanner of its own; accessibility checks come from community plugins or the paid Cypress Accessibility add-on, and visual comparison from plugins or third-party services; self-healing is described for cy.prompt steps (beta, Chromium-based browsers, a Cloud account required); Cloud is SaaS only, with no self-hosted option.
Unified platform (recorder, API, load, security, accessibility, visual, test data)
Teams that need UI, API, load, OWASP security and WCAG 2.2 evidence from one recorded flow, with every rescued locator shown in the run detail and test intelligence kept on local models inside the tenant.
Four flat tiers: Free $0, Solo Pro $39 a month, Team $249 a month for up to 16 seats, Enterprise from $799; no metering, no overage.
No real mobile devices; one execution region (India) and no on-premise installer yet; no SOC 2 report yet; locator rescues are accepted during the replay with no customer review queue yet, so someone should check the steps a run marks healed; a newer platform with a smaller ecosystem.
Dokimore product pages
Gatling
Load testing as code (Apache-2.0 engine) with Gatling Enterprise Edition
Load tests written in Java, Kotlin, Scala, JavaScript or TypeScript with HTTP, WebSocket, SSE, GraphQL, JMS, MQTT and gRPC scenarios, run locally for free or distributed from managed regions or your own cloud on Enterprise Edition.
Community Edition free (Apache-2.0 engine; the gRPC, MQTT, GraphQL and Postman components carry a separate component licence with limits); Enterprise Edition metered in credits, one credit being one minute of one load generator, with a quote-only top plan.
A load-testing tool: its Recorder and Studio turn browser traffic into load scenarios, and we found no UI-functional, accessibility, visual or security testing described; per its own comparison page, Community Edition runs locally with static HTML reports per run and no history or comparison; Studio exports Java and Maven projects for now.
Developer-owned load tests as JavaScript or TypeScript in git, gated in CI by thresholds, with a Chromium browser module and a managed runner that correlates results with Grafana metrics, logs and traces.
Open source (AGPL-3.0); Grafana Cloud k6 metered in virtual-user hours with a free-tier quota, a self-serve Pro plan and a quote-only Enterprise plan.
Load testing by scope: we found no security-scanner, visual-regression or accessibility-audit feature in its documentation; the browser module drives Chromium-based browsers; the vendor archived its JMeter and Postman converters in June 2026.
A longer-established unified platform with its own plugin store, for web, mobile, API and Windows desktop tests built by recording, from keywords, or in Groovy and Java.
A free edition for authoring and running tests locally, with no time limit stated on its pricing page; Professional per seat, annual billing on its pricing page; Enterprise by quote; command-line and CI runs need Runtime Engine licences, included for 30 days in the trial.
No built-in security scanner described on its product pages; accessibility through an axe-core custom keyword and load testing through OctoPerf, both documented by Katalon as not officially supported; built-in AI uses OpenAI, Anthropic or AWS Bedrock, with your own provider on Enterprise; no documented export of Studio test cases to plain Selenium or Playwright code (Katalon Recorder exports to Selenium-based code).
Load testing framework in Python (open source, MIT)
Load scenarios as plain Python with real program logic, scaled across cores and machines, run headless in CI with threshold exit codes, and self-hosted with pip, Docker or a Kubernetes operator.
Free, MIT. Its changelog (2.43.0) notes that the maintainers’ hosted service, locust.cloud, is shutting down, and the docs point to Microsoft’s Azure Load Testing for hosted runs.
HTTP built in; other protocols through experimental contrib clients or a custom client. HttpUser is not a real browser, and browser-driven users live in the separate locust-plugins package; its docs describe no accessibility, visual or security testing; no built-in recorder or codeless mode, with har2locust (beta) generating locustfiles from a HAR recording.
Agentic and low-code SaaS test automation platform
Hosted browser, mobile, API, accessibility and load tests authored by recording or from natural-language prompts, with auto-heal, failure analysis and test-impact analysis, wired into GitHub, GitLab, Jira, Slack and coding agents through a vendor-hosted MCP server.
Quote-only; cloud runs consume credits (allocated per year on current plans, with no rollover); local and CI-runner runs unmetered; a free trial.
Security scanning does not appear on its capability lists; API testing covers REST, SOAP and GraphQL but not gRPC or WebSocket at the transport layer; all customer data is stored in the United States; Playwright and Selenium import and export carry stated gaps.
Browser automation library and test runner (Microsoft, open source)
Code-first end-to-end testing in TypeScript, Python, .NET or Java across Chromium, Firefox and WebKit, with auto-wait, a Trace Viewer, Codegen, parallel runs by default and an MCP server for AI agents.
Free, Apache-2.0.
A library and test runner rather than a hosted platform: it ships reporters, API testing, visual comparisons and AI test agents, while scheduling, hosted capacity beyond your machines or CI, and load and security testing come from other tools.
API-first teams: Postman reports more than 50 million developers and 500,000 companies, its collections use an open-source format, and teams can publish public workspaces on the Postman API Network.
A free plan; paid Solo, Team (per user) and Enterprise (by quote) plans.
Centres on APIs: performance tests run from the Collection Runner (cloud runs on paid plans); UI tests are generated as Playwright code through Agent Mode and the Browser Tool; security checks focus on API definitions; accessibility testing is not described in its documentation.
Device and browser cloud (real and virtual devices) for Selenium, Appium, Playwright, Cypress, Espresso and XCUITest suites, with AI test-authoring and visual-testing products alongside
Running Selenium, Appium, Playwright, Cypress, Espresso and XCUITest suites on real devices and virtual browsers in several data-centre regions, with live testing and error reporting around them.
Subscriptions by concurrency tier (Live, Virtual Device Cloud, Real Device Cloud), monthly or annual; the pricing page lists paid plans and a free trial, and no permanent free plan.
Its core product runs the tests you bring, and Sauce AI for Test Authoring (Enterprise level) creates tests from natural-language prompts; its codeless product reached end of life in September 2025; its docs describe front-end performance testing rather than server load testing, and list no security-testing product; automated accessibility checks through Deque axe tooling; its FAQ describes Sauce API Quality while its products navigation lists no API-testing product.
Full control of a code-first stack with core bindings for Java, Python, C#, Ruby and JavaScript, two decades of community knowledge, and a self-hosted Grid, at no licence cost.
Free, open source (Apache 2.0).
WebDriver automates the browser and Grid runs tests in parallel on machines you host; test reporting, retries, test data and result dashboards come from frameworks and tools you add and maintain.
Natural-language test authoring that exports to Selenium, Playwright, Cypress or Appium, a real-device cloud, and execution that runs the script and the browser on one machine (HyperExecute), with SOC 2 Type II and ISO 27001 attestations published on its trust page.
Separately metered products (credits, parallel tests, minutes, scans, screenshots, virtual-user hours) with free tiers on most; SSO on Enterprise; private cloud for HyperExecute.
We found no OWASP web-application scanning on its pages (its Agent Assurance tests AI agents for prompt injection); it states it does not provide a VPAT; generative AI runs through third-party providers (OpenAI, Microsoft Azure) rather than local models.
Plain-English, locator-free test automation (SaaS)
Manual QA and non-engineers writing end-to-end tests in plain English across web, mobile, Windows desktop, API, e-mail, SMS and 2FA flows, run on the vendor’s servers with opt-in AI healing that labels what it changed.
Purchased parallel capacity rather than seats or runs: a free Public plan where every test and result is public, a paid Private plan with a trial, and quote-only Enterprise.
Execution is vendor-hosted unless you arrange its on-premise option, and AI is processed externally; exports are testRigor’s own language, with no Selenium or Playwright code export described; API steps are HTTP calls and its docs do not name GraphQL, gRPC, WebSocket or SOAP; we found no security scanner on its pages; its accessibility pages name no WCAG level.
Codeless and agentic test automation platform (SaaS, enterprise on-premise)
Natural-language and recorder-authored tests for web, mobile, API, desktop and enterprise apps such as Salesforce and SAP, with auto-healing, visual and WCAG-tagged accessibility checks, and a long list of CI tool guides.
Quote-only Pro and Enterprise editions; the vendor states paid plans scale with parallel runs and seats; a free trial.
Load testing and security scanning are not on its product, plan, docs or changelog pages (its load-testing guide lists Testsigma in a comparison table, so not checked rather than not offered); hosting is AWS in the United States per its DPA; API testing is REST and SOAP with GraphQL as a body type.
Web application security scanner (open source, "ZAP by Checkmarx")
A free, self-hosted DAST scanner with passive and active scans, a spider, AJAX spider and Client Spider, packaged Docker scans driven by an OpenAPI, SOAP or GraphQL definition, an automation framework for CI and an add-on marketplace, for a team with a security engineer to triage its alerts.
Free, Apache-2.0.
Alerts are potential vulnerabilities, each tied to the request that raised it and rated by risk and confidence for a person to review; ZAP does not mark them confirmed itself; no ownership verification before an active test (its docs ask for permission and offer a scope-limited Protected mode); security testing only.
Device and browser cloud (real and virtual devices) for Selenium, Appium, Playwright, Cypress, Espresso and XCUITest suites, with AI test-authoring and visual-testing products alongside.
Conflict of interest, stated: Dokimore is our product. It appears in the table with the same fields as every other tool, in alphabetical order, and is never ranked first; its own shortcomings are listed in the same column as everyone else’s.
What we read, not what we ran: every competitor cell comes from that vendor’s own product, pricing and security pages on the date shown in its row (the same audits that back our head-to-head comparison pages). We did not run the other tools to score them, and we say "not confirmed" or "we found no …" rather than guess.
No competitor prices: prices change and a stale number is a false claim. The table states each tool’s billing model; our own printed prices are on the pricing page.
Jobs, not ranks: "best for" names the job a tool is built for and why. No tool on this page is the right choice for every team, and none is placed above the others.
Dated and corrected: each row carries the date its claims were last checked and links the vendor page they were read from; every page has a correction link, we reply within five business days, each page lists what was corrected and when, and claims are re-read at least every quarter.
Questions buyers ask
Which test automation tool is best?
None of them for every team. The table above names the job each tool is built for: an API client for API-only work, Playwright or Selenium for code-first teams, a device cloud for real mobile devices, a scanner for security, a unified platform when one recorded flow has to produce several kinds of evidence. Pick by the job, then measure on your own application during a trial.
Why is Dokimore not ranked first on its own site?
Because a vendor roundup that ranks the vendor first is marketing, and answer engines and reviewers treat it as such. Dokimore is listed with the same fields as every other tool, in alphabetical order, with its own shortcomings in the same column. The head-to-head comparison pages say when the other tool is the better choice.
Where do the facts about other tools come from?
From each vendor’s own product, pricing and security pages, read on the date shown in the row, and from the audited rows of our comparison pages. We did not run the other tools to score them. If a row is out of date, the correction link at the foot of each comparison page reaches the people who maintain it.
Which of these are AI testing tools?
Five rows put AI in the product itself, which is what most people mean by AI test automation or AI QA: Applitools (visual AI checkpoints), mabl and Testsigma (agentic, natural-language and recorded authoring), TestMu AI (natural-language authoring that exports to code frameworks) and testRigor (tests written in plain English). Dokimore runs its own intelligence on local models inside the platform and tests AI applications through its LLM testing module; its heals come from fallback locators, are applied during the replay and are shown in the run. "AI testing tools" covers both meanings, so ask a vendor which one it means.
Are automated software testing tools and QA automation tools the same thing?
Mostly, yes: both names mean tools that run software tests by program instead of by hand. The split that matters is the job, as in the table: API clients, code frameworks, load tools, security scanners, device clouds and unified platforms. A QA team usually needs more than one kind, which is why the table groups them by job instead of ranking them.
Do these tools import each other’s assets?
Partly. Dokimore imports Postman, Bruno, Insomnia and Hoppscotch collections, OpenAPI specifications and cURL commands, with an import report of what was skipped or translated. It does not import Selenium, Cypress or Playwright test code or JMeter plans: UI flows are recorded again and load plans rebuilt. It exports Playwright, Selenium, Robot Framework and pytest code, and load profiles set up on the load screen as JSON and as a k6 script. It does not import ZAP reports, and the device clouds run the code you bring rather than import it.
What changed on this page
Our own rows updated after a change in the product: a load profile now exports from the load screen as a JSON file and as a k6 script, with its test data as a CSV file, and each virtual user can send different data. The answers that said load profiles have no export are corrected. There is still no JMX import or export.
Every row re-read against the vendors’ own pages and linked to the page it was read from. Corrected: Postman records browser sessions and runs load tests; Selenium Grid runs tests in parallel; Playwright ships reporters, accessibility scans and a healer agent; Katalon Recorder exports Selenium-based code; JMeter reports percentiles and Apdex in its built-in dashboard; BrowserStack Load Testing became generally available on 14 October 2026; Sauce Labs authors tests with Sauce AI; ZAP is no longer called OWASP ZAP, and JMeter is named Apache JMeter. Absences are now written as what we did not find, and our own rows no longer mention a heal review that customers do not have.
Our own claims corrected again, after checking what a signed-up customer can use: load profiles have no k6 export yet, and HAR files are shown for reading, not imported. The JMeter, Postman, k6 and load-testing rows now say so.
Eleven tools added the same day (Applitools, Artillery, Burp Suite, Cypress, Gatling, k6, Locust, mabl, Sauce Labs, testRigor, Testsigma), each row read from the vendor’s own pages by one reader and re-read by a second; a JMeter alternatives page; head-to-head comparisons with testRigor and mabl; the import and export claims about our own product corrected (no Selenium, Cypress, Playwright or JMeter importer; k6 export only).
First publication: eight tools, each row sourced from the vendor’s own pages on the date shown (the same audits behind the comparison pages); method and conflict of interest stated.
Spot something outdated or wrong about another tool? Tell us — we reply within 5 business days and note every correction on this page.
Measure it on your own application
Start on the Free plan, or bring one flow from your own product and we record it live.