AI testing: testing AI features, LLM APIs and agents
AI testing means two different things, and a page that does not say which is selling something. The first is testing software that contains a model: an assistant, an LLM API key, an agent that calls tools. Its answers are not deterministic, so a test asserts on structure, on bounds and on behaviour across many runs, and probes for prompt injection the way a security scan probes for injection. The second is using a model inside the testing process, where the honest question is what it measurably does better than a rule.
The pages below are split that way. For the first meaning: the LLM API key guide with its 60 probes, the assistant article and the gateway testing module. For the second: the benchmark of small local models on 250 cases, with its intervals, and the pages that say where a model is and is not used in the product.
What people search for
The literal queries this hub is written to answer; each leads to one of the pages below.
- llm testing
- llm evaluation tools
- ai testing tools
- prompt injection testing
- how to test an ai assistant
- ai agent testing
- agentic testing
Do it: test an AI feature
Executed guides with probe counts and outcomes.
How to test an LLM API key
Sixty probes across the OWASP LLM Top 10 2026 categories, and what each answer means.
Testing an AI assistant before release
Structure, bounds, refusals and injection, on a product that ships one.
LLM testing tool and AI gateway testing
The module, with its limits.
Prompt injection
Direct and indirect; why retrieved text never carries authority.
Learn: AI inside testing, measured
Numbers with intervals, and where a rule still wins.
Small local models on 250 cases
The benchmark behind every model choice here, with McNemar tests.
Privacy-first local AI
Why inference runs locally and what that costs.
Agentic testing
The strict definition, and why the product is deliberately not fully agentic.
pass@k
The metric for a non-deterministic answer.
Compare
Where the platform sits among AI testing tools.
Test automation tools compared by job
The AI column, with the meaning of "AI" stated per tool.
QA when coding agents open the pull requests
The review the author cannot do.
Questions people ask
How do you test something whose output is not deterministic?
Assert on what must hold across every run: the output is valid JSON with the required fields, numbers are within bounds, forbidden content is absent, the model refuses what it should refuse. Then measure across many runs rather than one, with pass@k or a pass rate with a confidence interval, and treat a probe that succeeds once as a finding.
What is prompt injection testing?
Sending the model text, directly or through content it retrieves, that tries to override its instructions, and checking whether it complied. The LLM API key guide runs sixty such probes grouped by the OWASP Top 10 for LLM applications (2026 edition) and reports each outcome; a tool that only checks the happy path has not tested the feature.
Does Dokimore use AI to write tests?
Natural-language authoring is in development; what ships today is deterministic: recording, replay, self-healing from stored locators, smart test selection with a written rationale, and local models for specific jobs where the benchmark showed they beat a rule. The benchmark article is the evidence, and it names the cases where a rule still won.
Try it on your own application
The Free plan asks for no card and runs for 14 days after the trial: record a flow, replay it, and read the evidence each step produced.