Skip to main content
LLM and AI gateway testing · updated 2026-10-05

LLM and AI gateway testing

A gateway for the model key you supply: injection and data-leak detectors, Pass^k reliability floors, a spend ceiling that returns 402, and evidence per call.

For: Teams shipping a feature on a language model who need to show what the gateway blocks, what it costs and how reliably it holds under attack, in the vocabulary of the OWASP Top 10 for LLM applications.

Capabilities

Injection detection

Direct and indirect prompt injection, control-tag tricks and system-prompt exfiltration attempts, with a two-speed detector path.

Data-leak detection

Eighteen pattern classes for personal data and secrets, including checksum-validated national and card identifiers, on the request and the response.

Pass^k floors, not accuracy

Twelve multi-turn scenarios run in both polarities over repeated trials and an injection-rate sweep; attack and benign floors are reported separately, with confidence intervals and a published known limitation.

OWASP vocabulary

Findings aligned to the OWASP Top 10 for LLM applications (2026) and the OWASP Top 10 for agentic applications.

Budget and rate gates

A spend ceiling enforced before the call (402 when exceeded), per-tenant rate limits with standard RateLimit headers, and a cost ledger priced from the provider’s own usage figures.

Capability profile

What a connected key’s model can do, measured by code across seventeen reasoning dimensions and an agentic layer, from a seeded corpus rather than a downloaded benchmark.

How it works

  1. Connect a keyYour provider, your key, never stored in plain text; or a self-hosted model.
  2. Route calls through the gatewayDetectors run on both directions; the envelope records what they found.
  3. Run the assurance suiteMulti-turn scenarios, both polarities, repeated trials; read the Pass^k floors.
  4. Set the budget and the limitsA spend ceiling and per-tenant rate limits, enforced before money leaves.

What you receive, and what we do not do

Evidence ledger
Per call
An envelope with detector findings, risk class, provider usage, cost, latency and the governance declarations.
Per suite run
Pass^k by polarity with confidence intervals, the scenarios evidenced, and the known limitations stated.
Per key
A capability profile across seventeen reasoning dimensions and an agentic layer.

Limits we state

  • Detectors are rule-based and bounded: the first 192 KiB and the last 64 KiB of a payload are scanned, and the envelope says so.
  • One scenario fails by design and is published as a known limitation: the crescendo detector is keyword-anchored.
  • Prompts go to the provider you chose, under your agreement with it; the platform records the call and never sends your data to any other provider.

Plans: Part of the API testing studio; the product shows the suite and budget limits that apply to your plan. See the metering table.

Questions buyers ask

Why Pass^k instead of accuracy?

Pass@k rises as you retry and measures the best case; Pass^k is the probability that every one of k trials passes, which is the floor a customer actually gets. Under injected faults the two diverge sharply, so the floor is the number worth reporting.

Does the platform see our prompts?

The gateway runs the detectors over the request and the response and records the envelope in your tenant. The prompt itself goes to the provider whose key you supplied, under your agreement with that provider, and to no other model.

What does the budget gate do?

It estimates the cost of a call before it is made, compares it with your windowed spend, and refuses the call with a 402 when the ceiling would be exceeded. Usage is recorded from the provider’s own figures afterwards.

Try it on your own application

The Free plan has no time limit and asks for no card. Or bring one flow from your product and we record it live.

Your choice about analytics

We use one optional analytics tool, run on our own servers, to see which pages help people. It stays off unless you turn it on. Storage needed to sign you in, keep the site secure and remember this choice is always on. Cookie Policy