In short
- Validate the shape, not only the values: compile a JSON Schema with Ajv and run it on every response the test receives. Against a release that renamed nothing and still broke four things, the validator reported "/ must have required property 'currency'", "/ must NOT have additional properties (etag)", "/total must be string" and "/items/0/qty must be integer" in one call.
- Use the API’s own contract where it publishes one: an OpenAPI 3.1 document carries a JSON Schema 2020-12 Schema Object for each response; fetch it, take components.schemas.Order, compile it, and the test checks the API against what the API promised. In our run that schema agreed with the hand-written one on both responses.
- A schema check is cheap and belongs on every request in the suite: the v1 request, validation included, took 3 ms through Playwright’s request context. The cost is writing the schema once; the return is every silent shape change for the life of the API.
About this article
- Format
- How-to guide: a procedure to follow, step by step.
- Written for
- SDETs and automation engineers · Developers who own their tests · QA leads and test managers
- Where it applies
- Every market. The rules that differ by country are on the country pages.
The failure mode is quiet. An API test asserts the status, reads two or three fields, compares them with expected values, passes. A release changes total from the string "12.50" to the number 12.5, drops currency because "it is always USD", adds an etag, and changes qty to a string somewhere in a nested array. The test still passes, because the two or three fields it reads are still there. The consumer that breaks is in production, and it is usually someone else’s.
This guide reproduces exactly that on a fixture API with a v1 and a v2 order endpoint, using Playwright 1.62.1’s request context (no browser) and Ajv 8.20.0, the JSON Schema validator this platform already ships. The error texts below are the validator’s output from that run; scripts and raw output are linked at the end.
The schema and the test
A JSON Schema says which properties are required, their types and constraints, and, with additionalProperties: false, that nothing else may appear. That last line is the one most teams leave out and the one that catches the field nobody meant to add.
export const ORDER_SCHEMA = {
type: 'object',
required: ['id', 'status', 'currency', 'total', 'items'],
additionalProperties: false,
properties: {
id: { type: 'integer', minimum: 1 },
status: { type: 'string', enum: ['pending', 'paid', 'refunded'] },
currency: { type: 'string', pattern: '^[A-Z]{3}$' },
total: { type: 'string', pattern: '^\\d+\\.\\d{2}$' }, // money as a decimal string, never a float
items: {
type: 'array', minItems: 1,
items: { type: 'object', required: ['sku', 'qty'], properties: { sku: { type: 'string' }, qty: { type: 'integer', minimum: 1 } } },
},
},
} as const;The test compiles the schema once and validates every response it receives, with the violations carried in the assertion message so a failure reads as a list of changes:
import { test, expect, request } from '@playwright/test';
import Ajv from 'ajv';
import { ORDER_SCHEMA } from './order.schema';
const ajv = new Ajv({ allErrors: true });
const isOrder = ajv.compile(ORDER_SCHEMA);
test('GET /api/orders/42 keeps the Order contract', async () => {
const api = await request.newContext({ baseURL: process.env.BASE_URL });
const response = await api.get('/api/orders/42');
expect(response.status()).toBe(200);
expect(response.headers()['content-type']).toContain('application/json');
const body = await response.json();
const valid = isOrder(body);
const errors = (isOrder.errors ?? []).map((e) => `${e.instancePath || '/'} ${e.message}`);
expect(valid, `schema violations:\n${errors.join('\n')}`).toBe(true); // the message carries every violation
expect(body.total).toBe('12.50'); // values still matter; they are the second check, not the only one
await api.dispose();
});What the validator said about v2
The v2 endpoint returned 200 with a body that any value check would have accepted: id 42, status paid, items with the right SKU. The validator, with allErrors on so that one run reports everything, returned false and four messages:
/ must have required property 'currency'
/ must NOT have additional properties (etag)
/total must be string
/items/0/qty must be integerEach line is a bug report with its location: the instancePath names the property, nested ones included, and the message names the rule. The first is the dropped field, the second the one that arrived unannounced, the third the type change that breaks every consumer doing string arithmetic on money, the fourth the nested change that only a schema walking the array would find. A status-and-values test reported none of them; the v1 and v2 statuses were both 200.
Use the contract the API publishes
When the API serves an OpenAPI document, the schema is already written. OpenAPI 3.1 defines its Schema Object as a superset of JSON Schema 2020-12, and response schemas live under each response’s media type or, shared, under components.schemas. The test can fetch the document and compile the Order schema from it, so the assertion is "the API does what its own documentation says". In our run the schema from the fixture’s /openapi.json accepted v1 and rejected v2 with the same four errors as the hand-written one.
const api = await request.newContext({ baseURL: process.env.BASE_URL });
const openapi = await (await api.get('/openapi.json')).json();
const isOrder = new Ajv({ allErrors: true }).compile(openapi.components.schemas.Order);
const body = await (await api.get('/api/orders/42')).json();
expect(isOrder(body), JSON.stringify(isOrder.errors, null, 2)).toBe(true);In the product
Two cautions. A schema that uses $ref across the document needs the references resolved before compiling (Ajv can take the whole document and resolve components), and a document that marks a field nullable or uses oneOf needs the same care in the schema as in the code. And the document can be wrong: a contract test against an OpenAPI file that nobody updates proves the API matches stale documentation. The fix is to generate the document from the code or to validate the document itself in the pipeline.
What to put in an API schema check
- required for every field a consumer reads, and additionalProperties: false unless the API explicitly allows extension; the extra field is the change nobody announces.
- type for every property, with integer rather than number where a count is meant, and string with a pattern for money, dates and identifiers.
- enum for status fields; a new state is a breaking change for every switch statement downstream.
- minItems and the item schema for arrays; the nested change was the one a shallow check missed.
- allErrors: true in the validator so one run reports every violation, and the errors in the assertion message so the report reads as a list of changes, not "expected true".
- The same check on error responses: a 404 body has a shape too, and clients parse it.
How the API studio does this
The API testing studio reads an OpenAPI or GraphQL schema and asserts a response against it as a step, so a chain of requests carries the contract check without a hand-written validator, and a failed assertion fails the run with the violations listed. A collection imported from Postman, Bruno, Insomnia or Hoppscotch, or built from an OpenAPI document or a cURL command through the import pages (a HAR opens there for reading), gets the same assertion added per request. The JSON validator is the quick check for a body that does not parse at all.
What to do next
- Write a JSON Schema for the three responses your consumers depend on most, with required, types, enum and additionalProperties: false, and compile it once per suite.
- Validate every response in the suite against its schema and put the violations in the assertion message; the first silent change will be reported with its path.
- If the API serves an OpenAPI document, compile the response schema from it in a contract test, and validate the document itself so it cannot drift from the code.
- Add the error responses: a 404 and a 422 have shapes clients parse too.
- Reproduce the v1-to-v2 run with the linked script against a copy of your own API to see which changes your current tests would miss.
Questions people ask
What is API schema validation testing?
Checking that a response has the shape the consumers depend on, independently of its values: required properties present, types right, enumerations respected, nothing unexpected added. A validator such as Ajv compiles a JSON Schema and reports every violation with its path; in our run it named four breaking changes in a response whose status and headline values were unchanged.
How is schema validation different from checking the response values?
A value check reads the fields it knows and ignores the rest, so a field that changed type elsewhere, a dropped field or an extra one goes unreported. A schema check walks the whole document. Keep both: the schema proves the contract, the value check proves the business logic for that request.
Can I validate an API response against its OpenAPI spec?
Yes. OpenAPI 3.1 Schema Objects are JSON Schema 2020-12, so the test can fetch the document, take the response schema from components.schemas or the response’s media type, compile it with Ajv and validate the live body. In our run the schema from the fixture’s own document rejected the broken release with the same four errors as the hand-written schema.
Does schema validation slow down an API test suite?
Not measurably. The v1 request in our run, fetch and validation included, took 3 ms through Playwright’s request context, and Ajv compiles a schema once and caches the function. The cost is writing the schema; the return is catching every silent shape change for the life of the API.
How we measured this
- Run on 2026-11-04 with Playwright 1.62.1’s request context (no browser launched) and Ajv 8.20.0, the versions this platform pins, networking disabled, fixture API on 127.0.0.1 serving /api/orders/42 (v1), /api/v2/orders/42 (the same order with total as a number, currency removed, an etag added and qty as a string) and /openapi.json. Scripts and raw output: docs/marketing/keyword-research/research/playwright-guides-2026-11-04 (e7-schema.js, out/e7-schema.json).
- The four error lines are Ajv’s instancePath and message for the v2 response, joined as the script joined them; the OpenAPI-sourced schema produced four errors on v2 and none on v1. The 3 ms is Date.now() around the v1 request and validation, one run.
- The snippets wrap the same calls in Playwright Test; the measurement used the library API.
- Dokimore’s behaviour is the product code on the article date: the API studio asserts a response against the OpenAPI or GraphQL schema it reads, and imports collections, OpenAPI, HAR and cURL.
Sources
- Ajv, Getting started: a schema compiles to a cached validation function; each call overwrites validate.errors read
- JSON Schema, Specification: 2020-12 is the current version read
- OpenAPI Specification 3.1.0: the Schema Object is a superset of JSON Schema 2020-12; response schemas under content media types and components.schemas read
- Playwright documentation, APIRequestContext: request contexts for API testing read