Testing strategy

Local verification

npm install
npm run check
npm test
npm run build

The unit suite uses mock providers and injected fake SDK clients, so it runs without API keys.

What is covered

  • Schema conversion: generic/strict objects, optional/default/nullable, tuples, unions, records, dates.
  • Retry engine: validation repair, retry prompt content, abort behavior, transport backoff.
  • Streaming: chunk passthrough, final validation, partial preview repair.
  • Providers: OpenAI chat tool_call, OpenAI Responses text/tool_call, Anthropic prompt/tool_call.

Real-provider integration tests

tests/openai.integration.spec.ts is skipped unless OPENAI_API_KEY is set.

$env:OPENAI_API_KEY="sk-..."
npm test

Before a production claim, extend the same pattern to a provider matrix:

  • OpenAI chat.completions: json_schema, json_object, tool_call
  • OpenAI Responses API: json_schema, tool_call
  • OpenAI-compatible gateway: json_object, prompt, and tool_call only if the gateway supports tools
  • Anthropic: prompt and tool_call
  • Google: json_schema/responseSchema

Track first-pass success rate, repaired-after-retry rate, latency, and token cost per schema family.