OpenAI API Compatibility Tester
Test your OpenAI-compatible endpoint for chat, streaming, tool calling, and JSON Mode. Enter a Base URL and model to get failure evidence, fixes, and reproduction code.
Six focused probes · No saved credentials · Local endpoints use browser-direct requests
Six checks for OpenAI API compatibility
A reachable endpoint or a successful chat reply only verifies part of an integration. LLMCompat sends real requests and checks the response contracts your client depends on, with a separate result for each capability.
01 · Model listing
Requests /models, checks the response shape, and looks for the model ID you entered.
02 · Chat completion
Sends a minimal request to /chat/completions and checks the choices, assistant message, and completion fields.
03 · SSE streaming
Checks content type, SSE framing, JSON events, delta content, finish reasons, and the [DONE] marker.
04 · Tool calling
Sends a function definition and checks the returned tool_calls, function name, call ID, and JSON arguments.
05 · Tool-result round trip
Returns a tool result using the call ID, then checks whether the endpoint accepts it and produces a final answer.
Check the complete round trip →06 · JSON Mode
Requests response_format: json_object and checks valid JSON, object shape, and the requested fields. Strict JSON Schema is outside this suite.
How to test an OpenAI-compatible API
- Enter the API root. Use the provider's Base URL, commonly
https://api.example.com/v1, rather than the full/chat/completionsoperation URL. - Add credentials and choose a model. Supply a key if the endpoint requires one. Use model lookup or enter the exact model ID exposed by your provider.
- Run the compatibility checks. Choose the probes you need. A connection preflight checks the route and model before the suite runs. Live generation requests can incur provider charges.
- Review and reproduce the result. Open a probe to see assertions, response evidence, and a suggested fix. Copy or download the report and use its reproduction code to investigate a failure.
To explore the output first, open the demo report in the tester. It uses an illustrative example and requires no endpoint or API key.
Start your compatibility test ↑Hosted APIs, gateways, and local model servers
Use the tester with endpoints that expose the OpenAI-style model-listing and Chat Completions routes. Actual support for streaming, tools, and JSON Mode depends on the model, runtime, and provider configuration.
- Hosted providers: enter the documented OpenAI-compatible Base URL and an API key authorized for the selected model.
- API gateways and reverse proxies: test the URL your application uses to check whether route rewriting or response handling changes the protocol.
- Local servers: test the OpenAI-compatible routes exposed by Ollama, vLLM, LM Studio, or llama.cpp. Localhost and private addresses use browser-direct mode and need a browser-compatible CORS policy.
Follow the local API setup guide for runtime configuration. Browser security rules may restrict HTTPS-to-localhost access; use the report's command-line reproduction to compare behavior outside the browser.
This suite does not test the Responses API, embeddings, image generation, audio, or every SDK. A PASS applies to the selected probes, endpoint, model, and time of the run.
Common API compatibility errors and what to check
| Symptom | Check next | Related guide |
|---|---|---|
| 401 or 403 | Credential validity, authorization header, and permission to use the model. | Base URL and authentication |
| 404 or an HTML response | The final request path, missing or duplicated /v1, and proxy routing. | API root configuration |
| curl works; browser fails | The OPTIONS preflight, allowed origin, methods, and authorization header. | CORS troubleshooting |
| Broken or incomplete stream | SSE frames, JSON parsing, buffering, finish reasons, and [DONE]. | Streaming troubleshooting |
| Missing tool_calls | Model support for tools, response shape, function arguments, and call IDs. | Tool calling troubleshooting |
| Invalid JSON output | Support for response_format, extra prose, code fences, or truncation. | JSON Mode troubleshooting |
How API keys and reports are handled
Your API key is held in tab memory; it is not saved to cookies, localStorage, or a database. Refreshing the page clears it. In browser-direct mode, requests go from your browser to your endpoint. In server-proxy mode, the key and request pass through the LLMCompat server to the public endpoint for that request.
The proxy does not log authorization headers, request bodies, or model output. Local and private endpoints always use browser-direct mode. Reports remain in the tab until you export them, with credentials redacted from exported evidence.
Cookieless analytics records page visits and selected actions such as starting a test. It does not receive your endpoint, API key, model name, request, response, or report content. See the full methodology and privacy policy.
Frequently asked questions
Is this just an API connectivity test?
No. Connectivity establishes whether a route responds. These probes also inspect response shape, streamed events, tool calls, tool results, and JSON output, so a successful HTTP response can still reveal a compatibility failure.
Do I need an API key to use the tester?
Live tests need whatever credentials your endpoint requires. A local server may accept requests without a key. You can view the illustrative demo report without supplying any credentials.
Can a model pass chat but fail tool calling?
Yes. These are separate capabilities. A provider can implement basic chat while rejecting tools or returning a response that does not follow the tool-call contract. The report gives each probe its own status and evidence.
Does JSON Mode passing prove strict schema support?
No. The JSON Mode probe requests a JSON object and checks syntax, shape, and specific fields. It does not exercise strict Structured Outputs or prove conformance to an arbitrary JSON Schema.
Does PASS mean my endpoint is fully OpenAI-compatible?
No. It means the selected probes passed for this endpoint and model at the time of the run. Other routes, schemas, prompts, and SDK versions may behave differently. Use the evidence alongside your application's own integration tests.
Know exactly what the result means.
Conclusions apply only to the tested model, endpoint, and moment in time. If you are still configuring the endpoint, start with the Base URL guide. The probe set is finite, so passing does not prove complete compatibility.
Methodology and privacy