How Credentials Are Handled
- Your API key only lives in this tab's memory. We do not write it to localStorage, cookies, or a database; refreshing the page clears it.
- In browser direct mode, credentials travel only between your machine and the target endpoint. They never pass through our server.
- In server proxy mode, credentials pass through our server only for the lifetime of the request and are forwarded to the target endpoint. They are not written to disk or logs. The proxy does not log Authorization, request bodies, or model output.
- Reports always render Authorization as
Bearer <redacted>. If an endpoint echoes the key in an error message, it is scrubbed before export. - This version does not provide share links and does not store reports on the server.
Traffic Analytics
- Production traffic is measured with PostHog in cookieless mode. PostHog does not write its analytics identity to cookies, localStorage, or sessionStorage.
- We record page views with query strings and URL fragments removed, plus guide-to-tester clicks, tool starts, completed suites, and report copy/download actions.
- Analytics never receives your Base URL, API key, model name, raw requests, raw responses, or report content.
- Automatic interaction capture, session replay, surveys, person profiles, heatmaps, performance capture, and exception capture are disabled.
- Browsers that send a supported Do Not Track signal are excluded from analytics.
Execution Routes
Browser direct
Requests are sent by your browser. This is the only route that can test localhost and private network addresses, but the target service must allow this site as a CORS origin. When direct mode fails, we send an additional no-cors probe to distinguish "service is down" from "service is running but blocked by CORS."
Server proxy
Requests are sent by our server, so timeouts, SSE handling, and error classification are consistent for every user and are not affected by browser CORS. The proxy validates the resolved IP when opening the socket, refuses loopback, private, link-local, and cloud metadata addresses, and never follows redirects. Local and private endpoints cannot use this route.
Status Meanings
Decision rule: each probe is made of assertions. Any blocking assertion failure means FAIL; only degraded assertion failures means PARTIAL; an endpoint that explicitly rejects a capability, such as returning 400 for an unsupported parameter, means UNSUPPORTED; a failed prerequisite means SKIPPED.
Current Probes
Model List P0
Reads /models, confirms models are enumerable, and checks whether the entered model ID is present.
Basic Chat Completion P0
Sends a minimal text request and checks whether the response shape can be parsed by OpenAI SDKs.
Streaming (SSE) P0
Checks SSE frame format, delta fields, end markers, and termination behavior.
Single Tool Calling P0
Provides one tool and checks whether the endpoint returns valid tool_calls with JSON-parseable arguments.
Tool Result Round Trip P0
Sends a tool result back and checks whether the endpoint can complete the second assistant turn from it.
JSON Mode P0
Uses response_format: json_object and checks whether the endpoint reliably returns valid JSON.
Before a run starts, LLMCompat performs a read-only preflight by loading /models and deciding whether to use the Base URL with /v1 or without it. If preflight fails, no billable requests are sent.
Limits
- Single-request timeout: 30s; streaming timeout: 60s; total run budget: 240s.
- Single responses are capped at 1000 KB; streams are capped at 2000 KB. Larger responses are truncated or stopped.
- Probes run sequentially to avoid load-testing the target service and to keep timing evidence independent.
Disclaimer
Test conclusions apply only to the model, endpoint, and moment used in that run. The probe set is finite, so all PASS does not mean the endpoint is fully OpenAI-compatible in every scenario. Likewise, a FAIL can come from a temporary network or load issue, so rerun to confirm. Only test services you are authorized to access.