The Short Answer
Set stream: true. Check the content type first. Then parse complete SSE events instead of parsing arbitrary network chunks.
- 1
Confirm an HTTP 200 response and a streaming content type.
- 2
Split events at blank lines, not at network chunk boundaries.
- 3
Parse only the value after each
data:prefix. - 4
Confirm a finish reason or the terminal
[DONE]marker.
Keep Four Streaming Layers Separate
HTTP delivers bytes. SSE groups those bytes into events. Each event contains an OpenAI-compatible JSON object. The final event ends the response.
One event can arrive in several chunks. Several events can also arrive in one chunk. Buffer bytes until you receive the event separator.
Inspect the Raw Stream
Use curl with buffering disabled. This command shows the frames in the order that the endpoint sends them.
curl -N https://api.example.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "your-model-id",
"messages": [{"role": "user", "content": "Reply with OK"}],
"stream": true
}'data: {"choices":[{"delta":{"role":"assistant"}}]}
data: {"choices":[{"delta":{"content":"OK"}}]}
data: {"choices":[{"delta":{},"finish_reason":"stop"}]}
data: [DONE]SSE frame rules are defined by MDN. OpenAI-compatible event fields follow the provider contract.
Diagnose the First Broken Layer
| Symptom | Likely layer | Check next |
|---|---|---|
| Normal JSON response | Request or server | stream: true and route support |
| Wrong content type | HTTP | Proxy and upstream headers |
| JSON parse errors | SSE parser | Frame buffering and data: removal |
| HTML in the stream | Proxy or route | Gateway error and final request URL |
| Long pause, then all text | Buffering | Reverse-proxy buffering |
| Early disconnect | Network or server | Timeout, finish event, upstream logs |
Verify the Parsed Events
LLMCompat stores the raw stream and the parsed events. It reports invalid frames, missing content, and incomplete termination separately.