Streaming
Server-sent events on every surface, and where usage and cost arrive on a stream.
このページはまだ翻訳されていません。英語版を表示しています。
Stream Chat Completions
stream: true returns SSE. Add stream_options: {"include_usage": true} to get the final usage chunk, which carries usage.cost and the top-level provider.
/v1/chat/completionscurl -N https://api.experientiallabs.ai/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-6-luna", "stream": true, "stream_options": {"include_usage": true},
"messages": [{"role": "user", "content": "Count to five."}]}'Where usage lands per surface
| Surface | Stream flag | Usage and cost |
|---|---|---|
| Chat Completions | stream: true | Final chunk when stream_options.include_usage is set |
| Responses | stream: true | response.completed event |
| Anthropic Messages | stream: true | Final message_delta (authoritative) |
| Gemini | :streamGenerateContent?alt=sse | Last frame's usageMetadata |
| Embeddings | not supported | One JSON body |
/v1/responseswith client.responses.stream(model="gpt-6-luna", input="Count to five.") as s:
for event in s:
if event.type == "response.output_text.delta":
print(event.delta, end="")
final = s.get_final_response()
print(final.usage)Failures mid-stream
Failover happens only before the first output delta. If the provider dies before any output, the gateway fails over to the next route; when none is left it answers an error (502 "provider stream failed"). If it dies after output, the stream never splices another route: it ends with the protocol's terminal error (a Chat error frame then [DONE], Messages event: error, Responses response.failed). A turn the model ended with nothing returns 200 with header x-gateway-warning: empty_completion; on a stream that already sent headers, the shape is in the terminal frames only. See Errors.
HTTP/2 200
x-gateway-warning: empty_completion
x-request-id: 3f2a...