Streaming

Server-sent events on every surface, and where usage and cost arrive on a stream.

Stream Chat Completions

stream: true returns SSE. Add stream_options: {"include_usage": true} to get the final usage chunk, which carries usage.cost and the top-level provider.

POST/v1/chat/completions
curl -N https://api.experientiallabs.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-6-luna", "stream": true, "stream_options": {"include_usage": true},
       "messages": [{"role": "user", "content": "Count to five."}]}'

Where usage lands per surface

SurfaceStream flagUsage and cost
Chat Completionsstream: trueFinal chunk when stream_options.include_usage is set
Responsesstream: trueresponse.completed event
Anthropic Messagesstream: trueFinal message_delta (authoritative)
Gemini:streamGenerateContent?alt=sseLast frame's usageMetadata
Embeddingsnot supportedOne JSON body
POST/v1/responses
with client.responses.stream(model="gpt-6-luna", input="Count to five.") as s:
    for event in s:
        if event.type == "response.output_text.delta":
            print(event.delta, end="")
    final = s.get_final_response()
    print(final.usage)

Failures mid-stream

Failover happens only before the first output delta. If the provider dies before any output, the gateway fails over to the next route; when none is left it answers an error (502 "provider stream failed"). If it dies after output, the stream never splices another route: it ends with the protocol's terminal error (a Chat error frame then [DONE], Messages event: error, Responses response.failed). A turn the model ended with nothing returns 200 with header x-gateway-warning: empty_completion; on a stream that already sent headers, the shape is in the terminal frames only. See Errors.

HTTP/2 200
x-gateway-warning: empty_completion
x-request-id: 3f2a...