Pricing and billing
Prompt caching
How cache reads and writes are priced, which routes cache, and how to turn caching off.
このページはまだ翻訳されていません。英語版を表示しています。
Prompt caching lets a provider reuse a prompt prefix it has seen recently, so the repeated part of your context costs less and answers sooner. It is a property of each route, not a switch: a route that publishes a cached input rate bills cache reads at that rate. The gateway helps by keeping a conversation on the route that holds its cache.
The three input legs
Every input token falls into exactly one leg. Usage reports each.
| Leg | Billed at | Typical ratio to input |
|---|---|---|
| Fresh input | The input rate | 1x |
| Cache read | The route's cached input rate | Often 0.1x |
| Cache write | The route's cache-write rate | 1.25x for a 5-minute write; 2x for a 1-hour write on Claude routes |
A route with a cache-write price lists it in cache_write_input_nano_usd_per_million (and the 1-hour variant). Writes cost more than plain input, so caching pays off when a prefix is read again before it expires.
{
"input_nano_usd_per_million": 4000000000,
"cached_input_nano_usd_per_million": 200000000,
"cache_write_input_nano_usd_per_million": 5000000000,
"cache_write_1h_input_nano_usd_per_million": 8000000000,
"output_nano_usd_per_million": 20000000000
}| Leg | Per million |
|---|---|
| Input | $4.00 |
| Cache read | $0.20 |
| 5-minute write | $5.00 |
| 1-hour write | $8.00 |
Which routes cache
| Provider behavior | What you do |
|---|---|
| Caches repeated prefixes automatically (most OpenAI, Gemini, DeepSeek and similar routes) | Nothing |
| Caches only when asked (Claude) | Add cache_control checkpoints; the gateway forwards them |
| No cached rate published | Repeats bill as fresh input |
A cached price describes billing, not a guaranteed hit. Hits depend on the model, prefix length and expiry. Check prompt_tokens_details.cached_tokens in usage to confirm.
{
"model": "claude-opus-5.5",
"max_tokens": 1024,
"system": [
{
"type": "text",
"text": "<long reference document>",
"cache_control": { "type": "ephemeral" }
}
],
"messages": [{ "role": "user", "content": "Summarize section 4." }]
}Caching and failover
The gateway sends each conversation a cache-affinity hint where the provider supports one and keeps it on the rung holding its cache. If that rung fails or throttles, the request fails over and pays a cache miss on the next rung. Availability wins over a warm cache. See Routing.
| Event | Cost effect |
|---|---|
| Same rung, prefix warm | Cache read price |
| Failover to another provider | Fresh input (and maybe a new write) |
Turn caching off
The organization prompt_caching setting (every plan) turns off everything the gateway adds: no affinity hint, no cache-based placement, and your cache_control markers and prompt_cache_key are removed before any provider sees the request. Providers that cache on their own may still do so, and the response says so in x-experiential-ignored-parameters. Expect higher input cost on long repeated prompts.
/api/orgs/{org_id}/provider-policycurl -X PUT https://api.experientiallabs.ai/api/orgs/$ORG_ID/provider-policy \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt_caching": false}'x-experiential-ignored-parameters: cache_control->not_forwarded(organization_prompt_caching_off)