Pricing and billing

Prompt caching

How cache reads and writes are priced, which routes cache, and how to turn caching off.

このページはまだ翻訳されていません。英語版を表示しています。

Prompt caching lets a provider reuse a prompt prefix it has seen recently, so the repeated part of your context costs less and answers sooner. It is a property of each route, not a switch: a route that publishes a cached input rate bills cache reads at that rate. The gateway helps by keeping a conversation on the route that holds its cache.

The three input legs

Every input token falls into exactly one leg. Usage reports each.

LegBilled atTypical ratio to input
Fresh inputThe input rate1x
Cache readThe route's cached input rateOften 0.1x
Cache writeThe route's cache-write rate1.25x for a 5-minute write; 2x for a 1-hour write on Claude routes

A route with a cache-write price lists it in cache_write_input_nano_usd_per_million (and the 1-hour variant). Writes cost more than plain input, so caching pays off when a prefix is read again before it expires.

{
  "input_nano_usd_per_million": 4000000000,
  "cached_input_nano_usd_per_million": 200000000,
  "cache_write_input_nano_usd_per_million": 5000000000,
  "cache_write_1h_input_nano_usd_per_million": 8000000000,
  "output_nano_usd_per_million": 20000000000
}
LegPer million
Input$4.00
Cache read$0.20
5-minute write$5.00
1-hour write$8.00

Which routes cache

Provider behaviorWhat you do
Caches repeated prefixes automatically (most OpenAI, Gemini, DeepSeek and similar routes)Nothing
Caches only when asked (Claude)Add cache_control checkpoints; the gateway forwards them
No cached rate publishedRepeats bill as fresh input

A cached price describes billing, not a guaranteed hit. Hits depend on the model, prefix length and expiry. Check prompt_tokens_details.cached_tokens in usage to confirm.

{
  "model": "claude-opus-5.5",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "<long reference document>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [{ "role": "user", "content": "Summarize section 4." }]
}

Caching and failover

The gateway sends each conversation a cache-affinity hint where the provider supports one and keeps it on the rung holding its cache. If that rung fails or throttles, the request fails over and pays a cache miss on the next rung. Availability wins over a warm cache. See Routing.

EventCost effect
Same rung, prefix warmCache read price
Failover to another providerFresh input (and maybe a new write)

Turn caching off

The organization prompt_caching setting (every plan) turns off everything the gateway adds: no affinity hint, no cache-based placement, and your cache_control markers and prompt_cache_key are removed before any provider sees the request. Providers that cache on their own may still do so, and the response says so in x-experiential-ignored-parameters. Expect higher input cost on long repeated prompts.

PUT/api/orgs/{org_id}/provider-policy
curl -X PUT https://api.experientiallabs.ai/api/orgs/$ORG_ID/provider-policy \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt_caching": false}'
x-experiential-ignored-parameters: cache_control->not_forwarded(organization_prompt_caching_off)