Routing and reliability

Rate limits

The tiers, windows and counting rules behind the gateway's own 429s.

このページはまだ翻訳されていません。英語版を表示しています。

Every organization has a rate-limit tier. The tier sets requests per minute and tokens per minute, hour and day for platform-funded traffic, counted across every key in the organization. Traffic on your own provider keys (BYOK) never counts against these windows.

Tiers

Your tier follows what you have bought. Settings > Rate limits shows your current tier and the exact figure for every window.

TierHow you get it
FreeEvery new organization
Tier 1A credit purchase of $5 or more
Tier 2The Pro plan
Tier 3The Max plan
Tier 4The Ultra plan
Tier 5On request, from Settings > Rate limits

A key can also carry its own requests and tokens per minute ceilings, set by an org admin under Budget & limits on the API keys page.

At a glance
ScopeWhole organization, all keys
CountedPlatform-funded traffic only
Not countedYour own keys and plans (BYOK)
Daily reset00:00 UTC

Windows and how they are counted

WindowCountsError prefix
Requests per minuteDispatches in a rolling 60 secondsorg_rate_limit
Tokens per minuteSettled tokens in a rolling 60 secondsorg_token_rate_limit
Tokens per hourSettled tokens in a rolling hourorg_token_rate_limit_hour
Tokens per daySettled tokens in the current UTC dayorg_token_rate_limit_day

Every dispatch counts, including one that later fails. A cached input token counts as at most a tenth of a token, so long cached prompts do not use up your token windows.

{
  "error": {
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "message": "org_rate_limit: your organization reached its tier's requests per minute..."
  }
}
Retry-After: 12

Which 429 is it

CodeWhose limitWhat to do
rate_limit_exceededYour tier's windowWait Retry-After, or move up a tier
admission_capacity_exceededToo many of your requests waiting at onceSpread bursts out; a higher tier admits more at a time
insufficient_quotaCredits or a spend cap, not a rateAdd credits or raise the cap. Retrying does not help
free_limit_reachedA free allowance's windowWait for the reset the message names

The message always names your tier, the next tier's value for that window and the page that raises it. A provider's own throttle is handled by routing first and reaches you only when no other rung can take the request.

import time, openai

try:
    reply = client.chat.completions.create(model="gpt-6-luna", messages=msgs)
except openai.RateLimitError as e:
    time.sleep(int(e.response.headers.get("retry-after", "5")))

Read your limits

A key can read its own effective ceilings. null means uncapped; source says whether a value was set on the key or comes from the default.

GET/api/gateway/keys/{api_key_id}/limits

The figures in the example are illustrative. See Errors for every code.

curl https://api.experientiallabs.ai/api/gateway/keys/$API_KEY_ID/limits \
  -H "Authorization: Bearer YOUR_API_KEY"
{
  "daily_spend_cap_nano_usd": null,
  "spend_cap_period": "daily",
  "requests_per_minute": 600,
  "tokens_per_minute": 2000000,
  "source": "default"
}