Rate limits

Concurrency per key

APIONE limits requests in flight, not requests or tokens per minute. Each key allows up to 6 concurrent requests by default, counted separately for API calls and web chat; contact us if you need more.

Over the limit, a request first waits briefly in a queue; only if it can't get a slot does it return 429 with Retry-After: 1:

{
  "error": {
    "message": "API token concurrent request limit exceeded",
    "type": "rate_limit_error",
    "code": "api_key_concurrency_exceeded",
    "retryable": true
  }
}

When you hit the limit

1. Back off and retry: honour Retry-After, add exponential backoff with jitter so instances don't all retry in lockstep.
2. Throttle on the client: cap batch concurrency below the limit with a semaphore or queue rather than discovering the limit by hitting it.
3. Split keys: run batch work and production traffic on different keys — each counts its own concurrency, so a saturated batch never starves user requests.
4. Shorten each request: trimming history and enabling prompt caching makes calls finish sooner, which frees slots.

import asyncio

sem = asyncio.Semaphore(4)   # below the key's limit, so you don't rate-limit yourself

async def ask(client, prompt):
    async with sem:
        return await client.chat.completions.create(
            model="deepseek-v4-flash",
            messages=[{"role": "user", "content": prompt}],
        )

Upstream limits

Providers have quotas of their own. When one is throttling, the gateway moves the request to another candidate and you generally never notice; only when no candidate is available do you get 429 rate_limit_exceeded or 503 provider_saturated — both are safe to retry after a backoff.