Failover

Provider failover (on by default)

When the first-choice provider returns a retryable error, the gateway moves to the next candidate within the same request — your client sees one successful response. Failover triggers on:

• Connection timeouts, TLS handshake failures, resets.
• Upstream 429 (that provider is rate-limited).
• Upstream 5xx.
• A stream that dies before any content is emitted.

It does not trigger where another provider would fail identically: malformed requests (400), content-policy blocks (403), context overflow (413).

Billing and retries

• Failed attempts during failover are not billed — only the successful response is.
• Failover happens inside the gateway: it counts as one request and holds one concurrency slot.
• It does add latency. Where latency matters most, set a shorter client timeout and retry yourself.

Observing it

Every response carries an X-Maxx-Request-ID header. Search it in console › Usage › Request log to see the call's status, duration and first-token latency. Start there when chasing intermittent slowness.