Rikaii Docs

Fallback routing ​

After routing picks a primary model and credential, the gateway tries to complete your request on that path. Fallback means a second attempt on a different execution path because the first attempt hit a retryable failure before returning a successful body.

This page explains what triggers a fallback, what runs next for PAYG (platform-metered) vs BYOK, and how you can tell a fallback succeeded.

Retryable failures (what counts as “worth retrying”) ​

The gateway retries only when the primary attempt fails with an error class that looks like transient infrastructure or capacity—not every 4xx.

Roughly:

  • HTTP 429 or 5xx from the upstream provider.
  • Timeouts and connection failures between the gateway and the provider.

Special BYOK rule: when the request is already using your provider key, 401 and 403 from the upstream are also treated as retryable. That matters because keys can be rotated, permissions change, or the vendor may reject a key intermittently—those cases can flow into Safety Net (below).

Non-retryable failures (for example most 400 validation errors from the provider) are returned to you immediately—no silent second model.

Streaming vs non-streaming (hard “point of no return”) ​

  • Non-streaming: If the primary call fails before a full response is returned, the gateway may run a fallback path and still answer with HTTP 200 and a normal completion body.
  • Streaming: If the primary stream has already emitted at least one SSE data: chunk, a mid-stream failure does not switch to another model—the client already received partial tokens. The error is propagated to the client.

Plan streaming clients to handle abrupt stream errors after partial output.

PAYG (platform-metered) traffic ​

Primary uses Rikaii’s catalog slug, platform upstream key, and your normal metering / spend guards.

When fallback triggers ​

If the primary upstream call fails with a retryable error and your Dashboard routing rule (or static routing) defines a non-empty fallback model slug for that rule, the gateway:

  1. Logs that the primary slug failed.
  2. Resolves the fallback slug the same way as any other catalog primary (fresh routing decision object for that slug).
  3. Dispatches again using the platform key appropriate for the fallback model.
  4. Recomputes overage protection completion caps for the fallback row (metered traffic only).

What you see ​

  • HTTP 200 with the fallback model’s completion in the body.
  • Response header X-Rikaii-Fallback: true on non-streaming success when a fallback hop was used (omitted when the primary succeeded on the first try).

If no fallback slug is configured, a retryable primary failure surfaces to your client as the upstream-normalized error (for example 502/503 with an api_error body)—there is no silent second model.

BYOK (your provider key on the primary) ​

Primary uses your stored secret when Use my key routing is active for that provider. Billing and failure modes split into two sub-cases.

A) BYOK with fallback / Safety Net disabled ​

Only your key is used. Retryable upstream failures are returned to you as-is (normalized into the OpenAI-style error JSON). No automatic switch to Rikaii’s keys.

B) BYOK with fallback / Safety Net enabled (Dashboard) ​

If the primary BYOK call fails with a retryable error:

  1. BYOK-only catalog models — If the primary slug is marked BYOK-only, the platform must not silently run on shared keys. You receive 403 with a clear message that Safety Net cannot use platform keys for that SKU. Fix the primary key or pick a different model.

  2. All other catalog models — The gateway may attempt Safety Net: a second call using Rikaii’s platform key for the same primary model slug (or the inferred provider in blind BYOK mode), so your users can still get a reply when your key is down.

Eligibility gate for Safety Net: the platform will not burn shared quota for free. Safety Net is skipped (original upstream error preserved) unless the account satisfies platform billing eligibility—for example Pro or positive wallet credits, and the workspace is not past the monthly spend cap where that cap would block Safety Net specifically.

Overage protection: Safety Net uses platform metering again; budget caps and monthly limits that apply to metered traffic still apply to that second hop.

What you see after a successful Safety Net ​

  • HTTP 200 with a normal completion body.
  • X-Rikaii-Fallback: true on non-streaming responses when the successful body came from Safety Net (same header convention as PAYG rule fallback).

Streaming does not add X-Rikaii-Fallback today even when Safety Net runs—treat stream success as opaque unless you correlate logs on your side.

Summary table ​

Traffic typePrimary fails (retryable)Fallback behavior
PAYGYes, and Dashboard rule has fallback slugSecond call to fallback slug on platform keys; header on non-stream success.
PAYGYes, no fallback slugError returned to client; no automatic second model.
BYOK, Safety Net offYesError returned; no platform retry.
BYOK, Safety Net on, BYOK-only slugYes403 — platform Safety Net not allowed.
BYOK, Safety Net on, normal slugYes, and eligibility passesRetry on platform key for same slug; metered guards apply.
Any, non-streaming (stream: false or omitted)Yes (retryable, before a full 200 JSON body is returned)Fallback is supported — the same PAYG / BYOK rules in the rows above apply. This mode is where X-Rikaii-Fallback: true is set when a fallback hop succeeds.
Any, streaming (stream: true)Failure after first SSE chunkNo mid-stream model switch; client sees stream error.

Rikaii API documentation

Copyright © Rikaii