Symptom: When using model: "auto" with free/no-auth providers (opencode, auggie, etc.), you intermittently get HTTP 429, 400, or 401 instead of answers. The requests succeed when retrying the same prompt moments later, but automation (cron jobs, agents, scripts) breaks on the first failure.
Root cause: Three independent failure modes stack up:
Provider rate-limit (429): Free tiers can enforce a per-window quota. A burst of parallel calls exhausts it, so the next request is refused until the window resets.
Broken model in passthrough (400/401): auto/* pools can include passthrough models from opencode that are registered in the catalog but have no live credentials (e.g. oc/north-mini-code-free → 401). The auto-router tries one, fails, and the error propagates before fallback kicks in.
Concurrency amplification (429 under load): When multiple agent/cron sessions hit auto at once, the aggregate request rate exceeds what free providers tolerate, so legitimate calls get flagged as abusive.
Verified fix (community-reported, 2026-08-10): tune three environment variables so that rotation, concurrency, and fallback absorb the free-tier churn instead of dying on it:
export OMNIROUTE_ROTATE_ON_400=true # hop to another model/provider on 400/401 (skips broken passthrough models)export OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT=4 # explicit heavyweight admission ceiling (unset by default: no request-count cap, see note below)export OMNIROUTE_CHAT_ADMISSION_QUEUE_MS=5000 # longer bounded wait for heavyweight capacity instead of an immediate retryable 503
Set these in the OmniRoute process environment (the daemon, e.g. via the LaunchAgent plist or systemctl edit), then restart OmniRoute. The rotation flag is the single highest-leverage lever: it converts a hard failure into a transparent retry against a healthy provider in the pool.
Note: OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT caps how many heavyweight — long-context — requests run at once; the bound is an admission gate, not a provider rate limiter. #503-fanout update: this var is no longer set by default (it now binds only when explicitly configured, as above) — heavyweight admission is instead gated by an auto-derived byte budget (OMNIROUTE_CHAT_MAX_INFLIGHT_BYTES) that scales itself from the host's real memory ceiling, so a fresh deployment should see far fewer 503 chat_admission_busy rejects without setting this var at all; explicitly setting it here still works exactly as documented. Explicit byte-budget overrides clamp to 8 MiB–2 GiB. A 413 body_exceeds_budget is not transient: increase that byte budget, lower OMNIROUTE_CHAT_HARD_MAX_BODY_BYTES, or increase the process memory ceiling. An inflight_bytes_budget shed is temporary contention and remains retryable. The per-provider rate limiting (open-sse/services/rateLimitManager.ts) is governed separately by RATE_LIMIT_MAX_WAIT_MS, RATE_LIMIT_MAX_QUEUE_DEPTH, and RATE_LIMIT_AUTO_ENABLE — see .env.example.
How to verify it worked: run your agent/cron twice in quick succession and confirm both succeed. Before the fix, the second run typically throws 429/401. After the fix, failures (if any) are retried transparently and the call completes. You can also curl /monitoring/health and watch the rateLimitedUntil field on the provider connections and the circuitBreakers.providerBreakers[].state for the affected providers — the state is one of CLOSED, DEGRADED, OPEN, or HALF_OPEN (see src/shared/utils/circuitBreaker.ts), and a provider that keeps failing will flip CLOSED → DEGRADED → OPEN before the reset window lets a probe through (HALF_OPEN).
If you still see 429: the active account for that provider has genuinely exhausted its quota (not just rate). Add a second account for the same provider in the OmniRoute dashboard → Providers → Accounts, or mix in another free provider (e.g. routeway, auggie). Rotation only helps with transient rate/400/401; a hard quota exhaustion requires a second credential or a different provider.
If you see 403 on vision models (auto/vision, bazaarlink/*): the connected account lacks a paid plan that includes vision, or the API key has insufficient permissions. Verify in the provider dashboard that the key scope includes vision/multimodal, or connect a paid tier account and keep it as the vision target.
When you run npm install -g omniroute, you may see a wall of warnings like npm warn ERESOLVE, peer-dependency notices, and deprecated messages. These are expected and harmless. Your install succeeded if you see added <N> packages in the output.
To suppress the peer-dependency resolution warnings, use OmniRoute's supported install form:
npm install -g omniroute --legacy-peer-deps
--legacy-peer-deps suppresses ERESOLVE and peer-dependency notices only. Deprecation notices remain visible because they come from transitive third-party packages; they do not indicate that the install failed.
The warnings come from stale peer-dependency ranges in third-party packages OmniRoute doesn't control:
marked-terminal wants marked >=1 <16, found marked@18 — works fine in practice; the upstream peer range is just stale.
deprecated prebuild-install@7.1.3 — a transitive native-binary fetch helper. It is not
used to install the pinned wreq-js transport binding and does not indicate that web-cookie
provider transport setup failed.
No action needed — the warnings cannot be fully silenced without forking upstream packages.
This is a false positive. Nothing is infected, and no action is required.
Avast and AVG run a heuristic that flags plain-text/Markdown files containing many
HTTP-request-looking links. OmniRoute's README.md ships inside the npm package (it is
listed in package.json → files), so it lands at node_modules/omniroute/README.md on
a global install — and it contains ~15 http://localhost:20128/... examples (the MCP
HTTP/SSE endpoints, the A2A .well-known URL, and curl snippets). That link density is
enough to trip the heuristic.
If this started only recently: the file did not change in kind. The README grew its
endpoints table (MCP HTTP + SSE + A2A were added) and more curl examples, which pushed
it past the threshold.
The file is inert documentation with zero executable content. You can safely restore it
from quarantine.
What to do:
Stop the notifications — exclude the install directory in your antivirus
(Avast: Settings → Exceptions), adding your global node_modules path and/or the
OmniRoute data dir (~/.omniroute/).
Report the false positive — https://www.avast.com/false-positive-file-form.php,
attaching the quarantined README.md. This is the fix that helps everyone, since it is
the vendor's heuristic overreacting to a text file.
Why we do not "fix" this on our side: the examples are all http://localhost, and
localhost cannot be https without self-signed-certificate friction. Mangling the docs to
dodge one vendor's heuristic would hurt every reader to satisfy a scanner bug.
This is a false positive from a behavioral heuristic. Nothing is infected. Kaspersky's
PDM: prefix means the verdict comes from its Proactive Defense Module (System Watcher),
which judges what the installer does rather than matching it against known malware. When
it fires, Kaspersky "rolls back" the whole installation — deleting files it had already
written — so the app ends up broken or missing.
The files it flags are stock parts of declared, open-source dependencies bundled with the
desktop app, for example:
resources/app/.build/next/node_modules/playwright-<hash>/lib/…/agentParser.js and
workerProcessEntry.js — Playwright, the browser-automation
library used for in-app provider login and browser-backed chat.
resources/app/.build/next/node_modules/@wreq-js/binding-win32-<arch>-msvc-<hash>/wreq-js.win32-<arch>-msvc.node
— the pinned wreq-js native binding used for browser-fingerprinted HTTP on web-cookie
providers (<arch> is x64 or arm64).
Why it fires: the Windows installer is not yet code-signed, so an unsigned NSIS
installer has zero reputation and behavioral heuristics run at maximum aggression. Combined
with a bundled native DLL and hundreds of .js files written under
%LOCALAPPDATA%\Programs\OmniRoute (including hash-suffixed package directories from the
Next.js standalone build), that is enough to trip the heuristic. Code signing is planned;
until it lands, new releases can repeat this.
What to do:
Verify your download first (rules out a tampered file). Every release publishes
latest.yml, whose sha512 field (base64) covers the OmniRoute.Setup.<version>.exe
installer. In PowerShell, from the folder containing the installer:
The output must match latest.yml → sha512. If it does not, delete the file and
re-download only from the GitHub releases page.
Restore + exclude — restore the rolled-back items from quarantine and add an exclusion
for %LOCALAPPDATA%\Programs\OmniRoute (Kaspersky → Settings → Threats and Exclusions),
then reinstall.
Cause: You are running a Node.js version outside OmniRoute's approved secure runtime floor. The most common case is running an older Node 22 or 24 patch level that falls below the patched security floor OmniRoute requires.
Symptoms:
Login page shows a blank screen or a server error
Console shows Error: Module did not self-register or similar native binding errors
The login page shows an orange warning banner with your Node version if the runtime is outside the supported secure policy
Fix:
Install a supported Node.js LTS release (recommended: Node.js 24.x):
nvm install 24nvm use 24
Verify your version: node --version should show v24.0.0 or newer on the 24.x LTS line
Reinstall OmniRoute: npm install -g omniroute
Restart: omniroute
Supported secure versions:>=22.22.2 <23 or >=24.0.0 <27. Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported.
Cause: npm v11 (shipped with Node.js 24+) blocks install scripts for optional
dependencies by default. Since better-sqlite3 is listed in optionalDependencies
and requires native compilation (node-gyp rebuild), npm silently skips it.
Symptoms:
Server crashes on startup with Cannot find module 'better-sqlite3'
ls node_modules/better-sqlite3 shows "No such file or directory"
Cause: After a global npm install -g omniroute, the better-sqlite3 native binary inside the package may have been compiled for a different architecture or Node.js ABI than what is running locally. This is common on macOS (both Apple Silicon and Intel) when the pre-built binary does not match your environment.
Symptoms:
Server fails immediately on startup with a dlopen error
Error contains slice is not valid mach-o file
Full example:
dlopen(/Users/<user>/.nvm/versions/node/v24.14.1/lib/node_modules/omniroute/app/node_modules/better-sqlite3/build/Release/better_sqlite3.node, 0x0001): tried: '...' (slice is not valid mach-o file)
Fix — rebuild for your local environment (no Node.js downgrade required):
cd $(npm root -g)/omniroute/appnpm rebuild better-sqlite3omniroute
Note: This recompiles the native binding against your local Node.js version and CPU architecture, resolving the binary mismatch. The officially supported runtime range is >=22.22.2 <23 or >=24.0.0 <27 (SUPPORTED_NODE_RANGE in src/shared/utils/nodeRuntimeSupport.ts, aligned with the package.jsonengines field). Node.js 24.x LTS (Krypton) and Node.js 26 are fully supported with better-sqlite3 v12.x.
Cause: The API key validation endpoint (POST /api/providers/validate) was previously bypassing proxy configuration, causing failures in environments that require proxy routing.
Fix (v3.5.5+): This is now fixed. Provider validation routes through runWithProxyContext, honoring provider-level and global proxy settings automatically.
Cause: The MITM proxy and its CA certificate install into the environment where OmniRoute runs. Under WSL that environment is the Linux guest, while the AI desktop apps (Kiro, Trae, Copilot, Zed, …) run on the Windows host. The host apps do not trust the guest's certificate store and do not route through the guest's system proxy, so desktop interception does not engage there.
Recommendation: Run OmniRoute natively on the same OS as the desktop apps you want to intercept (Windows for Windows apps; macOS/Linux likewise). Keeping OmniRoute inside WSL while targeting host apps requires manually trusting the generated CA certificate on the Windows host and pointing each host app's network/proxy settings at the WSL proxy endpoint — an unsupported, fragile setup.
Cause: Kiro's backend enforces a single active session per OIDC client registration.
When two accounts share the same registered client (connections imported before v3.8.0),
refreshing one account's token invalidates the other's refresh token.
Fix (v3.8.0+): Re-import affected connections.
Starting with v3.8.0, every new Kiro connection created via Import Token,
Google/GitHub social login, or Auto-Import automatically registers its own
dedicated OIDC client. The connection is therefore fully isolated and refreshing one
account has no effect on any other account.
Connections that were imported before v3.8.0 do not carry a per-connection client
registration. Those connections continue to use the shared social-auth refresh endpoint.
To gain isolation, delete the old connection from Dashboard → Providers and re-add it
via any of the three import flows.
For full details and step-by-step instructions for adding two Kiro accounts side by side,
see docs/guides/KIRO_SETUP.md.
Symptoms:curl http://localhost:20128/v1/models returns curl: (56) Recv failure: Connection reset by peer. Dashboard and unauthenticated endpoints work, but authenticated endpoints fail — it looks like an auth problem but isn't.
Cause:docker run -p 20128:20128 publishes on both 0.0.0.0 (IPv4) and :: (IPv6), but the process inside the container listens on IPv4 only. On hosts where localhost resolves to ::1 first, the connection lands on the IPv6 published port with no listener behind it → connection reset.
Fix:
Quick diagnostic: Run curl -4 http://localhost:20128/v1/models. If it works with -4 but fails without, you have an IPv6 bind mismatch.
Permanent fix: Bind to IPv4 explicitly by using -p 127.0.0.1:20128:20128 in your docker run command:
Set APP_LOG_TO_FILE=true in your .env file. Application logs are written under logs/.
Request artifacts are stored under ${DATA_DIR}/call_logs/ when the call log pipeline is
enabled in settings.
When pipeline capture is enabled, set CALL_LOG_PIPELINE_CAPTURE_STREAM_CHUNKS=false to omit
stream chunk payloads, or tune CALL_LOG_PIPELINE_MAX_SIZE_KB to change the artifact cap in KB.
Use a model id whose first segment is a provider you have credentials for (openai/whisper-1, openrouter/deepgram/nova-3). Bare deepgram/nova-3 requires a native Deepgram key.
Verify the provider is connected in Dashboard → Providers
Thinking tags not appearing — Check if the target provider supports thinking and the thinking budget setting
Tool calls dropping — Some format translations may strip unsupported fields; verify in Playground mode
System prompt missing — Claude and Gemini handle system prompts differently; check translation output
SDK returns raw string instead of object — Resolved in v1.x; response sanitizer strips non-standard fields (x_groq, usage_breakdown, etc.) that cause OpenAI SDK Pydantic validation failures. If you still see this on v3.x+, please file an issue.
GLM/ERNIE rejects system role — Resolved in v1.x; role normalizer automatically merges system messages into user messages for incompatible models. If you still see this on v3.x+, please file an issue.
developer role not recognized — Resolved in v1.x; automatically converted to system for non-OpenAI providers. If you still see this on v3.x+, please file an issue.
json_schema not working with Gemini — Resolved in v1.x; response_format is now converted to Gemini's responseMimeType + responseSchema. If you still see this on v3.x+, please file an issue.
When many concurrent requests hit a rate-limited provider, OmniRoute uses mutex + auto rate-limiting to serialize requests and prevent cascading failures. This is automatic for API key providers.
The chat completions endpoint returns a retryable 503 response whose error code is
chat_admission_busy.
The response includes Retry-After. Since #12135 the value is derived from observed
occupancy — the larger of the OMNIROUTE_CHAT_ADMISSION_QUEUE_MS window the request already
waited and the time the current heavyweight leases have been held — rounded up to whole
seconds and capped at 60. On an idle gate it keeps the historical floors: 2 seconds on the
byte-based path, 1 second on the structure-based path (which also includes
reason: "structure_limit").
This can happen while another heavyweight chat or long-running streaming response is still
in flight.
The structure-based response uses the same type and code, with the message
Local chat admission capacity is busy for this structurally heavy request; upstream provider routing was not attempted. Retry shortly.
and reason: "structure_limit".
At the default thresholds, a request is structurally heavy when it has at least 200 messages,
at least 64 tools, or at least 32,000 estimated tokens, or when bounded structure estimation
exhausts its bounds of 10,000 visited nodes or depth 12.
Cause: This is deliberate load shedding inside OmniRoute, not an upstream-provider failure.
Each process uses a process-local guard to reserve limited heavyweight capacity before retaining
and parsing a large request body. A heavyweight lease remains held for the lifetime of an SSE
response.
#503-fanout: before this fix, the guard capped concurrency at a fixed request COUNT
(OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT, default 1) regardless of host memory, so coding-agent
fan-out (multiple subagents/CLIs, bodies routinely > 256 KB) collapsed to an effective
concurrency of ~1 and 503'd under completely normal load. The guard now self-tunes: it is gated
by an auto-derived ingest BYTE budget (OMNIROUTE_CHAT_MAX_INFLIGHT_BYTES) sized from the
process's real memory ceiling, and it also consults a live resource-pressure signal — so it
only sheds when the host is genuinely under memory pressure, not merely because more than one
heavy request arrived at once. The old count cap (OMNIROUTE_CHAT_MAX_HEAVY_IN_FLIGHT) is
still honored, but only if you explicitly set it.
When capacity is busy, a heavyweight request first waits up to
OMNIROUTE_CHAT_ADMISSION_QUEUE_MS (default 2000, 0 disables the wait) for a slot to free up
before answering the retryable 503. The bounded wait exists so agent-style clients
(OpenCode, Claude Code, Cursor) that fan out heavy sub-requests concurrently serialize the burst
instead of burning their whole retry budget on immediate rejections and dying mid-task.
Current heavyweight lease occupancy, the resolved byte budget, and live pressure severity are
surfaced at GET /api/monitoring/health → chatAdmission (inflightBytes, maxInflightBytes,
budgetSource, pressureSeverity, countCapEnabled) — check these before touching any env var.
Settings → Resilience → Request Queue → Concurrent Requests does not control this; that setting
governs a separate provider request-queue mechanism.
Fix:
Retry first. Clients should honor Retry-After and use backoff rather than immediately
repeating the request.
Check /api/monitoring/health → chatAdmission before tuning anything. countCapEnabled: false and a generous maxInflightBytes mean the auto-derived budget is already doing its
job; a pressureSeverity of high/critical means the host is genuinely low on memory —
that is not fixable by an admission env var, it needs more RAM or a smaller workload.
Only if /api/monitoring/health shows the auto-derived budget is genuinely too small for
your host (rare — it already scales from container to bare-metal), override it directly with
OMNIROUTE_CHAT_MAX_INFLIGHT_BYTES rather than falling back to the legacy request-count cap.
Some OmniRoute users place the gateway in front of RAG or agent stacks. In those setups it is common to see a strange pattern: OmniRoute looks healthy (providers up, routing profiles ok, no rate limit alerts) but the final answer is still wrong.
In practice these incidents usually come from the downstream RAG pipeline, not from the gateway itself.
If you want a shared vocabulary to describe those failures you can use the WFGY ProblemMap, an external MIT license text resource that defines sixteen recurring RAG / LLM failure patterns. At a high level it covers:
retrieval drift and broken context boundaries
empty or stale indexes and vector stores
embedding versus semantic mismatch
prompt assembly and context window issues
logic collapse and overconfident answers
long chain and agent coordination failures
multi agent memory and role drift
deployment and bootstrap ordering problems
The idea is simple:
When you investigate a bad response, capture:
user task and request
route or provider combo in OmniRoute
any RAG context used downstream (retrieved documents, tool calls, etc)
Map the incident to one or two WFGY ProblemMap numbers (No.1 … No.16).
Store the number in your own dashboard, runbook, or incident tracker next to the OmniRoute logs.
Use the corresponding WFGY page to decide whether you need to change your RAG stack, retriever, or routing strategy.
Full text and concrete recipes live here (MIT license, text only):
Very short or immediate cooldowns on ModelScope after a small burst of requests
Combo routing skips ModelScope earlier than expected
Cause: ModelScope emits provider-specific Retry-After headers. v3.8.0 ships dedicated handling for those headers, so older versions misread them as generic rate-limit hints.
Fix:
Ensure you are on v3.8.0 or later
Verify the useUpstream429BreakerHints toggle is enabled under Settings → Resilience
Warning logged: background mode degraded to synchronous
A background: true request returns a normal synchronous response instead of a background job handle
Cause: v3.8.0 intentionally degrades background: true on the Responses API to synchronous execution while emitting a warning. Full async background execution is a future deliverable.
Fix:
Adjust the client to call without background, or
Wait for a later release that ships full async background mode (track the changelog)
If the CLI prints ⚠ Server did not respond within 60s but the server is
actually working, the readiness probe budget is too short for your environment.
This commonly happens on Windows (antivirus, filesystem watchers) or containers
with heavy startup workloads.
Fix — raise the budget:
# Via env var (persists across starts):export OMNIROUTE_READY_TIMEOUT_MS=180000 # 3 minutesomniroute serve# Via CLI flag (one-off):omniroute serve --ready-timeout 180000
The default is 60 000 ms (60 s). The warning is informational only; the server
continues starting in the background and will be reachable once boot completes.