Open-source, MIT-licensed AI gateway and control plane that routes, governs and observes every LLM request behind one OpenAI-compatible endpoint.

ARBR works by combining OpenAI-Compatible Routing: A single drop-in endpoint over every major provider, with rules, difficulty-aware selection, cost guardrails and automatic fallback choosing the model per request., In-Path Governance: Budgets, rate limits, output guardrails, prompt-injection checks and kill switches enforce policy before inference rather than auditing it afterwards., Structured Observability: Cost, latency, tokens and routing decisions are emitted as structured events attributed by application, team, model and user, viewable in local dashboards or exported to OpenTelemetry backends such as Datadog, Grafana and Prometheus., LLM-Judge Evaluation: A sample of live traffic is scored for quality so requests can be routed to the cheapest model that provably clears the bar, rather than optimising on price alone., Safe Model Deployment: Canary and shadow new models against real traffic with regression gates that block promotion until evaluations pass, plus instant rollback. to help users with LLM Cost Reduction: Route summarisation and extraction traffic to cheap small models while reserving frontier models for analysis, cutting spend without hand-editing every call site., AI Spend Attribution: Give finance and engineering a per-application, per-team and per-user breakdown of token spend so AI budgets can be owned by the groups that generate them., Enterprise AI Governance: Enforce departmental budgets, rate limits and kill switches in the request path so a runaway agent cannot exhaust a quarter's inference budget., Provider Risk Mitigation: Keep applications provider-neutral behind one endpoint with automatic fallback, so a single vendor outage or price change does not require a code change., Model Migration Testing: Shadow or canary a newly released model against production traffic and let regression gates decide whether it is promoted..

Key features include OpenAI-Compatible Routing: A single drop-in endpoint over every major provider, with rules, difficulty-aware selection, cost guardrails and automatic fallback choosing the model per request., In-Path Governance: Budgets, rate limits, output guardrails, prompt-injection checks and kill switches enforce policy before inference rather than auditing it afterwards., Structured Observability: Cost, latency, tokens and routing decisions are emitted as structured events attributed by application, team, model and user, viewable in local dashboards or exported to OpenTelemetry backends such as Datadog, Grafana and Prometheus., LLM-Judge Evaluation: A sample of live traffic is scored for quality so requests can be routed to the cheapest model that provably clears the bar, rather than optimising on price alone., Safe Model Deployment: Canary and shadow new models against real traffic with regression gates that block promotion until evaluations pass, plus instant rollback..

ARBR is useful for anyone interested in LLM Cost Reduction: Route summarisation and extraction traffic to cheap small models while reserving frontier models for analysis, cutting spend without hand-editing every call site., AI Spend Attribution: Give finance and engineering a per-application, per-team and per-user breakdown of token spend so AI budgets can be owned by the groups that generate them., Enterprise AI Governance: Enforce departmental budgets, rate limits and kill switches in the request path so a runaway agent cannot exhaust a quarter's inference budget., Provider Risk Mitigation: Keep applications provider-neutral behind one endpoint with automatic fallback, so a single vendor outage or price change does not require a code change., Model Migration Testing: Shadow or canary a newly released model against production traffic and let regression gates decide whether it is promoted..

ARBR is free to use.

Visit https://projectarbr.org/ to sign up and explore ARBR.