LiteLLM Alternatives: 10 AI Gateways Compared for 2026

The best LiteLLM alternatives in 2026 are Bifrost, Portkey, Kong AI Gateway, Agent Router (formerly Envoy AI Gateway), TrueFoundry, Cloudflare AI Gateway, Vercel AI Gateway, OpenRouter and Helicone, plus Coworker if you want routing built into a finished AI platform instead of a proxy you run. Which one fits depends on why you are leaving: throughput points to Bifrost, Kubernetes-native control to Agent Router or Kong, managed governance to Portkey or TrueFoundry, and zero infrastructure to Cloudflare, Vercel or OpenRouter.

I checked every vendor's own site, docs, pricing page or GitHub repo on October 6, 2026; the prices and counts below are theirs. New to gateways? Start with the LLM gateway guide.

LiteLLM alternatives at a glance

What is LiteLLM, and why do teams look for alternatives?

LiteLLM, from BerriAI, is an open-source Python SDK and proxy that puts one OpenAI-compatible API in front of many model providers. Its homepage claims 140+ providers and 1,800+ models, and its GitHub repository has more than 60,000 stars. The free edition includes virtual keys, budgets, spend tracking, fallbacks, request logging and Prometheus metrics, per its pricing page, plus latency-based and cost-based routing. Teams still leave for four reasons.

Throughput at high volume. LiteLLM's own benchmarks page shows stock v1.101.0 settling near 190 requests per second, with 92.07% client-side success, on 50K to 100K-token prompts, while a high-throughput profile in nightly builds reached 3,000. The project is moving its core to Rust: 5% of major APIs as of v1.104.0-rc.1, with a goal of 100% by December 31, 2026.

Licensed enterprise features. SSO is free for up to 5 users. Beyond that, and for SCIM, audit logs, virtual key rotation and multi-region deployment, you need an enterprise license, quoted on annual request capacity.

Upgrade cadence. A new minor version ships roughly every week, and since June 29, 2026 only the four most recent stable minor lines receive fixes.

The March 2026 supply-chain incident. On March 24, 2026, PyPI releases 1.82.7 and 1.82.8 shipped with a credential stealer. LiteLLM's security update says they were live from 10:39 UTC for about 40 minutes before PyPI quarantined them, and that users of the official Docker image, which pins dependencies, were not affected. Datadog Security Labs ties it to a wider campaign that began with the Trivy scanner on March 19.

When should you stay on LiteLLM?

Stay if you want an MIT-licensed gateway you control, need the widest provider list here, and your volume fits the Python path. Run the official Docker image, pin versions, and plan for monthly upgrades.

Which open-source LiteLLM alternatives can you self-host?

1. Bifrost (by Maxim AI)

Bifrost is an Apache 2.0 gateway written in Go that covers 20+ providers, per its docs. The free edition runs on Docker, Kubernetes or as a single binary, with fallbacks, semantic caching, virtual-key budgets, custom routing rules, OpenTelemetry metrics and traces, and an MCP gateway, per its pricing page. Enterprise (custom pricing) adds guardrails, cluster mode, adaptive load balancing, SSO and audit logs, with VPC, on-prem or air-gapped deployment.

Best for: teams leaving over latency or memory. Maxim's own benchmark, first published in June 2025, reports a P99 of 1.68 seconds for Bifrost against 90.72 seconds for LiteLLM at 500 requests per second on a 2 vCPU AWS t3.medium. It does not name the LiteLLM version and predates LiteLLM's 2026 performance work, so rerun it yourself. A LiteLLM compatibility plugin eases migration. Trade-off: SSO and audit logs are paid, as with LiteLLM.

2. Agent Router (formerly Envoy AI Gateway)

Envoy AI Gateway became Agent Router when it joined the Agentic AI Foundation on September 10, 2026, per the project blog. The code, maintainers and Apache 2.0 license are unchanged. Per its site, it routes OpenAI-compatible requests to 17 providers out of the box, with provider fallback, model name virtualization, token limits per team or app, and a router for MCP tools. Telemetry follows OpenTelemetry GenAI conventions.

Best for: Kubernetes platform teams that manage policy as configuration. Tetrate offers a hosted version. Trade-off: beyond the laptop CLI, its getting-started guide deploys on Kubernetes with Envoy Gateway, which is heavy if you only need one proxy.

3. Kong AI Gateway

Kong extends its API gateway to LLM, MCP and agent-to-agent traffic. The open-source Kong Gateway (Apache 2.0) ships the basic AI Proxy plugin. AI Proxy Advanced, which balances traffic across models by lowest latency, lowest usage, semantic match or priority failover, is a licensed enterprise plugin, as is semantic caching. Kong's docs list 19 providers, from OpenAI to vLLM.

Pricing: Konnect Plus starts at $25 a month plus usage, each LLM proxied costs $100 a month (up to 5 on Plus), and each million requests past the first costs $200, per Kong's pricing page. Fully self-hosted deployment needs Kong Gateway Enterprise.

Best for: companies where Kong already fronts their APIs. Trade-off: routing across 5 models adds $500 a month on Plus before request charges.

Which managed gateways replace LiteLLM's governance features?

4. Portkey (now part of Palo Alto Networks)

Portkey pairs a gateway with observability, guardrails and prompt management, claims 1,600+ LLMs on its homepage, and supports conditional routing, fallbacks, load balancing and canary tests. On its pricing page, Developer is free for 10,000 logs a month but not suitable for production, Production is $49 a month for 100,000 logs plus $9 per extra 100,000 requests, and Enterprise adds VPC hosting, SSO and HIPAA.

Palo Alto Networks completed its acquisition of Portkey on May 29, 2026 and launched it as Prisma AIRS AI Gateway on July 16. As of October 6, the last commit to the MIT-licensed open-source gateway was on May 25, 2026.

Best for: managed controls backed by a security vendor. Trade-off: weigh that inactivity if you self-host the gateway.

5. TrueFoundry

TrueFoundry sells AI, MCP and agent gateways. Its gateway page claims 1,600+ models, latency-based routing, weighted load balancing, automatic fallback and sub-3ms internal latency. On its pricing page, Developer is $0 for up to 3 users, Pro is $25 per user per month with 20,000 requests per user plus $20 per additional 100,000, and Enterprise is custom.

Best for: regulated companies that want budgets and guardrails per team inside their own cloud. Trade-off: VPC, on-prem and air-gapped deployment are Enterprise only.

6. Helicone (now part of Mintlify)

Helicone is open-source (Apache 2.0) LLM observability with an OpenAI-compatible gateway covering 100+ models, automatic fallbacks and credits at 0% markup, per its docs. On its pricing page, Hobby is free for 10,000 requests, Pro is $79 a month and Team is $799 a month, plus usage.

Watch out: Mintlify acquired Helicone in March 2026, and the announcement says services stay live in maintenance mode, with security updates and new models still shipping. That suits logging you already rely on, not a new long-term gateway.

Which hosted gateways need no infrastructure at all?

7. Cloudflare AI Gateway

Cloudflare AI Gateway runs on Cloudflare's network with nothing to deploy. Analytics, caching and rate limiting are free on all plans, retries and model fallbacks are built in, and dynamic routing (in beta) adds conditional, percentage and budget steps. Its docs list 24 providers. Per the pricing page, Unified Billing adds a 5% fee on purchased credits with no markup on provider tokens.

Best for: apps already on Cloudflare. Trade-off: you cannot self-host it, and spend limits and guardrails are still in beta.

8. Vercel AI Gateway

Vercel AI Gateway is managed and callable from any infrastructure, with hundreds of models, provider and model fallbacks, budgets per team, project or key, and a log of every routing attempt. Pricing is the provider's list price with no markup or platform fee on tokens, including when you bring your own keys.

Best for: teams on Vercel or the AI SDK. Trade-off: it is managed only, and some controls, such as team-wide zero data retention at $0.10 per 1,000 requests, are paid add-ons.

9. OpenRouter

OpenRouter offers 500+ models from 80+ providers through one API, with automatic provider fallback and an Auto Router. Provider prices pass through without markup, plus a platform fee of 5.5% on the pay-as-you-go Standard plan and 8% on Business, per its pricing page. Bring-your-own-key use is fee-free up to $25,000 of list-price inference a month, then 5%.

Best for: many models on one bill. Trade-off: it replaces model access, not a self-hosted control plane. See the OpenRouter pricing guide and our OpenRouter alternative page.

What if you would rather not run a gateway at all?

10. Coworker

Coworker is not a drop-in, self-hosted proxy. It is a finished AI platform your team uses, not an API layer for your own app, with routing built in. Coworker's LLM gateway picks a model per task across Anthropic, OpenAI and Google, plus open-weight models from Moonshot and Z.ai, all US-hosted.

The other half is context. OM2, Coworker's organizational memory, draws on 50+ connectors such as Slack, Salesforce, Jira and GitHub, and respects the access controls already set in each tool. Routing changes which model answers, while OM2 narrows context before any model sees it. In Coworker's 100-task benchmark, OM2 plus model routing cut token cost 51x versus Claude with its native connectors (statistically significant benchmarks comparing Coworker MCP vs. Claude Native Tooling).

Best for: teams that want AI with company context, in Coworker's apps or through Coworker MCP in Claude, ChatGPT or Cursor. Trade-off: if you are building your own product and need an OpenAI-compatible endpoint with per-key budgets, pick a gateway above.

How do you choose between LiteLLM alternatives?

Start from the reason you are leaving.

What should you test before you switch?

1. Replay a day of production traffic, including streaming and tool calls, and measure p50 and p99 overhead yourself.
2. Check what triggers a fallback, and alert on the fallback rate.
3. Reconcile the gateway's cost report with one provider invoice, using the LLM cost calculator for a baseline.
4. Map keys, budgets and team permissions before cutover, and pin versions for anything you self-host.

For tracing beyond what a gateway logs, see LLM observability.

If the real goal is AI in your team's daily work rather than another proxy to run, book a demo. Or request a Context Impact Report: Coworker runs the same benchmark on your company's data and covers the token costs.

Frequently asked questions

What is the best open-source alternative to LiteLLM?

Bifrost and Agent Router (formerly Envoy AI Gateway), both Apache 2.0. Bifrost is a single Go service with fallbacks, semantic caching and budgets for free. Agent Router is built on Envoy for Kubernetes platform teams. Kong Gateway is also Apache 2.0, but multi-model load balancing needs an enterprise license.

Is LiteLLM free for commercial use?

Yes. The core is MIT licensed and free to self-host, with virtual keys, budgets, fallbacks and logging. The repository's enterprise directory carries a commercial license, and SSO beyond 5 users, SCIM and audit logs require a paid license.

Is LiteLLM safe to use after the March 2026 supply-chain attack?

The compromise hit two PyPI releases, 1.82.7 and 1.82.8, on March 24, 2026, and PyPI quarantined them. LiteLLM says its official Docker image was not affected. It has rebuilt its release pipeline and publishes SHA-256 checksums for audited releases. If either version ran in your environment, LiteLLM advises rotating every secret on those systems.

What is the difference between LiteLLM and OpenRouter?

LiteLLM is software you host in front of your own provider accounts. OpenRouter is a hosted service with 500+ models on one account and one bill, passing provider prices through plus a platform fee. Pick LiteLLM for control over keys and traffic, and OpenRouter for model breadth with nothing to run.

What is the difference between LiteLLM and Portkey?

Both offer an OpenAI-compatible gateway with fallbacks, load balancing and budgets. LiteLLM is mainly self-hosted open source. Portkey is mainly a managed platform with observability, guardrails and prompt management, owned by Palo Alto Networks since May 2026.

What is the best Portkey alternative?

TrueFoundry is the closest managed match, with routing, guardrails, budgets and prompt management, and it runs in your own cloud on the Enterprise plan. Bifrost is the closest open-source option, with a release shipped the day I checked. If you mainly used Portkey for logs and spend tracking, Cloudflare or Vercel cover that with nothing to host.

Can Coworker replace LiteLLM?

Only for giving people AI with routing handled for them. Coworker is a finished platform your team uses, not a drop-in self-hosted proxy for your application's API traffic. It routes across Anthropic, OpenAI, Google, Moonshot and Z.ai models and adds organizational memory across 50+ tools.

Related reading

• Coworker LLM gateway
• What is an LLM gateway?
• OpenRouter alternative
• OpenRouter pricing
• LLM cost calculator
• LLM observability
• What is MCP?
• Coworker benchmarks