BerriAI/LiteLLM
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
LiteLLM is an open-source Python SDK and proxy server from BerriAI that lets you call 100+ LLM APIs through one OpenAI-compatible interface. It adds cost tracking, rate limiting, load balancing, fallbacks, and logging in front of providers like OpenAI, Anthropic, Bedrock, and Vertex AI.
Our Review
LiteLLM, from BerriAI, has 48,000 GitHub stars and has quietly become plumbing for a huge share of production AI apps. Its job is unglamorous but vital: give every model provider the same interface, so swapping or load-balancing across vendors is a config change rather than a rewrite.
What LiteLLM does:
- •One interface for 100+ providers call OpenAI, Anthropic, Google, Bedrock, Azure, Vertex, local models, and more in the OpenAI format.
- •Proxy server (AI gateway) run a central gateway so every team and app shares one endpoint, key management, and budget.
- •Cost tracking and budgets track spend per key, user, or team, and enforce limits to avoid runaway bills.
- •Reliability features automatic retries, fallbacks, and load balancing across providers and keys.
- •Guardrails and logging apply safety checks and log requests for debugging and audit.
- •Python SDK or standalone use it as a library in code or as a deployed proxy in front of your stack.
Getting started:
Install with pip install litellm, then call litellm.completion(model="...", messages=[...]), or run the proxy with litellm --config to expose one OpenAI-compatible endpoint. Docs at docs.litellm.ai.
Limitations:
LiteLLM moves extremely fast, so frequent releases and a large open issue count mean occasional breaking changes and rough edges. The proxy adds an operational component you run and scale yourself, plus a small latency hop. Advanced enterprise features such as SSO and fine-grained governance sit behind a paid tier, and supporting 100+ providers means provider-specific quirks still surface.
Our Verdict
LiteLLM is close to a default in 2026 for any team calling more than one LLM provider. If you are tired of writing separate clients for OpenAI, Anthropic, and Bedrock, or you need central cost control and fallbacks across vendors, LiteLLM solves exactly that, with 48,000 stars and a permissive core license.
For developers, you can adopt it incrementally: start with the Python SDK to normalize calls, then add the proxy when you want one gateway with shared keys, budgets, retries, and logging for a whole team. Because everything speaks the OpenAI format, existing code and tools work with minimal changes.
Skip LiteLLM if you only ever call one provider and value the lowest possible latency; a direct SDK avoids the extra layer. If you want a fully hosted gateway with no infrastructure to run, OpenRouter offers a similar multi-provider experience as a managed service.
Frequently Asked Questions
What is LiteLLM?
LiteLLM is an open-source Python SDK and proxy server from BerriAI that lets you call more than 100 LLM providers through a single OpenAI-compatible interface. It adds cost tracking, budgets, rate limiting, load balancing, fallbacks, guardrails, and logging, so applications and teams can use many models without writing provider-specific code for each one.
Is LiteLLM free and open source?
Yes. LiteLLM's core is open source under the MIT license and free to use as both a Python library and a self-hosted proxy as of 2026. Advanced enterprise features such as SSO and fine-grained governance are part of a paid tier, but the core SDK and gateway cost nothing to run.
Which LLM providers does LiteLLM support?
LiteLLM supports more than 100 providers as of 2026, including OpenAI, Anthropic, Google Gemini and Vertex AI, AWS Bedrock and SageMaker, Azure OpenAI, Cohere, Hugging Face, NVIDIA NIM, and local models via vLLM and Ollama. All of them are exposed in the OpenAI request and response format, so you can switch providers with a config change.
What is the LiteLLM proxy?
The LiteLLM proxy, also called the AI gateway, is a server you run in front of your LLM providers. It gives every app and team one endpoint and one set of virtual keys, with central budgets, rate limits, retries, fallbacks, and logging. It is how organizations standardize and govern LLM access instead of each service holding its own provider keys.
How is LiteLLM different from OpenRouter?
Both give you many models through one interface. LiteLLM is open source: you self-host the SDK or proxy and bring your own provider keys, keeping full control. OpenRouter is a managed hosted gateway with its own billing. Choose LiteLLM for self-hosting and data control; choose OpenRouter for a zero-ops, fully managed multi-model API.
How do I install LiteLLM?
Visit the GitHub repository at https://github.com/BerriAI/litellm for installation instructions.
What license does LiteLLM use?
LiteLLM uses the MIT license.
What are alternatives to LiteLLM?
Explore related tools and alternatives on My AI Guide.
Open source & community-verified
MIT licensed: free to use in any project, no strings attached. 49,199 developers have starred this, meaning the community has reviewed and trusted it.
Reviewed by My AI Guide for relevance, quality, and active maintenance before listing.
Topics