LiteLLM Cheat Sheet

LiteLLM cheat sheet

LiteLLM gives you a single, OpenAI-shaped interface to call 100+ providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, Groq, Ollama and more — by just changing the model string. Same request format, same response object (response.choices[0].message.content), everywhere. Use it as a Python SDK (completion()), a Router for load-balancing/fallbacks across deployments, or a standalone proxy gateway. This sheet targets LiteLLM 1.x.

core calls streaming / async providers & params router / reliability cost / proxy gotcha ★ most common

Verified 2026-08-24 against the official docs at docs.litellm.ai (LiteLLM 1.x). Responses follow the OpenAI schema; the same code runs against any supported provider by swapping the model prefix.

1Install & completion()the one call

pip install litellm★
SDK only. Add litellm[proxy] for the gateway server.
from litellm import completion r = completion(model="openai/gpt-4o", messages=msgs)★
The universal call. Swap the model string to change provider — everything else stays identical.
r.choices[0].message.content★
Response follows the OpenAI schema for every provider: choices, message, usage, id, model.
export OPENAI_API_KEY=... ANTHROPIC_API_KEY=... GEMINI_API_KEY=...
Keys come from env vars by default (per provider), or pass api_key= explicitly.

2Messages, params, responsethe request

msgs = [{"role":"system","content":"..."}, {"role":"user","content":"Hi"}]★
Standard chat-message list. Multimodal content parts (image/audio) supported for capable models.
completion(model, messages, temperature=0.7, max_tokens=500)★
OpenAI-style params (top_p, stop, n, seed) are translated to each provider's native format.
completion(..., response_format={"type":"json_object"})
JSON mode / structured output; pass a Pydantic model as response_format= where supported.
completion(..., tools=tools, tool_choice="auto")
Function/tool calling with the OpenAI schema; read r.choices[0].message.tool_calls.

3Streaming & asyncnon-blocking

for chunk in completion(model, messages, stream=True): print(chunk.choices[0].delta.content or "", end="")★
Streaming yields OpenAI-style delta chunks for every provider.
from litellm import acompletion r = await acompletion(model, messages)★
Async twin of completion. stream=True gives an async iterator.
from litellm import stream_chunk_builder full = stream_chunk_builder(chunks, messages=msgs)
Reassemble streamed chunks into a single completion object (with usage).

4Embeddings & moreother endpoints

from litellm import embedding embedding(model="openai/text-embedding-3-small", input=["txt"])★
Same unified shape for embeddings. aembedding for async.
litellm.transcription(...) · image_generation(...) · rerank(...)
Audio, image and rerank endpoints where the provider supports them.
litellm.completion(model="...", mock_response="hi")
Return a canned response — test your app without hitting (or paying) a provider.

7Router — load balancingmany deployments

from litellm import Router router = Router(model_list=[...])★
A drop-in that spreads calls across multiple deployments of the same logical model — with retries, cooldowns, fallbacks & cost tracking built in.
model_list=[{"model_name":"gpt", "litellm_params":{"model":"azure/dep1", "api_key":...}}, ...]★
Group several real deployments under one model_name; callers just ask for "gpt".
router.completion(model="gpt", messages=msgs)★
Same signature as completion (and router.acompletion). The router picks a healthy deployment.
Router(..., routing_strategy="latency-based-routing")
Also simple-shuffle, least-busy, usage-based-routing (respects per-deployment TPM/RPM limits).

8Fallbacks & retriesfailover

Router(..., fallbacks=[{"gpt": ["claude"]}])★
On a 429/500 for gpt, automatically retry the request on claude. General-error failover.
context_window_fallbacks=[...] # needs enable_pre_call_checks=True
Route to a bigger-context model when the input overflows the current one.
content_policy_fallbacks=[...]
On a content-policy refusal, send to a different provider instead of failing.
Router(..., num_retries=3, cooldown_time=60, allowed_fails=2)
Retries with backoff; a deployment that fails too often is put in cooldown and skipped.

!Common gotchasread before shipping

unsupported params error by default
Sending e.g. logprobs to a provider that lacks it raises. Set litellm.drop_params=True for portable code.
right key for the right prefix
Each provider reads its own env var (ANTHROPIC_API_KEY, GEMINI_API_KEY, ...). Wrong-key errors usually mean a missing/mismatched var.
Azure/Bedrock need extra config
Azure wants api_base+api_version and the deployment name; Bedrock/Vertex use cloud credentials, not a simple key.
costs depend on the price map
completion_cost relies on LiteLLM's model-price table — a brand-new model may be missing until you update or supply pricing.
capabilities still differ
A unified API doesn't unify features — tool calling, JSON mode, vision and context limits vary by model. Check per-model support.

Worth memorizing

one call, any providercompletion(model="provider/name", messages=...) with an OpenAI-shaped response

prefix picks the backendopenai/ anthropic/ gemini/ bedrock/ azure/ ollama/ ...

response is always OpenAI schemar.choices[0].message.content; r.usage.total_tokens

stream=True yields deltaschunk.choices[0].delta.content — same for every provider

acompletion / aembeddingasync twins of the sync calls

drop_params=True for portabilityquietly drop params a provider doesn't support

Router = load balance + failovermodel_list, fallbacks, retries, cooldowns, cost tracking

completion_cost + callbacksper-call spend; one-line Langfuse/Prometheus/OTel logging

proxy = OpenAI-compatible gatewaylitellm --config; point any OpenAI SDK at it

LiteLLM cheat sheet

Outline

Call

Providers

Reliability

Ops

Call Any Model

Providers & Params

Router & Reliability

Cost, Observability & Proxy

Gotchas

Worth memorizing