Load Balance And Fall Back Across Model Providers

LiteLLM is an open-source AI gateway and Python SDK that calls 100+ LLM providers (OpenAI, Anthropic, Bedrock, Vertex, vLLM, Ollama) in OpenAI format. The proxy adds virtual keys, spend tracking, guardrails, load balancing, logging, plus MCP tool and A2A agent routing.

AI Frameworks | Python · ai-gateway · llm-gateway