Vibe Coding Discoverlitellm
View on GitHubThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
LiteLLM is an open-source AI gateway and Python SDK that calls 100+ LLM providers (OpenAI, Anthropic, Bedrock, Vertex, vLLM, Ollama) in OpenAI format. The proxy adds virtual keys, spend tracking, guardrails, load balancing, logging, plus MCP tool and A2A agent routing.
Use Cases
Call 100+ LLM providers through one OpenAI-compatible APISelf-host a centralized LLM proxy for a team or orgTrack LLM spend and enforce per-key budgetsCreate virtual API keys with rate limits for usersLoad balance and fall back across model providersExpose MCP servers as tools to any LLM via a gatewayRoute and invoke A2A agents behind one endpointApply guardrails and content moderation on LLM trafficLog and observe LLM requests for complianceMigrate an existing OpenAI/Azure app to other providersServe embeddings, images, audio, batches and rerank endpointsUse as a Python SDK for direct provider-agnostic completions
Built With
- Language
- Python
- Frameworks
- FastAPI · Pydantic · OpenAI Python SDK · LangChain · vLLM · Ollama · MCP SDK · a2a-sdk · OpenTelemetry · Prisma · React · uvicorn · httpx · tiktoken · boto3
Tags
ai-gateway · llm-gateway · llmops · openai-compatible · proxy-server · cost-tracking · guardrails · load-balancing · mcp-gateway · a2a · multi-provider · python-sdk · rust · observability · virtual-keys · rate-limiting