Serve Local Or Hosted Language Models Through A Unified Api

Rust CLI and server for running local or hosted models and connecting them to coding agents. Packages models as OCI artifacts, supports llama.cpp, vLLM, SGLang, and MLX-LM, and exposes compatible Ollama, OpenAI, and Anthropic APIs.

AI Frameworks | Rust · local inference · LLM serving