Pool Inference Capacity Across Multiple Machines

Rust CLI and server for running local or hosted models and connecting them to coding agents. Packages models as OCI artifacts, supports llama.cpp, vLLM, SGLang, and MLX-LM, and exposes compatible Ollama, OpenAI, and Anthropic APIs.

AI Frameworks | Rust · local inference · LLM serving