Run Gguf Language Models Locally

Rust-native local inference backend and chat app for tested GGUF models, with Metal, CUDA, and CPU execution paths. Includes an OpenAI-compatible API, embeddings and reranking, and approval-gated coding tools.

AI Frameworks | Rust · local inference · GGUF