Self-host An Openai-compatible Inference Server

Harbor is a CLI and companion app that boots a fully pre-wired local LLM stack via Docker Compose. One command starts backends (Ollama, llama.cpp, vLLM, MLX), frontends (Open WebUI, LibreChat), and services like SearXNG, Speaches, and ComfyUI, with cross-service config handled for you.

AI Tools | Python · llm-stack · docker-compose