Provide A Streaming Chat UI For Served Models

Python framework for packaging and serving predictive models and LLMs as production APIs. Includes OpenAI-, Anthropic-, and Ollama-compatible endpoints, a built-in chat UI, and native MCP support.

AI Frameworks | Python · model serving · LLM inference