Serve Models Through An Openai-compatible Api

Published projects tagged with this use case.

1 project

gufo

★ 442

A C++ inference engine optimized for AMD Strix Halo systems with ROCm. It serves supported language, audio, image, and video models through a local OpenAI-compatible API.

AI Frameworks | C++ · LLM inference · local AI

View Project →