Export Quantized Models To Gguf For Local Inference

Soup is a Python CLI and web UI for configuring, fine-tuning, evaluating, and exporting LLMs. It supports methods such as LoRA and QLoRA, with layer streaming designed to train large models on low-VRAM GPUs.

AI Tools | Python · LLM fine-tuning · QLoRA