Quantize Models For Faster Inference And Smaller Checkpoints

Python library for optimizing PyTorch, Hugging Face, and ONNX models with quantization, pruning, distillation, sparsity, NAS, and speculative decoding. Exports optimized checkpoints for deployment with TensorRT-LLM, TensorRT, vLLM, and SGLang.

AI Tools | Python · model optimization · quantization