Export Optimized Models For Tensorrt-llm, Vllm, Or Sglang Deployment

Python library for optimizing PyTorch, Hugging Face, and ONNX models with quantization, pruning, distillation, sparsity, NAS, and speculative decoding. Exports optimized checkpoints for deployment with TensorRT-LLM, TensorRT, vLLM, and SGLang.

AI Tools | Python · model optimization · quantization