Model Quantizations

Olive supports multiple Quantization algorithms. Olive support PyTorch and ONNX model quantization.

• PyTorch
  • AutoGPTQ
  • AutoAWQ
  • QuaRot
• ONNX
  • Quantize with onnxruntime
  • Quantize with Intel® Neural Compressor
  • Quantize with AMD Vitis AI Quantizer
  • NVIDIA TensorRT Model Optimizer-Windows