How Tos#

Installation and Setup#

• How to install Olive

Olive Command Lines#

The Olive CLI provides a set of primitives such as quantize, finetune, onnx-graph-capture, auto-opt that enable you to easily optimize select models and experiment with different cutting-edge optimization strategies without the need to define workflows.

-How to use the olive optimize command to optimize a Pytorch model

• How to use the olive auto-opt command to take a PyTorch/Hugging Face model and turn it into an optimized ONNX model

• how to use the olive finetune command to create (Q)LoRA adapters

• How to use the olive quantize command to quantize your model with different precisions and techniques such as AWQ

• How to use the olive run command to execute an Olive workflow.

Olive Python API#

• How to use Python API

Customize Workflow (aka Recipes)#

• How to write a new workflow from scratch

• How to define input model for a new workflow

• How to customize a pass parameters

• How to setup custom dataset for calibration and evaluation

• How to define evaluation metrics such as accuracy, latency, throughput, and your own custom metrics

• How to package output model for deployment

• How to define host or target systems

• How to configure Olive engine

Extend Olive#

• Olive design overview

• How to add a new Pass

• How to add custom model evaluator

• How to add custom scripts to load datasets