OliveModels

The following models are available in Olive.

Model Configuration

class olive.model.ModelConfig(*, type: str, config: dict)

ONNX Model

class olive.model.ONNXModel(model_path: str | Path | ResourcePath | ResourcePathConfig | None = None, onnx_file_name: str | None = None, inference_settings: dict | None = None, use_ort_extensions: bool = False, model_attributes: Dict[str, Any] | None = None)

CompositeOnnxModel Model

class olive.model.CompositeOnnxModel(model_components: List[ONNXModel | Dict[str, Any]], model_component_names: List[str], model_attributes: Dict[str, Any] | None = None)

CompositeOnnxModel represents multi component models. Whisper is an example composite model that has encoder and decoder components. CompositeOnnxModel is a collection of OnnxModels.

DistributedOnnxModel Model

class olive.model.DistributedOnnxModel(model_filepaths: List[Path | str] = [], inference_settings: dict | None = None, use_ort_extensions: bool = False, model_attributes: Dict[str, Any] | None = None)

OpenVINO Model

class olive.model.OpenVINOModel(model_path: str | Path | ResourcePath | ResourcePathConfig | None, model_attributes: Dict[str, Any] | None = None)

PyTorch Model

class olive.model.PyTorchModel(model_path: str | Path | ResourcePath | ResourcePathConfig | None = None, model_file_format: ModelFileFormat = ModelFileFormat.PYTORCH_ENTIRE_MODEL, model_loader: str | Callable | None = None, model_script: Path | str | None = None, script_dir: Path | str | None = None, io_config: Dict[str, Any] | IOConfig | None = None, dummy_inputs_func: str | Callable | None = None, hf_config: Dict[str, Any] | HFConfig | None = None, adapter_path: str | Path | ResourcePath | ResourcePathConfig | None = None, model_attributes: Dict[str, Any] | None = None)

SNPE Model

class olive.model.SNPEModel(input_names: List[str], input_shapes: List[List[int]], output_names: List[str], output_shapes: List[List[int]], model_path: str | Path | ResourcePath | ResourcePathConfig | None = None, model_attributes: Dict[str, Any] | None = None)