Engine

Engine

class olive.engine.Engine(config: Dict[str, Any] | EngineConfig | None = None, search_strategy: SearchStrategy | None = None, host: OliveSystem | None = None, target: OliveSystem | None = None, evaluator_config: OliveEvaluatorConfig | None = None, execution_providers: List[str] | None = None)

The engine executes the registered Olive Steps, facilitate evaluation of the output models using provided evaluation criteria and produces output model(s).

register(pass_type: Type[Pass], config: Dict[str, Any] | None = None, disable_search=False, name: str | None = None, host: OliveSystem | None = None, evaluator_config: OliveEvaluatorConfig | None = None, clean_run_cache: bool = False, output_name: str | None = None)

Register a pass configuration so that it could be instantiated and executed later.

run(input_model_config: ModelConfig, data_root: str | None = None, packaging_config: PackagingConfig | None = None, output_dir: str | None = None, output_name: str | None = None, evaluate_input_model: bool = True)

Run all the registered Olive passes on the input model and produce one or more candidate models.

Parameters:

• input_model_config – input Olive model configuration

• packaging_config – packaging configuration, if packaging_config is provided, the output model will be packaged into a zip file.

• output_dir – output directory for the output model

• output_name – output name for the output model, if output_name is provided, the output model will be saved to engine’s output_dir with the prefix of output_name.

• evaluate_input_model – if evaluate_input_model is True, run the evaluation on the input model.

Returns:

if search strategy is None, all passes are run in the order they were registered.

1. Final model -> {output_dir}/{output_name}_{AcceleratorSpec}_model.onnx

2. JSON file -> {output_dir}/{output_name}_{AcceleratorSpec}_model.json

3. Evaluation results of the final model -> {output_dir}/{output_name}_{AcceleratorSpec}_metrics.json

Return footprint/zip(packaging_config) of the final model and evaluation results of the final model.

if search strategy is not None, run the search strategy to find candidate models. Return footprint/zip(packaging_config) of candidate models and evaluation results.

Note

All parameters that of type ...Config or ConfigBase class can be assigned dictionaries with keys corresponding to the fields of the class.

EngineConfig

pydantic settings olive.engine.EngineConfig

field search_strategy: SearchStrategyConfig | bool = None

field host: SystemConfig = None

field target: SystemConfig = None

field execution_providers: List[str] = None

field evaluator: OliveEvaluatorConfig = None

field azureml_client_config: AzureMLClientConfig | None = None

field packaging_config: PackagingConfig = None

field cache_dir: Path | str = '.olive-cache'

field clean_cache: bool = False

field clean_evaluation_cache: bool = False

field plot_pareto_frontier: bool = False

SearchStrategyConfig

pydantic settings olive.strategy.search_strategy.SearchStrategyConfig

field execution_order: str [Required]

field search_algorithm: str [Required]

field search_algorithm_config: ConfigBase = None

field output_model_num: int = None

field stop_when_goals_met: bool = False

field max_iter: int = None

field max_time: int = None

SystemConfig

pydantic settings olive.systems.system_config.SystemConfig

field type: SystemType [Required]

field config: TargetUserConfig = None

create_system()

OliveEvaluatorConfig

pydantic settings olive.evaluator.olive_evaluator.OliveEvaluatorConfig

field metrics: List[Metric] = []

SearchStrategy

class olive.strategy.search_strategy.SearchStrategy(config: Dict[str, Any] | SearchStrategyConfig)

Search strategy