Configuring OliveSystem¶

This document describes how to configure the different types of OliveSystems

Local System¶

{
    "type": "LocalSystem",
    "config": {
        "device": "cpu"
    }
}

from olive.systems.local import LocalSystem
from olive.system.common import Device

local_system = LocalSystem(device=Device.CPU)

Please refer to LocalTargetUserConfig for more details on the config options.

AzureML System¶

Prerequisites¶

azureml extra dependencies installed.

pip install olive-ai[azureml]

or

pip install azure-ai-ml azure-identity

2. AzureML Workspace with necessary compute created. Refer to this for more details. Download the workspace config json.

System Configuration¶

{
    "type": "AzureML",
    "config": {
        "aml_compute": "cpu-cluster",
        "aml_docker_config": {
            "base_image": "mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
            "conda_file_path": "conda.yaml"
        }
    }
}

from olive.systems.azureml import AzureMLDockerConfig, AzureMLSystem

docker_config = AzureMLDockerConfig(
    base_image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
    conda_file_path="conda.yaml",
)
aml_system = AzureMLSystem(
    aml_compute="cpu-cluster",
    aml_docker_config={
            "base_image": "mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
            "conda_file_path": "conda.yaml"
        }
)

Please refer to this example for "conda.yaml".

Important

The AzureML environment must have olive-ai installed!

Please refer to AzureMLTargetUserConfig for more details on the config options.

Docker System¶

Prerequisites¶

Docker Engine installed on the host machine.

docker extra dependencies installed.

pip install olive-ai[docker]

or

pip install docker

System Configuration¶

{
    "type": "Docker",
    "config": {
        "local_docker_config": {
            "image_name": "olive",
            "build_context_path": "docker",
            "dockerfile": "Dockerfile"
        }
    }
}

from olive.systems.docker import DockerSystem, LocalDockerConfig

local_docker_config = LocalDockerConfig(
    image_name="olive",
    build_context_path="docker",
    dockerfile="Dockerfile",
)
docker_system = DockerSystem(local_docker_config=local_docker_config)

Please refer to this example for "docker" and "Dockerfile".

Important

The docker container must have olive-ai installed!

Please refer to DockerTargetUserConfig for more details on the config options.

Python Environment System¶

{
    "type": "PythonEnvironment",
    "config": {
        "python_environment_path": "/home/user/.virtualenvs/myenv",
        "device": "cpu"
    }
}

from olive.systems.python_environment import PythonEnvironmentSystem
from olive.system.common import Device

python_environment_system = PythonEnvironmentSystem(
    python_environment_path = "/home/user/.virtualenvs/myenv",
    device = Device.CPU
)

Important

The python environment system can only be used to evaluate onnx models. It must have onnxruntime installed!

Please refer to PythonEnvironmentTargetUserConfig for more details on the config options.