.. _command_line_tools:

Olive provides command line tools that can be invoked using the ``olive`` command.

Run
===

Run Olive workflow defined in the input .json configuration file.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: run

Finetune
========

Fine-tune a model on a dataset using HuggingFace peft. Huggingface training arguments can be provided along with the defined options.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: finetune

Auto-Optimization
=================

Automatically optimize the input model for the given target and precision.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: auto-opt

Quantization
============

Quantize PyTorch or ONNX model using various Quantization algorithms.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: quantize

Capture Onnx Graph
==================

Capture ONNX graph using PyTorch Exporter or Model Builder from the Huggingface model or PyTorch model.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: capture-onnx-graph

Generate Adapters
==================

Generate ONNX model with adapters as inputs.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: generate-adapter


Convert Adapters
================

Convert LoRA adapter weights to a file that will be consumed by ONNX models generated by Olive ExtractedAdapters pass.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: convert-adapters

Tune OnnxRuntime Session Params
===============================

Automatically tune the OnnxRuntime session parameters for a given onnx model. Currently, for onnx model converted from huggingface model and used for generative tasks, user can simply provide the --model onnx_model_path --hf_model_name hf_model_name --device device_type to get the tuned session parameters.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: tune-session-params

Qualcomm SDK
============

Configure Qualcomm SDK.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: configure-qualcomm-sdk

AzureML
=======

Manage the AzureML Compute resources.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: manage-aml-compute

Shared Cache
=============

Delete Olive model cache stored in the cloud.

.. argparse::
    :module: olive.cli.launcher
    :func: get_cli_parser
    :prog: olive
    :path: shared-cache

Providing Input Models
======================

There are more than one way to supply input model to the Olive commands.

1. HuggingFace model can be directly used as an input model. For example ``-m microsoft/Phi-3-mini-4k-instruct``.

2. A model produced by a Olive command can be directly used as an input model. You can specify the model file path using the ``-m <output_model>`` option, where ``<output_model>`` is the output folder defined by ``-o <output_model>`` in the previous Olive command.

3. Olive commands also accept a local PyTorch model as an input model. You can specify the model file path using the ``-m model.pt`` option, and the associated model script using the ``--model_script script.py`` option. For example, ``olive capture-onnx-graph -m model.pt --model_script script.py``.

4. A model from AzureML registry can be directly used as an input model. For example ``-m azureml://registries/<registry_name>/models/<model_name>/versions/<version>``.

5. An ONNX model available locally can also be used as an input for the Olive commands that accept ONNX model as an input.

Olive reserves several function names to provide specific inputs for the PyTorch model. These functions should be defined in your model script:

- **Model Loader Function (`_model_loader`)**:
  Loads the PyTorch model. If the model file path is provided using the `-m` option, it takes higher priority than the model loader function.

  .. code-block:: python

      def _model_loader():
          ...
          return model

- **IO Config Function (`_io_config`)**:
  Returns the IO configuration for the model. Either `_io_config` or `_dummy_inputs` is required for the `capture-onnx-graph` CLI command.

  .. code-block:: python

      def _io_config(model: PyTorchModelHandler):
          ...
          return io_config

- **Dummy Inputs Function (`_dummy_inputs`)**:
  Provides dummy input tensors for the model. Either `_io_config` or `_dummy_inputs` is required for the `capture-onnx-graph` CLI command.

  .. code-block:: python

      def _dummy_inputs(model: PyTorchModelHandler):
          ...
          return dummy_inputs

- **Model Format Function (`_model_file_format`)**:
  Specifies the format of the model. The default value is `PyTorch.EntireModel`. For more available options, refer to `this <https://github.com/microsoft/Olive/blob/main/olive/constants.py#L23-L26>`_.

  .. code-block:: python

      def _model_file_format():
          ...
          return model_file_format