MLOS Package APIs Overview

This is a list of major functions and classes provided by the MLOS packages.

mlos-core API

This is a list of major functions and classes provided by mlos_core.

Optimizers

OptimizerType(value[, names, module, ...])

Enumerate supported MlosCore optimizers.

OptimizerFactory()

Simple factory class for creating BaseOptimizer-derived objects

OptimizerFactory.create(*, parameter_space)

Create a new optimizer instance, given the parameter space, optimizer type, and potential optimizer options.

BaseOptimizer(*, parameter_space[, ...])

Optimizer abstract base class defining the basic interface.

RandomOptimizer(*, parameter_space[, ...])

Optimizer class that produces random suggestions.

FlamlOptimizer(*, parameter_space[, ...])

Wrapper class for FLAML Optimizer: A fast library for AutoML and tuning.

BaseBayesianOptimizer(*, parameter_space[, ...])

Abstract base class defining the interface for Bayesian optimization.

SmacOptimizer(*, parameter_space[, ...])

Wrapper class for SMAC based Bayesian optimization.

Spaces

Converters

configspace_to_flaml_space(config_space)

Converts a ConfigSpace.ConfigurationSpace to dict.

Space Adapters

SpaceAdapterType(value[, names, module, ...])

Enumerate supported MlosCore space adapters.

SpaceAdapterFactory()

Simple factory class for creating BaseSpaceAdapter-derived objects

SpaceAdapterFactory.create(*, parameter_space)

Create a new space adapter instance, given the parameter space and potential space adapter options.

BaseSpaceAdapter(*, orig_parameter_space)

SpaceAdapter abstract class defining the basic interface.

IdentityAdapter(*, orig_parameter_space)

Identity (no-op) SpaceAdapter class.

LlamaTuneAdapter(*, orig_parameter_space[, ...])

Implementation of LlamaTune, a set of parameter space transformation techniques, aimed at improving the sample-efficiency of the underlying optimizer.

mlos-bench API

This is a list of major functions and classes provided by mlos_bench.

Main

run.py

The script to run the benchmarks or the optimization loop.

Also available as mlos_bench command line tool.

Note

The are json config examples and json schemas on the main source code repository site.

Benchmark Environments

Status(value[, names, module, qualname, ...])

Enum for the status of the benchmark/environment.

Environment(*, name, config[, ...])

An abstract base of all benchmark environments.

CompositeEnv(*, name, config[, ...])

Composite benchmark environment.

MockEnv(*, name, config[, global_config, ...])

Scheduler-side environment to mock the benchmark results.

Local Environments

LocalEnv(*, name, config[, global_config, ...])

Scheduler-side Environment that runs scripts locally.

LocalFileShareEnv(*, name, config[, ...])

Scheduler-side Environment that runs scripts locally and uploads/downloads data to the shared file storage.

Remote Environments

RemoteEnv(*, name, config[, global_config, ...])

Environment to run benchmarks and scripts on a remote host OS.

OSEnv(*, name, config[, global_config, ...])

OS Level Environment for a host.

VMEnv(*, name, config[, global_config, ...])

Remote VM/host environment.

HostEnv(*, name, config[, global_config, ...])

Remote host environment.

Tunable Parameters

Tunable(name, config)

A tunable parameter definition and its current value.

TunableGroups([config])

A collection of covariant groups of tunable parameters.

Service Mix-ins

Service([config, global_config, parent, methods])

An abstract base of all Environment Services and used to build up mix-ins.

FileShareService([config, global_config, ...])

An abstract base of all file shares.

ConfigPersistenceService([config, ...])

Collection of methods to deserialize the Environment, Service, and TunableGroups objects.

Local Services

LocalExecService([config, global_config, ...])

Collection of methods to run scripts and commands in an external process on the node acting as the scheduler.

Remote Azure Services

AzureVMService([config, global_config, ...])

Helper methods to manage VMs on Azure.

AzureFileShareService([config, ...])

Helper methods for interacting with Azure File Share

Optimizer Adapters

Optimizer(tunables, config[, global_config, ...])

An abstract interface between the benchmarking framework and mlos_core optimizers.

MockOptimizer(tunables, config[, ...])

Mock optimizer to test the Environment API.

MlosCoreOptimizer(tunables, config[, ...])

A wrapper class for the mlos_core optimizers.

Storage

Base Runtime Backends

Storage(config[, global_config, service])

An abstract interface between the benchmarking framework and storage systems (e.g., SQLite or MLFLow).

from_config(config_file[, global_configs])

Create a new storage object from JSON5 config file.

SQL DB Storage Backend

SqlStorage(config[, global_config, service])

An implementation of the Storage interface using SQLAlchemy backend.

Analysis Client Access APIs

ExperimentData(*, experiment_id, ...)

Base interface for accessing the stored experiment benchmark data.

TrialData(*, experiment_id, trial_id, ...)

Base interface for accessing the stored experiment benchmark trial data.

TunableConfigData(*, tunable_config_id)

Base interface for accessing the stored experiment benchmark (tunable) config data.

TunableConfigTrialGroupData(*, ...[, ...])

Base interface for accessing the stored experiment benchmark tunable config trial group data.

mlos-viz API

This is a list of major functions and classes provided by mlos_viz.

MlosVizMethod(value[, names, module, ...])

What method to use for visualizing the experiment results.

plot([exp_data, results_df, objectives, ...])

Plots the results of the experiment.