mlos_core.optimizers package

Basic initializer module for the mlos_core optimizers.

class mlos_core.optimizers.BaseOptimizer(*, parameter_space: ConfigurationSpace, space_adapter: BaseSpaceAdapter | None = None)

Bases: object

Optimizer abstract base class defining the basic interface.

Attributes:

space_adapter

Get the space adapter instance (if any).

Methods

cleanup()	Remove temp files, release resources, etc.
get_best_observation()	Returns the best observation so far as a dataframe.
get_observations()	Returns the observations as a dataframe.
register(configurations, scores[, context])	Wrapper method, which employs the space adapter (if any), before registering the configurations and scores.
register_pending(configurations[, context])	Registers the given configurations as "pending".
suggest([context, defaults])	Wrapper method, which employs the space adapter (if any), after suggesting a new configuration.

cleanup() → None

Remove temp files, release resources, etc. after use. Default is no-op. Redefine this method in optimizers that require cleanup.

get_best_observation() → DataFrame

Returns the best observation so far as a dataframe.

Returns:

best_observationpd.DataFrame

Dataframe with a single row containing the best observation. The columns are parameter names and “score” for the score.

get_observations() → DataFrame

Returns the observations as a dataframe.

Returns:

observationspd.DataFrame

Dataframe of observations. The columns are parameter names and “score” for the score, each row is an observation.

register(configurations: DataFrame, scores: Series, context: DataFrame | None = None) → None

Wrapper method, which employs the space adapter (if any), before registering the configurations and scores.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

scorespd.Series

Scores from running the configurations. The index is the same as the index of the configurations.

contextpd.DataFrame

Not Yet Implemented.

abstract register_pending(configurations: DataFrame, context: DataFrame | None = None) → None

Registers the given configurations as “pending”. That is it say, it has been suggested by the optimizer, and an experiment trial has been started. This can be useful for executing multiple trials in parallel, retry logic, etc.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

property space_adapter: BaseSpaceAdapter | None

Get the space adapter instance (if any).

suggest(context: DataFrame | None = None, defaults: bool = False) → DataFrame

Wrapper method, which employs the space adapter (if any), after suggesting a new configuration.

Parameters:

contextpd.DataFrame

Not Yet Implemented.

defaultsbool

Whether or not to return the default config instead of an optimizer guided one. By default, use the one from the optimizer.

Returns:

configurationpd.DataFrame

Pandas dataframe with a single row. Column names are the parameter names.

class mlos_core.optimizers.FlamlOptimizer(*, parameter_space: ConfigurationSpace, space_adapter: BaseSpaceAdapter | None = None, low_cost_partial_config: dict | None = None, seed: int | None = None)

Bases: BaseOptimizer

Wrapper class for FLAML Optimizer: A fast library for AutoML and tuning.

Parameters:

parameter_spaceConfigSpace.ConfigurationSpace

The parameter space to optimize.

space_adapterBaseSpaceAdapter

The space adapter class to employ for parameter space transformations.

low_cost_partial_configdict

A dictionary from a subset of controlled dimensions to the initial low-cost values. More info: https://microsoft.github.io/FLAML/docs/FAQ#about-low_cost_partial_config-in-tune

seedOptional[int]

If provided, calls np.random.seed() with the provided value to set the seed globally at init.

Attributes:

space_adapter

Get the space adapter instance (if any).

Methods

cleanup()	Remove temp files, release resources, etc.
get_best_observation()	Returns the best observation so far as a dataframe.
get_observations()	Returns the observations as a dataframe.
register(configurations, scores[, context])	Wrapper method, which employs the space adapter (if any), before registering the configurations and scores.
register_pending(configurations[, context])	Registers the given configurations as "pending".
suggest([context, defaults])	Wrapper method, which employs the space adapter (if any), after suggesting a new configuration.

register_pending(configurations: DataFrame, context: DataFrame | None = None) → None

Registers the given configurations as “pending”. That is it say, it has been suggested by the optimizer, and an experiment trial has been started. This can be useful for executing multiple trials in parallel, retry logic, etc.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

class mlos_core.optimizers.OptimizerFactory

Bases: object

Simple factory class for creating BaseOptimizer-derived objects

Methods

create(*, parameter_space[, optimizer_type, ...])

Create a new optimizer instance, given the parameter space, optimizer type, and potential optimizer options.

static create(*, parameter_space: ConfigurationSpace, optimizer_type: OptimizerType = OptimizerType.FLAML, optimizer_kwargs: dict | None = None, space_adapter_type: SpaceAdapterType = SpaceAdapterType.IDENTITY, space_adapter_kwargs: dict | None = None) → ConcreteOptimizer

Create a new optimizer instance, given the parameter space, optimizer type, and potential optimizer options.

Parameters:

parameter_spaceConfigSpace.ConfigurationSpace

Input configuration space.

optimizer_typeOptimizerType

Optimizer class as defined by Enum.

optimizer_kwargsOptional[dict]

Optional arguments passed in Optimizer class constructor.

space_adapter_typeOptional[SpaceAdapterType]

Space adapter class to be used alongside the optimizer.

space_adapter_kwargsOptional[dict]

Optional arguments passed in SpaceAdapter class constructor.

Returns:

optimizerConcreteOptimizer

Instance of concrete optimizer class (e.g., RandomOptimizer, FlamlOptimizer, SmacOptimizer, etc.).

class mlos_core.optimizers.RandomOptimizer(*, parameter_space: ConfigurationSpace, space_adapter: BaseSpaceAdapter | None = None)

Bases: BaseOptimizer

Optimizer class that produces random suggestions. Useful for baseline comparison against Bayesian optimizers.

Parameters:

parameter_spaceConfigSpace.ConfigurationSpace

The parameter space to optimize.

Attributes:

space_adapter

Get the space adapter instance (if any).

Methods

cleanup()	Remove temp files, release resources, etc.
get_best_observation()	Returns the best observation so far as a dataframe.
get_observations()	Returns the observations as a dataframe.
register(configurations, scores[, context])	Wrapper method, which employs the space adapter (if any), before registering the configurations and scores.
register_pending(configurations[, context])	Registers the given configurations as "pending".
suggest([context, defaults])	Wrapper method, which employs the space adapter (if any), after suggesting a new configuration.

register_pending(configurations: DataFrame, context: DataFrame | None = None) → None

Registers the given configurations as “pending”. That is it say, it has been suggested by the optimizer, and an experiment trial has been started. This can be useful for executing multiple trials in parallel, retry logic, etc.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

class mlos_core.optimizers.SmacOptimizer(*, parameter_space: ConfigurationSpace, space_adapter: BaseSpaceAdapter | None = None, seed: int | None = 0, run_name: str | None = None, output_directory: str | None = None, max_trials: int = 100, n_random_init: int | None = None, max_ratio: float | None = None, use_default_config: bool = False, n_random_probability: float = 0.1)

Bases: BaseBayesianOptimizer

Wrapper class for SMAC based Bayesian optimization.

Parameters:

parameter_spaceConfigSpace.ConfigurationSpace

The parameter space to optimize.

space_adapterBaseSpaceAdapter

The space adapter class to employ for parameter space transformations.

seedOptional[int]

By default SMAC uses a known seed (0) to keep results reproducible. However, if a None seed is explicitly provided, we let a random seed be produced by SMAC.

run_nameOptional[str]

Name of this run. This is used to easily distinguish across different runs. If set to None (default), SMAC will generate a hash from metadata.

output_directoryOptional[str]

The directory where SMAC output will saved. If set to None (default), a temporary dir will be used.

max_trialsint

Maximum number of trials (i.e., function evaluations) to be run. Defaults to 100. Note that modifying this value directly affects the value of n_random_init, if latter is set to None.

n_random_initOptional[int]

Number of points evaluated at start to bootstrap the optimizer. Default depends on max_trials and number of parameters and max_ratio. Note: it can sometimes be useful to set this to 1 when pre-warming the optimizer from historical data. See Also: mlos_bench.optimizer.bulk_register

max_ratioOptional[int]

Maximum ratio of max_trials to be random configurations to be evaluated at start to bootstrap the optimizer. Useful if you want to explicitly control the number of random configurations evaluated at start.

use_default_config: bool

Whether to use the default config for the first trial after random initialization.

n_random_probability: float

Probability of choosing to evaluate a random configuration during optimization. Defaults to 0.1. Setting this to a higher value favors exploration over exploitation.

Attributes:

n_random_init

Gets the number of random samples to use to initialize the optimizer’s search space sampling.

space_adapter

Get the space adapter instance (if any).

Methods

acquisition_function(configurations[, context])	Invokes the acquisition function from this Bayesian optimizer for the given configuration.
cleanup()	Remove temp files, release resources, etc.
get_best_observation()	Returns the best observation so far as a dataframe.
get_observations()	Returns the observations as a dataframe.
register(configurations, scores[, context])	Wrapper method, which employs the space adapter (if any), before registering the configurations and scores.
register_pending(configurations[, context])	Registers the given configurations as "pending".
suggest([context, defaults])	Wrapper method, which employs the space adapter (if any), after suggesting a new configuration.
surrogate_predict(configurations[, context])	Obtain a prediction from this Bayesian optimizer's surrogate model for the given configuration(s).

acquisition_function(configurations: DataFrame, context: DataFrame | None = None) → ndarray[Any, dtype[_ScalarType_co]]

Invokes the acquisition function from this Bayesian optimizer for the given configuration.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

cleanup() → None

Remove temp files, release resources, etc. after use. Default is no-op. Redefine this method in optimizers that require cleanup.

property n_random_init: int

Gets the number of random samples to use to initialize the optimizer’s search space sampling.

Note: This may not be equal to the value passed to the initializer, due to logic present in the SMAC. See Also: max_ratio

Returns:

int

The number of random samples used to initialize the optimizer’s search space sampling.

register_pending(configurations: DataFrame, context: DataFrame | None = None) → None

Registers the given configurations as “pending”. That is it say, it has been suggested by the optimizer, and an experiment trial has been started. This can be useful for executing multiple trials in parallel, retry logic, etc.

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

surrogate_predict(configurations: DataFrame, context: DataFrame | None = None) → ndarray[Any, dtype[_ScalarType_co]]

Obtain a prediction from this Bayesian optimizer’s surrogate model for the given configuration(s).

Parameters:

configurationspd.DataFrame

Dataframe of configurations / parameters. The columns are parameter names and the rows are the configurations.

contextpd.DataFrame

Not Yet Implemented.

class mlos_core.optimizers.SpaceAdapterType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Enumerate supported MlosCore space adapters.

IDENTITY = <class 'mlos_core.spaces.adapters.identity_adapter.IdentityAdapter'>

A no-op adapter will be used

LLAMATUNE = <class 'mlos_core.spaces.adapters.llamatune.LlamaTuneAdapter'>

An instance of LlamaTuneAdapter class will be used

Subpackages

• mlos_core.optimizers.bayesian_optimizers package
  • BaseBayesianOptimizer
    • BaseBayesianOptimizer.acquisition_function()
    • BaseBayesianOptimizer.surrogate_predict()
  • SmacOptimizer
    • SmacOptimizer.acquisition_function()
    • SmacOptimizer.cleanup()
    • SmacOptimizer.n_random_init
    • SmacOptimizer.register_pending()
    • SmacOptimizer.surrogate_predict()
  • Submodules
    • mlos_core.optimizers.bayesian_optimizers.bayesian_optimizer module
      • BaseBayesianOptimizer
    • mlos_core.optimizers.bayesian_optimizers.smac_optimizer module
      • SmacOptimizer

Submodules

• mlos_core.optimizers.flaml_optimizer module
  • EvaluatedSample
    • EvaluatedSample.config
    • EvaluatedSample.score
  • FlamlOptimizer
    • FlamlOptimizer.register_pending()
• mlos_core.optimizers.optimizer module
  • BaseOptimizer
    • BaseOptimizer.cleanup()
    • BaseOptimizer.get_best_observation()
    • BaseOptimizer.get_observations()
    • BaseOptimizer.register()
    • BaseOptimizer.register_pending()
    • BaseOptimizer.space_adapter
    • BaseOptimizer.suggest()
• mlos_core.optimizers.random_optimizer module
  • RandomOptimizer
    • RandomOptimizer.register_pending()