mlos_bench.optimizers.mock_optimizer

Mock optimizer for mlos_bench.

Mostly intended for testing and validation. This optimizer produces random suggestions. The range of the suggestions can be controlled by a config.

See the test cases or example json configs for more details.

Classes

MockOptimizer

Mock optimizer to test the Environment API.

Module Contents

class mlos_bench.optimizers.mock_optimizer.MockOptimizer(tunables: mlos_bench.tunables.tunable_groups.TunableGroups, config: dict, global_config: dict | None = None, service: mlos_bench.services.base_service.Service | None = None)[source]

Bases: mlos_bench.optimizers.track_best_optimizer.TrackBestOptimizer

Mock optimizer to test the Environment API.

Create a new optimizer for the given configuration space defined by the tunables.

Parameters:

  • tunables (TunableGroups) – The tunables to optimize.

  • config (dict) – Free-format key/value pairs of configuration parameters to pass to the optimizer.

  • global_config (dict | None)

  • service (Service | None)

bulk_register(configs: collections.abc.Sequence[dict], scores: collections.abc.Sequence[dict[str, mlos_bench.tunables.tunable_types.TunableValue] | None], status: collections.abc.Sequence[mlos_bench.environments.status.Status] | None = None) → bool[source]

Pre-load the optimizer with the bulk data from previous experiments.

Parameters:

  • configs (Sequence[dict]) – Records of tunable values from other experiments.

  • scores (Sequence[Optional[dict[str, TunableValue]]]) – Benchmark results from experiments that correspond to configs.

  • status (Optional[Sequence[Status]]) – Status of the experiments that correspond to configs.

Returns:

is_not_empty – True if there is data to register, false otherwise.

Return type:

bool

suggest() → mlos_bench.tunables.tunable_groups.TunableGroups[source]

Generate the next (random) suggestion.

Return type:

mlos_bench.tunables.tunable_groups.TunableGroups