mlos_bench.optimizers.grid_search_optimizer

Grid search optimizer for mlos_bench.

Classes

GridSearchOptimizer

Grid search optimizer.

Module Contents

class mlos_bench.optimizers.grid_search_optimizer.GridSearchOptimizer(tunables: mlos_bench.tunables.tunable_groups.TunableGroups, config: dict, global_config: dict | None = None, service: mlos_bench.services.base_service.Service | None = None)[source]

Bases: mlos_bench.optimizers.track_best_optimizer.TrackBestOptimizer

Grid search optimizer.

Create a new optimizer for the given configuration space defined by the tunables.

Parameters:

  • tunables (TunableGroups) – The tunables to optimize.

  • config (dict) – Free-format key/value pairs of configuration parameters to pass to the optimizer.

  • global_config (Optional[dict])

  • service (Optional[Service])

bulk_register(configs: Sequence[dict], scores: Sequence[Dict[str, mlos_bench.tunables.tunable.TunableValue] | None], status: Sequence[mlos_bench.environments.status.Status] | None = None) → bool[source]

Pre-load the optimizer with the bulk data from previous experiments.

Parameters:

  • configs (Sequence[dict]) – Records of tunable values from other experiments.

  • scores (Sequence[Optional[Dict[str, TunableValue]]]) – Benchmark results from experiments that correspond to configs.

  • status (Optional[Sequence[Status]]) – Status of the experiments that correspond to configs.

Returns:

is_not_empty – True if there is data to register, false otherwise.

Return type:

bool

not_converged() → bool[source]

Return True if not converged, False otherwise.

Base implementation just checks the iteration count.

Return type:

bool

register(tunables: mlos_bench.tunables.tunable_groups.TunableGroups, status: mlos_bench.environments.status.Status, score: Dict[str, mlos_bench.tunables.tunable.TunableValue] | None = None) → Dict[str, float] | None[source]

Register the observation for the given configuration.

Parameters:

  • tunables (TunableGroups) – The configuration that has been benchmarked. Usually it’s the same config that the .suggest() method returned.

  • status (Status) – Final status of the experiment (e.g., SUCCEEDED or FAILED).

  • score (Optional[Dict[str, TunableValue]]) – A dict with the final benchmark results. None if the experiment was not successful.

Returns:

value – Benchmark scores extracted (and possibly transformed) from the dataframe that’s being MINIMIZED.

Return type:

Optional[Dict[str, float]]

suggest() → mlos_bench.tunables.tunable_groups.TunableGroups[source]

Generate the next grid search suggestion.

Return type:

mlos_bench.tunables.tunable_groups.TunableGroups

property pending_configs: Iterable[Dict[str, mlos_bench.tunables.tunable.TunableValue]][source]

Gets the set of pending configs in this grid search optimizer.

Return type:

Iterable[Dict[str, TunableValue]]

property suggested_configs: Iterable[Dict[str, mlos_bench.tunables.tunable.TunableValue]][source]

Gets the set of configs that have been suggested but not yet registered.

Return type:

Iterable[Dict[str, TunableValue]]