mlos_bench.storage.sql.storage

Saving and restoring the benchmark data in SQL database.

Classes

SqlStorage

An implementation of the Storage interface using SQLAlchemy

Module Contents

class mlos_bench.storage.sql.storage.SqlStorage(config: dict, global_config: dict | None = None, service: mlos_bench.services.base_service.Service | None = None)[source]

Bases: mlos_bench.storage.base_storage.Storage

An implementation of the Storage interface using SQLAlchemy backend.

Create a new storage object.

Parameters:
__repr__() → str[source]
Return type:

str

experiment(*, experiment_id: str, trial_id: int, root_env_config: str, description: str, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, opt_targets: Dict[str, Literal['min', 'max']]) → mlos_bench.storage.base_storage.Storage.Experiment[source]

Create a new experiment in the storage.

We need the opt_target parameter here to know what metric to retrieve when we load the data from previous trials. Later we will replace it with full metadata about the optimization direction, multiple objectives, etc.

Parameters:

  • experiment_id (str) – Unique identifier of the experiment.

  • trial_id (int) – Starting number of the trial.

  • root_env_config (str) – A path to the root JSON configuration file of the benchmarking environment.

  • description (str) – Human-readable description of the experiment.

  • tunables (TunableGroups)

  • opt_targets (Dict[str, Literal["min", "max"]]) – Names of metrics we’re optimizing for and the optimization direction {min, max}.

Returns:

experiment – An object that allows to update the storage with the results of the experiment and related data.

Return type:

Storage.Experiment

property experiments: Dict[str, mlos_bench.storage.base_experiment_data.ExperimentData][source]

Retrieve the experiments’ data from the storage.

Returns:

experiments – A dictionary of the experiments’ data, keyed by experiment id.

Return type:

Dict[str, ExperimentData]