mlos_bench.storage.sql.trial

Storage.Trial interface implementation for saving and restoring the benchmark trial data using SQLAlchemy backend.

Classes

Trial

Store the results of a single run of the experiment in SQL database.

Module Contents

class mlos_bench.storage.sql.trial.Trial(*, engine: sqlalchemy.engine.Engine, schema: mlos_bench.storage.sql.schema.DbSchema, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, experiment_id: str, trial_id: int, config_id: int, opt_targets: Dict[str, Literal['min', 'max']], config: Dict[str, Any] | None = None)

Bases: mlos_bench.storage.base_storage.Storage.Trial

Store the results of a single run of the experiment in SQL database.

Parameters:

• engine (sqlalchemy.engine.Engine)

• schema (mlos_bench.storage.sql.schema.DbSchema)

• tunables (mlos_bench.tunables.tunable_groups.TunableGroups)

• experiment_id (str)

• trial_id (int)

• config_id (int)

• opt_targets (Dict[str, Literal['min', 'max']])

• config (Optional[Dict[str, Any]])

update(status: mlos_bench.environments.status.Status, timestamp: datetime.datetime, metrics: Dict[str, Any] | None = None) → Dict[str, Any] | None

Update the storage with the results of the experiment.

Parameters:

• status (Status) – Status of the experiment run.

• timestamp (datetime.datetime) – Timestamp of the status and metrics.

• metrics (Optional[Dict[str, Any]]) – One or several metrics of the experiment run. Must contain the (float) optimization target if the status is SUCCEEDED.

Returns:

metrics – Same as metrics, but always in the dict format.

Return type:

Optional[Dict[str, Any]]

update_telemetry(status: mlos_bench.environments.status.Status, timestamp: datetime.datetime, metrics: List[Tuple[datetime.datetime, str, Any]]) → None

Save the experiment’s telemetry data and intermediate status.

Parameters:

• status (Status) – Current status of the trial.

• timestamp (datetime.datetime) – Timestamp of the status (but not the metrics).

• metrics (List[Tuple[datetime.datetime, str, Any]]) – Telemetry data.

Return type:

None