mlos_bench.storage.sql.trial

Storage.Trial interface implementation for saving and restoring the benchmark trial data using SQLAlchemy backend.

Classes

Trial

Store the results of a single run of the experiment in SQL database.

Module Contents

class mlos_bench.storage.sql.trial.Trial(*, engine: sqlalchemy.engine.Engine, schema: mlos_bench.storage.sql.schema.DbSchema, tunables: mlos_bench.tunables.tunable_groups.TunableGroups, experiment_id: str, trial_id: int, config_id: int, trial_runner_id: int | None = None, opt_targets: dict[str, Literal['min', 'max']], config: dict[str, Any] | None = None, status: mlos_bench.environments.status.Status = Status.UNKNOWN)[source]

Bases: mlos_bench.storage.base_storage.Storage.Trial

Store the results of a single run of the experiment in SQL database.

Parameters:
set_trial_runner(trial_runner_id: int) → int[source]

Assign the trial to a specific TrialRunner.

Parameters:

trial_runner_id (int)

Return type:

int

update(status: mlos_bench.environments.status.Status, timestamp: datetime.datetime, metrics: dict[str, Any] | None = None) → dict[str, Any] | None[source]

Update the storage with the results of the experiment.

Parameters:

  • status (Status) – Status of the experiment run.

  • timestamp (datetime.datetime) – Timestamp of the status and metrics.

  • metrics (Optional[dict[str, Any]]) – One or several metrics of the experiment run. Must contain the (float) optimization target if the status is SUCCEEDED.

Returns:

metrics – Same as metrics, but always in the dict format.

Return type:

Optional[dict[str, Any]]

update_telemetry(status: mlos_bench.environments.status.Status, timestamp: datetime.datetime, metrics: list[tuple[datetime.datetime, str, Any]]) → None[source]

Save the experiment’s telemetry data and intermediate status.

Parameters:
Return type:

None