mlos_bench.storage.base_trial_data module¶

Base interface for accessing the stored benchmark trial data.

class mlos_bench.storage.base_trial_data.TrialData(*, experiment_id: str, trial_id: int, tunable_config_id: int, ts_start: datetime, ts_end: datetime | None, status: Status)¶

Bases: object

Base interface for accessing the stored experiment benchmark trial data.

A trial is a single run of an experiment with a given configuration (e.g., set of tunable parameters).

Attributes:

experiment_id: ID of the experiment this trial belongs to.
metadata_df: Retrieve the trials’ metadata parameters as a dataframe.
metadata_dict: Retrieve the trials’ metadata parameters as a dict.
results_df: Retrieve the trials’ results from the storage.
results_dict: Retrieve the trials’ results from the storage as a dict.
status: Status of the trial.
telemetry_df: Retrieve the trials’ telemetry from the storage as a dataframe.
trial_id: ID of the trial.
ts_end: End timestamp of the trial (UTC).
ts_start: Start timestamp of the trial (UTC).
tunable_config: Retrieve the trials’ tunable configuration data from the storage.
tunable_config_id: ID of the (tunable) configuration of the trial.
tunable_config_trial_group: Retrieve the trial’s (tunable) config trial group data from the storage.

property experiment_id: str¶: ID of the experiment this trial belongs to.

abstract property metadata_df: DataFrame¶

Retrieve the trials’ metadata parameters as a dataframe.

Note: this corresponds to the Trial object’s “config” property.

Returns:

metadatapandas.DataFrame: An optional dataframe with the metadata associated with the trial. It has two str columns, “parameter” and “value”. Returns an empty dataframe if there is no metadata.

property metadata_dict: dict¶

Retrieve the trials’ metadata parameters as a dict.

Note: this corresponds to the Trial object’s “config” property.

Returns:

metadatadict

abstract property results_df: DataFrame¶

Retrieve the trials’ results from the storage.

Returns:

resultspandas.DataFrame: A dataframe with the trial results. It has two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, the dataframe is empty.

property results_dict: Dict[str, int | float | str | None]¶

Retrieve the trials’ results from the storage as a dict.

Returns:

resultsdict

property status: Status¶: Status of the trial.

abstract property telemetry_df: DataFrame¶

Retrieve the trials’ telemetry from the storage as a dataframe.

Returns:

configpandas.DataFrame: A dataframe with the trial telemetry, if there is any. It has one datetime column, “ts”, and two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, or there is no telemetry data, the dataframe is empty.

property trial_id: int¶: ID of the trial.

property ts_end: datetime | None¶: End timestamp of the trial (UTC).

property ts_start: datetime¶: Start timestamp of the trial (UTC).

abstract property tunable_config: TunableConfigData¶

Retrieve the trials’ tunable configuration data from the storage.

Note: this corresponds to the Trial object’s “tunables” property.

Returns:

tunable_configTunableConfigData: A TunableConfigData object.

property tunable_config_id: int¶: ID of the (tunable) configuration of the trial.

abstract property tunable_config_trial_group: TunableConfigTrialGroupData¶: Retrieve the trial’s (tunable) config trial group data from the storage.