mlos_bench.storage.base_trial_data
Base interface for accessing the stored benchmark trial data.
A single trial is a single run of an experiment with a given configuration (e.g., set of tunable parameters).
See also
mlos_bench.storageThe base storage module for mlos_bench, which includes some basic examples in the documentation.
Classes
Base interface for accessing the stored experiment benchmark trial data. |
Module Contents
- class mlos_bench.storage.base_trial_data.TrialData(*, experiment_id: str, trial_id: int, tunable_config_id: int, ts_start: datetime.datetime, ts_end: datetime.datetime | None, status: mlos_bench.environments.status.Status)[source]
Base interface for accessing the stored experiment benchmark trial data.
A trial is a single run of an experiment with a given configuration (e.g., set of tunable parameters).
- Parameters:
experiment_id (str)
trial_id (int)
tunable_config_id (int)
ts_start (datetime.datetime)
ts_end (Optional[datetime.datetime])
- property metadata_df: pandas.DataFrame[source]
- Abstractmethod:
- Return type:
Retrieve the trials’ metadata parameters as a dataframe.
Note: this corresponds to the Trial object’s “config” property.
- Returns:
metadata – An optional dataframe with the metadata associated with the trial. It has two str columns, “parameter” and “value”. Returns an empty dataframe if there is no metadata.
- Return type:
- property metadata_dict: dict[source]
Retrieve the trials’ metadata parameters as a dict.
Note: this corresponds to the Trial object’s “config” property.
- Returns:
metadata
- Return type:
- property results_df: pandas.DataFrame[source]
- Abstractmethod:
- Return type:
Retrieve the trials’ results from the storage.
- Returns:
results – A dataframe with the trial results. It has two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, the dataframe is empty.
- Return type:
- property results_dict: Dict[str, mlos_bench.tunables.tunable.TunableValue | None][source]
Retrieve the trials’ results from the storage as a dict.
- Returns:
results
- Return type:
- property status: mlos_bench.environments.status.Status[source]
Status of the trial.
- Return type:
- property telemetry_df: pandas.DataFrame[source]
- Abstractmethod:
- Return type:
Retrieve the trials’ telemetry from the storage as a dataframe.
- Returns:
config – A dataframe with the trial telemetry, if there is any. It has one datetime column, “ts”, and two str columns, “metric” and “value”. If the trial status is not SUCCEEDED, or there is no telemetry data, the dataframe is empty.
- Return type:
- property ts_end: datetime.datetime | None[source]
End timestamp of the trial (UTC).
- Return type:
Optional[datetime.datetime]
- property ts_start: datetime.datetime[source]
Start timestamp of the trial (UTC).
- Return type:
- property tunable_config: mlos_bench.storage.base_tunable_config_data.TunableConfigData[source]
- Abstractmethod:
- Return type:
mlos_bench.storage.base_tunable_config_data.TunableConfigData
Retrieve the trials’ tunable configuration data from the storage.
Note: this corresponds to the Trial object’s “tunables” property.
- Returns:
tunable_config – A TunableConfigData object.
- Return type:
- property tunable_config_id: int[source]
ID of the (tunable) configuration of the trial.
- Return type:
- property tunable_config_trial_group: mlos_bench.storage.base_tunable_config_trial_group_data.TunableConfigTrialGroupData[source]
- Abstractmethod:
- Return type:
mlos_bench.storage.base_tunable_config_trial_group_data.TunableConfigTrialGroupData
Retrieve the trial’s (tunable) config trial group data from the storage.