mlos_bench.storage.base_tunable_config_trial_group_data

Base interface for accessing the stored benchmark config trial group data.

Since a single config may be used by multiple trials, we can group them together for easier analysis.

See also

mlos_bench.storage

The base storage module for mlos_bench, which includes some basic examples in the documentation.

Classes

TunableConfigTrialGroupData

Base interface for accessing the stored experiment benchmark tunable config trial

Module Contents

class mlos_bench.storage.base_tunable_config_trial_group_data.TunableConfigTrialGroupData(*, experiment_id: str, tunable_config_id: int, tunable_config_trial_group_id: int | None = None)

Base interface for accessing the stored experiment benchmark tunable config trial group data.

A (tunable) config is used to define an instance of values for a set of tunable parameters for a given experiment and can be used by one or more trial instances (e.g., for repeats), which we call a (tunable) config trial group.

Parameters:

• experiment_id (str)

• tunable_config_id (int)

• tunable_config_trial_group_id (Optional[int])

__eq__(other: Any) → bool

Parameters:

other (Any)

Return type:

bool

__repr__() → str

Return type:

str

property experiment_id: str

ID of the experiment.

Return type:

str

property results_df: pandas.DataFrame

Abstractmethod:

Return type:

pandas.DataFrame

Retrieve all results for this (tunable) config trial group as a single DataFrame.

Returns:

results – A DataFrame with configurations and results from all trials of the experiment. Has columns [trial_id, config_id, ts_start, ts_end, status] followed by tunable config parameters (prefixed with “config.”) and trial results (prefixed with “result.”). The latter can be NULLs if the trial was not successful.

Return type:

pandas.DataFrame

See also

mlos_bench.storage.base_experiment_data.ExperimentData.results_df

property trials: Dict[int, mlos_bench.storage.base_trial_data.TrialData]

Abstractmethod:

Return type:

Dict[int, mlos_bench.storage.base_trial_data.TrialData]

Retrieve the trials’ data for this (tunable) config trial group from the storage.

Returns:

trials – A dictionary of the trials’ data, keyed by trial id.

Return type:

Dict[int, TrialData]

property tunable_config: mlos_bench.storage.base_tunable_config_data.TunableConfigData

Abstractmethod:

Return type:

mlos_bench.storage.base_tunable_config_data.TunableConfigData

Retrieve the (tunable) config data for this (tunable) config trial group from the storage.

Return type:

TunableConfigData

property tunable_config_id: int

ID of the config.

Return type:

int

property tunable_config_trial_group_id: int

The unique ID (within this experiment) of the (tunable) config trial group.

This is a unique identifier for all trials in this experiment using the given config_id, and typically defined as the the minimum trial_id for the given config_id.

Return type:

int