onlineml.trial_runner
OnlineTrialRunner Objects
class OnlineTrialRunner()
Class for the OnlineTrialRunner.
__init__
def __init__(max_live_model_num: int, searcher=None, scheduler=None, champion_test_policy="loss_ucb", **kwargs)
Constructor.
Arguments:
max_live_model_num
- The maximum number of 'live'/running models allowed.searcher
- A class for generating Trial objects progressively. The ConfigOracle is implemented in the searcher.scheduler
- A class for managing the 'live' trials and allocating the resources for the trials.champion_test_policy
- A string to specify what test policy to test for champion. Currently can choose from ['loss_ucb', 'loss_avg', 'loss_lcb', None].
champion_trial
@property
def champion_trial() -> Trial
The champion trial.
running_trials
@property
def running_trials()
The running/'live' trials.
step
def step(data_sample=None, prediction_trial_tuple=None)
Schedule one trial to run each time it is called.
Arguments:
data_sample
- One data example.prediction_trial_tuple
- A list of information containing (prediction_made, prediction_trial).
get_top_running_trials
def get_top_running_trials(top_ratio=None, top_metric="ucb") -> list
Get a list of trial ids, whose performance is among the top running trials.
get_trials
def get_trials() -> list
Return the list of trials managed by this TrialRunner.
add_trial
def add_trial(new_trial)
Add a new trial to this TrialRunner. Trials may be added at any time.
Arguments:
new_trial
Trial - Trial to queue.
stop_trial
def stop_trial(trial)
Stop a trial: set the status of a trial to be Trial.TERMINATED and perform other subsequent operations.
pause_trial
def pause_trial(trial)
Pause a trial: set the status of a trial to be Trial.PAUSED and perform other subsequent operations.
run_trial
def run_trial(trial)
Run a trial: set the status of a trial to be Trial.RUNNING and perform other subsequent operations.