Search Algorithms#

BANANAS#

class archai.discrete_search.algos.bananas.MoBananasSearch(search_space: BayesOptSearchSpace, search_objectives: SearchObjectives, output_dir: str, surrogate_model: Predictor | None = None, num_iters: int | None = 10, init_num_models: int | None = 10, num_parents: int | None = 10, mutations_per_parent: int | None = 5, num_candidates: int | None = 10, clear_evaluated_models: bool = True, save_pareto_weights: bool = False, seed: int | None = 1)[source]#

Multi-objective version of BANANAS algorithm.

It has been proposed in Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization.

Reference:

https://arxiv.org/abs/2105.01015

get_surrogate_iter_dataset(all_pop: List[ArchaiModel]) → Tuple[ndarray, ndarray][source]#

Get the surrogate dataset for the current iteration.

Parameters:

all_pop – All population models.

Returns:

Tuple of encoded architectures and target values.

sample_models(num_models: int, patience: int | None = 30) → List[ArchaiModel][source]#

Sample models from the search space.

Parameters:

  • num_models – Number of models to sample.

  • patience – Number of tries to sample a valid model.

Returns:

List of sampled models.

mutate_parents(parents: List[ArchaiModel], mutations_per_parent: int | None = 1, patience: int | None = 30) → List[ArchaiModel][source]#

Mutate parents to generate new models.

Parameters:

  • parents – List of parent models.

  • mutations_per_parent – Number of mutations to apply to each parent.

  • patience – Number of tries to sample a valid model.

Returns:

List of mutated models.

predict_expensive_objectives(archs: List[ArchaiModel]) → Dict[str, MeanVar][source]#

Predict expensive objectives for archs using surrogate model.

Parameters:

archs – List of architectures.

Returns:

Dictionary of predicted expensive objectives.

thompson_sampling(archs: List[ArchaiModel], sample_size: int, pred_expensive_objs: Dict[str, MeanVar], cheap_objs: Dict[str, ndarray]) → List[int][source]#

Get the selected architecture list indices from Thompson Sampling.

Parameters:

  • archs – List of architectures.

  • sample_size – Number of architectures to select.

  • pred_expensive_objs – Predicted expensive objectives.

  • cheap_objs – Cheap objectives.

Returns:

List of selected architecture indices.

search() → SearchResults[source]#

Search for the best architecture.

Returns:

Search results.

Evolution Pareto#

class archai.discrete_search.algos.evolution_pareto.EvolutionParetoSearch(search_space: EvolutionarySearchSpace, search_objectives: SearchObjectives, output_dir: str, num_iters: int | None = 10, init_num_models: int | None = 10, initial_population_paths: List[str] | None = None, num_random_mix: int | None = 5, max_unseen_population: int | None = 100, mutations_per_parent: int | None = 1, num_crossovers: int | None = 5, clear_evaluated_models: bool = True, save_pareto_model_weights: bool = True, seed: int | None = 1)[source]#

Evolutionary multi-objective search algorithm that greedily evolves Pareto frontier models.

It starts from an evaluated random subset of models. In each iteration, the algorithm evaluates new subset of models generated from mutations (mutation_per_parent) and crossovers (num_crossovers) of the current pareto frontier, and a new random subset of models (num_random_mix). The process is repeated until num_iters is reached.

sample_models(num_models: int, patience: int | None = 5) → List[ArchaiModel][source]#

Sample models from the search space.

Parameters:

  • num_models – Number of models to sample.

  • patience – Number of tries to sample a valid model.

Returns:

List of sampled models.

mutate_parents(parents: List[ArchaiModel], mutations_per_parent: int | None = 1, patience: int | None = 20) → List[ArchaiModel][source]#

Mutate parents to generate new models.

Parameters:

  • parents – List of parent models.

  • mutations_per_parent – Number of mutations to apply to each parent.

  • patience – Number of tries to sample a valid model.

Returns:

List of mutated models.

crossover_parents(parents: List[ArchaiModel], num_crossovers: int | None = 1, patience: int | None = 30) → List[ArchaiModel][source]#

Crossover parents to generate new models.

Parameters:

  • parents – List of parent models.

  • num_crossovers – Number of crossovers to apply.

  • patience – Number of tries to sample a valid model.

Returns:

List of crossovered models.

on_calc_task_accuracy_end(current_pop: List[ArchaiModel]) → None[source]#

Callback function called right after calc_task_accuracy().

on_search_iteration_start(current_pop: List[ArchaiModel]) → None[source]#

Callback function called right before each search iteration.

select_next_population(current_pop: List[ArchaiModel]) → List[ArchaiModel][source]#

Select the next population from the current population

Parameters:

current_pop – Current population.

Returns:

Next population.

search() → SearchResults[source]#

Search for the best architecture.

Returns:

Search results.

Regularized Evolution#

class archai.discrete_search.algos.regularized_evolution.RegularizedEvolutionSearch(search_space: EvolutionarySearchSpace, search_objectives: SearchObjectives, output_dir: str, num_iters: int | None = 10, init_num_models: int | None = 10, initial_population_paths: List[str] | None = None, pareto_sample_size: int | None = 40, history_size: int | None = 100, clear_evaluated_models: bool | None = True, save_pareto_model_weights: bool = True, seed: int | None = 1)[source]#

Regularized Evolution algorithm.

It has been proposed in Regularized Evolution for Image Classifier Architecture Search.

Reference:

https://arxiv.org/abs/1802.01548v7.

sample_models(num_models: int, patience: int | None = 5) → List[ArchaiModel][source]#

Sample models from the search space.

Parameters:

  • num_models – Number of models to sample.

  • patience – Number of tries to sample a valid model.

Returns:

List of sampled models.

mutate_parents(parents: List[ArchaiModel], mutations_per_parent: int | None = 1, patience: int | None = 20) → List[ArchaiModel][source]#

Mutate parents to generate new models.

Parameters:

  • parents – List of parent models.

  • mutations_per_parent – Number of mutations to apply to each parent.

  • patience – Number of tries to sample a valid model.

Returns:

List of mutated models.

search() → SearchResults[source]#

Search for the best architecture.

Returns:

Search results.

Successive Halving#

class archai.discrete_search.algos.successive_halving.SuccessiveHalvingSearch(search_space: DiscreteSearchSpace, objectives: SearchObjectives, dataset_provider: DatasetProvider, output_dir: str, num_iters: int | None = 10, init_num_models: int | None = 10, init_budget: float | None = 1.0, budget_multiplier: float | None = 2.0, seed: int | None = 1)[source]#

Successive Halving algorithm

search() → SearchResults[source]#

Search for the best architecture.

Returns:

Search results.