NasBench-101

Base Operators

class archai.supergraph.algos.nasbench101.base_ops.ConvBnRelu(in_channels, out_channels, kernel_size=1, stride=1, padding=0)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class archai.supergraph.algos.nasbench101.base_ops.Conv3x3BnRelu(in_channels, out_channels)

3x3 convolution with batch norm and ReLU activation.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class archai.supergraph.algos.nasbench101.base_ops.Conv1x1BnRelu(in_channels, out_channels)

1x1 convolution with batch norm and ReLU activation.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class archai.supergraph.algos.nasbench101.base_ops.MaxPool3x3(in_channels, out_channels, kernel_size=3, stride=1, padding=1)

3x3 max pool with no subsampling.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

Operators

class archai.supergraph.algos.nasbench101.nasbench101_op.NasBench101Op(op_desc: OpDesc, arch_params: ArchParams | None, affine: bool)

forward(x: List[Tensor])

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

Configuration

Configuration flags.

archai.supergraph.algos.nasbench101.config.build_config()

Build config from flags defined in this module.

Graph (Utilities)

Utility functions used by generate_graph.py.

archai.supergraph.algos.nasbench101.graph_util.gen_is_edge_fn(bits)

Generate a boolean function for the edge connectivity.

Given a bitstring FEDCBA and a 4x4 matrix, the generated matrix is

[[0, A, B, D], [0, 0, C, E], [0, 0, 0, F], [0, 0, 0, 0]]

Note that this function is agnostic to the actual matrix dimension due to order in which elements are filled out (column-major, starting from least significant bit). For example, the same FEDCBA bitstring (0-padded) on a 5x5 matrix is

[[0, A, B, D, 0], [0, 0, C, E, 0], [0, 0, 0, F, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]

Parameters:

bits – integer which will be interpreted as a bit mask.

Returns:

vectorized function that returns True when an edge is present.

archai.supergraph.algos.nasbench101.graph_util.is_full_dag(matrix)

Full DAG == all vertices on a path from vert 0 to (V-1).

i.e. no disconnected or “hanging” vertices.

It is sufficient to check for:

1. no rows of 0 except for row V-1 (only output vertex has no out-edges)

2. no cols of 0 except for col 0 (only input vertex has no in-edges)

Parameters:

matrix – V x V upper-triangular adjacency matrix

Returns:

True if the there are no dangling vertices.

archai.supergraph.algos.nasbench101.graph_util.num_edges(matrix)

Computes number of edges in adjacency matrix.

archai.supergraph.algos.nasbench101.graph_util.hash_module(matrix, labeling)

Computes a graph-invariance MD5 hash of the matrix and label pair.

Parameters:

• matrix – np.ndarray square upper-triangular adjacency matrix.

• labeling – list of int labels of length equal to both dimensions of matrix.

Returns:

MD5 hash of the matrix and labeling.

archai.supergraph.algos.nasbench101.graph_util.permute_graph(graph, label, permutation)

Permutes the graph and labels based on permutation.

Parameters:

• graph – np.ndarray adjacency matrix.

• label – list of labels of same length as graph dimensions.

• permutation – a permutation list of ints of same length as graph dimensions.

Returns:

np.ndarray where vertex permutation[v] is vertex v from the original graph

archai.supergraph.algos.nasbench101.graph_util.is_isomorphic(graph1, graph2)

Exhaustively checks if 2 graphs are isomorphic.

Model Builder

archai.supergraph.algos.nasbench101.model_builder.build(desc_matrix: List[List[int]], vertex_ops: List[str], device=None, stem_out_channels=128, num_stacks=3, num_modules_per_stack=3, num_labels=10) → Module

Model Description Builder

class archai.supergraph.algos.nasbench101.nasbench101_model_desc_builder.NasBench101CellBuilder

pre_build(conf_model_desc: Config) → None

hook for accomplishing any setup before build starts

build_cell(in_shapes: List[List[List[int | float]]], conf_cell: Config, cell_index: int) → CellDesc

build_nodes(stem_shapes: List[List[int | float]], conf_cell: Config, cell_index: int, cell_type: CellType, node_count: int, in_shape: List[int | float], out_shape: List[int | float]) → Tuple[List[List[int | float]], List[NodeDesc]]

Model Matrix

archai.supergraph.algos.nasbench101.model_matrix.prune(model_matrix: ndarray, vertex_ops: List[str])

Prune the extraneous parts of the graph.

General procedure:

1. Remove parts of graph not connected to input.

2. Remove parts of graph not connected to output.

3. Reorder the vertices so that they are consecutive after steps 1 and 2.

These 3 steps can be combined by deleting the rows and columns of the vertices that are not reachable from both the input and output (in reverse).

Model Metrics (PB-2)

Model Specification

Model specification for module connectivity individuals.

This module handles pruning the unused parts of the computation graph but should avoid creating any TensorFlow models (this is done inside model_builder.py).

class archai.supergraph.algos.nasbench101.model_spec.ModelSpec(matrix, ops, data_format='channels_last')

Model specification given adjacency matrix and labeling.

hash_spec(canonical_ops)

Computes the isomorphism-invariant graph hash of this spec.

Parameters:

canonical_ops – list of operations in the canonical ordering which they were assigned (i.e. the order provided in the config[‘available_ops’]).

Returns:

MD5 hash of this spec which can be used to query the dataset.

visualize()

Creates a dot graph. Can be visualized in colab directly.

archai.supergraph.algos.nasbench101.model_spec.is_upper_triangular(matrix)

True if matrix is 0 on diagonal and below.

Model

Builds the Pytorch computational graph.

Tensors flowing into a single vertex are added together for all vertices except the output, which is concatenated instead. Tensors flowing out of input are always added.

If interior edge channels don’t match, drop the extra channels (channels are guaranteed non-decreasing). Tensors flowing out of the input as always projected instead.

class archai.supergraph.algos.nasbench101.model.Network(spec, stem_out_channels, num_stacks, num_modules_per_stack, num_labels)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class archai.supergraph.algos.nasbench101.model.Cell(spec, in_channels, out_channels)

Builds the model using the adjacency matrix and op labels specified. Channels controls the module output channel count but the interior channels are determined via equally splitting the channel count whenever there is a concatenation of Tensors.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

archai.supergraph.algos.nasbench101.model.Projection(in_channels, out_channels)

1x1 projection (as in ResNet) followed by batch normalization and ReLU.

archai.supergraph.algos.nasbench101.model.Truncate(inputs, channels)

Slice the inputs to channels if necessary.

archai.supergraph.algos.nasbench101.model.ComputeVertexChannels(in_channels, out_channels, matrix)

Computes the number of channels at every vertex.

Given the input channels and output channels, this calculates the number of channels at each interior vertex. Interior vertices have the same number of channels as the max of the channels of the vertices it feeds into. The output channels are divided amongst the vertices that are directly connected to it. When the division is not even, some vertices may receive an extra channel to compensate.

Returns:

list of channel counts, in order of the vertices.

Experiment Runner

class archai.supergraph.algos.nasbench101.nasbench101_exp_runner.PetridishExperimentRunner(config_filename: str, base_name: str, clean_expdir=False)

model_desc_builder() → PetridishModelBuilder

trainer_class() → Type[ArchTrainer] | None