opto.trace

Subpackages

• opto.trace.propagators
  • opto.trace.propagators.propagators
  • opto.trace.propagators.graph_propagator

Package Contents

Classes

stop_tracing	A contextmanager to disable tracing.
Node	A data node in a directed graph, this is a basic data structure of Trace.
Module	Module is a ParameterContainer which has a forward method.
NodeContainer	An identifier for a container of nodes.

Functions

node	Create a Node object from data. If the data is already a Node, it will be returned as is. This function is provided for the convenience of the user and should be used instead of directly invoking the Node class.
model	Wrap a class with this decorator. This helps collect parameters for the optimizer. This decorated class cannot be pickled.
apply_op	A broadcasting operation that applies an op to container of Nodes.

Data

GRAPH

API

node(data, name=None, trainable=False, description=None, constraint=None)

Create a Node object from data. If the data is already a Node, it will be returned as is. This function is provided for the convenience of the user and should be used instead of directly invoking the Node class.

data: The data to create the Node from.

name: (optional) The name of the Node.

trainable: (optional) A boolean indicating whether the Node is trainable or not. Default is False.

description: (optional) A string describing the data.

constraint: (optional) A string describing any constraint that the data should obey.

The node function allows users to create Node objects from data. The function first checks if the trainable parameter is True. If it is, it checks if the data is already a Node. If it is, it extracts the underlying data and updates the name if a new name is provided. It then creates a ParameterNode object with the extracted data, name, trainable set to True, and the provided constraint. If the message is not already a Node, it creates a new ParameterNode object with the message as the data, the provided name, trainable set to True, and the provided constraint.

If the trainable parameter is False, the function checks if the message is already a Node. If it is, it checks if a name is provided. If a name is provided, it issues a warning that the name is ignored because the message is already a Node. It then returns the message as is. If the message is not already a Node, it creates a new Node object with the message as the data, the provided name, and the provided constraint.

class stop_tracing

A contextmanager to disable tracing.

GRAPH = 'Graph(...)'

class Node(value: Any, *, name: str = None, trainable: bool = False, description: str = '[Node] This is a node in a computational graph.', constraint: Union[None, str] = None, info: Union[None, Dict] = None)

Bases: opto.trace.nodes.AbstractNode[opto.trace.nodes.T]

A data node in a directed graph, this is a basic data structure of Trace.

trainable: A boolean indicating whether the node is trainable or not. _feedback: A dictionary of feedback from children nodes. _description: A string describing the node. _constraint: A string describing all constraints that the data in the node should satisfy. _backwarded: A boolean indicating whether the backward method has been called. _info: A dictionary containing additional information about the node. _dependencies: A dictionary of dependencies on parameters and expandable nodes.

The Node class extends the AbstractNode class to represent a data node in a directed graph. It includes additional attributes and methods to handle feedback, constraints, and dependencies. The node can be marked as trainable, and it can store feedback from children nodes. The node has a description and additional information associated with it. The node can also track dependencies on parameters and expandable nodes, which are nodes that depend on parameters not visible in the current graph level.

The Node class is meant to be subclassed and extended to create specific types of nodes. The feedback mechanism is analogous to gradients in machine learning and is used to propagate information back through the graph. The feedback mechanism is designed to support non-commutative aggregation, so feedback should be handled carefully to maintain the correct order of operations.

Initialization

Initialize an instance of the Node class.

value: The value to be assigned to the node. name: The name of the node (optional). trainable: A boolean indicating whether the node is trainable or not (optional). description: A string describing the node (optional). constraint: A string describing constraints on the node (optional). info: A dictionary containing additional information about the node (optional).

zero_feedback()

Zero out the feedback of the node. zero_feedback should be used judiciously within the feedback propagation process to avoid unintended loss of feedback data. It is specifically designed to be used after feedback has been successfully propagated to parent nodes.

property feedback

The feedback from children nodes.

property description

A textual description of the node.

property info

Additional information about the node.

property type

The type of the data stored in the node.

property parameter_dependencies

The depended parameters.

Ensure that the ‘_dependencies’ attribute is properly initialized and contains a ‘parameter’ key with a corresponding value before calling the parameter_dependencies function to avoid potential KeyError exceptions.

property expandable_dependencies

The depended expandable nodes, where expandable nodes are those who depend on parameters not visible in the current graph level.

Ensure that the ‘_dependencies’ attribute is properly initialized and contains an ‘expandable’ key with a corresponding value before calling the expandable_dependencies function to avoid potential KeyError exceptions

backward(feedback: Any = '', propagator=None, retain_graph=False, visualize=False, simple_visualization=True, reverse_plot=False, print_limit=100)

Performs a backward pass in a computational graph. This function propagates feedback from the current node to its parents, updates the graph visualization if required, and returns the resulting graph.

feedback: The feedback given to the current node. propagator: A function that takes in a node and a feedback, and returns a dict of {parent: parent_feedback}. If not provided, a default GraphPropagator object is used. retain_graph: If True, the graph will be retained after backward pass. visualize: If True, the graph will be visualized using graphviz. simple_visualization: If True, identity operators will be skipped in the visualization; otherwise, they will be included. reverse_plot: if True, plot the graph in reverse order (from child to parent). print_limit: The maximum number of characters to print for node descriptions and content.

The function checks if the current node has already been backwarded. If it has, an AttributeError is raised. Otherwise, the function adds the feedback to the current node by calling the _add_feedback method of the node object. The feedback is initialized with a special “FEEDBACK_ORACLE” node and the propagated feedback from the propagator object.

If the current node has no parents, indicating that it is a root node, the function checks if visualization is enabled. If it is, the current node is added to the digraph object with the appropriate style attributes. Finally, the function returns the digraph object.

If the current node has parents, indicating that it is not a root node, the function initializes a priority queue. The priority queue is used to process the nodes in the correct order during the backward pass. The function enters a loop that continues until the queue is empty. In each iteration, a node is popped from the queue and processed. The node is checked to ensure it has parents and is an instance of the MessageNode class. If not, an AttributeError is raised.

The function propagates information from the current node to its parents by calling the propagator object with the current node as the argument. The propagator object computes the propagated feedback based on the child node’s description, data, and feedback. The propagated feedback is then added to the parents of the current node by calling the _add_feedback method of each parent node.

After processing the parents of the current node, the _backwarded attribute of the current node is updated to indicate that it has been backwarded. This attribute is set to True unless the retain_graph parameter is set to True.

The loop continues until the queue is empty, indicating that all the nodes have been processed. Finally, the function returns the digraph object.

clone()

Create and return a duplicate of the current Node object.

detach()

Create and return a deep copy of the current instance of the Node class.

getattr(key)

Get the attribute of the node with the specified key.

key: The key of the attribute to get.

call(fun: str, *args, **kwargs)

Call the function with the specified arguments and keyword arguments.

fun: The function to call. args: The arguments to pass to the function. kwargs: The keyword arguments to pass to the function.

len()

Return the length of the node.

We overload magic methods that return a value. This method returns a MessageNode.

eq(other)

Check if the node is equal to another value.

other: The value to compare the node to.

If a logic operator is used in an if-statement, it will return a boolean value. Otherwise, it will return a MessageNode.

neq(other)

Check if the node is not equal to another value.

other: The value to compare the node to.

If a logic operator is used in an if-statement, it will return a boolean value. Otherwise, it will return a MessageNode.

format(*args, **kwargs)

capitalize()

lower()

upper()

swapcase()

title()

split(sep=None, maxsplit=-1)

strip(chars=None)

replace(old, new, count=-1)

items()

values()

keys()

pop(__index=-1)

append(*args, **kwargs)

class Module

Bases: opto.trace.containers.ParameterContainer

Module is a ParameterContainer which has a forward method.

abstract forward(*args, **kwargs)

save(file_name)

Save the parameters of the model to a file.

load(file_name)

Load the parameters of the model from a file.

class NodeContainer

An identifier for a container of nodes.

model(cls)

Wrap a class with this decorator. This helps collect parameters for the optimizer. This decorated class cannot be pickled.

apply_op(op, output, *args, **kwargs)

A broadcasting operation that applies an op to container of Nodes.

Args:

op (callable): the operator to be applied. output (Any): the container to be updated. *args (Any): the positional inputs of the operator. **kwargs (Any): the keyword inputs of the operator.