mscclpp.default_algos.reducescatter_multi_nodes

mscclpp.default_algos.reducescatter_multi_nodes(spec, thread_block_group_size=1)

Build a hierarchical ReduceScatter across nodes and local GPUs.

Return type:

CollectiveProgram

Parameters:

• spec (AlgoSpec)

• thread_block_group_size (int)