mscclpp.language.default_algos.allreduce_2nodes

mscclpp.language.default_algos.allreduce_2nodes(spec, thread_block_group_size)

Implements a multi-node AllReduce using a hierarchical approach: 1. Intra-node allreduce 2. Inter-node exchange (exchange reduced data between nodes) 3. Intra-node allreduce

Return type: