mscclpp.default_algos.allgather_multi_nodes

mscclpp.default_algos.allgather_multi_nodes(spec, thread_block_group_size=1)

Build a hierarchical AllGather across nodes and local GPUs.

thread_block_group_size partitions each transfer and local copy across that many thread blocks. The default of one preserves the original thread block layout.