mscclpp.language.collectives

Classes

AllGather(num_ranks, chunk_factor, inplace)	An AllGather collective communication pattern.
AllReduce(num_ranks, chunk_factor, inplace)	An AllReduce collective communication operation.
AllToAll(num_ranks, chunk_factor, inplace)
Collective(num_ranks, chunk_factor, inplace)	Base class for defining collective communication patterns.
ReduceScatter(num_ranks, chunk_factor, inplace)	A ReduceScatter collective communication operation.
TestCollective(num_ranks, input_size, ...)	A test collective for validation and testing purposes.

class mscclpp.language.collectives.AllGather(num_ranks, chunk_factor, inplace)

Bases: Collective

An AllGather collective communication pattern.

The AllGather operation is a collective communication pattern where each rank begins with a unique block of data and, by the end of the operation, every process holds the concatenation of all data blocks from all ranks.

This operation creates input buffers sized by chunk_factor and output buffers sized to hold data from all ranks (num_ranks * chunk_factor).

init_buffers()

Initialize buffers for the AllGather operation.

Creates input buffers sized by chunk_factor and output buffers sized to hold data from all ranks (num_ranks * chunk_factor).

Returns:

A list of buffer dictionaries, one for each rank.

Return type:

list

class mscclpp.language.collectives.AllReduce(num_ranks, chunk_factor, inplace)

Bases: Collective

An AllReduce collective communication operation.

The AllReduce operation combines data from all ranks using a specified reduction operation (e.g., sum, max, min) and then distributes the final reduced result back to all ranks.

This operation creates input and output buffers both sized to hold the complete dataset (num_ranks * chunk_factor)

init_buffers()

Initialize buffers for the AllReduce operation.

Creates input and output buffers both sized to hold the complete dataset (num_ranks * chunk_factor) since AllReduce operates on the full data from all ranks.

Returns:

A list of buffer dictionaries, one for each rank.

Return type:

list

class mscclpp.language.collectives.AllToAll(num_ranks, chunk_factor, inplace)

Bases: Collective

class mscclpp.language.collectives.Collective(num_ranks, chunk_factor, inplace)

Bases: object

Base class for defining collective communication patterns.

Collective serves as the foundation for implementing various collective communication algorithms like AllGather, AllReduce, and ReduceScatter. It defines the common interface and behavior that all collective operations must implement.

num_ranks

The number of ranks participating in the collective.

Type:

int

chunk_factor

The chunk factor for data subdivision.

Type:

int

inplace

Whether the collective operates in-place.

Type:

bool

name

The name of the collective operation.

Type:

str

class mscclpp.language.collectives.ReduceScatter(num_ranks, chunk_factor, inplace)

Bases: Collective

A ReduceScatter collective communication operation.

ReduceScatter performs a reduction operation across all ranks and then scatters the result, with each rank receiving a unique portion of the reduced data. This is the inverse of AllGather.

This operations creates input buffers sized to hold the complete dataset (num_ranks * chunk_factor) and output buffers sized to hold each rank’s portion (chunk_factor).

init_buffers()

Initialize buffers for the ReduceScatter operation.

Creates input buffers sized to hold the complete dataset (num_ranks * chunk_factor) and output buffers sized to hold each rank’s portion (chunk_factor).

Returns:

A list of buffer dictionaries, one for each rank.

Return type:

list

class mscclpp.language.collectives.TestCollective(num_ranks, input_size, output_size)

Bases: Collective

A test collective for validation and testing purposes.

TestCollective provides a simple collective implementation used for testing the DSL functionality with custom input and output buffer sizes.

input_size

The size of the input buffer.

Type:

int

output_size

The size of the output buffer.

Type:

int

init_buffers()

Initialize input and output buffers for the test collective.

Creates input and output buffers with the specified sizes for each rank.

Returns:

A list of buffer dictionaries, one for each rank.

Return type:

list