Table of Contents
Scatter and gather (MPI)
The basic pattern for distributing work is: rank 0 holds the full dataset, divides it into N equal chunks, and sends one to each process. Without collectives, that is a loop of sends:
if (rank == 0) for (int i = 0; i < size; i++) MPI_Send(&data[i * chunk], chunk, MPI_INT, i, 0, MPI_COMM_WORLD); else MPI_Recv(local, chunk, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
This serialises all N sends on rank 0. MPI_Scatter expresses the same pattern as a single collective call that all processes participate in simultaneously, dividing a root's buffer into equal-sized chunks and delivering one to each process including the root itself. MPI_Gather is the inverse: each process sends a chunk back and root assembles the result.
int send_buf[size]; // root fills this int recv_val; MPI_Scatter(send_buf, 1, MPI_INT, &recv_val, 1, MPI_INT, 0, MPI_COMM_WORLD);
MPI_Scatterv and MPI_Gatherv are the variable-length variants, where each process sends or receives a different number of elements. They take a counts array and a displacements array, so chunks do not need to be equal in size or contiguous in the root's buffer. This is the right tool for distributing irregular domain decompositions where the workload per process is not uniform.
