Site Tools


mpi-message-ordering

Table of Contents

MPI Message ordering

Message ordering in MPI guarantees FIFO delivery: messages from A to B on the same communicator and tag arrive in order.

// rank 1 is guaranteed to receive 1 first, then 2
if (rank == 0) {
    MPI_Send(&one, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
    MPI_Send(&two, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
} else if (rank == 1) {
    int a, b;
    MPI_Recv(&a, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // a == 1
    MPI_Recv(&b, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // b == 2
}

The guarantee applies only to point-to-point messages between the same pair of processes on the same communicator with the same tag. It does not prevent overtaking when multiple senders are involved: if rank 0 and rank 2 both send to rank 1, their messages can arrive in any order relative to each other. Collectives have no ordering guarantees relative to point-to-point messages.

mpi-message-ordering.md · Last modified: by 127.0.0.1