Site Tools


message-ordering-mpi

Table of Contents

Message ordering (MPI)

TCP guarantees that bytes sent on a stream arrive in order. If you send() “hello” and then send() “world” on the same socket, the receiver gets “hello” first. MPI provides the same guarantee for point-to-point messages: messages sent from process A to process B on the same communicator with the same tag arrive in the order they were sent. If A calls MPI_Send twice to B with tag 0, B's first matching MPI_Recv gets the first message.

// rank 1 is guaranteed to receive 1 first, then 2
if (rank == 0) {
    MPI_Send(&one, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
    MPI_Send(&two, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
} else if (rank == 1) {
    int a, b;
    MPI_Recv(&a, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // a == 1
    MPI_Recv(&b, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // b == 2
}

The guarantee applies only to point-to-point messages between the same pair of processes on the same communicator with the same tag. It does not prevent overtaking when multiple senders are involved: if rank 0 and rank 2 both send to rank 1, their messages can arrive in any order relative to each other. Collectives have no ordering guarantees relative to point-to-point messages.

message-ordering-mpi.txt · Last modified: by 127.0.0.1