Table of Contents
Message ordering (MPI)
TCP guarantees that bytes sent on a stream arrive in order. If you send() “hello” and then send() “world” on the same socket, the receiver gets “hello” first. MPI provides the same guarantee for point-to-point messages: messages sent from process A to process B on the same communicator with the same tag arrive in the order they were sent. If A calls MPI_Send twice to B with tag 0, B's first matching MPI_Recv gets the first message.
// rank 1 is guaranteed to receive 1 first, then 2 if (rank == 0) { MPI_Send(&one, 1, MPI_INT, 1, 0, MPI_COMM_WORLD); MPI_Send(&two, 1, MPI_INT, 1, 0, MPI_COMM_WORLD); } else if (rank == 1) { int a, b; MPI_Recv(&a, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE); // a == 1 MPI_Recv(&b, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE); // b == 2 }
The guarantee applies only to point-to-point messages between the same pair of processes on the same communicator with the same tag. It does not prevent overtaking when multiple senders are involved: if rank 0 and rank 2 both send to rank 1, their messages can arrive in any order relative to each other. Collectives have no ordering guarantees relative to point-to-point messages.
