Ivan's wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
# Message ordering (MPI)
TCP guarantees that bytes sent on a stream arrive in order. If you `send()` "hello" and then `send()` "world" on the same socket, the receiver gets "hello" first. MPI provides the same guarantee for point-to-point messages: messages sent from process A to process B on the same communicator with the same tag arrive in the order they were sent. If A calls `MPI_Send` twice to B with tag 0, B's first matching `MPI_Recv` gets the first message.

```c
// rank 1 is guaranteed to receive 1 first, then 2
if (rank == 0) {
    MPI_Send(&one, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
    MPI_Send(&two, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
} else if (rank == 1) {
    int a, b;
    MPI_Recv(&a, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // a == 1
    MPI_Recv(&b, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);  // b == 2
}
```

The guarantee applies only to point-to-point messages between the same pair of processes on the same communicator with the same tag. It does not prevent overtaking when multiple senders are involved: if rank 0 and rank 2 both send to rank 1, their messages can arrive in any order relative to each other. Collectives have no ordering guarantees relative to point-to-point messages.