Persistent communication (MPI)

HTTP/1.0 opened a new TCP connection for every request, paying a three-way handshake each time. HTTP/1.1 introduced keep-alive: one connection, many requests, the setup cost paid once. MPI persistent communication applies the same idea to message passing. In an iterative algorithm where the same process pair exchanges data every iteration with the same buffer and count — a halo exchange in a stencil code, for example — MPI_Send_init and MPI_Recv_init register the communication pattern once. Each iteration calls MPI_Start to trigger it and MPI_Wait to complete it, skipping argument validation and buffer registration on every subsequent pass.

MPI_Request reqs[2];
MPI_Send_init(send_buf, N, MPI_DOUBLE, right, 0, MPI_COMM_WORLD, &reqs[0]);
MPI_Recv_init(recv_buf, N, MPI_DOUBLE, left,  0, MPI_COMM_WORLD, &reqs[1]);
 
for (int iter = 0; iter < MAX_ITER; iter++) {
    pack_halo(send_buf);
    MPI_Startall(2, reqs);
    compute_interior();
    MPI_Waitall(2, reqs, MPI_STATUSES_IGNORE);
    use_halo(recv_buf);
}
 
MPI_Request_free(&reqs[0]);
MPI_Request_free(&reqs[1]);

The implementation can cache internal state and skip repeated argument validation, which reduces per-iteration overhead for small messages. On high-latency networks the benefit is modest since latency dominates; for many small messages in a tight loop it adds up. Persistent requests follow the same non-blocking semantics as MPI_Isend/MPI_Irecv: the buffer must not be modified between MPI_Start and MPI_Wait.