# MPI

**MPI** (Message Passing Interface) is a standard for distributed-memory parallel programming in C, C++, and Fortran. Launch N independent copies of your program via `mpirun -n N ./program`; each gets its own address space and communicates by explicitly sending and receiving messages. Unlike [[openmp|OpenMP]], processes don't share memory, but MPI scales from a laptop to clusters with thousands of nodes without code changes.

The execution model is **SPMD** (single program, multiple data): all processes start together and execute the same binary, but each plays a different role based on its **rank** (0 to N-1). Every MPI program must call `MPI_Init` first and `MPI_Finalize` last. Compile with `mpicc` (C) or `mpicxx` (C++).

```c
// compile: mpicc -o ping ping.c
// run: mpirun -n 2 ./ping
// description: rank 0 sends a value to rank 1

#include <mpi.h>
#include <stdio.h>

int main(int argc, char **argv) {
    MPI_Init(&argc, &argv);
    int rank, value = 0;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    if (rank == 0) {
        value = 42;
        MPI_Send(&value, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
        printf("rank 0: sent %d\n", value);
    } else {
        MPI_Recv(&value, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
        printf("rank 1: received %d\n", value);
    }

    MPI_Finalize();
    return 0;
}
```

Rank 0 blocks on `MPI_Send` until rank 1 reaches `MPI_Recv`. This blocking behaviour is fundamental—MPI communication is synchronous. From here, [[mpi-point-to-point|point-to-point communication]] is the natural next concept, then [[mpi-collectives|collectives]].

## Concepts

 1. [[mpi-communicators|Communicators]]
 2. [[mpi-point-to-point|Point-to-point communication]]
 3. [[mpi-blocking|Blocking and non-blocking]]
 4. [[mpi-send-modes|Send modes]]
 5. [[mpi-message-ordering|Message ordering]]
 6. [[mpi-probing|Probing for messages]]
 7. [[mpi-deadlock|Deadlock]]
 8. [[mpi-performance-model|Performance model]]
 9. [[mpi-collectives|Collectives]]
 10. [[mpi-reduction|Reduction]]
 11. [[mpi-scatter-and-gather|Scatter and gather]]
 12. [[mpi-prefix-reductions|Prefix reductions]]
 13. [[mpi-nonblocking-collectives|Non-blocking collectives]]
 14. [[mpi-persistent-communication|Persistent communication]]
 15. [[mpi-derived-datatypes|Derived datatypes]]
 16. [[mpi-virtual-topologies|Virtual topologies]]
 17. [[mpi-process-groups|Process groups]]
 18. [[mpi-communicator-duplication|Communicator duplication]]
 19. [[mpi-one-sided|One-sided communication]]
 20. [[mpi-shared-memory-windows|Shared memory windows]]
 21. [[mpi-parallel-io|Parallel I/O]]
 22. [[mpi-time-measurement|Time measurement]]
 23. [[mpi-hybrid-openmp|Hybrid MPI+OpenMP]]
 24. [[mpi-overview|Overview]]