Differences

This shows you the differences between two versions of the page.

--- mpi [June 10, 2026 at 21:55] – Ivan Janevski
+++ mpi [August 22, 2026 at 15:22] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 # MPI
-**MPI** (or **Message Passing Interface**) is a parallel computing API used for distributed parallelism.
-## Overview
+**MPI** (Message Passing Interface) is a standard for distributed-memory parallel programming in C, C++, and Fortran. Launch N independent copies of your program via `mpirun -n N ./program`; each gets its own address space and communicates by explicitly sending and receiving messages. Unlike [[openmp|OpenMP]], processes don't share memory, but MPI scales from a laptop to clusters with thousands of nodes without code changes.
-C programming language:
+The execution model is **SPMD** (single program, multiple data): all processes start together and execute the same binary, but each plays a different role based on its **rank** (0 to N-1). Every MPI program must call `MPI_Init` first and `MPI_Finalize` last. Compile with `mpicc` (C) or `mpicxx` (C++).
 ```c
-int MPI_Init(...);
+// compile: mpicc -o ping ping.c
-int MPI_Finalize(...);
+// run: mpirun -n 2 ./ping
-int MPI_Send (...);
+// description: rank 0 sends a value to rank 1
-int MPI_Recv (...);
-int MPI_Bcast(...);
-int MPI_Reduce(...);
-int MPI_Group_size(...);
-int MPI_Group_rank(...);
-int MPI_Comm_rank (...);
-int MPI_Comm_size (...);
-int MPI_Type_commit(...);
-int MPI_Type_contiguous(...);
-int MPI_Type_free(...);
-```
-C++ programming language:
+#include <mpi.h>
-```cpp
+#include <stdio.h>
-namespace MPI {
-    class Comm {...};
+int main(int argc, char **argv) {
-    class Intracomm : public Comm {...};
+    MPI_Init(&argc, &argv);
-    class Graphcomm : public Intracomm {...};
+    int rank, value = 0;
-    class Cartcomm : public Intracomm {...};
+    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
-    class Intercomm : public Comm {...};
-    class Datatype {...};
+    if (rank == 0) {
-    class Errhandler {...};
+        value = 42;
-    class Exception {...};
+        MPI_Send(&value, 1, MPI_INT, 1, 0, MPI_COMM_WORLD);
-    class Group {...};
+        printf("rank 0: sent %d\n", value);
-    class Op {...};
+    } else {
-    class Request {...};
+        MPI_Recv(&value, 1, MPI_INT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
-    class Prequest : public Request {...};
+        printf("rank 1: received %d\n", value);
-    class Status {...};
+    }
-};
+    MPI_Finalize();
+    return 0;
+}
 ```
+Rank 0 blocks on `MPI_Send` until rank 1 reaches `MPI_Recv`. This blocking behaviour is fundamental—MPI communication is synchronous. From here, [[mpi-point-to-point|point-to-point communication]] is the natural next concept, then [[mpi-collectives|collectives]].
+## Concepts
+. [[mpi-communicators|Communicators]]
+. [[mpi-point-to-point|Point-to-point communication]]
+. [[mpi-blocking|Blocking and non-blocking]]
+. [[mpi-send-modes|Send modes]]
+. [[mpi-message-ordering|Message ordering]]
+. [[mpi-probing|Probing for messages]]
+. [[mpi-deadlock|Deadlock]]
+. [[mpi-performance-model|Performance model]]
+. [[mpi-collectives|Collectives]]
+. [[mpi-reduction|Reduction]]
+. [[mpi-scatter-and-gather|Scatter and gather]]
+. [[mpi-prefix-reductions|Prefix reductions]]
+. [[mpi-nonblocking-collectives|Non-blocking collectives]]
+. [[mpi-persistent-communication|Persistent communication]]
+. [[mpi-derived-datatypes|Derived datatypes]]
+. [[mpi-virtual-topologies|Virtual topologies]]
+. [[mpi-process-groups|Process groups]]
+. [[mpi-communicator-duplication|Communicator duplication]]
+. [[mpi-one-sided|One-sided communication]]
+. [[mpi-shared-memory-windows|Shared memory windows]]
+. [[mpi-parallel-io|Parallel I/O]]
+. [[mpi-time-measurement|Time measurement]]
+. [[mpi-hybrid-openmp|Hybrid MPI+OpenMP]]
+. [[mpi-overview|Overview]]