Differences

This shows you the differences between two versions of the page.

--- openmp [June 11, 2026 at 09:35] – Ivan Janevski
+++ openmp [June 13, 2026 at 03:13] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 # OpenMP
-**OpenMP** is a shared-memory parallelism API for C, C++, and Fortran. It lets a single program exploit multiple CPU cores by distributing work across a team of threads that all share the same address space. This is in contrast to distributed-memory models like [[mpi|MPI]], where each process has its own memory. OpenMP is implemented via compiler directives (`#pragma omp` in C/C++), a small runtime library (`libomp`), and a set of environment variables.
+**OpenMP** is a shared-memory parallelism API for C, C++, and Fortran. If you have a loop that takes too long and you want it to use all the cores on your machine instead of just one, OpenMP is usually the shortest path there. It works via compiler directives (`#pragma omp` in C/C++), a small runtime library (`libomp`), and a set of environment variables. In contrast to distributed-memory models like [[mpi|MPI]], where each process has its own memory, OpenMP threads all share the same address space — you get parallelism without touching your data layout.
-The execution model is **fork-join**: the program starts as a single master thread. When it hits a `#pragma omp parallel` block, it forks into a team of worker threads; all threads execute the block concurrently, then join back into one thread at the closing brace. The compiler inserts the thread creation, synchronization, and teardown code. The programmer only writes directives. If the compiler does not support OpenMP, it silently ignores all `#pragma omp` directives and the program runs serially, which is a useful property during debugging.
+The execution model is **fork-join**: the program starts as a single thread. When it hits a `#pragma omp parallel` block, it forks into a team of worker threads that all execute the block concurrently, then join back into one thread at the closing brace. You do not write any thread creation, management, or teardown code — the compiler inserts all of that. If the compiler does not support OpenMP, it silently ignores all `#pragma omp` directives and the program runs serially, which is useful for debugging: drop `-fopenmp` from the compile command and you get a clean serial build without changing a line of source.
 ```c
@@ Line 13: / Line 13: @@
 ```
-The number of threads defaults to the number of logical CPU cores. It can be overridden with the `OMP_NUM_THREADS` environment variable or the `num_threads(N)` clause on the pragma. Output order is non-deterministic — threads are scheduled by the OS. Compile with `-fopenmp` and include `<omp.h>` to use the `omp_*` runtime functions.
+The thread count defaults to the number of logical cores. You can override it at runtime with `OMP_NUM_THREADS=N ./prog` without recompiling, or with `omp_set_num_threads(n)` inside the program, or with a `num_threads(N)` clause directly on the pragma. Output order from a parallel region is non-deterministic — threads are scheduled by the OS, not by rank. Compile with `-fopenmp` and include `<omp.h>` for the `omp_*` runtime functions.
-Adding more threads does not always mean proportionally faster code. [[amdahls-law|Amdahl's law]] states that if a fraction $s$ of the program is inherently serial, the maximum possible speedup is $1/s$ regardless of how many threads are used. A loop that accounts for 80% of runtime can at best give 5× speedup no matter how many cores are available. This makes reducing the serial fraction more impactful than simply increasing thread count. Eliminating overhead sources like false sharing, load imbalance, and unnecessary synchronisation compounds that further.
+Adding more threads does not always mean proportionally faster code. [[amdahls-law|Amdahl's law]] says that if a fraction $s$ of the program is inherently serial, the maximum speedup is $1/s$ no matter how many threads you add. A loop that accounts for 80% of runtime can at best give 5× speedup. This is why reducing the serial fraction matters more than just adding more cores, and why eliminating overhead like false sharing, load imbalance, and unnecessary barriers compounds those gains.
+## Practice
+Compile the hello-world above and run it:
+```bash
+$ gcc -fopenmp -o hello hello.c
+$ OMP_NUM_THREADS=4 ./hello
+thread 2 of 4
+thread 0 of 4
+thread 3 of 4
+thread 1 of 4
+```
+The output order is non-deterministic — run it a few times and you will get different permutations. Now try dropping `-fopenmp`:
+```bash
+$ gcc -o hello hello.c
+$ ./hello
+thread 0 of 1
+```
+Without the flag, every `#pragma omp` directive is silently ignored and the program runs as a single thread. This is the most useful OpenMP debugging technique: if your parallel output looks wrong, remove `-fopenmp` and check whether the serial output is correct first.
+Here is a more realistic example: parallelising a sum of a billion integers with a single pragma.
+```c
+// compile: gcc -O2 -fopenmp -o sum sum.c
+// run: OMP_NUM_THREADS=4 ./sum
+// description: parallel reduction; try with -fopenmp omitted for serial baseline
+#include <omp.h>
+#include <stdio.h>
+int main(void) {
+    long n = 1000000000L, sum = 0;
+    double t = omp_get_wtime();
+    #pragma omp parallel for reduction(+:sum)
+    for (long i = 0; i < n; i++)
+        sum += i;
+    printf("sum=%ld  time=%.2fs\n", sum, omp_get_wtime() - t);
+    return 0;
+}
+```
+On a 4-core machine this runs roughly 4× faster with `-fopenmp` than without. The only additions to the loop are the pragma line and the `reduction(+:sum)` clause. Without the clause you would get a data race and a wrong answer — the clause is what makes it correct. When it works this cleanly, it genuinely feels like cheating.
+Try varying `OMP_NUM_THREADS` from 1 up to your core count and beyond. Speedup will plateau or even decline past the hardware thread count — that is thread management overhead and Amdahl's law at work.
 ## Concepts
@@ Line 81: / Line 129: @@
 | `OMP_DISPLAY_ENV` | `false` | Print OpenMP version and active settings at startup: `TRUE` or `VERBOSE` |
-## Info
-*This article is synthetic*
-Up: [[home]]