Differences

This shows you the differences between two versions of the page.

--- parallel-computing [June 15, 2026 at 15:33] – Ivan Janevski
+++ parallel-computing [August 22, 2026 at 15:22] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 # Parallel computing
-**Parallel computing** is a style of programming where a computation is broken into parts that run simultaneously across multiple processors, cores, or machines. The motivation is straightforward: a single core has a clock speed ceiling, and modern CPUs gain performance by adding more cores rather than running each core faster. To take advantage of that, programs have to be written with parallelism in mind.
-Not every program benefits equally. [[amdahls-law|Amdahl's law]] shows that the sequential fraction of a program — the part that cannot be parallelized — sets a hard ceiling on speedup regardless of how many cores you add. [[gustafsons-law|Gustafson's law]] is the more optimistic counterpart: if you scale the problem size alongside the hardware, speedup grows linearly. In practice, HPC workloads follow Gustafson's regime — you buy more nodes to solve a bigger problem, not just to solve the same one faster.
+**[Parallel computing](https://en.wikipedia.org/wiki/Parallel_computing)** is a computational model where work is broken into parts that execute simultaneously across multiple processors, cores, or machines. Modern CPUs gain performance through additional cores rather than clock speed increases, so exploiting parallelism is essential for performance.
-## Map of parallel computing
+[[amdahls-law|Amdahl's law]] shows the sequential fraction limits speedup; [[gustafsons-law|Gustafson's law]] shows that scaling problem size with hardware yields near-linear speedup.
+## Example
+This example shows a simple parallel computation using OpenMP.
+```c
+// compile: gcc -fopenmp -o parallel parallel.c
+// run: ./parallel
+// description: parallel loop computing array sum
+#include <omp.h>
+#include <stdio.h>
+int main() {
+    int arr[100];
+    for (int i = 0; i < 100; i++) arr[i] = i;
+    int sum = 0;
+#pragma omp parallel for reduction(+:sum)
+    for (int i = 0; i < 100; i++) {
+        sum += arr[i];
+    }
+    printf("Sum: %d\n", sum);
+    return 0;
+}
+```
- - [[introduction-to-parallel-computing]]
- - [[saxpy]]
- - [[synchronization-primitve]]
-   - [[sync-semaphore]]
-   - [[sync-mutex]]
-   - [[sync-monitor]]
-   - [[sync-linda]]
-   - [[sync-csp]]
-   - [[sync-mbox]]
- - [[numbers-every-programmer-should-know]]
- - [[amdahls-law]]
- - [[gustafsons-law]]
- - [[cache]]
-   - [[l1-cache]]
-   - [[l2-cache]]
-   - [[l3-cache]]
-   - [[cache-coherence]]
-     - [[cache-snoopy-protocols]]
-        - [[wti]]
-        - [[msi]]
-        - [[mesi]]
-        - [[moesi]]
-        - [[dragon]]
-        - [[firefly]]
-     - [[cache-directory-protocols]]
- - [[cuda]]
- - [[openmp]]
- - [[mpi]]
- - [[queuing-theory]]
- - [[lock-free-queue]]
- - [[numa]]
- - [[false-sharing]]
- - [[aba-problem]]
- - [[trace-monoid]]
- - [[hazard-pointer]]
- - [[cache-coherence]]
- - [[roofline-model]]
- - [[numa]]
- - [[embarrassingly-parallel]]
- - [[fork-join-model]]
- - [[rcu]]
- - [[lock]]
- - [[lock-convoy]]
- - [[lock-contention]]
- - [[spinlock]]
- - [[smp]]
- - [[memory-order]]
- - [[cas]]
- - [[treiber-stack]]
- - [[cas]]
- - [[michael-scott-queue]]
- - [[bespoke-algorithm]]