Differences

This shows you the differences between two versions of the page.

--- parallel-computing [June 10, 2026 at 21:46] – Ivan Janevski
+++ parallel-computing [August 22, 2026 at 15:22] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 # Parallel computing
-**Parallel computing** is a type of software engineering where you use parallelism (more threads, more cores, more computers) to increase the amount of computation power against a given task. There are essentially three spheres of parallel computing: 1. CPU parallelism ([[openmp|OpenMP]]), 2. distributed parallelism ([[mpi|MPI]]), and 3. GPU parallelism ([[cuda|CUDA]]).
-Generally speaking, the potential speed up you'd get by parallelizing increasing $N$ cores is governed by Amdahl's law
+**[Parallel computing](https://en.wikipedia.org/wiki/Parallel_computing)** is a computational model where work is broken into parts that execute simultaneously across multiple processors, cores, or machines. Modern CPUs gain performance through additional cores rather than clock speed increases, so exploiting parallelism is essential for performance.
-$$S(\text{N}) = \frac{1}{(1 - P) + \frac{P}{N}}$$
-## List of parallel computing concepts
+[[amdahls-law|Amdahl's law]] shows the sequential fraction limits speedup; [[gustafsons-law|Gustafson's law]] shows that scaling problem size with hardware yields near-linear speedup.
+## Example
+This example shows a simple parallel computation using OpenMP.
+```c
+// compile: gcc -fopenmp -o parallel parallel.c
+// run: ./parallel
+// description: parallel loop computing array sum
+#include <omp.h>
+#include <stdio.h>
+int main() {
+    int arr[100];
+    for (int i = 0; i < 100; i++) arr[i] = i;
+    int sum = 0;
+#pragma omp parallel for reduction(+:sum)
+    for (int i = 0; i < 100; i++) {
+        sum += arr[i];
+    }
+    printf("Sum: %d\n", sum);
+    return 0;
+}
+```
- - [[openmp]]
- - [[mpi]]
- - [[cuda]]