hpc
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| hpc [March 21, 2026 at 18:37] – yanevskiv | hpc [August 26, 2026 at 15:46] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | # HPC | ||
| + | |||
| + | **High-Performance Computing (HPC)** is the practice of extracting maximum computational power from available hardware. Write code that scales across multiple cores, optimize memory access patterns, and profile performance bottlenecks to maximize throughput. HPC spans multi-core CPUs, GPUs, and distributed clusters—from a laptop running OpenMP to supercomputers with thousands of nodes. | ||
| + | |||
| + | HPC is fundamentally about hardware-software co-design. You can't optimize without understanding the machine: cache hierarchies, | ||
| + | |||
| + | ```bash | ||
| + | # Compile openmp program | ||
| + | $ gcc -O3 -fopenmp -o app app.c | ||
| + | |||
| + | # Run with 4 threads | ||
| + | $ OMP_NUM_THREADS=4 ./app | ||
| + | |||
| + | # Profile with perf | ||
| + | $ perf record ./app | ||
| + | $ perf report | ||
| + | ``` | ||
| + | |||
| + | Performance scales across three dimensions: parallelism (use all cores), memory efficiency (minimize stalls), and communication (reduce latency and bandwidth waste). Most real bottlenecks live at the intersection of these. A fast algorithm on the wrong hardware, or perfect parallelism with pathological cache behavior, both fail. | ||
| + | |||
| + | ## Concepts | ||
| + | |||
| + | 1. [[hpc-hardware|Hardware architectures]] | ||
| + | 2. [[hpc-parallelism|Parallelism models]] | ||
| + | 3. [[openmp|OpenMP]] | ||
| + | 4. [[mpi|MPI]] | ||
| + | 5. [[cuda|CUDA]] | ||
| + | 6. [[hpc-memory-hierarchy|Memory hierarchy]] | ||
| + | 7. [[hpc-cache-optimization|Cache optimization]] | ||
| + | 8. [[hpc-vectorization|Vectorization]] | ||
| + | 9. [[hpc-profiling|Profiling]] | ||
| + | 10. [[hpc-roofline-model|Roofline model]] | ||
| + | 11. [[amdahls-law|Amdahl' | ||
| + | 12. [[hpc-communication-patterns|Communication patterns]] | ||
| + | 13. [[hpc-load-balancing|Load balancing]] | ||
| + | 14. [[hpc-numerical-libraries|Numerical libraries]] | ||
| + | 15. [[hpc-cluster-computing|Cluster computing]] | ||
| + | 16. [[hpc-debugging-distributed|Debugging distributed applications]] | ||
| + | 17. [[hpc-performance-bottlenecks|Performance bottlenecks]] | ||
| + | 18. [[hpc-heterogeneous-computing|Heterogeneous computing]] | ||