High-Performance Computing (HPC) is the practice of extracting maximum computational power from available hardware. Write code that scales across multiple cores, optimize memory access patterns, and profile performance bottlenecks to maximize throughput. HPC spans multi-core CPUs, GPUs, and distributed clusters—from a laptop running OpenMP to supercomputers with thousands of nodes.

HPC is fundamentally about hardware-software co-design. You can't optimize without understanding the machine: cache hierarchies, memory bandwidth, floating-point units, and communication latencies. It's not enough to parallelize; you must parallelize efficiently relative to the hardware underneath.

# Compile OpenMP program
$ gcc -O3 -fopenmp -o app app.c
 
# Run with 4 threads
$ OMP_NUM_THREADS=4 ./app
 
# Profile with perf
$ perf record ./app
$ perf report

Performance scales across three dimensions: parallelism (use all cores), memory efficiency (minimize stalls), and communication (reduce latency and bandwidth waste). Most real bottlenecks live at the intersection of these. A fast algorithm on the wrong hardware, or perfect parallelism with pathological cache behavior, both fail.

1. Hardware architectures
2. Parallelism models
3. OpenMP
4. MPI
5. CUDA
6. Memory hierarchy
7. Cache optimization
8. Vectorization
9. Profiling
10. Roofline model
11. Amdahl's law
12. Communication patterns
13. Load balancing
14. Numerical libraries
15. Cluster computing
16. Debugging distributed applications
17. Performance bottlenecks
18. Heterogeneous computing