Parallel computing is a type of software engineering where you use parallelism (e.g. more threads, more processes, more nodes, etc.) to increase the amount of computation power to tackle a problem.

There are essentially three spheres of parallel computing: 1. CPU parallelism (OpenMP), 2. distributed parallelism (MPI), and 3. GPU parallelism (CUDA).

Parallel computing is related to performance engineering. This makes sense, because having say 12 CPU cores working on a problem. $$S(\text{N-cores}) = \frac{1}{(1 - P) + \frac{P}{N}}$$

• $S(N)$ - Speed up
• $P$ - Length