Parallel computing is a type of software engineering where you use parallelism (e.g. more threads, more processes, more nodes, etc.) to increase the amount of computation power to tackle a problem.

There are essentially three spheres of parallel computing: 1. CPU parallelism (OpenMP), 2. distributed parallelism (MPI), and 3. GPU parallelism (CUDA). I would classify FPGA

Parallel computing is related to performance engineering. This makes sense, because usually increasing parallelism increases performance, but this is not always so.

$$S(\text{N-cores}) = \frac{1}{(1 - P) + \frac{P}{N}}$$

• $S(N)$ - Speed up
• $P$ - Fraction of (percentage)
• $1 - P$ - Fraction of the program that