Parallel computing is a type of software engineering where you use parallelism (more threads, more cores, more computers) to increase the amount of computation power against a given task. There are essentially three spheres of parallel computing: 1. CPU parallelism (OpenMP), 2. distributed parallelism (MPI), and 3. GPU parallelism (CUDA).

Generally speaking, the potential speed up you'd get by parallelizing increasing $N$ cores is governed by Amdahl's law $$S(\text{N}) = \frac{1}{(1 - P) + \frac{P}{N}}$$

• OpenMP
• MPI
• cuda