Parallel computing is a type of software engineering where you use parallelism (more threads) to increase the amount of computation power to tackle a problem.

There are essentially three spheres of parallel computing: 1. CPU parallelism (OpenMP), 2. distributed parallelism (MPI), and 3. GPU parallelism (CUDA). I would carve out a fourth category, which is 4.

Parallel computing is related to performance engineering. This makes sense, because usually increasing parallelism increases performance, but this is not always so. Not every piece of code can be parallelized and sometimes the overhead of creating threads can overweight the potential speedup.

Generally speaking, the potential speed up you'd get by parallelizing increasing $N$ cores is governed by Amdahl's law $$S(\text{N-cores}) = \frac{1}{(1 - P) + \frac{P}{N}}$$