# Introduction to parallel computing

**[Introduction to parallel computing](https://en.wikipedia.org/wiki/Parallel_computing)** defines core concepts: **parallelism** (multiple things executing simultaneously, requiring multiple cores), **concurrency** (multiple logically independent tasks, on one or many cores), **shared memory** (threads on one machine communicating via variables), and **distributed memory** (processes on separate machines communicating via messages).

The two fundamental scalability models are [[amdahls-law|Amdahl's law]] (fixed-size problem, bounded speedup) and [[gustafsons-law|Gustafson's law]] (growing problem size, near-linear speedup).

## Example

This example illustrates the difference between shared and distributed memory communication.

```c
// Shared memory: multiple threads, one address space
#include <omp.h>
#pragma omp parallel
{
    int shared_var = 0;  // visible to all threads
#pragma omp critical
    shared_var++;
}

// Distributed memory: separate processes, explicit messages
// (pseudocode: actual MPI shown in [[mpi]] article)
// process_0: MPI_Send(&data, 1, MPI_INT, 1, ...)
// process_1: MPI_Recv(&data, 1, MPI_INT, 0, ...)
```