Introduction to parallel computing defines core concepts: parallelism (multiple things executing simultaneously, requiring multiple cores), concurrency (multiple logically independent tasks, on one or many cores), shared memory (threads on one machine communicating via variables), and distributed memory (processes on separate machines communicating via messages).

The two fundamental scalability models are Amdahl's law (fixed-size problem, bounded speedup) and Gustafson's law (growing problem size, near-linear speedup).

This example illustrates the difference between shared and distributed memory communication.

// Shared memory: multiple threads, one address space
#include <omp.h>
#pragma omp parallel
{
    int shared_var = 0;  // visible to all threads
#pragma omp critical
    shared_var++;
}
 
// Distributed memory: separate processes, explicit messages
// (pseudocode: actual MPI shown in [[mpi]] article)
// process_0: MPI_Send(&data, 1, MPI_INT, 1, ...)
// process_1: MPI_Recv(&data, 1, MPI_INT, 0, ...)