# numactl OpenMP

**Combining numactl with OpenMP** ensures threads land on intended cores with local memory. OpenMP's affinity mechanism (`OMP_PLACES`, `OMP_PROC_BIND`) specifies which threads run on which cores; numactl enforces it system-wide and controls memory allocation.

**Basic setup:**

```bash
export OMP_NUM_THREADS=8
export OMP_PLACES="{0:4},{4:4}"       # threads 0-3 on core 0, threads 4-7 on core 4
export OMP_PROC_BIND=close
numactl --cpunodebind=0 --membind=0 ./app
```

`OMP_PLACES` specifies thread placement (cores 0-3 and 4-7). `OMP_PROC_BIND=close` keeps threads close to their initial placement. `numactl` enforces that the entire process stays on node 0.

**More complex affinity:**

```bash
export OMP_PLACES="{0,1,2,3},{4,5,6,7}"  # explicit core lists
export OMP_PROC_BIND=spread              # spread threads across places
```

`OMP_PLACES` can list cores explicitly (comma-separated) or use ranges. `OMP_PROC_BIND=spread` distributes threads evenly across places; `OMP_PROC_BIND=close` clusters them.

**Per-NUMA-node binding:**

```bash
export OMP_NUM_THREADS=8
export OMP_PLACES="{0:4},{4:4}"
export OMP_PROC_BIND=close
numactl --cpunodebind=0-1 --membind=0-1 --interleave=0-1 ./app
```

Threads 0-3 run on cores 0-3 (node 0), threads 4-7 on cores 4-7 (node 1). Memory is interleaved across both nodes. Each thread's private data (stack, thread-local storage) lives on its home node.

**Nested OpenMP with NUMA:**

```bash
export OMP_NUM_THREADS=4
export OMP_PLACES="{0,1,2,3}"
export OMP_PROC_BIND=close
numactl --cpunodebind=0 --membind=0 ./app
```

Bind the entire process to node 0, then let OpenMP do fine-grained affinity within that node.

**Debugging affinity:**

```bash
export OMP_DISPLAY_ENV=true
numactl --cpunodebind=0 --membind=0 ./app
```

`OMP_DISPLAY_ENV=true` prints OpenMP settings at startup, showing which cores threads are pinned to.

**Performance tip:** Ensure `OMP_NUM_THREADS` matches the cores you're pinning to. Pinning to 4 cores but setting `OMP_NUM_THREADS=8` causes contention and defeats the purpose.

**Note:** OpenMP affinity is a hint; numactl is a guarantee. numactl overrides OS scheduling, ensuring the process stays pinned even if OpenMP's affinity isn't perfectly configured.