# numactl Memory Policies

**Memory allocation policies** control where `malloc`, `new`, and page faults allocate memory. `--membind` is strict; `--preferred` is flexible; `--interleave` spreads memory across nodes.

**`--membind` (strict binding):**

```bash
numactl --cpunodebind=0 --membind=0 ./app
```

All memory comes from node 0. If node 0 runs out, the allocation fails (even if other nodes have free memory). Use when you know the memory footprint and want to guarantee local access.

**`--preferred` (fallback):**

```bash
numactl --cpunodebind=0 --preferred=0 ./app
```

Try to allocate from node 0, but fall back to other nodes if node 0 is full. Allocation never fails due to NUMA constraints. Useful for workloads of variable size or when you can tolerate some cross-socket access.

**`--interleave` (round-robin):**

```bash
numactl --cpunodebind=0-1 --interleave=0,1 ./app
numactl --cpunodebind=all --interleave=all ./app
```

Spread memory pages round-robin across specified nodes. Each allocation alternates: first page from node 0, second from node 1, third from node 0, etc. Good for large shared data structures accessed uniformly by threads on all nodes. Bad for working sets that fit on one node—you pay cross-socket penalties unnecessarily.

**Choosing a policy:**

1. **Single-socket workload:** `--membind=N` with `--cpunodebind=N`. Everything local, no cross-socket access.

2. **Multi-socket workload with private per-thread data:** Bind threads to their home node's CPUs, use `--preferred` or `--membind` on that node. Each thread's stack and private data stay local.

3. **Shared data structure accessed by all nodes:** Use `--interleave=all` to spread the structure evenly. Reduces hot spots on any one memory controller.

4. **Unknown or variable memory footprint:** Use `--preferred=N` with `--cpunodebind=N`. Tries to keep memory local but doesn't fail if the workload grows large.

**Example: MPI with per-rank binding:**

```bash
# Rank 0 on node 0, rank 1 on node 1
mpirun -n 2 bash -c 'numactl --cpunodebind=$((OMPI_COMM_WORLD_RANK % 2)) \
    --membind=$((OMPI_COMM_WORLD_RANK % 2)) ./app'
```

Each MPI rank binds to its assigned node. Rank 0 → node 0, rank 1 → node 1. Ensures locality within each rank.

**Interleave vs non-interleaved:** Interleave is a trade-off. It avoids memory controller hot spots and load imbalance but costs more cross-socket access if the working set could fit on one node. Benchmark both.