The L2 cache sits between the fast, tiny L1 cache and the large, shared L3 cache. It is a compromise: bigger than L1 (typically 256 KB to 2 MB per core on modern x86 designs) so it catches more of L1's misses, but still small and close enough to the core to answer in tens of cycles rather than the hundred-plus cycles a trip to L3 or DRAM would cost.

Unlike L1, L2 is normally unified, holding both instructions and data in the same array rather than splitting them. Whether it is private to one core or shared across a small cluster of cores depends on the microarchitecture. Intel's client designs have historically kept L2 private per core; some server and mobile designs share an L2 across a cluster of two to four cores to amortize its area cost.

A cache hierarchy can be inclusive (every line in L1 is guaranteed to also be in L2), exclusive (a line lives in exactly one of L1 or L2, never both), or non-inclusive (no guarantee either way). Inclusive L2s simplify Cache coherence: a snoop or directory lookup only has to check L2 to know whether any core's L1 might hold a line, instead of checking every L1 directly. The tradeoff is wasted capacity, since every byte cached in L1 is also occupying space in L2.