Site Tools


wiki:flops

FLOPS

FLOPS (floating-point operations per second) measures the rate at which a processor performs floating-point arithmetic, and is the standard unit for comparing raw compute throughput in HPC. Unlike clock speed, FLOPS accounts for how much actual arithmetic a chip can do per cycle, which depends on how many floating-point units it has, how wide its SIMD lanes are, and whether it has dedicated hardware like fused multiply-add.

Peak vs achieved

Peak FLOPS is the theoretical maximum a chip can sustain if every functional unit is busy on every cycle, computed directly from the hardware spec: cores × clock speed × FLOPs per cycle per core. Achieved FLOPS is what a real program actually gets, and it is almost always well below peak, since real code spends cycles on memory stalls, branches, and instructions that aren't floating-point arithmetic at all. The gap between the two is exactly what the Roofline model is built to explain: whether a given kernel is capped by arithmetic throughput or by how fast data can be fed to the arithmetic units.

Peak FLOPS = cores × clock (Hz) × FLOPs/cycle/core (from SIMD width and FMA)

Precision matters

FLOPS figures are meaningless without specifying precision, since a chip's throughput differs enormously between double precision (FP64), single precision (FP32), and lower-precision formats (FP16, bfloat16, INT8) used in machine learning. A GPU's advertised “peak FLOPS” for FP16 tensor-core matrix multiplication can be an order of magnitude higher than the same chip's FP64 peak, because dedicated lower-precision hardware trades numerical range and accuracy for raw throughput. Comparing two chips' FLOPS numbers without matching precision is a common and misleading mistake.

Scale prefixes

Real systems' FLOPS values span many orders of magnitude, so the number is almost always quoted with an SI prefix: GFLOPS ($10^9$) for a single core or modest GPU kernel, TFLOPS ($10^{12}$) for a modern GPU or a small cluster, up to petaflops ($10^{15}$) and exaflops ($10^{18}$) for the largest supercomputers.

wiki/flops.md · Last modified: by 127.0.0.1