Site Tools


tflops

Table of Contents

TFLOPS

TFLOPS is FLOPS measured in units of $10^{12}$ (teraflops) floating-point operations per second, the scale at which modern GPUs and small clusters are quoted. A single high-end datacenter GPU exceeds tens of TFLOPS in FP64 and over a hundred TFLOPS in lower-precision formats used for machine learning.

Precision matters enormously: a GPU's FP16 tensor throughput can be 10x higher than its FP64 throughput, and quoting “TFLOPS” without precision is meaningless.

Example

GPU TFLOPS scales through massive parallelism, not single-thread speed.

GPU TFLOPS:  thousands of narrow lanes × lower precision
CPU GFLOPS:  handful of wide SIMD units × high precision
tflops.md · Last modified: by 127.0.0.1