Sampling is the core mechanism of perf record. Instead of counting every event (which would be prohibitively expensive), perf samples at a fixed rate and records what was happening at each sample point. This gives statistical profiling with low overhead.
perf record ./program # sample every 4000 cycles (default) perf record -c 10000 ./program # sample every 10000 cycles perf record -F 99 ./program # sample 99 times per second
-c N samples every N events (cycles by default). -F Hz samples at Hz times per second. -F 99 means sample 99 times per second, regardless of execution speed—useful for consistent sampling rate across different workloads.
Overhead: Sampling adds minimal overhead (typically 1-5%) compared to instrumentation-based profiling (10-50% or more). Each sample involves a context switch, recording the instruction pointer and stack, and resuming—expensive operations. The sampling rate controls the trade-off between accuracy and overhead.
Higher sampling rates produce more samples and better statistical coverage but add overhead. -F 99 is typical for production profiling; -F 1000 (1000 Hz) is for detailed analysis on development machines but adds noticeable overhead.
Event-based sampling:
perf record -e cycles ./program # sample on cycles (default) perf record -e cache-misses ./program # sample on cache misses (rare events) perf record -e L1-dcache-load-misses -c 100 ./program # every 100 L1 misses
By default, perf samples on cycles. You can change to other events. Sampling on rare events (like cache misses) produces fewer samples but focuses on specific phenomena. When sampling on an infrequent event, adjust the count threshold to get enough samples: a 0.1% miss rate means only 1 in 1000 accesses misses, so c 100 samples every 100 misses (roughly 10,000 accesses).
Precise sampling (PEBS and IBS):
perf record -e cycles:ppp ./program # PEBS (Precise Event-Based Sampling, Intel) perf record -e L1-dcache-load-misses:ibs ./program # IBS (Instruction-Based Sampling, AMD)
Intel PEBS and AMD IBS provide precise instruction-level sampling without skid (error in recording the exact instruction). The :ppp suffix enables PEBS; :ibs enables IBS. These give the most accurate profiling data but require CPU support.
Call-graph sampling:
perf record -g ./program # capture call stacks perf record -g --call-graph dwarf ./program # DWARF-based stack unwinding perf record -g --call-graph fp ./program # frame-pointer unwinding
Without -g, perf only records the leaf instruction. With -g, each sample includes the full call stack—how the program got to that instruction. DWARF unwinding is more accurate on optimized code; frame-pointer unwinding requires special compilation flags.
Data collection considerations: Long-running programs produce large perf.data files (can be gigabytes). For continuous monitoring, use system-wide profiling (-a) or profile specific processes (-p PID). Time-limiting with a wall-clock timeout is sometimes necessary: perf record -a sleep 60 profiles everything system-wide for 60 seconds.