fenv.h gives you access to the floating-point environment: the status flags that record when something unusual happened during a computation (overflow, division by zero, invalid result like NaN), and the rounding mode that controls how inexact results are rounded. Before C99 formalised this header, there was no standard way to check whether a computation had silently produced a NaN or infinity.

#include <fenv.h>
#include <math.h>
 
feclearexcept(FE_ALL_EXCEPT);      // clear all pending exception flags
 
double r = sqrt(-1.0);             // invalid: produces NaN
 
if (fetestexcept(FE_INVALID))
    puts("invalid operation occurred");

The standard exception flags:

Rounding mode is set with fesetround and queried with fegetround. The four IEEE 754 modes are FE_TONEAREST (default, ties round to even), FE_TOWARDZERO (truncate), FE_UPWARD, and FE_DOWNWARD.

You can save and restore the entire floating-point environment — flags plus rounding mode — with fegetenv/fesetenv. This is useful when you want to do speculative computation without leaving exception residue that the caller might misinterpret.

// compile: gcc -o fenvdemo fenvdemo.c -lm
// run: ./fenvdemo
// description: trigger each standard FP exception and check which flag was set
 
#include <fenv.h>
#include <math.h>
#include <stdio.h>
 
static void check(const char *label) {
    printf("%-22s invalid=%d divbyzero=%d overflow=%d underflow=%d inexact=%d\n",
           label,
           !!fetestexcept(FE_INVALID),   !!fetestexcept(FE_DIVBYZERO),
           !!fetestexcept(FE_OVERFLOW),  !!fetestexcept(FE_UNDERFLOW),
           !!fetestexcept(FE_INEXACT));
    feclearexcept(FE_ALL_EXCEPT);
}
 
int main(void) {
    volatile double x;
    x = sqrt(-1.0);       check("sqrt(-1)");
    x = 1.0 / 0.0;        check("1.0/0.0");
    x = 1e308 * 1e308;    check("1e308*1e308");
    x = 1e-308 / 1e10;    check("1e-308/1e10");
    x = 1.0 / 3.0;        check("1.0/3.0");
    return 0;
}

Each line triggers a different flag. The volatile stops the compiler from folding the computations at compile time. Notice that 1.0/3.0 only sets FE_INEXACT — it is a valid result, just not exactly representable in binary. And 1.0/0.0 sets FE_DIVBYZERO, not FE_INVALID: IEEE 754 considers it a valid (infinite) result.