# nsight

## What is Nsight?
**Nsight** is NVIDIA's suite of profiling and debugging tools for CUDA and GPU-accelerated applications. It is really two separate tools sharing a brand name: **Nsight Systems**, a timeline profiler that shows how CPU and GPU work overlaps across an entire application run, and **Nsight Compute**, a kernel-level profiler that dissects a single CUDA kernel launch down to instruction throughput, memory bandwidth, and occupancy.

[[perf]] and [[valgrind]] were built for CPU execution and have no visibility into what happens once work is dispatched onto a GPU; the GPU's execution is asynchronous relative to the CPU and runs its own scheduler across thousands of concurrent threads. Nsight exists because profiling a heterogeneous CPU+GPU application needs a tool that understands CUDA streams, kernel launches, and memory transfers as first-class events, not just CPU-side function calls.

The two tools answer different questions and are used in sequence. Nsight Systems answers "where is the overall time going, and is the GPU actually staying busy or waiting on the CPU," which is the first thing to check on any new workload. Nsight Compute answers "why is this specific kernel slow," and is reached for only after Systems has identified which kernel is worth that level of scrutiny.

## Install
Nsight Systems and Nsight Compute ship with the CUDA Toolkit, or can be installed standalone from NVIDIA's developer site. On Debian and Ubuntu, once the NVIDIA CUDA APT repository is configured:

```bash
sudo apt install nsight-systems nsight-compute
```

Both tools require an NVIDIA GPU and driver on the target machine; profiling a remote GPU node typically means running the CLI collection step on the remote machine over [[ssh]] and copying the resulting report file back to view in the GUI locally.

```bash
nsys --version
ncu --version
```

## Practice
Nsight Systems wraps a whole program invocation and records a timeline of everything that happened, CPU threads, CUDA API calls, kernel execution, memory copies:

```bash
nsys profile -o report ./my_cuda_app
```

This produces `report.nsys-rep`, opened in the Nsight Systems GUI to see a timeline view: CPU activity on one row, each CUDA stream on its own row below, so gaps where the GPU sits idle waiting on the CPU (or vice versa) are immediately visible as literal empty space in the timeline.

Nsight Compute profiles one kernel invocation at a time in much greater depth:

```bash
ncu -o kernel_report ./my_cuda_app
```

By default this profiles every kernel launch in the program, which is slow; narrowing to a specific kernel by name and launch count is standard practice once Systems has identified the target:

```bash
ncu --kernel-name my_hot_kernel --launch-count 1 -o kernel_report ./my_cuda_app
```

## Concepts

### Nsight Systems: the timeline view
The core insight Nsight Systems gives is whether the GPU is actually the bottleneck at all. A common surprise on a first profiling pass is a timeline where the GPU rows show frequent gaps, meaning the application is CPU-bound (waiting on data preparation, disk I/O, or single-threaded host code) rather than GPU-bound, and no amount of kernel optimisation will help until the CPU-side stall is fixed.

```bash
nsys profile --trace=cuda,nvtx,osrt -o report ./app
```

`--trace=nvtx` picks up NVTX range annotations the application itself can emit to label logical phases (like "data load" vs "training step") directly in the timeline, which is far more useful than trying to infer program phases from raw kernel names alone.

### Nsight Compute: kernel-level metrics
Nsight Compute reports metrics like achieved occupancy (fraction of the GPU's thread-scheduling capacity actually in use), memory throughput as a percentage of the device's theoretical peak bandwidth, and compute throughput as a percentage of peak FLOPs. A kernel with low occupancy and low memory throughput both is usually latency-bound rather than bandwidth- or compute-bound, often from too few threads in flight to hide instruction latency.

```bash
ncu --set full --kernel-name my_hot_kernel -o full_report ./app
```

`--set full` collects the complete metric set, including the roofline-adjacent breakdown of whether the kernel is compute-bound or memory-bound, at the cost of running the kernel many times (once per metric group) to collect it all.

### Roofline analysis
Nsight Compute can generate a roofline plot for a profiled kernel, plotting its achieved performance against the theoretical peak the GPU can deliver at that kernel's arithmetic intensity (FLOPs per byte moved from memory). A kernel plotted well below the roofline ceiling for its intensity has room to improve; whether that means restructuring memory access patterns or increasing arithmetic intensity per byte loaded depends on which side of the roofline's knee the kernel sits on.

### Remote profiling workflow
The typical HPC pattern is to collect a report on the remote GPU node over [[ssh]] with the CLI tools (`nsys`/`ncu`), then transfer the resulting `.nsys-rep` or `.ncu-rep` file back to a local machine to open in the GUI, since the GUI itself doesn't need GPU access, only the collection step does.

```bash
ssh gpu-node "nsys profile -o /tmp/report ./app"
scp gpu-node:/tmp/report.nsys-rep .
```