> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rightnowai.co/llms.txt
> Use this file to discover all available pages before exploring further.

# CUDA Setup

> Configure CUDA Toolkit and profiling for optimal performance

## System Requirements

### CUDA Toolkit

**Required**: CUDA Toolkit 11.0-12.5

Download from [NVIDIA Developer](https://developer.nvidia.com/cuda-toolkit)

### Nsight Compute

**Bundled**: Included with CUDA Toolkit

Provides authentic hardware metrics and profiling capabilities.

## Automatic Detection

RightNow AI automatically detects and configures your CUDA environment:

### CUDA Environment Detection

* CUDA Toolkit installation path and version
* `nvcc` compiler availability and configuration
* NCU (`nv-nsight-cu-cli`) installation and version
* GPU compute capability and specifications

### Hardware Detection

* NVIDIA GPU models and specifications (RTX 40/30/20, GTX 16/10 series)
* GPU memory capacity and bandwidth
* SM (Streaming Multiprocessor) count
* Architecture type (Turing, Ampere, Ada Lovelace, Hopper)

## Manual Configuration

### CUDA Path Setup

If automatic detection fails:

1. **Go to Settings** → **CUDA Configuration**
2. **Set CUDA Toolkit path manually**:
   * **Windows**: `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x`
   * **Linux**: `/usr/local/cuda` or `/opt/cuda`
3. **Verify NCU installation** at: `[CUDA_PATH]/nsight-compute-2024.x.x/`
4. **Test profiling** with a simple kernel

### Windows UAC/Permissions

**Automatic UAC Handling**:

* RightNow AI handles UAC elevation automatically for performance counter access
* Automatic UAC prompt when profiling requires elevated permissions
* Registry fixes for Windows performance counters
* No manual setup required

**Manual Registry Fix** (if needed):

```batch theme={null}
# Run as Administrator
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance" /v "Disable Performance Counters" /t REG_DWORD /d 0 /f
```

### Linux Permissions & Setup

**Required Permissions**:

```bash theme={null}
# Add user to GPU access groups
sudo usermod -a -G video $USER
sudo usermod -a -G render $USER

# For NCU profiling (some distributions)
sudo usermod -a -G perf_users $USER

# Reboot after group changes
sudo reboot
```

**GPU Metrics Access**:

```bash theme={null}
# Enable GPU performance counters (NVIDIA driver 470+)
sudo nvidia-smi -pm 1
sudo nvidia-smi -lgc 0,1980

# Verify nvidia-smi access
nvidia-smi
```

## Multi-GPU Configuration

### Automatic Multi-GPU Detection

* RightNow AI detects all NVIDIA GPUs in the system
* GPU selection dropdown in profiling interface
* Cross-GPU performance comparison

### Device Selection

1. Open profiling panel
2. Select target GPU from device dropdown
3. Profile kernels with `cudaSetDevice()` targeting specific GPU
4. Compare performance across different GPU architectures

### Supported Configurations

* **Single GPU**: RTX 4090, RTX 4080, RTX 3090, etc.
* **Multi-GPU**: SLI/NVLink configurations
* **Mixed Architectures**: Different GPU generations in same system

## Troubleshooting Common Issues

### NCU Not Found

1. Ensure CUDA Toolkit is installed (not just runtime)
2. Check NCU binary exists: `nv-nsight-cu-cli --version`
3. Add CUDA bin directory to PATH

### Permission Denied

1. **Windows**: Run RightNow AI as Administrator or allow UAC prompt
2. **Linux**: Ensure user is in correct groups (video, render, perf\_users)
3. Check GPU driver supports profiling (R470+ recommended)

### Profiling Fails

1. Verify GPU supports compute capability 3.5+
2. Close other GPU-intensive applications
3. Check CUDA application compiles and runs normally
4. Try profiling with simple kernel first

<Check>
  **Tip**: Use `nvidia-smi` to monitor GPU status and verify your setup is working correctly.
</Check>