CUDA Setup

​

System Requirements

​

CUDA Toolkit

​

Nsight Compute

​

Automatic Detection

​

CUDA Environment Detection

• CUDA Toolkit installation path and version
• nvcc compiler availability and configuration
• NCU (nv-nsight-cu-cli) installation and version
• GPU compute capability and specifications

​

Hardware Detection

• NVIDIA GPU models and specifications (RTX 40/30/20, GTX 16/10 series)
• GPU memory capacity and bandwidth
• SM (Streaming Multiprocessor) count
• Architecture type (Turing, Ampere, Ada Lovelace, Hopper)

​

Manual Configuration

​

CUDA Path Setup

1. Go to Settings → CUDA Configuration
2. Set CUDA Toolkit path manually:
   • Windows: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x
   • Linux: /usr/local/cuda or /opt/cuda
3. Verify NCU installation at: [CUDA_PATH]/nsight-compute-2024.x.x/
4. Test profiling with a simple kernel

​

Windows UAC/Permissions

• RightNow AI handles UAC elevation automatically for performance counter access
• Automatic UAC prompt when profiling requires elevated permissions
• Registry fixes for Windows performance counters
• No manual setup required

# Run as Administrator
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance" /v "Disable Performance Counters" /t REG_DWORD /d 0 /f

​

Linux Permissions & Setup

# Add user to GPU access groups
sudo usermod -a -G video $USER
sudo usermod -a -G render $USER

# For NCU profiling (some distributions)
sudo usermod -a -G perf_users $USER

# Reboot after group changes
sudo reboot

# Enable GPU performance counters (NVIDIA driver 470+)
sudo nvidia-smi -pm 1
sudo nvidia-smi -lgc 0,1980

# Verify nvidia-smi access
nvidia-smi

​

Multi-GPU Configuration

​

Automatic Multi-GPU Detection

• RightNow AI detects all NVIDIA GPUs in the system
• GPU selection dropdown in profiling interface
• Cross-GPU performance comparison

​

Device Selection

1. Open profiling panel
2. Select target GPU from device dropdown
3. Profile kernels with cudaSetDevice() targeting specific GPU
4. Compare performance across different GPU architectures

​

Supported Configurations

• Single GPU: RTX 4090, RTX 4080, RTX 3090, etc.
• Multi-GPU: SLI/NVLink configurations
• Mixed Architectures: Different GPU generations in same system

​

Troubleshooting Common Issues

​

NCU Not Found

1. Ensure CUDA Toolkit is installed (not just runtime)
2. Check NCU binary exists: nv-nsight-cu-cli --version
3. Add CUDA bin directory to PATH

​

Permission Denied

1. Windows: Run RightNow AI as Administrator or allow UAC prompt
2. Linux: Ensure user is in correct groups (video, render, perf_users)
3. Check GPU driver supports profiling (R470+ recommended)

​

Profiling Fails

1. Verify GPU supports compute capability 3.5+
2. Close other GPU-intensive applications
3. Check CUDA application compiles and runs normally
4. Try profiling with simple kernel first