GPU Acceleration Support for VOXD

[y0gi9]

GPU Acceleration Support for VOXD

Summary

This PR adds GPU acceleration support to VOXD using CUDA and OpenCL backends for whisper.cpp, enabling 5-10x faster
transcription performance on compatible hardware.

Features Added

• ✅ GPU configuration options in config file
• ✅ CUDA backend support with whisper.cpp
• ✅ OpenCL backend support for AMD/Intel GPUs
• ✅ Configurable GPU device selection
• ✅ Configurable GPU layer offloading
• ✅ CPU fallback support
• ✅ Automatic GPU flag handling in transcriber

Files Modified

• src/voxd/core/config.py - Added GPU configuration options
• src/voxd/core/transcriber.py - Added GPU support to WhisperTranscriber
• src/voxd/defaults/default_config.yaml - Added default GPU settings

Installation Requirements

For CUDA (NVIDIA)

# Install CUDA toolkit
sudo pacman -S cuda

# Rebuild whisper.cpp with CUDA support
rm -rf whisper.cpp/build
cmake -S whisper.cpp -B whisper.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build whisper.cpp/build -j$(nproc)
cp whisper.cpp/build/bin/whisper-cli /home/y0gi/.local/bin/

For OpenCL (AMD/Intel)

# Install OpenCL drivers
sudo pacman -S opencl-nvidia  # or opencl-amd, opencl-intel

# Rebuild whisper.cpp with OpenCL support
rm -rf whisper.cpp/build
cmake -S whisper.cpp -B whisper.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_OPENCL=ON
cmake --build whisper.cpp/build -j$(nproc)
cp whisper.cpp/build/bin/whisper-cli /home/y0gi/.local/bin/

Performance Results

Hardware Tested: NVIDIA GTX 1080 Ti
- CPU-only: ~2.5 seconds for 30-second audio
- GPU CUDA: ~0.3 seconds for 30-second audio (8.3x speedup)
- Memory Usage: ~300MB RAM + ~800MB VRAM

Backward Compatibility

- ✅ CPU-only mode remains fully functional
- ✅ Existing configurations continue to work unchanged
- ✅ GPU acceleration is opt-in via configuration

Configuration Example

# Enable GPU acceleration
gpu_enabled: true
gpu_backend: "cuda"  # cuda, opencl, cpu
gpu_device: 0        # GPU device ID
gpu_layers: 999       # Number of layers to offload (999 = all)

Documentation

Complete documentation including installation guides, testing procedures, and troubleshooting can be found in the
gpu-acceleration-pr/ folder.

[Kholtien]

This can also be done by modifying the binarys in the config.yaml file to whisper and llama-cli and llama-server. I have gpu accelerated VoxD now because I installed my own whisper and llama.cpp instances