Skip to content

fix(extensions): create AMD GPU overlay files for 5 services#386

Merged
Lightheartdevs merged 1 commit intoLight-Heart-Labs:resources/devfrom
yasinBursali:fix/b3b4-create-amd-overlays
Mar 18, 2026
Merged

fix(extensions): create AMD GPU overlay files for 5 services#386
Lightheartdevs merged 1 commit intoLight-Heart-Labs:resources/devfrom
yasinBursali:fix/b3b4-create-amd-overlays

Conversation

@yasinBursali
Copy link
Contributor

@yasinBursali yasinBursali commented Mar 18, 2026

What

Create compose.amd.yaml overlay files for 5 services that support AMD GPUs per their manifests, providing ROCm device passthrough.

Why

These services declare gpu_backends: [amd, nvidia] in their manifests but had no AMD overlay — meaning AMD GPU users could not utilize GPU acceleration.

How

Created compose.amd.yaml for each service with:

  • /dev/dri and /dev/kfd device passthrough (ROCm)
  • group_add for video/render groups
  • HSA_OVERRIDE_GFX_VERSION environment variable

Services

Service Image swap Notes
ollama No Official image supports AMD natively
text-generation-webui Yes → default-rocm-v4.0 (SHA pinned) Base uses NVIDIA-specific image tag
xtts No Device passthrough only; ROCm compatibility unverified
invokeai No Official image has ROCm support
rvc No Device passthrough only; ROCm compatibility unverified

Excluded services (with rationale)

  • fooocus: runpod/fooocus:2.5.3 is CUDA-only, no ROCm variant exists
  • immich: GPU accel requires immich-machine-learning sidecar which is not yet in the extensions library compose

Known limitations

  • rvc and xtts: Base Docker images may be CUDA-built. AMD device passthrough is provided but GPU acceleration may silently fall back to CPU if the image lacks ROCm PyTorch. Should be verified before production promotion.

Merge Order

#379-#383 (manifest fixes) → #385 (NVIDIA extraction) → THIS PR → #387 (structural fixes)

This PR MUST merge after #385 (NVIDIA extraction). Three base compose files (ollama, text-gen-webui, rvc) currently contain driver: nvidia device reservations. #385 removes those. If this PR merges first, Docker Compose would merge both NVIDIA and AMD device entries, causing runtime errors on AMD systems.

Testing

  • docker compose config validation passed for all 5 services (base + AMD overlay)
  • Service IDs verified matching base compose.yaml keys
  • text-gen-webui uses correct default-rocm-v4.0 tag (verified exists on Docker Hub) with SHA pin
  • Critique Guardian: APPROVED WITH WARNINGS (image compatibility for rvc/xtts noted above)

Platform Impact

  • AMD: GPU acceleration now available for 5 services
  • NVIDIA: No change (separate overlay)
  • macOS / CPU-only: No change (overlay not applied)

Add compose.amd.yaml with ROCm device passthrough (/dev/dri, /dev/kfd)
for services that support AMD GPUs per their manifests.

Services: ollama, text-generation-webui, xtts, invokeai, rvc

text-generation-webui overlay includes image swap to ROCm variant
(default-rocm-v4.0) since the base image is NVIDIA-specific. Image
pinned with sha256 digest per project convention.

Excluded fooocus (CUDA-only image, no ROCm variant exists) and immich
(GPU accel requires immich-machine-learning sidecar not yet in compose).
Copy link
Collaborator

@Lightheartdevs Lightheartdevs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — Correct AMD GPU overlay files.

What's good

  • Creates compose.amd.yaml for 5 services: invokeai, ollama, rvc, text-generation-webui, xtts
  • AMD pattern is correct: /dev/dri + /dev/kfd device passthrough, VIDEO_GID/RENDER_GID group adds, HSA_OVERRIDE_GFX_VERSION env var
  • text-generation-webui AMD overlay correctly swaps to the ROCm image variant (atinoda/text-generation-webui:default-rocm-v4.0)
  • Follows established pattern from core stack's docker-compose.amd.yml

Merge after #385 (which extracts NVIDIA overlays from the same base compose files).

Copy link
Collaborator

@Lightheartdevs Lightheartdevs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — Correct AMD GPU overlay files.

Creates compose.amd.yaml for 5 services (invokeai, ollama, rvc, text-generation-webui, xtts) with the correct AMD pattern:

  • /dev/dri + /dev/kfd device passthrough
  • VIDEO_GID/RENDER_GID group adds with sensible defaults
  • HSA_OVERRIDE_GFX_VERSION env var for GPU compatibility
  • text-generation-webui correctly swaps to the ROCm image variant

Follows established pattern from core stack's docker-compose.amd.yml. LGTM. Merge after #385.

@Lightheartdevs Lightheartdevs merged commit ab2fd10 into Light-Heart-Labs:resources/dev Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants