Add pygfx/WGPU backend for 2D/3D plotting

[physwkim]

OpenGL has been deprecated on macOS. This PR adds a new rendering backend based on pygfx/WGPU, enabling native Metal (macOS), Vulkan (Linux), and DirectX (Windows) support.

Drop-in alternative to the existing OpenGL backend for both 2D and 3D plotting.

2D (silx.gui.plot)

• Curves, images, scatter, markers, shapes, error bars, color bar
• GPU colormap pipeline (scalar texture + shader)
• GPU compute shaders for min/max reduction and histogram
• Async compute for non-blocking streaming updates
• Efficient image streaming update path

3D (silx.gui.plot3d)

• SceneWindow with backend="pygfx" support
• Mesh, scatter, volume (isosurface via marching cubes, cut planes), image, heightmap
• Object parameters tree (SceneModel reuse via GroupItem)
• 3D axes (pygfx Ruler) and bounding box wireframe
• ClipPlane support for GroupItem

Key changes

• BackendPygfx.py — 2D backend implementation
• Plot3DWidgetPygfx.py, SceneWidgetPygfx.py — 3D backend implementation
• _PlotFrameCore.py — shared plot frame logic between OpenGL and pygfx
• _pygfx_sync.py — Item3D to pygfx object sync layer
• Parametrized tests for both backends

---

Benchmarks

Two benchmarks were added to evaluate the backend performance.

---

Benchmarks

Benchmarks were run on a MacBook M4 Pro.

• compareBackendsFPS.py for live line update performance
• imageStreamingBenchmark.py for image streaming throughput

Image streaming benchmark

Example result for 4096 × 4096 streaming images:

Size   Norm    FPS     plot_ms   other   total
1024   linear  507.3   1.56      0.02    1.58
2048   linear  308.1   2.64      0.03    2.67
4096   linear  125.7   6.77      0.04    6.81

The backend maintains >120 FPS for 4K images during continuous streaming updates.

---

Curve update benchmark

Live curve updates comparison between backends:

matplotlib : ~136 FPS
OpenGL : ~120 FPS
pygfx/WGPU : ~244 FPS

The pygfx backend shows ~2× higher performance than the OpenGL backend for dynamic curve updates.

---

Platform Status

Platform	Status
macOS (Metal)	Fully supported
Linux (X11 + Vulkan)	Works well
Linux (Wayland)	Not working
Windows	Expected to work via DirectX

---

Motivation

• OpenGL is deprecated on macOS
• Modern GPU APIs (Metal / Vulkan / DX12) are now the standard
• WebGPU/WGPU provides a cross-platform abstraction layer

Using pygfx allows silx to benefit from:

• modern GPU rendering
• better macOS compatibility
• improved performance for streaming visualization

AI Disclosure

Claude Code was used to generate a significant portion of the code in this PR, including the pygfx backend, synchronization layer, and associated tests.

The author guided the implementation by providing high-level design direction and examples, and performed iterative validation through manual testing, debugging, and benchmarking.

[loichuder]

Wow, thanks for the big contribution, that looks very promising!

We'll get back to you but know that the review may take a bit of time 😉

[t20100]

Thanks for the contribution!

We'll come back to you and iterate over this PR.

A first quick comment, we aim at keeping plot backends playing an equal role, so adding a plot backend should only modify PlotWidget and not other part of the code like plot items or ColormapDialog.
If optimizations of other parts of the code are needed, let's to do it separately.

[physwkim]

Unlike the previous PR, this one has been cleaned up to contain only the backend-related changes. A separate PR for the addImage fast path optimization has been created at #4524.

[t20100]

Thanks for the split!

I had a quick look at the code and a first try, here are some more early comments:

• Could you add the "pygfx" backend to the tests?
  To do so, search for "gl" in src/silx/gui/plot/test, there's so ("mpl", "gl") (or ("gl", "mpl")) test parameters where you can add "pygfx".
  Also in src/silx/gui/plot/test/test_plotwidget.py, there is some classes of this kind:

@pytest.mark.usefixtures("use_opengl")
class TestPlotWidget_Gl(TestPlotWidget):
    backend = "gl"

Add similar ones for pygfx.
This will enable the tests with the pygfx backend.

• I tested in on Linux (ubuntu24.04) with python3.14 and up-to-date dependencies.
  I managed to run the backend with PyQt5 (with some warnings) but it panics with both PyQt6 and PySide6:

from silx import sx
w = sx.PlotWidget(backend="pygfx")

Error:

WARNING:wgpu:Unable to find extension: VK_EXT_physical_device_drm
WARNING:wgpu:Detected skylake derivative running on mesa i915. Clears to srgb textures will use manual shader clears.

thread '<unnamed>' (595070) panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wgpu-hal-27.0.4/src/vulkan/instance.rs:452:18:
XlibSurface::create_xlib_surface() failed: ERROR_OUT_OF_HOST_MEMORY
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread '<unnamed>' (595070) panicked at library/core/src/panicking.rs:230:5:
panic in a function that cannot unwind

Do you know what could go wrong?

[t20100]

To complete previous comment, it also panics when setting QT_QPA_PLATFORM=wayland with PyQt5 so wayland support is probably the root cause.

[physwkim]

Thanks for testing!

I've added the "pygfx" backend to the test parametrizations and added the _Pygfx test classes — that's already included in the latest push.

About the Wayland crash — yeah, I was able to reproduce it too. This is a known upstream issue in wgpu. The wgpu-py docs mention that Wayland support is currently broken, so it'll probably take some time on their side to fix it. For now I've added a guard in PlotWidget that raises a clear error for Wayland sessions instead of letting it panic.

While adding the tests, I also found and fixed a couple of issues — image/triangle picking was stubbed out (return None # TODO), and processEvents() wasn't reliably flushing pending item updates due to rendercanvas's deferred draw scheduling. Both are fixed in the latest commits.

You mentioned recently that AI-generated code should be disclosed. Could you clarify where you'd like it noted — in the commit messages, PR description, or somewhere in the source code?

[physwkim]

The Linux CI doesn't have a GPU, so wgpu device creation fails. OpenGL works fine there because Mesa provides a software renderer (llvmpipe), but wgpu doesn't have that kind of fallback. I added a check that tries to create a wgpu device before running pygfx tests — if it fails, the tests are skipped.

To run pygfx tests on Linux CI, one option would be using a GPU-equipped runner, or installing mesa-vulkan-drivers (lavapipe) for software Vulkan. I'm not sure how to set that up in the CI pipeline though.

[loichuder]

You mentioned recently that AI-generated code should be disclosed. Could you clarify where you'd like it noted — in the commit messages, PR description, or somewhere in the source code?

Just a quick sentence in the PR description is enough. Something like:

I used ChatGPT to generate the docstrings.

or

This part of the code was written with Copilot autocompletion.

[physwkim]

3D plotting seems to be coming along nicely as well.

• examples/pygfx_backend/13_3d_scatter.py

• examples/pygfx_backend/14_3d_volume.py

[t20100]

Hi,

Sorry for the delayed reply.

We are currently preparing a major release after a very long time and we want to have this available as soon as possible, so we will not integrate your PR before then.
Don't worry, we will come back to you once this is done and we aim at resuming a shorter and regular release cycle. In the meantime, you can already use your backend in your application by passing the class to the plots: plot = PlotWidget(backend=MyBackendClass).

For the review process, it would be best to focus on one feature at a time: Reviews are still done by humans in this project. It would be much simpler to go step by step and have a first PR for the 2d plot backend only (you can keep this one opened to demo 2d/3d plot and open another one providing only the 2d plot backend).