Integrate with Imgui

[panxinmiao]

This PR integrates ImGui using the imgui_bundle library.
the imgui_bundle provides Python bindings for ImGui.

I implemented an ImGui backend based on wgpu-py, as well as an ImGui renderer using the wgpu-py Canvas.

The main challenge I faced in this work was the encapsulation of the API. Ultimately, I abstracted two classes: ImguiWgpuBackend and ImguiRenderer.

• ImguiWgpuBackend is a lower-level API that is independent of any GUI framework. It encapsulates the core logic for rendering ImGui using wgpu. Essentially, it wraps the GPU DrawCall required for rendering ImGui, allowing it to be embedded in any Render Pass, — You only need to insert the GPU DrawCall at the appropriate place in your application's rendering loop. Note that UI events need to be handled yourself (via backend.io).

• ImguiRenderer uses the wgpu-py Canvas as its concrete implementation and interfaces with the wgpu-py event system. It draws ImGui using an independent render pass, rendering directly to the Canvas' surface texture. Therefore, we need to be able to retrieve the same surface texture multiple times in different render passes within the same frame.

For specific usage, please refer to the examples included in this PR.

Additionally, I am still considering the integration and API design with pygfx. It could be treated as a special world object in pygfx or deeply integrated with the pygfx renderer to leverage features such as anti-aliasing.

Currently, in pygfx, you can also use the ImguiRenderer implementation to integrate ImGui into the pygfx scene rendering. This is entirely independent of the pygfx rendering logic, sharing only the Canvas instance. Therefore, any methods or features in pygfx will not affect ImGui. (such as snapshot() method in the pygfx.renderer )

Example:

sea.mp4

[kushalkolar]

This looks really nice! 😄 . I'll play with it in pygfx over the next few days :)

The API from the example looks nice so far.

[Korijn]

I had never heard of imgui_bundle before. Based on stars it seems pyimgui is a bit more popular. Can you explain the difference between the two libraries and why you picked this one?

I tried to integrate with pyimgui before but I really struggled.

[panxinmiao]

Can you explain the difference between the two libraries and why you picked this one?

When starting this work, I investigated the Python binding libraries listed in the imgui project. I found that the imgui_bundle project was relatively newer and more active. In contrast, pyimgui was last updated in April of last year. Additionally, imgui_bundle also includes bindings for many third-party extensions based on imgui, such as ImPlot, ImGuizmo, and other libraries. I thought that using imgui_bundle integration might help in obtaining a more powerful ecosystem integration.

Overall, however, I didn't particularly carefully consider this choice, and don't have a particular preference. I just tried starting with imgui_bundle, and it worked out quite well. 😅

[Korijn]

A word on the dependency:

• pygfx currently requires python >= 3.8, but imgui-bundle latest version only supports python >= 3.10
• imgui-bundle 1.0.0 is the last version with py 3.9 support
• imgui-bundle 0.6.6 is the last version with py 3.8 support

Version 1.2.1 is the minimum version supported by your code in my testing. So that means this imgui backend requires at least python 3.10.

I've pushed a commit to your PR with a change to setup.py to make the requirement explicit

@almarklein can we drop python 3.8 yet?

[Korijn]

Apologies for the delay. The changes look great to me. I'm OK with the location of the new modules in wgpu.utils.

[almarklein]

Finally took the time to do a proper review. I made a few comments.

The new event requires some thought.

How portable is this code for using other imgui libs (alternative Python wrappers)? Just to get an impression to know how easy it would be to adjust of the ecosystem changes.

I think it makes sense to put the imgui stuff in a utils.imgui subpackage:

from wgpu.utils.imgui import ImguiRenderer
from wgpu.utils.imgui import ImguiWgpuBackend

[panxinmiao]

I think it makes sense to put the imgui stuff in a utils.imgui subpackage:

Updated.

How portable is this code for using other imgui libs (alternative Python wrappers)? Just to get an impression to know how easy it would be to adjust of the ecosystem changes.

I haven't rigorously tested it, but I believe integration should be quite easy. Since we've implemented a generic backend for imgui, these imgui-based extension libraries should naturally receive support.

I tried implot, and it works well.

Edit:
Sorry, I misunderstood the question earlier and thought we were discussing mgui extension libraries. 😅

I believe switching to other Python wrappers shouldn't be too difficult, although I'm not entirely sure. The basic logic should be similar, though some methods might differ and require adaptation. Overall, I don't think it will be a significant issue.

[almarklein]

This is looking great! Awesome work!

The only hurdle is that this relies on the new char event, which means it does not yet work via jupyter_rfb, sorry @kushalkolar. I created #530 to track this, as it requires some more thought and work. I don't think we need to hold this PR for that. We can merge with desktop-only support, and add full support later.

[kushalkolar]

The only hurdle is that this relies on the new char event, which means it does not yet work via jupyter_rfb, sorry @kushalkolar. I created #530 to track this, as it requires some more thought and work. I don't think we need to hold this PR for that. We can merge with desktop-only support, and add full support later.

yea been playing with it in jupyter_rfb, and yes of course one step at a time, we can do this fully later 😄