How does Dear ImGui handle Windows event processing concurrently with internal update/rendering?

[xgamercpp]

Hello, I was unsure of any other medium to ask this, so I am asking this here:

I'm currently working on a Win32 application that utilizes Dear ImGui's multi-viewport support. One aspect that really puzzles me is how Dear ImGui manages to process Windows events (such as resizing, moving, and input events) while still rendering frames without noticeable delays or stutters.

When I resize my Win32 OpenGL window, it doesn't paint until resized. Not the case with ImGui viewports.

• How do you ensure that window events (e.g., resizing or user input) are processed promptly without stalling the rendering?

Also, I have ran into several multi threading issues in my projects, so unclear what to do exactly, as I can call wglMakeCurrent for imgui_opengl3_impl related render calls on separate thread, but then viewports were causing issues by not rendering any viewport outside the main window.

• Given that Win32 requires that message processing occur on the same thread that created the window, how could you exactly do win32 main event processing on one thread and render the dear imgui viewports on a separate game thread?

I’m trying to gain a deeper understanding of how I can achieve fluid multi threaded integration between event handling on one thread, and all game related stuff for GUI and rendering on its own thread, so that I can apply similar principles to my own projects. Any insights, examples, or references you can provide would be extremely helpful.

Thanks for your time and for all the great work on ImGui.

[PathogenDavid]

This is definitely off-topic for this forum, but I can try and quickly answer your questions.

One aspect that really puzzles me is how Dear ImGui manages to process Windows events (such as resizing, moving, and input events) while still rendering frames without noticeable delays or stutters.

The unsatisfying answer here is that for secondary viewports, Dear ImGui is not processing any windows events for moving/resizing because Windows isn't resizing secondary viewports. Dear ImGui is handling resizing and moving internally. From a Windows perspective, secondary viewports are just bare rectangles with no window decorations.

To put it another way: Since Dear ImGui internally handles the movement and resizing, WM_WINDOWSPOSCHANGED (and by association WM_SIZE and WM_MOVE) are not sent to secondary viewports.

You can actually see the behavior difference for yourself by disabling ImGuiIO::ConfigViewportsNoDecoration. If you move/resize using the Windows window decorations you'll see things stall, if you move/resize using Dear ImGui's window decorations they will not. (Resizing is a little tricky since the border will be razor-thin.)

This approach is not without downsides. On Windows it prevents using Aero Snap with secondary viewports (IE: dragging the window to screen edges or using Win+Left/Right to snap them to the sides.) Linux treats borderless windows fundamentally differently in ways that break multi-viewports (random example), and Wayland on Linux in general just hates applications wanting to control windows for themselves (IIRC it's literally impossible to query the location or size of your window.) It also means we loose out on certain affordances expected by OS users (for example, double-clicking the title bar on Windows windows to maximize them, hovering the maximize button on macOS or Windows 11.)

You can get some of these things back by explicitly handling them, but it relies on cooperation with Windows that Dear ImGui doesn't currently do. (And in fact, most applications which draw their own non-client area do not get this correct. For example, did you know that if you double click the app's icon in the title bar that it will close? That one is almost always overlooked, even VSCode misses it. That used to be the main way I closed windows :/)

Nik highlighted some of the workarounds he uses in ImHex here: #8268 (comment)

How do you ensure that window events (e.g., resizing or user input) are processed promptly without stalling the rendering?

This isn't caused by failing to handle the events promptly, it's an unfortunate design decision of how Windows message loops work that dates back to the oldest versions of Windows.

There's no great documentation on this AFAIK, but moving/sizing windows put the message loop into a modal loop state where the WndProc doesn't return until the move/size gesture has completed. (This loop starting is signaled by WM_ENTERSIZEMOVE.)

So really the question is "How do you keep redrawing the window when the window is in a modal loop state?"

The way Windows "expects" you to do this is to handle the WM_PAINT message to redraw your app. This makes sense for GDI-era Windows UIs that only redraw lazily and rely on dirty rects and such, but doesn't generally mesh well with how your typical game engine is structured.

(In GLFW apps, this is what glfwSetWindowRefreshCallback does. In SDL you handle SDL_WINDOWEVENT_EXPOSED, but you need to use SDL_AddEventWatch since your SDL event loop is stalled during the modal message loop.)

It is worth noting that you should be aware that doing anything complicated in a WndProc in a 32-bit app on 64-bit Windows comes with some risks to your sanity as far as debugging goes. (Short version: Windows will sometimes completely swallow exceptions thrown in a WndProc handler - including access violations. They will not reach your debugger, your app will not crash, but you will go insane. Long version: WNDPROC remarks. Worth noting that the remarks make it sound like this is reliable behavior, but it actually only happens in some contexts.)

how could you exactly do win32 main event processing on one thread and render the dear imgui viewports on a separate game thread?

Not sure if you're asking this because you guessed this might be what Dear ImGui is doing or because you want to use it as a solution.

As you hopefully understand from above, this is not how Dear ImGui avoids the issue. So I'll assume the latter.

It is possible to have your render loop and your message pump be on separate threads, and it is a possible solution to this problem without using WM_PAINT.

I've done it, and it's not trivial. You can't use the Dear ImGui-provided backends so you must make your own, and you need to carefully marshal things between the two threads.

This approach is also not without downsides. In particular, secondary viewports don't appear to resize as fluidly anymore because the message pump and rendering are on different timelines. (It still updates as you're resizing, but the window and rendered size will not always be perfectly in sync.) You can kind-of hide the effect on D3D by configuring the swap chain to not stretch when the window size and swap chain size are mismatched, but I don't think similar functionality exists in OpenGL.

I would never consider this approach with OpenGL simply because it's an uncommon situation that I'd expect graphics drivers to handle poorly. (It requires swapping buffers from the rendering thread rather than the message pump thread.)

If you were wanting your main viewport to live on the main thread and have secondary viewports live on another, you cannot do this with Dear ImGui without major shenanigans.

Also, I have ran into several multi threading issues in my projects

OpenGL does not play nice with multiple threads in general. I'd avoid having it interact with multiple threads entirely.

Judging by your profile it looks like you're just starting out with things, which is great! I'd strongly recommend avoiding multithreading for a while though and focus on developing rigor and discipline in a single-threaded context first. Concurrency is extremely unforgiving.

I’m trying to gain a deeper understanding of how I can achieve fluid multi threaded integration between event handling on one thread, and all game related stuff for GUI and rendering on its own thread, so that I can apply similar principles to my own projects.

Even though I don't personally love the structural changes it imposes on you, WM_PAINT is the way to go here.

However, since you're (seemingly) still starting out I'd also strongly recommend you try to avoid the allure of chasing this level of polish and focus on learning more productive things. If your goal is to tinker on game engines professionally some day, there's about a thousand things more valuable than learning the finer details of Win32 junk and it's probably better to just use SDL. (I say this as someone who has wasted way too much time learning the finer details of Win32 junk and has historically avoided using SDL. Save learning Win32 junk for when someone is paying you to do it.)

If you go searching, you'll find that many (maybe even most) games/engines just don't handle this at all. (SDL actually didn't even officially support handling it until November 2023.)

Sorry this turned into a huge wall of text, hopefully it was useful! 😅

[xgamercpp]

Hello! Thank you for your such detailed and insightful responses to my questions! I really appreciate it. Seriously!

Let me address some of the points you made.

The unsatisfying answer here is that for secondary viewports, Dear ImGui is not processing any windows events for moving/resizing because Windows isn't resizing secondary viewports. Dear ImGui is handling resizing and moving internally. From a Windows perspective, secondary viewports are just bare rectangles with no window decorations. ... To put it another way: Since Dear ImGui internally handles the movement and resizing, WM_WINDOWSPOSCHANGED (and by association WM_SIZE and WM_MOVE) are not sent to secondary viewports.

• I would like to understand a bit more about this, if you have any additional details.

You can actually see the behavior difference for yourself by disabling ImGuiIO::ConfigViewportsNoDecoration. If you move/resize using the Windows window decorations you'll see things stall, if you move/resize using Dear ImGui's window decorations they will not. (Resizing is a little tricky since the border will be razor-thin.)

• That is pretty cool!

... did you know that if you double click the app's icon in the title bar that it will close? ... even VSCode misses it.

• Not sure if it matters, but my VSCode is not doing that, but Notepad is xD

So really the question is "How do you keep redrawing the window when the window is in a modal loop state?"

• I really appreciate your detailed answers, and I also believe that this should've been one of my original discussion's points.

It is possible to have your render loop and your message pump be on separate threads, and it is a possible solution to this problem without using WM_PAINT ... I've done it, and it's not trivial. You can't use the Dear ImGui-provided backends so you must make your own, and you need to carefully marshal things between the two threads.

• Thinking about this for that past few weeks, and having come across several online repositories looking for solutions, I found some way to do this, using @cmuratori 's DTC Example. I implemented in an example application (~700 LOC) and now everything is fine so far; I am rendering while getting messages through some major trickery suggested by Casey Muratori in the DTC example; viewports work fine now!

Judging by your profile it looks like you're just starting out with things, which is great! I'd strongly recommend avoiding multithreading for a while though and focus on developing rigor and discipline in a single-threaded context first. Concurrency is extremely unforgiving.

• Yes definitely! Multithreading has been a wild ride--just have to learn to work with it not against it!

Sorry this turned into a huge wall of text, hopefully it was useful! 😅

• Absolutely no problem! I love getting to learn and know about these things!

[PathogenDavid]

Glad you found all that to be helpful!

(Apologies in advance for any typos, I didn't proofread this much 😅)

I would like to understand a bit more about this, if you have any additional details.

Sure!

First I should correct myself slightly: WM_WINDOWSPOSCHANGED, etc are still sent to the secondary viewport windows. They just aren't sent by Windows as a result of the user dragging on the titlebar, etc. They're actually indirectly sent by Dear ImGui itself, I'll show an example later.

I think it's easiest to see how this all works by looking at ImGuiPlatformIO, specifically the portion dedicated to multi-viewport support. This is the interface that backends implement to allow Dear ImGui to interact with them directly (as opposed to things like ImGui_ImplWin32_NewFrame which are only ever called by user code.)

The best way to learn about how Dear ImGui interacts with backends through this interface is to right-click on one of the members and click "Find All References" (or the equivalent in your IDE.)

If you trace it in both directions, you'll find it is only called in ImGui::UpdatePlatformWindows and is implemented by ImGui_ImplWin32_SetWindowPos in imgui_impl_win32.cpp (and similar functions in other backends):

// Apply Position and Size (from ImGui to Platform/Renderer backends)
if ((viewport->LastPlatformPos.x != viewport->Pos.x || viewport->LastPlatformPos.y != viewport->Pos.y) && !viewport->PlatformRequestMove)
    g.PlatformIO.Platform_SetWindowPos(viewport, viewport->Pos);
if ((viewport->LastPlatformSize.x != viewport->Size.x || viewport->LastPlatformSize.y != viewport->Size.y) && !viewport->PlatformRequestResize)
    g.PlatformIO.Platform_SetWindowSize(viewport, viewport->Size);

(Note that if you look at the full context, the above logic only applies to secondary viewports.)

You can see (conceptually) that if Dear ImGui's representation of the viewport has a different position than the last known position of the viewport it tells the backend to move the actual platform viewport, which has a pretty simple implementation on Windows:

imgui/backends/imgui_impl_win32.cpp

Lines 1204 to 1213 in a4ebe3d

	static void ImGui_ImplWin32_SetWindowPos(ImGuiViewport* viewport, ImVec2 pos)
	{
	ImGui_ImplWin32_ViewportData* vd = (ImGui_ImplWin32_ViewportData*)viewport->PlatformUserData;
	IM_ASSERT(vd->Hwnd != 0);
	RECT rect = { (LONG)pos.x, (LONG)pos.y, (LONG)pos.x, (LONG)pos.y };
	if (viewport->Flags & ImGuiViewportFlags_OwnedByApp)
	ImGui_ImplWin32_UpdateWin32StyleFromWindow(viewport); // Not our window, poll style before using
	::AdjustWindowRectEx(&rect, vd->DwStyle, FALSE, vd->DwExStyle);
	::SetWindowPos(vd->Hwnd, nullptr, rect.left, rect.top, 0, 0, SWP_NOZORDER | SWP_NOSIZE | SWP_NOACTIVATE);
	}

The main thing to note here is the SetWindowPos. IE: It's programmatically moving the window from Dear ImGui, not from some sort of user interaction handled by Windows.

If we trace ImGuiViewportP::Pos it's...well it's used all over. If you dig through the writes to it though, you'll see the common theme is that it's synchronized with the position of the ImGuiWindow that it owns.

If you trace ImGuiWindow::Pos around, it's similarly used all over. If you dig around long enough through, you'll find that mouse drag movement is primarily handled by ImGui::UpdateMouseMovingWindowNewFrame:

ImVec2 pos = g.IO.MousePos - g.ActiveIdClickOffset;
if (moving_window->Pos.x != pos.x || moving_window->Pos.y != pos.y)
{
    SetWindowPos(moving_window, pos, ImGuiCond_Always);
    if (moving_window->Viewport && moving_window->ViewportOwned) // Synchronize viewport immediately because some overlays may relies on clipping rectangle before we Begin() into the window.
    {
        moving_window->Viewport->Pos = pos;
        moving_window->Viewport->UpdateWorkRect();
    }
}

One thing really worth highlighting here is that this function is what moves windows even when they're merged into the main viewport or in cases when multi-viewports aren't enabled. (IE: The same logic exists on the main branch, just without the viewport sync part.) A good way to confirm this is to plop down a message breakpoint on the ImGui::SetWindowPos call.

I'll stop tracing things around at this point. Hopefully you're satisfied that this is the way windows get moved via mouse interactions within Dear ImGui, and you can see how those interactions eventually propagate to calling Win32's SetWindowPos function to move the secondary viewport window on your screen.

Although I will give you one tip in case you find yourself wanting to trace things around even more: I didn't find ImGui::UpdateMouseMovingWindowNewFrame via ImGuiWindow::Pos since it has an overwhelming number of results. I instead found it via ImGuiContext::MovingWindow which I found via ImGui::UpdateMouseMovingWindowEndFrame which I found via ImGuiIO::ConfigWindowsMoveFromTitleBarOnly -- which has very few results and is much easier to digest. The reason ImGuiWindow::Pos has so many hits is that there's just a lot of ways to change it. Since I knew I wanted to focus on mouse drag movement, I went to see how a feature more directly related to it was handled--even though that feature was unrelated to my question. This strategy is a really good way to navigate unfamiliar codebases. For example: Wanna learn where Dear ImGui renders the window decorations? The easiest way to find it is to look for where ImGuiWindowFlags_NoBackground is handled.

---

Earlier I mentioned WM_WINDOWSPOSCHANGED/etc are still sent in this codepath, and they just weren't sent as a result of an interaction handled by Windows. If you stick a breakpoint in ImGui_ImplWin32_WndProcHandler_PlatformWindow's handling of WM_SIZE and move a secondary viewport, you'll see the following callstack:

(Tip: Right-click a stack frame row and select "Load Symbols" if you see less details than this screenshot. You may need to enable the Microsoft Symbol Servers.)

Some things to highlight in this callstack:

You can see that the bottom of the stack (A) from main is ImGui::UpdatePlatformWindows calling ImGui_ImplWin32_SetWindowPos calling Win32's SetWindowPos (called by its internal name of NtUserSetWindowPos here)...and eventually our WndProc is getting called to handle the WM_SIZE message we put the breakpoint in (C).

You might've assumed your WndProc is only called in response to DispatchMessage in your message pump, but that's not actually the case. That's only for queued messages, which are mostly messages from outside of your process, other threads, or explicitly queued via PostMessage.

This is what I meant by the messages being sent indiredctly via Dear ImGui. Win32's SetWindowPos.

You might also be wondering about the WndProc frame at (B). This is handling message WM_WINDOWPOSCHANGED (message 71), which goes to DefWndProc at the end of ImGui_ImplWin32_WndProcHandler_PlatformWindow which eventually leads to the WM_SIZE (message 3) handled in the top frame (C).

If you've read the documentation remarks of WM_WINDOWPOSCHANGED or WM_SIZE, this probably isn't too surprising since WM_SIZE is defined as being a helper message generated by the DefWndProc for WM_WINDOWPOSCHANGED. Just wanted to highlight how simple this interaction is and that it's pretty visible.

You'll actually find a lot of windows messages are like this. If you put a breakpoint on a WM_SIZE handler you'll actually see it comes from a WM_WINDOWPOSCHANGED which comes from a WM_SYSCOMMAND which comes from a WM_NCLBUTTONDOWN.

The NC in WM_NCLBUTTONDOWN means "non-client", IE: The user is clicking on the border of the window (or something like that.) The WM_SYSCOMMAND feels a little funky if you read its documentation but that message is pretty funky in general. You might assume it has wParam = 0xF000 (SC_SIZE) but it's actually 0xF008 (which is not documented.)

Either way, if you prod at it a bit, you can figure out that the DefWndProc for WM_SYSCOMMAND in this situation appears to be the implementation of the modal loop at the core of this discussion. (I say "appears" because this is obviously not documented, and not something I've ever cared to investigate before so this impromptu investigation wasn't particularly exhaustive.)

---

At this point you probably have enough sense to see why things act differently with ImGuiIO::ConfigViewportsNoDecoration and the instead user moves the viewport via dragging on the Windows titlebar of the window. In that case the WM_MOVE of ImGui_ImplWin32_WndProcHandler_PlatformWindow sets ImGuiViewport::PlatformRequestMove which is evntually checked in ImGui::UpdateViewportsNewFrame which queries the position of the window via ImGuiPlatformIO::Platform_GetWindowPos which calls ImGui_ImplWin32_GetWindowPos which calls the Win32 function ClientToScreen to get the window position.

(With your knowledge of how Dear ImGui is indirectly causing WM_MOVE to be dispatched, you might wonder if this causes any sort of feedback. It doesn't since ImGuiViewport::PlatformRequestMove is cleared via ImGuiViewportP::ClearRequestFlags via ImGui::UpdatePlatformWindows at the end of the frame. As the name implies, ImGui::UpdateViewportsNewFrame happens at the start of a new frame.)

Not sure if it matters, but my VSCode is not doing that, but Notepad is xD

My point was it doesn't close in VSCode even though it should :)

Back in the ancient times, Windows did not have a dedicated close button. You used to click on the control-menu button (which was eventually replaced by the program icon) and select "Close". As a shortcut you could just double-click the control-menu button. (Try it for yourself!)

• I found some way to do this, using @​cmuratori 's DTC Example.

Definitely a cool approach, not sure if I've seen it done that way before. Thanks for sharing!

I would definitely be careful about this strategy though, some window messages use points for wParam/lParam and those pointers would no longer be valid. You also can't return a result other than 0 with this approach.

Totally fine for a lot of messages though (including the ones in the example.) SendMessage instead of PostThreadMessage could solve those limitations, but would want to sit down and figure out why Casey didn't already use that approach (maybe just to avoid having another service window owned by the "main" thread since it wasn't super necessary for his use-case.)

It's also good to be mindful about making sure all the messages that ImGui_ImplWin32_WndProcHandler wants get passed across that thread boundary, especially when you update Dear ImGui. We've had issues in the past with people experiencing problems caused by them filtering which messages they send to the backend (and this solution is implicitly filtering them.)

BTW, Windows actually has a struct that's equivalent to the_baby, it's called CREATESTRUCT.

[cmuratori]

Totally fine for a lot of messages though (including the ones in the example.) SendMessage instead of PostThreadMessage could solve those limitations, but would want to sit down and figure out why Casey didn't already use that approach (maybe just to avoid having another service window owned by the "main" thread since it wasn't super necessary for his use-case.)

SendMessage is blocking, while PostThreadMessage is not. The reason you would not want to use SendMessage in this case is to ensure that both threads run independently: your rendering will not block your system message processing, and your system message processing will not block your rendering.

If you do not care about your message processing thread, and are OK with it blocking on your rendering, you should be able to use SendMessage if you so desire. I have never tested that method myself, though, so there may be some hidden gotchas that I am unaware of.

- Casey

[PathogenDavid]

SendMessage is blocking, while PostThreadMessage is not.

Yup, in the context of my warning it's desirable for the message processing to wait for the rendering thread.

Although I'm not seeing where it'd allow the message thread to block the rendering thread, the messages sent from rendering->message are already using SendMessage since they need to wait for the result.

Actually thinking about it a bit more, you'd definitely need some mechanism to avoid only draining one message per frame, and something like that could pretty easily cause a modal loop on the message thread to hold the render thread hostage--which is maybe that's what you were getting at.

Thanks for chiming in (and for sharing the demo back in 2021!)

[cmuratori]

It's worth remembering that DTC was just a small demonstration I did for Jonathan Blow, since he was asking about this. In real code, I use neither PostThreadMessage nor SendMessage, since the better thing to do is to directly handle the messages where they arrive and deal with any resulting thread communication yourself via your own code.

So I would recommend against thinking too hard about Post vs. Send - you probably shouldn't be shipping code with either. I am just explaining why DTC specifically was using PostThreadMessage, and why it is preferable if for some reason you didn't want to have your own interthread communication. There really isn't much reason to use SendMessage unless you need information that can only be accessed at the time of the message (which is not just the pointers you mentioned, but also things like GetKeyState, etc., that are tied to the message queue state). In my opinion, once you get to that point, the correct next step is to use your own thread communication that better supports more flexible data transfer, because SendMessage is a) blocking and b) won't let you actually call things like GetKeyState and have it work properly on the other thread anyway.

- Casey

[PathogenDavid]

Yeah for sure, that's what I do in real code as well. @xgamercpp is still learning about this sort of stuff and liked your demo so I was mainly trying to give them an idea of limitations to watch out for since they aren't Jonathan Blow.