SDL 3 Cursor misbehavior switching from Fullscreen to Windowed in Wayland when cursor is hidden

[Dragon-Baroque]

Followup of #9799 which fixed the switch from Windowed to Fullscreen, but incompletely the switch from Fullscreen to Windowed.

When switching Exult from Fullscreen to Windowed, no cursor is ever visible on the whole screen

• Exult applies a SDL_HideCursor to prevent the the display of the system cursor and displays its own cursor to track the position of the mouse when inside its window, but the mouse it outside the window,
• The Linux Plasma desktop does not display any cursor either as the Exult window claims the cursor as grabbed by and focused into its window even though the mouse is actually outside the window.

This misbehavior appears to be related with the Mouse Warp emulation in Wayland. Thanks to Kontrabant to identify this.

In that case, is simply the Mouse Warp emulation missing a check to ensure that the mouse position is within the bounds of the window ?

---

From the discussion that went in exult/exult#379.

Try setting the SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP hint to 0.

Yes, that solves the problem.

Meanwhile, I built a new reproducible case with testwm.
With a patch to testwm to toggle SDL_ShowCursor and SDL_HideCursor on the right click of the mouse - borrowed from testcursor in testwm.zip. You might consider adding that permanently, as using a hidden cursor is not uncommon.

• Rebuild SDL 3,
• Run testwm --grab ( without --grab this works ) in Linux Fedora 40 Plasma over Wayland,
• Bring the cursor into the testwm window,
• Hide it with right click,
• Switch to large fullscreen with Alt-Enter,
• The cursor is still invisible, but move it to the bottom right of the screen, following the SDL_GetState for that,
• Switch back to windowed, then the cursor is invisible, the window state contains MOUSE_GRABBED and MOUSE_FOCUS, the mouse state is way beyond the window size at 640x480, and does not change as you move the mouse.

With the SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP set to 0, the last step changes to

• Switch back to windowed, then the cursor is visible and out of the testwm window, the window state contains MOUSE_GRABBED and but not MOUSE_FOCUS, the mouse state is way beyond the window size at 640x480, and does not change as you move the mouse, but you can bring the cursor into the testwm window and this sets MOUSE_FOCUS and GetState changes.

NB My previous explanation about the cursor being confined to the game window was that, a MouseGrab. Thus the --grab option.
Exult only performs MouseGrabs between MOUSE_BUTTON_DOWN and MOUSE_BUTTON_UP. But Exult performs the switch to Fullscreen and back to Windowed in the callback of a MOUSE_BUTTON_UP ( button click ). I have removed the MouseGrabs of Exult and retried, but the problem still shows.

[Kontrabant]

This sounds like a bug in KDE/KWin, as it should be either sending a pointer leave event when the cursor is outside of the window (as Weston does), or warping the cursor back into the grab region (as GNOME does).

If the pointer winds up outside of the window and the window still reports mouse focus, that means SDL never got a pointer leave event, which is a compositor bug.

[Kontrabant]

Switch back to windowed, then the cursor is invisible, the window state contains MOUSE_GRABBED and MOUSE_FOCUS, the mouse state is way beyond the window size at 640x480, and does not change as you move the mouse.

Yes, because it's hitting the warp emulation path in SDL and switching to relative mode, which doesn't send coordinates, just relative motion events.

With the SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP set to 0, the last step changes to

• Switch back to windowed, then the cursor is visible and out of the testwm window, the window state contains MOUSE_GRABBED and but not MOUSE_FOCUS, the mouse state is way beyond the window size at 640x480, and does not change as you move the mouse, but you can bring the cursor into the testwm window and this sets MOUSE_FOCUS and GetState changes.

I just tested it on KDE and this is correct behavior. The mouse left the window, so MOUSE_FOCUS is lost, but the window still has the grab flag set since the pointer will be confined as soon as it re-enters the window.

[Dragon-Baroque]

With your commit b1e01b9 I can report that

• testwm --grab does no longer show the cursor misbehavior above,
• Exult does no longer loose its cursor back from Fullscreen.

Would you like the following patch to SDL testing that adds a ShowCursor / HideCursor toggle on Ctrl-h in SDL_test_common.h and SDL_test_common.c, and adds an option --hide-cursor ?
SDLTestCommon.zip
It has been very handy to me.
As far as I can tell, only sources in test/ use SDL_test_common.
Among those, only test/testmodal.c uses SDLK_h but it does not use SDLTest_CommonInit - it has its own event loop.

[Kontrabant]

Glad it's working for you now. While working on this, I did find a bug in KDE, where pointer confinement is lost after leaving fullscreen until the pointer leaves and re-enters the window. If you happen to run into it, there is already an upstream bug report opened: https://bugs.kde.org/show_bug.cgi?id=487189

Would you like the following patch to SDL testing that adds a ShowCursor / HideCursor toggle on Ctrl-h in SDL_test_common.h and SDL_test_common.c, and adds an option --hide-cursor ?

Sure, anything that helps reproduce bugs without having to manually hack up the tests is always helpful. Feel free to make a pull.

[Dragon-Baroque]

Sure, anything that helps reproduce bugs without having to manually hack up the tests is always helpful. Feel free to make a pull.

My first PR towards SDL. This is #9848.
I followed the contributing README, and there are no changes in the testautomation run.
However when I tried to apply a clang-format, I got many many changes, so I did not include them in the commit.
Is the .clang-format ready for SDL 3 ?

[Kontrabant]

Any other cursor related problems, or can this issue be closed?

[Dragon-Baroque]

You may close the issue.

Just a question : Exult switches correctly from Windowed to Fullscreen and back, whether or not the SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP is set to 0. Exult sets it to 0 at this moment.
What is your preferred choice ? With or without the hint ?

[slouken]

In general, leaving the hints unset is the good thing to do unless you need specific behavior.

Thanks!