Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pen input not being friendly with touch #6509

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

frenzibyte
Copy link
Member

Fixes pen & touch osu! playstyle on mobile tablets no longer working in the past release.


Back before SDL was bumped, SDL was propagating pen input on iOS in two forms:

  • mouse move input when the pen is within the screen's region but not touching it.
  • touch input when the pen is directly touching the screen.

This was working perfectly fine with our implementation of OsuTouchInputMapper in osu! for an elegant pen + touch playstyle. But since we bumped SDL and it became handling pen input natively on iOS and now became directly mapped to mouse input by our implementation in SDL3Window / #6488, everything fell apart.

I initially began on this by looking into updating the implementation of OsuTouchInputMapper to expect pen input by mouse events, but bailed out after finding multiple catastrophic input-level issues by the idea of sending pen input as mouse input (see #6508 / https://discord.com/channels/188630481301012481/589331078574112768/1332383057642000426).

I then attempted to dive right into the o!f input subsystem and try to get things in a right place, then I realised I'm essentially about to propose a top-down refactor of major input components, and I'm absolutely in no place, time, or even interest in doing any of that (nor do I think that anyone will be supporting this effort right at this very moment).

Finally, I decided to just remind myself of how osu! has always been working until now and mimicked that behaviour for now, calling it a day and allowing ourselves to focus on much better and prioritised tasks. In fact, I think the behaviour in this PR might even make more sense with regards to the idea that we want pen and touch input to work harmoniously.

I have tested this on my iPad and it's working perfectly fine everywhere.

@frenzibyte frenzibyte requested review from Susko3 and a team January 25, 2025 08:14
@Walavouchey
Copy link
Member

was sort of hoping this would also fix my use case of surface pro pen + touch input on windows (screen capture), but that has never worked unlike the ios pen

ideally i think mouse aim + touch tapping should be supported as well but we'll see

@frenzibyte
Copy link
Member Author

Are you testing with SDL3?

@Walavouchey
Copy link
Member

hmm, same results. made a discussion #6510 and put the logs there

@hwsmm
Copy link
Contributor

hwsmm commented Jan 25, 2025

You may need to wait for the next SDL3-CS release since SDL3 added Windows Ink support after our last update.

I also tried it on Android, but I think Pen+Touch playstyle never worked on Android (at least on my device, Galaxy Tab S7 FE) due to how palm rejection is implemented. My device just rejects all touch inputs while the pen is near the screen.

On osu!, this PR enables Touch device mod while pen is down, and disables it while pen is hovering. I think it may need a fix.

Copy link
Member

@Susko3 Susko3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this change, but it seems this is the simplest fix to make osu! work, so it can go in as a hack. I would like to see the following:

  • Add a FrameworkSetting flag that games (osu!) can set a default
  • Place the pen handling introduced in this PR behind the flag
  • Restore master behaviour when the flag is disabled
  • Add pen-as-mouse-and-touch tests to TestSceneOsuTouchInput
    • It should be easy to modify the tests so they check actual pen input (i.e. make a helper function that calls InputManager.BeginTouch(), so that it can be substituted for InputManager.PressPen() when someone starts to implement proper pen input)

@frenzibyte
Copy link
Member Author

frenzibyte commented Jan 25, 2025

  • Add a FrameworkSetting flag that games (osu!) can set a default
  • Place the pen handling introduced in this PR behind the flag
  • Restore master behaviour when the flag is disabled

I don't know about any of this. Why are we bringing back something that must be working perfectly fine with this PR, it's just introducing a flag that I can't imagine anyone will be bothered enough to use it?

If anything, I can move the application of touch input within PenHandler itself so as to not bother the SDL implementation enough with that, but that's code quality at best.

  • Add pen-as-mouse-and-touch tests to TestSceneOsuTouchInput

    • It should be easy to modify the tests so they check actual pen input (i.e. make a helper function that calls InputManager.BeginTouch(), so that it can be substituted for InputManager.PressPen() when someone starts to implement proper pen input)

I have something like that prepared already, I'll PR it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot play osu! with pen + touch input
4 participants