Skip to content

Add support for using thumb stick as simulated mouse #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
duhrer opened this issue Dec 24, 2023 · 6 comments
Open

Add support for using thumb stick as simulated mouse #139

duhrer opened this issue Dec 24, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@duhrer
Copy link
Contributor

duhrer commented Dec 24, 2023

If we make a transparent overlay and absolutely position a cursor image within it, we should be able to simulate a mouse pointer. We would need an approach to detect the foremost object under the cursor.

With a cursor and an awareness of what's under it, we could simulate mouse clicks in a way similar to what we do with the "button tab navigation", but with a MouseEvent and as many of its properties as we can simulate. I'm not sure if we can also simulate mouse hover, but it's worth investigating.

If at all possible this should also be usable with our modals.

This could also lay the groundwork for #77, although additional work to simulate the look of dragging content would be necessary.

@duhrer duhrer added the enhancement New feature or request label Dec 24, 2023
@duhrer
Copy link
Contributor Author

duhrer commented Dec 24, 2023

One concern here is that there are a lot of thumb stick actions and only two thumb sticks.

I was thinking we might address this by having a single "smart" action with an associated mode variable, and have an action to toggle through modes. For example, "smart" thumb stick navigation might scroll by default, and depressing the thumb stick (or using the action menu) might switch to "mouse pointer" mode.

@duhrer
Copy link
Contributor Author

duhrer commented Dec 24, 2023

We could also have auto scrolling when the simulated mouse pointer reaches the edge of the window.

@duhrer
Copy link
Contributor Author

duhrer commented Dec 24, 2023

From reading through this and other threads on stackoverflow, it seems like you can't programatically simulate a hover. You can definitely simulate mouseover, mouseenter, and mouseleave/mouseout events.

One approach in the linked thread was to inspect the current styles looking for any that apply to :hover, and then generate a workalike style that uses a named class instead.

@duhrer
Copy link
Contributor Author

duhrer commented Dec 24, 2023

I assume we'd need a performant way to come up with a short list of things to check, and to me that short list should be based on visibility.

There are pure JS approaches that use getBoundingClientRect on every element and check parents to see if they're displayed. Although I'm reluctant to rely further on jQuery than we can avoid, they do provide a :visible pseudoselector that might be easier to use.

As we'll probably be polling quite often, I was thinking perhaps of caching the list of visible elements and only updating it if the window size or scrolling change. However, that wouldn't catch things that appear / disappear based on other interactions, so we'll probably just have to see how expensive it is to make the list.

@duhrer
Copy link
Contributor Author

duhrer commented Dec 24, 2023

Given that we can't change the hover of whatever we're clicking on as easily, we could also just change the pointer, like from a circle (not hovering) to a caret (hovering, pointing at whatever it is).

@duhrer
Copy link
Contributor Author

duhrer commented Sep 26, 2024

Although this is a long term goal, for now there are quite a few good options if you're willing to use another program in combination with (or instead of) the Gamepad Navigator. I've started writing these up on the wiki.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant