 |
 |
 |
 |
|---|---|---|---|
| OpenCV | Mediapipe | Pillow | Python |
Developed using OpenCV and Mediapipe, this app will allow you to use your computer without a physical mouse by allowing you to perform basic mouse operations by using certain hand gestures and movements in front of your webcam. If you wish to have better FPS while using this app, use the CUI based version
To run this app, you can download the latest release where you will find a .exe file. Run the .exe file and that's it! However if you wish to use the source code to run the app, follow these simple steps:
- See that you have
Pythoninstalled. - Clone this branch (or the repository if you wish to).
- Create a new Python environment and install the dependencies in it. This can be done as follows:
- Open
command promptandcdto the directory where you have cloned the repo. - Run the command
Python -m venv envto create a new virtual environment namedenv - Activate the enviroment by running the command
env\Scripts\activate.bat - Install depencies by running the command
pip install -r requirements.txt
- Open
The above steps will create a python virtual environment and install the required depndencies in it. Now whenever you want to run the app, follow these steps:
- Open
command promptandcdto the directory where you have cloned the repo. - Activate the python environment by running the command
env\Scripts\activate.bat. - Run the app by the command
Python GUI_Virtual_Mouse.py.
Upon running the app, your webcam will be turned on and will be used to track your hand and fingers. By default, this app assumes that the distance between your hand and your webcam is about 20cm. This can be changed by using the options menu to change the webcam distance as per your preference. As of now, you can only use your right hand to use this app. Support for left hand is coming soon. This app works in 4 modes:
- Pointer Mode
- Left Click Mode
- Right Click Mode
- Scrolling Mode
This mode will be triggered when only your index finger is up, while the rest are down. In this mode you can move around your index finger to move the cursor on screen. The red circle on the index finger indicates that you are in pointer mode.
This mode will be triggered when your index and middle fingers are up while rest are down. While in this mode, bring your index and middle fingers closer to perform a typical mouse left-click. The red circle on the index and the middle finger indicates that you are in left-click mode.
This mode will be triggered when your index finger and thumb are up while rest of the fingers are down. While in this mode, bring your index finger and thumb closer to perform a typical mouse right-click. The red circle on the index finger and the thumb indicates that you are in right-click mode.
This mode will be triggered when your index, ring and middle fingers are up while rest of the fingers are down. While in this mode, move your ring finger away from your middle finger to scroll up and move your index finger away from your middle finger to scroll down. The red circle on the index, ring and the middle finger indicates that you are in scrolling-click mode.
See Screenshots for better understanding of different modes. If this app does not work properly for you, customising it may remove any the problems that you face.
You can customise some aspects of this app based on your preferences by using the options menu that appears on the top left corner of the window upon running the app. Few of the things you can change are:
- Assumed distance between your hand and your webcam (This setting will greatly influence how effective your clicks are registered).
- Sensitivity of the cursor
- Amount of smoothening applied to movement of cursor
- Enable/Disbale FPS
 |
|---|
| Hand Detection |
 |
| Pointer Mode |
 |
| Left Click Mode |
 |
| Bring fingers closer to trigger mouse left-click. The circle between the fingers will turn green when a click is triggered |
 |
| Right Click Mode |
 |
| Bring index finger and thumb closer to trigger mouse right-click. The circle between them will turn green when a click is triggered |
 |
| Scrolling Mode |
 |
| Scroll up by moving your ring finger away from Middle finger |
 |
| Scroll Down by moving your index finger away from your middle finger |
 |
|---|
| Demo GIF |
Sometimes the position of thumb is such that its hard to detect whether it is up or down as it doesn't actually go down like rest of the fingers. This may cause issues. So keep in mind that the correct position of thumb when it will be treated as up or down or not active is as follows:
 |
|---|
| Thumb Up |
 |
| Thumb Down |