HandPointer

HandPuntator is a Python-based system that enables hands-free computer control using real-time hand tracking and gesture recognition powered by MediaPipe, translating finger landmarks into precise cursor movement and configurable actions like clicking, dragging, and scrolling for accessible, touch-free interaction scenarios .

Key contributions

• Implemented robust real-time hand landmark tracking and gesture mapping with MediaPipe Tasks to support low-latency, smooth pointer control and discrete actions .

• Designed a modular input layer that maps index-finger kinematics to cursor motion and pinch-orientation gestures to clicks and drags, making behaviors easy to extend or swap .

• Built an experimentation pipeline with OpenCV for video capture and visualization, enabling rapid iteration on gesture thresholds, smoothing, and UX tuning . What it showcases

• Computer vision integration: applying MediaPipe’s Gesture Recognizer/Hands pipeline to interface design and human-computer interaction .

• Systems thinking: translating continuous hand pose data into stable, user-friendly controls through filtering, debouncing, and gesture state machines .

• Accessibility and HCI: enabling touch-free control for mobility and hygiene contexts, and prototyping alternative input methods .

Gesture set

• Hand open: move the pointer by tracking the index fingertip position in real time, with smoothing to ensure stable motion.
• Index + middle finger touch: perform a left-click when the index fingertip contacts the middle fingertip, with a short debounce to prevent accidental multiple clicks.

Run Locally

you will need python 3.12 to use

1: Clone the git repo and go to directory

  git clone https://github.com/Zunza5/HandPointer.git
  cd HandPointer

2: Create and activate a virtual environment

macOS/Linux:

  python3.12 -m venv .venv
  source .venv/bin/activate

windows:

  python3.12 -m venv .venv
  ..venv\Scripts\Activate.ps1

3: Install dependencies

  pip install -r requirements.txt

4: run main.py

  python3.12 main.py

Authors

• @Zunza5