A professional, beginner-friendly web application for real-time sign language recognition using Flask, OpenCV, and MediaPipe. Perfect for a second-year computer engineering student's project.
This project detects 12 common sign language gestures using a webcam and converts them to text in real-time. The system is built with a clean web interface and stores all recognized signs in a database.
Key Features:
- ✅ Real-time webcam gesture detection
- ✅ 12 common sign language gestures recognized
- ✅ Web-based interface (HTML/CSS/JavaScript)
- ✅ SQLite database for storing predictions
- ✅ Dashboard with statistics
- ✅ Clean, modular code structure
- ✅ Fully commented for learning
sign_language_converter/
├── app.py # Main Flask application (entry point)
├── config.py # Configuration settings
├── database.py # Database operations
├── gesture_model.py # Gesture recognition logic
├── camera_module.py # Webcam capture and streaming
├── requirements.txt # Python dependencies
├── templates/
│ └── index.html # Main web page
├── static/
│ ├── css/
│ │ └── style.css # Styling and layout
│ └── js/
│ └── script.js # Frontend interactivity
├── uploads/ # Folder for captured frames (optional)
└── sign_language_database.db # SQLite database (auto-created)
- THUMBS UP - Thumb extended upward
- THUMBS DOWN - Thumb pointing downward
- OK SIGN - Thumb and index forming circle
- PEACE SIGN - Index and middle fingers extended
- OPEN PALM - All five fingers extended
- CLOSED FIST - All fingers folded
- POINTING - Only index finger extended
- ROCK SIGN - Index and pinky extended
- LOVE YOU - Index, middle, and pinky extended
- CALL ME - Thumb and pinky forming phone shape
- VICTORY - Middle and ring fingers extended
- THUMBS SIDEWAYS - Thumb pointing sideways
# Navigate to project folder
cd sign_language_converter
# Install required packages
pip install -r requirements.txtWhat gets installed:
Flask- Web framework for backendopencv-python- Computer vision librarymediapipe- Hand detection and trackingnumpy- Numerical computing
python app.pyExpected output:
============================================================
Sign Language to Text Converter - Flask App
============================================================
Starting server...
Open browser and go to: http://localhost:5000
Press CTRL+C to stop the server
============================================================
Go to: http://localhost:5000
- Click "Start Camera" button
- Make hand gestures in front of webcam
- Recognized signs appear in the list
- View statistics on the dashboard
✅ Lightweight - Perfect for college projects
✅ Python-based - Easy to understand and modify
✅ Great documentation - Lots of tutorials available
✅ Suitable for beginners - Not overly complex
✅ Can scale - Can be deployed to production later
✅ No server needed - Database is just a file
✅ Built into Python - No setup required
✅ Perfect for college projects - Lightweight
✅ Easy to backup - Single file format
✅ Good enough for this scale - Thousands of records
✅ Industry standard - Used in real computer vision
✅ Works with all webcams - Universal compatibility
✅ Fast and efficient - Good performance on laptops
✅ Easy to use - Clear API and documentation
✅ Widely used - Many examples and tutorials
✅ Pre-trained model - Doesn't require GPU
✅ Fast detection - Works smoothly on CPU
✅ Accurate hand tracking - 21 key points per hand
✅ Easy integration - Clean Python API
✅ Google-backed - Continuously improved
User clicks "Start Camera"
↓
JavaScript calls /start_camera endpoint
↓
Flask opens webcam and starts streaming
↓
Video displayed in <img> tag (MJPEG stream)
↓
JavaScript polls /api/detect_gesture every 100ms
↓
MediaPipe detects hand and landmarks
↓
Gesture logic analyzes finger positions
↓
If gesture detected → Save to SQLite database
↓
JSON response sent to JavaScript with gesture name
↓
JavaScript updates UI with detected gesture
↓
Database predictions fetched and displayed
┌─────────────────────────────────────────────────────┐
│ WEB BROWSER (Frontend) │
│ ┌────────────────────────────────────────────┐ │
│ │ HTML (Structure) │ │
│ │ CSS (Styling) │ │
│ │ JavaScript (Interactivity) │ │
│ └────────────────────────────────────────────┘ │
└────────────────┬──────────────────────────────────┘
│ HTTP Requests/Responses
┌────────────────▼──────────────────────────────────┐
│ FLASK BACKEND (Python) │
│ ┌──────────────────────────────────────────┐ │
│ │ app.py (Main application) │ │
│ │ Routes: /start_camera, /video_feed, etc │ │
│ └──────────────────────────────────────────┘ │
└────────────────┬──────────────────────────────────┘
│ Imports
┌────────┴────────┬──────────────┬────────────┐
▼ ▼ ▼ ▼
┌─────────┐ ┌──────────────┐ ┌──────────┐ ┌────────┐
│ config │ │ database.py │ │ camera │ │gesture │
│ .py │ │ │ │_module.py│ │_model │
│ │ │ SQLite Ops │ │ OpenCV │ │.py │
└─────────┘ │ Storing Data │ │ Streaming│ │MediaPipe
└──────────────┘ └──────────┘ │Detection
▼
┌─────────────┐
│ SQLite │
│ Database │
│ (.db file) │
└─────────────┘
Hand Detection (MediaPipe):
- Input: Video frame from webcam
- Process: Detects hand and extracts 21 key points
- Output: Coordinates of finger tips, joints, wrist
Finger State Analysis:
- Compare each finger's tip position with its PIP joint
- If tip is further than PIP → finger is extended
- Return boolean for each finger (true/false)
Gesture Logic (Rule-based):
- Combine finger states with distance calculations
- Example: PEACE SIGN = Index AND Middle extended, Ring AND Pinky NOT extended
- Pattern matching approach (not ML classification)
For College Project:
✅ Easier to understand and explain
✅ Doesn't require training data
✅ Works immediately (no training time)
✅ Easy to modify/add new gestures
✅ Perfect for demonstration
For Production:
❌ Less flexible if rules get complex
❌ Harder to handle edge cases
❌ Manual tuning needed
Each detection is stored as:
├─ id: Unique identifier (1, 2, 3, ...)
├─ gesture: Text of recognized sign ("THUMBS UP")
├─ timestamp: When it was detected (2024-02-05 10:30:45)
└─ confidence: How confident (0.9)
Query Examples:
- Get all predictions: SELECT * FROM predictions
- Count by gesture: SELECT gesture, COUNT(*) FROM predictions GROUP BY gesture
- Get recent: SELECT * FROM predictions LIMIT 10 ORDER BY timestamp DESC
A:
- Background lighting affects hand detection (solved by better lighting)
- Fast hand movements can miss detection (solved by frame buffering)
- False positives from similar gestures (solved by confidence threshold)
A:
- Pattern matching with MediaPipe is simpler and faster
- Doesn't require GPU (works on normal laptops)
- Google's pre-trained model is reliable and accurate
- Easy to debug and modify (no black-box ML model)
A:
- JavaScript sends HTTP requests (GET/POST)
- Flask backend processes requests and returns JSON
- JavaScript updates HTML dynamically with responses
- No page reload needed (AJAX - Asynchronous JavaScript)
A:
- No server setup required
- Perfect for college projects (simple deployment)
- Single file database (easy backup)
- Good enough for this scale
- If needed more users, can migrate to MySQL later
A:
Yes! To add a new gesture:
1. Identify which fingers should be up/down
2. Add new condition in detect_gesture() function
3. Update gesture list in config.py
4. Done! No retraining needed
- Open
gesture_model.py - Add new condition in
detect_gesture()function:
# New gesture: SHAKA SIGN (pinky and thumb extended)
if (fingers['thumb'] and fingers['pinky'] and
not fingers['index'] and not fingers['middle'] and not fingers['ring']):
return "SHAKA SIGN"- Add to
config.pygesture list - Done!
- Adjust confidence thresholds in
config.py - Modify detection logic in
gesture_model.py - Test with different hand positions
- Check terminal output (already has detailed logging)
- Add more
print()statements for debugging - Check console in browser (F12 → Console)
- Check if another app is using camera
- Try closing browser and restarting
- Restart Python application
- Check if camera is connected
- Ensure good lighting
- Show gesture clearly to camera
- Check if gesture is in supported list
- Adjust hand distance from camera
- Camera might not be started
- Check browser console for errors (F12)
- Check Flask terminal for error messages
- Refresh page and try again
- Close other applications
- Reduce video resolution (edit config.py)
- Reduce gesture detection frequency
- Check CPU usage
- Lines of Code: ~2000 (including comments)
- Number of Modules: 6 (modular design)
- Number of Gestures: 12 (extensible)
- UI Responsiveness: Mobile, Tablet, Desktop
- Database Scalability: Up to 100,000+ records
- No GPU Required: Runs on standard laptop
- Modular architecture (separation of concerns)
- Clear function documentation
- Comprehensive comments for learning
- Proper error handling
- Database abstraction
- Backend (Python Flask)
- Frontend (HTML/CSS/JavaScript)
- Database (SQLite)
- Real-time communication (HTTP APIs)
- Easy to understand and explain
- Real-world applicable
- Demonstrates proper coding practices
- Professional UI/UX
- Scalable architecture
- Code is well-commented
- Architecture is clearly explained
- Easy to answer technical questions
- Can show live demo
- Easy to modify and extend during viva
python app.pypip install -r requirements.txthttp://localhost:5000
1. Browser Console (F12 → Console)
2. Flask Terminal (where you ran python app.py)
3. SQLite Database (sign_language_database.db)
This project demonstrates:
- Full-stack development (backend + frontend + database)
- Computer vision (hand detection and gesture recognition)
- Web development (Flask, REST APIs, HTML/CSS/JS)
- Software engineering (modular design, documentation)
- Problem-solving (real-time processing, accuracy improvements)
Perfect for a college project that will impress your evaluators! 🚀
This project is for educational purposes. Free to use and modify.
Last Updated: February 2024
Version: 1.0
Status: Production Ready for College Projects ✅