[connect and meet] 🎯 Basic Details Team Name: Kernel Team Members Member 1: Saniya V S - SSET Member 2: Zeba Saithalavi - SSET Hosted Project Link [mention your project hosted link here]
Project Description In today’s digital world, most video communication platforms are designed mainly for people who can hear and speak, which creates challenges for individuals who are deaf, hard of hearing, or speech-impaired. These users often rely on sign language, while others depend on spoken communication, leading to a communication gap during online meetings.Many platforms do not provide integrated support for both speech-to-text conversion and sign language recognition in a single system. As a result, inclusive and accessible real-time communication remains limited. Therefore, there is a need for a platform that enables seamless interaction between users by converting speech and basic sign language gestures into real-time captions during video calls
The Problem statement Communication during online video meetings is often difficult for people who are deaf, hard of hearing, or speech-impaired. Most video calling platforms mainly support spoken communication and do not provide built-in support for sign language recognition. As a result, users who rely on sign language may struggle to communicate effectively with others who depend on speech. Additionally, not all platforms offer accurate real-time speech-to-text captions. This creates a communication gap and limits inclusivity in digital meetings. Therefore, there is a need for a system that integrates real-time speech-to-text conversion and basic sign language recognition within a single video communication platform to ensure accessible and inclusive interaction for all users
The Solution The proposed solution is to develop an inclusive video communication platform called that integrates real-time speech-to-text conversion and basic sign language recognition within a single system. The platform uses the user’s microphone to capture spoken words and convert them into live captions using browser-based speech recognition. At the same time, the camera captures hand gestures and identifies five predefined sign language gestures, converting them into text displayed on the screen. These captions are shared instantly between users during the video call, ensuring smooth and accessible communication. By combining video streaming, speech recognition, and gesture detection, the system reduces communication barriers and creates a more inclusive and user-friendly digital meeting environment.
Technical Details Technologies/Components Used For Software:
Languages used: JavaScript, HTML, CSS Frameworks used: React.js, Express.js Libraries used: Socket.IO, WebRTC, Web Speech API Tools used: VS Code, Git, Node.js, npm, Chrome Browser
Features List the key features of your project:
Feature 1: Speech-to-text conversion that converts spoken words into live captions. Feature 2: Real-time caption sharing between users using Socket.IO. Feature 3: Sign language detection for hand gestures converted into text
Implementation For Software: Installation Backend: cd backend npm install Frontend: cd frontend npm install
Run Backend: npm start Frontend: npm run dev
Project Documentation For Software: Screenshots (Add at least 3)  Add caption explaining what this shows
 Add caption explaining what this shows
 Add caption explaining what this shows
Diagrams System Architecture The system follows a full‑stack web architecture. The frontend is built using React.js, where the user accesses the camera and microphone through the browser. WebRTC handles real‑time video communication between users. The Web Speech API converts spoken words into text captions in real time. The video frames are captured and sent to the sign detection module, which analyzes predefined hand gestures (5 selected signs). Socket.IO enables real‑time transmission of captions between connected users. The backend is built using Node.js and Express.js, which manages socket connections and real-time messaging.
Application Workflow When the user opens the application, the browser requests camera and microphone permission. The live video stream is displayed using WebRTC. If the user speaks, the Web Speech API converts speech into text captions instantly. If the user performs one of the predefined hand signs, the Sign Detector captures the frame and identifies the sign. The detected text (either speech or sign) is displayed as captions and shared in real time with connected users through Socket.IO.
For Hardware: Schematic & Circuit  Add caption explaining connections
 Add caption explaining the schematic
Build Photos 
 List out all components shown
 Explain the build steps
 Explain the final build
Additional Documentation For Web Projects with Backend: API Documentation Base URL: http://localhost:5000
Endpoints GET /api/endpoint
Description: [What it does] Parameters: param1 (string): [Description] param2 (integer): [Description] GET /health Description: Checks if backend server is running.
Response:
{ "status": "success", "message": "Server running" } POST /detect-sign Description: Sends captured image frame for sign detection.
Request Body:
{ "image": "base64_encoded_image_string" } Response:
{ "status": "success", "prediction": "Hello" } [Add more endpoints as needed...]
For Mobile Apps: App Flow Diagram App Flow Explain the user flow through your application User opens the website → Grants camera & microphone access → Video stream starts → Speech converts to text → Sign gestures convert to text → Captions displayed → Captions shared with other user. Installation Guide For Android (APK):
Download the APK from [Release Link] Enable "Install from Unknown Sources" in your device settings: Go to Settings > Security Enable "Unknown Sources" Open the downloaded APK file Follow the installation prompts Open the app and enjoy! For iOS (IPA) - TestFlight:
Download TestFlight from the App Store Open this TestFlight link: [Your TestFlight Link] Click "Install" or "Accept" Wait for the app to install Open the app from your home screen Building from Source:
flutter build apk
./gradlew assembleDebug
flutter build ios
xcodebuild -workspace App.xcworkspace -scheme App -configuration Debug For Hardware Projects: Bill of Materials (BOM) Component Quantity Specifications Price Link/Source Arduino Uno 1 ATmega328P, 16MHz ₹450 [Link] LED 5 Red, 5mm, 20mA ₹5 each [Link] Resistor 5 220Ω, 1/4W ₹1 each [Link] Breadboard 1 830 points ₹100 [Link] Jumper Wires 20 Male-to-Male ₹50 [Link] [Add more...] Total Estimated Cost: ₹[Amount]
Assembly Instructions Step 1: Prepare Components
Gather all components listed in the BOM Check component specifications Prepare your workspace Step 1 Caption: All components laid out Step 2: Build the Power Supply
Connect the power rails on the breadboard Connect Arduino 5V to breadboard positive rail Connect Arduino GND to breadboard negative rail Step 2 Caption: Power connections completed Step 3: Add Components
Place LEDs on breadboard Connect resistors in series with LEDs Connect LED cathodes to GND Connect LED anodes to Arduino digital pins (2-6) Step 3 Caption: LED circuit assembled Step 4: [Continue for all steps...]
Final Assembly: Final Build Caption: Completed project ready for testing
For Scripts/CLI Tools: Command Reference Basic Usage:
python script.py [options] [arguments] Available Commands:
command1 [args] - Description of what command1 does command2 [args] - Description of what command2 does command3 [args] - Description of what command3 does Options:
-h, --help - Show help message and exit -v, --verbose - Enable verbose output -o, --output FILE - Specify output file path -c, --config FILE - Specify configuration file --version - Show version information Examples:
python script.py input.txt
python script.py -v input.txt
python script.py -o output.txt input.txt
python script.py -c config.json --verbose input.txt Demo Output Example 1: Basic Processing
Input:
This is a sample input file with multiple lines of text for demonstration purposes Command:
python script.py sample.txt Output:
Processing: sample.txt Lines processed: 3 Characters counted: 86 Status: Success Output saved to: output.txt Example 2: Advanced Usage
Input:
{ "name": "test", "value": 123 } Command:
python script.py -v --format json data.json Output:
[VERBOSE] Loading configuration... [VERBOSE] Parsing JSON input... [VERBOSE] Processing data... { "status": "success", "processed": true, "result": { "name": "test", "value": 123, "timestamp": "2024-02-07T10:30:00" } } [VERBOSE] Operation completed in 0.23s Project Demo Video [Add your demo video link here - YouTube, Google Drive, etc.]
Explain what the video demonstrates - key features, user flow, technical highlights
Additional Demos [Add any extra demo materials/links - Live site, APK download, online demo, etc.]
AI Tools Used (Optional - For Transparency Bonus) If you used AI tools during development, document them here for transparency:
Tool Used: [e.g., GitHub Copilot, v0.dev, Cursor, ChatGPT, Claude]
Purpose: [What you used it for]
Example: "Generated boilerplate React components" Example: "Debugging assistance for async functions" Example: "Code review and optimization suggestions" Key Prompts Used:
"Create a REST API endpoint for user authentication" "Debug this async function that's causing race conditions" "Optimize this database query for better performance" Percentage of AI-generated code: [Approximately X%]
Human Contributions:
Architecture design and planning Custom business logic implementation Integration and testing UI/UX design decisions Note: Proper documentation of AI usage demonstrates transparency and earns bonus points in evaluation!
Team Contributions Saniya V S: Frontend development, WebRTC integration, UI design Zeba Saithalavi: Backend development, Socket.IO integration, API design
License This project is licensed under the [LICENSE_NAME] License - see the LICENSE file for details.
Common License Options:
MIT License (Permissive, widely used) Apache 2.0 (Permissive with patent grant) GPL v3 (Copyleft, requires derivative works to be open source) Made with ❤️ at TinkerHub