-
Notifications
You must be signed in to change notification settings - Fork 148
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1461 from LJ-Hao/docusaurus-version
add:pose_based_light_control_with_nodered_and_rpi_with_aikit
- Loading branch information
Showing
2 changed files
with
178 additions
and
1 deletion.
There are no files selected for viewing
176 changes: 176 additions & 0 deletions
176
...ons/Computer-Vision/pose_based_light_control_with_nodered_and_rpi_with_aikit.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| --- | ||
| description: This wiki demonstrates how to use pose estimation with yolov8 to control your lights. | ||
| title: Pose-Based Light Control with Node-Red and Raspberry Pi with AIkit | ||
| keywords: | ||
| - Edge | ||
| - reComputer r1000 | ||
| - Object detecton | ||
| - AI kit | ||
| image: https://files.seeedstudio.com/wiki/wiki-platform/S-tempor.png | ||
| slug: /pose_based_light_control_with_nodered_and_rpi_with_aikit | ||
| last_update: | ||
| date: 07/26/2024 | ||
| author: Jiahao | ||
|
|
||
| no_comments: false # for Disqus | ||
| --- | ||
|
|
||
| # Pose-Based Light Control with Node-Red and Raspberry Pi with AIkit | ||
|
|
||
| ## Introduction | ||
| This wiki will guide you on how to run YOLOv8 using an AI kit, use YOLOv8 to monitor your posture, and ultimately control your lights based on your posture. In this project, a USB camera captures your pose, and yolov8n run on [AI kit](https://www.seeedstudio.com/Raspberry-Pi-AI-Kit-p-5900.html) with [reComputer R1000](https://www.seeedstudio.com/reComputer-R1000-Series-Optional-Accessories.html) to detect your pose. The processed video, displaying the detected pose, is then streamed in real-time to [reTerminal DM](https://www.seeedstudio.com/reTerminal-DM-CM4104032-p-5898.html) using [gstreamer](https://gstreamer.freedesktop.org/). Meanwhile, the joint coordinates are sent using [mqtt](https://mqtt.org/) to [Node-RED](https://nodered.org/) which deploy on the reComputer R1000. At last, the Node-RED flow controls the smart lights based on the joint coordinates. | ||
|
|
||
| <p style={{textAlign: 'center'}}><img src="https://files.seeedstudio.com/wiki/reComputer-R1000/pose_control_light/pose_control.jpeg" alt="pir" width={1000} height="auto"/></p> | ||
|
|
||
| ## Prepare Hardware | ||
|
|
||
| <div class="table-center"> | ||
| <table align="center"> | ||
| <tr> | ||
| <th>reComputer r1000</th> | ||
| <th>reTerminal DM</th> | ||
| <th>Raspberry Pi AI Kit</th> | ||
| </tr> | ||
| <tr> | ||
| <td><div style={{textAlign:'center'}}><img src="https://media-cdn.seeedstudio.com/media/catalog/product/cache/bb49d3ec4ee05b6f018e93f896b8a25d/1/-/1-113991274-recomputer-r1025-10-0.jpg" style={{width:600, height:'auto'}}/></div></td> | ||
| <td><div style={{textAlign:'center'}}><img src="https://media-cdn.seeedstudio.com/media/catalog/product/cache/bb49d3ec4ee05b6f018e93f896b8a25d/s/k/sku114070262.jpg" style={{width:600, height:'auto'}}/></div></td> | ||
| <td><div style={{textAlign:'center'}}><img src="https://media-cdn.seeedstudio.com/media/catalog/product/cache/bb49d3ec4ee05b6f018e93f896b8a25d/1/-/1-113060086-raspberry-pi-ai-kit-45font.jpg" style={{width:600, height:'auto'}}/></div></td> | ||
| </tr> | ||
| <tr> | ||
| <td><div class="get_one_now_container" style={{textAlign: 'center'}}> | ||
| <a class="get_one_now_item" href="https://www.seeedstudio.com/reComputer-R1000-Series-Optional-Accessories.html"> | ||
| <strong><span><font color={'FFFFFF'} size={"4"}> Get One Now 🖱️</font></span></strong> | ||
| </a> | ||
| </div></td> | ||
| <td><div class="get_one_now_container" style={{textAlign: 'center'}}> | ||
| <a class="get_one_now_item" href="https://www.seeedstudio.com/reTerminal-DM-CM4104032-p-5898.html"> | ||
| <strong><span><font color={'FFFFFF'} size={"4"}> Get One Now 🖱️</font></span></strong> | ||
| </a> | ||
| </div></td> | ||
| <td><div class="get_one_now_container" style={{textAlign: 'center'}}> | ||
| <a class="get_one_now_item" href="https://www.seeedstudio.com/Raspberry-Pi-AI-Kit-p-5900.html"> | ||
| <strong><span><font color={'FFFFFF'} size={"4"}> Get One Now 🖱️</font></span></strong> | ||
| </a> | ||
| </div></td> | ||
| </tr> | ||
| </table> | ||
| </div> | ||
|
|
||
| ## Run this project | ||
|
|
||
| ### On reComputer R1000 | ||
|
|
||
| #### Step 1: Install AI kit | ||
|
|
||
| Pleses refer to [this wiki](https://wiki.seeedstudio.com/yolov8_object_detection_on_recomputer_r1000_with_hailo_8l/) and make sure you install the AI kit to your reComputer R1000. | ||
|
|
||
| #### Step 2: Install project | ||
|
|
||
| Download the project with the command below: | ||
| ``` | ||
| git clone https://github.com/LJ-Hao/Pose-Based_Light_Control_with_Node-Red_and_Raspberry_Pi_with_AIkit.git && cd Pose-Based_Light_Control_with_Node-Red_and_Raspberry_Pi_with_AIkit | ||
| ``` | ||
|
|
||
| #### Step 3: Run pose estimation | ||
|
|
||
| Edit the ```pose_estimation.py``` file and change the ```mqtt_server``` to your reTerminal DM's IP address. And change the ```Gstreamer pipeline``` to your reTerminal DM's IP address. And then run the command below to start the pose estimation. | ||
|
|
||
| ``` | ||
| bash run.sh pose_estimation.py | ||
| ``` | ||
|
|
||
| #### Step 4: Install Node-RED | ||
|
|
||
| Install Node-RED. You can download Node-RED with just one line of command: | ||
|
|
||
| ``` | ||
| bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered) | ||
| ``` | ||
|
|
||
| #### Step 5: Import flows.json | ||
|
|
||
| Import project files. Click ```Settings => Import``` in the upper right corner, ```click select a file``` to import in the new pop-up window, select ```flows.jeson```, and finally click ```Import```. | ||
|
|
||
| <p style={{textAlign: 'center'}}><img src="https://files.seeedstudio.com/wiki/reComputer-R1000/pose_control_light/nodered_import.gif" alt="pir" width={1000} height="auto"/></p> | ||
|
|
||
| #### Step 6: Download the required controls | ||
|
|
||
| After importing the project file, the system may prompt that your node-red is missing some controls. You need to download the following controls: | ||
|
|
||
| ``` | ||
| @bartbutenaers/node-red-rtsp-to-mjpeg | ||
| @chirpstack/node-red-contrib-chirpstack | ||
| @flowfuse/node-red-dashboard | ||
| node-bacnet-contrib-extended | ||
| node-red-contrib-buffer-parser | ||
| node-red-contrib-image-output | ||
| node-red-contrib-image-tools | ||
| node-red-contrib-modbus | ||
| node-red-contrib-msg-speed | ||
| node-red-contrib-multipart-stream-encoder | ||
| node-red-contrib-onvif | ||
| node-red-contrib-play-audio | ||
| node-red-dashboard | ||
| node-red-node-base64 | ||
| node-red-node-mysql | ||
| node-red-node-pi-gpio | ||
| node-red-node-ping | ||
| node-red-node-serialport | ||
| node-red-node-smooth | ||
| node-red-sensecap-paas | ||
| node-red-node-random | ||
| ``` | ||
|
|
||
| Take ```node-red-node-random``` as an example, click ```Settings => Manage Palette``` in the upper right corner, select the ```Install``` column in the new window, enter ```node-red-node-random```, and finally click ```Install``` Button. | ||
|
|
||
| <p style={{textAlign: 'center'}}><img src="https://files.seeedstudio.com/wiki/reComputer-R1000/pose_control_light/nodered_dowload_patelle.gif" alt="pir" width={1000} height="auto"/></p> | ||
|
|
||
| #### Step 7: Deploy the project | ||
|
|
||
| Click the deploy button in the upper right corner and the system will run normally. | ||
|
|
||
| <p style={{textAlign: 'center'}}><img src="https://files.seeedstudio.com/wiki/reComputer-R1000/pose_control_light/nodered_deploy.png" alt="pir" width={1000} height="auto"/></p> | ||
|
|
||
| ### On reTerminal DM | ||
|
|
||
| #### Step 1: Install the gstreamer on reTerminal DM | ||
|
|
||
| Download the gstreamer with the command below: | ||
|
|
||
| ``` | ||
| sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio | ||
| ``` | ||
|
|
||
| #### Step 2: Install the repository | ||
|
|
||
| Download the repository with the command below: | ||
| ``` | ||
| git clone https://github.com/LJ-Hao/Pose-Based_Light_Control_with_Node-Red_and_Raspberry_Pi_with_AIkit.git && cd Pose-Based_Light_Control_with_Node-Red_and_Raspberry_Pi_with_AIkit | ||
| ``` | ||
| #### Step 3: Run video reciver | ||
|
|
||
| ``` | ||
| python3 video_receiver.py | ||
| ``` | ||
|
|
||
| ## Result | ||
|
|
||
| We showed this demonstration in a youtube live broadcast which begin at ```19:47```. And you can see the light will turn on when the host clap her hands, then the light will turn off when the host wave her hand, enjoy this demo: | ||
|
|
||
| <div align="center"> | ||
| <iframe width="800" height="400" src="https://www.youtube.com/embed/v-S6_tH75NI" title="Making Next Gadget: AI Boosted RPi for Real World Applications" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| </div> | ||
|
|
||
| ## Tech Support & Product Discussion | ||
|
|
||
| Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs. | ||
|
|
||
| <div class="button_tech_support_container"> | ||
| <a href="https://forum.seeedstudio.com/" class="button_forum"></a> | ||
| <a href="https://www.seeedstudio.com/contacts" class="button_email"></a> | ||
| </div> | ||
|
|
||
| <div class="button_tech_support_container"> | ||
| <a href="https://discord.gg/eWkprNDMU7" class="button_discord"></a> | ||
| <a href="https://github.com/Seeed-Studio/wiki-documents/discussions/69" class="button_discussion"></a> | ||
| </div> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters