naoqi-pose-retargeting

This repository provides scripts to capture 3D human pose using Mediapipe Pose and retarget it onto Pepper and Nao robots using the NAOqi SDK. It works alongside the receiver and uses parts of the code for retargeting and socket communication from this repository.

Installation and Usage

To install, use a Python 3.8 environment and do

pip install -r requirements.txt

To analyze a video and visualize the resulting joint angles, do:

python teleop.py --video VIDEOFILE --fps 10 --plot_angle_trace

To teleoperate Pepper from video, start pepper_gui.py from this fork of the original code (within Python 2.7), and then execute

python teleop.py --video VIDEOFILE --fps 10 --enable_teleop

To teleoperate Pepper from webcam, use

python teleop.py --enable_teleop

Notes

Mediapipe Landmark mapping (source).

OpenPose to Mediapipe Body Mapping

body_mapping = {'0':  "Nose",      -> 0
                '1':  "Neck",      -? 11+12

                '2':  "RShoulder", -> 12
                '3':  "RElbow",    -> 14
                '4':  "RWrist",    -> 16

                '5':  "LShoulder", -> 11
                '6':  "LElbow",    -> 13
                '7':  "LWrist",    -> 15

                '8':  "MidHip"      -> 23+24}

Todo

• Fix hip movement
• Write qiBullet simulation part