A pipeline for estimating human attention toward objects with on-board cameras on the iCub humanoid robot
This is the official repository of the paper titled "A pipeline for estimating human attention toward objects with on-board cameras on the iCub humanoid robot".
This research report introduces a learning system designed to detect the object that humans are gazing at, using solely visual feedback. By incorporating face detection, human attention prediction, and online object detection the system enables the robot to perceive and interpret human gaze accurately, thereby facilitating the establishment of joint attention with human partners. Additionally, a novel dataset collected with the humanoid robot iCub is introduced, comprising over 22, 000 images from ten participants gazing at different annotated objects. This dataset serves as a benchmark for human gaze estimation in table-top human-robot interaction (HRI) contexts. In this work, we use it to assess the proposed pipeline’s performance and examine each component’s effectiveness. Furthermore, the developed system is deployed on the iCub, and a supplementary video showcases its functionality. The results demonstrate the potential of the proposed approach as a first step to enhance social awareness and responsiveness in social robotics. This advancement can enhance assistance and support in collaborative scenarios, promoting more efficient human-robot collaborations.
The proposed dataset depicts human-robot interactions in a table-top scenario where the human partner gazes at different objects. The ObjectAttention dataset consists of 250 videos (22, 732 frames), depicting 10 participants in 2 different trials for 5 sessions, gazing at the different objects.
The following modules are required for the proposed pipeline.
This is the requirement of Human Attention Estimation pathway to detect anatomical key-points. Please follow the instructions provided in the OpenPose repository for proper installation.
This is the erequirement of Object Detection pathway to detect objects. Please follow the instructions provided in the Online detection repository for proper installation.