Because I want my SfM point-cloud model to be properly aligned with my CAD Model canonically so I can render stuff relative to the canonical pose of my CAD model.
- Added pipeline for realtime visualization.
- Added SuperPoint rotation limtation fix.
OnePose: One-Shot Object Pose Estimation without CAD Models
Jiaming Sun*, Zihao Wang*, Siyu Zhang*, Xingyi He, Hongcheng Zhao, Guofeng Zhang, Xiaowei Zhou
CVPR 2022
Refer to the original repository for setup instructions.
Images used to construct the SfM point-cloud model are generated from Blender using the desired CAD Model. Camera intrinsics and object pose (relative to the camera) are collected and processed directly using Blender's python interface. The 3D bounding box is also manually annotated in Blender, and the reprojected 2D bounding box is collected after some processing using Blender's python interface.
The 3D model is obtained using an Artec 3D scanner with some postprocessing using Artec's software. NVIDIA MoMa is a good and cheap alternative if you cannot get your hands on a high-end 3D scanner.
Combined with the graph attention network, SuperPoint descriptors have a rotation limitation of about 45 degrees; Tilting further than 45 degress on all axes will result in the loss of 2D-3D correspondences.
To solve this limitation, the grayscale input to SuperPoint is rotated before being passed through SuperPoint. Subsequently, the 2D coordinates of each interest point are unrotated back to the input's original orientation.
Modified for private use-case.
- Added YoloV5 detector, weights located at data/yolov5/
- Added realtime scripts cv2_real_time_improved.py and cv2_real_time_improved.sh
- Added a modified feature-matching object detector (modified for realtime, but too slow)
- Added Open3D Pipeline for RGB-D Cameras as realsense.py, requires realsense cameras.
- Synthetic data generation with Blender and NVIDIA MoMa setup.
This repository uses work from YoloV5 and OnePose, neither of which I am originally involved in.