[Perception] Detect and classify obstacles

[gxywang]

Label obstacles for interaction purposes (avoid craters, navigate to berm, etc)

Classes: (rocks & column) vs (craters) vs (berms)

Steps:

1. Write 3D detection algorithm
2. Simulate RGB-D sensor on simple robot with a “crater” and “rocks”

• Use nav+sim’s gazebo world?
• Lidar sensor assuming our pointcloud reconstruction is aight; Or directly get camera frames?

3. Test algorithm on simulation
4. Reroute nodes to connect with realsense node & rtabmap
5. Test algorithm in reality (floor with something vaguely round for rock; trash can for crater)
6. Addtl: Conditional activation in excavation (& construction?) zone

Helpful Resources:

• (nvidia)
• (blog tutorial / github)
• (YOLO basics)

Detection Algorithm Ideas:

• Craters will have an edge in the ground with -y direction, can look for that via depth sensor input from Realsense
• Berm will be created by us, can: mark area we want to build berm in (same start condition as mechanical dump), and draw contour after finished “dumping” current amt of sand
• Rocks & column will have +y direction; whenever there is an occlusion that isn’t a berm (or in the obstacle zone), mark it as a rock/column

[gxywang]

Navigation team says RTAB-Map seems to have decent obstacle detection in simulation, so may not be necessary. Verify with more testing