This code release accompanies the following paper:
Daphne Barretto; Collaborators: Nate Simon, Jimmy Wu, Anirudha Majumdar, Szymon Rusinkiewicz
Abstract: Autonomous heterogeneous multi-robot systems have the potential to drastically improve the success rates of search and rescue missions, by decreasing the dependence on human efforts and thereby decreasing response times. In this work, we build upon spatial action maps -- action representations aligned with state representations for more efficient learning and improved performance -- to learn collaborative behaviors for autonomous heterogeneous multi-robot search and rescue via reinforcement learning. We develop and experiment with new robot types with various capabilities and new maps that communicate and analyze relevant information about robot location and rescue target location in the state representation. We successfully demonstrate learning collaborative behaviors for autonomous heterogeneous multi-robot search and rescue, including coordinated robot navigation for effective searching and reduced collisions, and coordinated responses from robots with rescue action capabilities to navigate to rescue targets found by other robots. We analyze the learned behaviors from each map, and we report quantitative performance improvements when using maps communicating robot location and rescue target location. We demonstrate the benefits of robots limited to search when effective collaborative behaviors are learned, and the benefits of diverse capabilities, particularly between unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs), by showing scenarios where heterogeneous multi-robot systems can outperform homogeneous ones. We conduct robustness and generalization tests on the method of using spatial action maps for autonomous heterogeneous multi-robot search and rescue, by considering new environments, numbers of rescue targets, and robot configurations. We demonstrate that the method is robust and that models generalize fairly well to unseen scenarios under some limitations. Finally, we propose future work that demonstrates scenarios where heterogeneous multi-robot systems use learned collaborative behaviors to consistently perform better than homogeneous ones. Overall, this work provides insight into learning collaborative behaviors for autonomous heterogeneous multi-robot search and rescue using spatial action maps, with potential implications for real-world search and rescue.