1. Pytorch implementation SphereVLAD and SphereVLAD++ proposed in the papers:
SeqSphereVLAD: Sequence Matching Enhanced Orientation-invariant Place Recognition
Peng Yin, Fuying Wang, Anton Egorov, Jiafan Hou, Ji Zhang, Howie Choset
SphereVLAD++: Attention-based and Signal-enhanced Viewpoint Invariant Descriptor
Shiqi Zhao, Peng Yin*, Ge Yi, Sebastian Scherer
2. Core library for our iSimLoc, AutoMerge and BioSLAM works:
iSimLoc: Visual Global Localization for Previously Unseen Environments With Simulated Images
Peng Yin; Ivan Cisneros; Shiqi Zhao; Ji Zhang; Howie Choset; Sebastian Scherer
AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale Environments
Peng Yin; Shiqi Zhao; Haowen Lai; Ruohai Ge; Ji Zhang; Howie Choset; Sebastian Scherer
BioSLAM: A Bioinspired Lifelong Memory System for General Place Recognition
Peng Yin; Abulikemu Abuduweili; Shiqi Zhao; Lingyun Xu; Changliu Liu; Sebastian Scherer
System
The codes are tested in:
- Ubuntu20.04+CUDA11.6+python3.8.10
- Ubuntu18.04+CUDA10.2+python3.6.9
Python Environment Dependency
- s2cnn
- torch==1.13.0+cu116 torchaudio==0.13.0+cu116 torchvision==0.14+cu116
- cupy-cuda11x==11.6.0
It is highly recommended too create a virtual environment by conda or virtualenv.
# create virtual env
virtualenv /path_to_env/spherevlad -p /usr/bin/python3
source /path_to_env/spherevlad/bin/activate
# or
conda create -n spherevlad python==3.8
conda activate spherevlad
# install pytorch and cupy
pip3 install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116
pip3 install cupy-cuda11x==11.6.0
# install s2cnn
pip3 install pynvrtc joblib
mkdir third && cd third
git clone https://github.com/jonkhler/s2cnn.git
cd s2cnn
python setup.py install
# install all others packages
pip3 install -r requirements.txtData Structure
├── data
├── results
│ └── model_name
│ ├── DATA.yaml
│ ├── MODEL.yaml
│ ├── log.txt
│ └── pth
│ ├── model_xxx.pth
│ ├── ...
└── dataset
├── PITT
│ ├── DATA
│ │ ├── Train1
│ │ │ ├── cloudGlobal.pcd
│ │ │ └── poses.txt
│ │ ├── ...
│ │ └── Trainxx
Note
Modify the PYTHONPATH environment variable to include absolute path to the project root folder:
export PYTHONPATH=$PATHONPATH:/path_to_spherevlad/SphereVLAD
Data Preparation
Download the sample data from Dropbox and put unzipped folder in the /data/PITT. The data structure should be like the same mentioned in Data Structure section.
mkdir data
python generating_sph/gene_pitt.py --config-network=config/network/spherevlad.yaml --config-dataset=config/dataset/PITT.yamlTraining on Alita Urban Dataset
# spherevlad
python3 train_lcd.py --config-network=config/network/spherevlad.yaml --config-dataset=config/dataset/PITT.yaml
# spherevlad2
python3 train_lcd.py --config-network=config/network/spherevlad2.yaml --config-dataset=config/dataset/PITT.yaml-
If there are issues with the s2cnn, please download the version we have modified and tested.
-
The generated weight will be saved at /data/results/SphereVLAD-{DatasetName}-{DD}-{MM}-{YYYY}-{TIME}, the folder also contains the log and configuration of the model and dataset.
Testing on Alita Urban Dataset
Pretrained weight on Alita Urban Dataset: Dropbox. Please unzip it and put in /data/results
Please change the trajectory numbers in line 45 and line 68 in eval/evaluate_pitt.py first.
# spherevlad
python eval/evaluate_pitt.py --model-path=data/results/SphereVLAD --epoch=666 --noise=1 --type=recall --trans-noise=2 --rot-noise=180 --log=False
# spherevlad2
python eval/evaluate_pitt.py --model-path=data/results/SphereVLAD2 --epoch=666 --noise=1 --type=recall --trans-noise=2 --rot-noise=180 --log=False- --model-path: the folder contains the configuration and trained weights
- --epoch: the epoch of trained weights
- Modify the dataloader/pittsburgh.py to fit the data structure of your own dataset
- Modify the config/dataset/PITT.yaml to change the configuration of the dataset
- Add new evaluation methods in eval/eval_utils.py and import it in train_lcd.py
@INPROCEEDINGS{9341727,
author={Yin, Peng and Wang, Fuying and Egorov, Anton and Hou, Jiafan and Zhang, Ji and Choset, Howie},
booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
title={SeqSphereVLAD: Sequence Matching Enhanced Orientation-invariant Place Recognition},
year={2020},
volume={},
number={},
pages={5024-5029},
doi={10.1109/IROS45743.2020.9341727}}
@ARTICLE{9956017,
author={Zhao, Shiqi and Yin, Peng and Yi, Ge and Scherer, Sebastian},
journal={IEEE Robotics and Automation Letters},
title={SphereVLAD++: Attention-Based and Signal-Enhanced Viewpoint Invariant Descriptor},
year={2023},
volume={8},
number={1},
pages={256-263},
doi={10.1109/LRA.2022.3223555}}
@article{yin2022alita,
title={Alita: A large-scale incremental dataset for long-term autonomy},
author={Yin, Peng and Zhao, Shiqi and Ge, Ruohai and Cisneros, Ivan and Fu, Ruijie and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
journal={arXiv preprint arXiv:2205.10737},
year={2022}
}
@ARTICLE{10035430,
author={Yin, Peng and Cisneros, Ivan and Zhao, Shiqi and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
journal={IEEE Transactions on Robotics},
title={iSimLoc: Visual Global Localization for Previously Unseen Environments With Simulated Images},
year={2023},
volume={39},
number={3},
pages={1893-1909},
doi={10.1109/TRO.2023.3238201}}
@ARTICLE{10203034,
author={Yin, Peng and Zhao, Shiqi and Lai, Haowen and Ge, Ruohai and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
journal={IEEE Transactions on Robotics},
title={AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale Environments},
year={2023},
volume={},
number={},
pages={1-19},
doi={10.1109/TRO.2023.3290448}}
@ARTICLE{10242249,
author={Yin, Peng and Abuduweili, Abulikemu and Zhao, Shiqi and Xu, Lingyun and Liu, Changliu and Scherer, Sebastian},
journal={IEEE Transactions on Robotics},
title={BioSLAM: A Bioinspired Lifelong Memory System for General Place Recognition},
year={2023},
volume={},
number={},
pages={1-20},
doi={10.1109/TRO.2023.3306615}}