Caution
Commercial use of RoScenes is strictly forbidden.
[2024-07-14] You can now download the dataset at ModelScope.
[2024-07-13] Devkit for RoScenes released.
[2024-07-01] Paper accepted to ECCV 2024! 🥳
[2024-05-28] Please stay tuned for the updates! We are doing final checks on data privacy.
Note
Please refer to ModelScope for downloading the dataset.
After download and extract, the dataset folder should be organized as follows:
. [DATA_ROOT] # Dataset root folder
├── 📂train # training set
│ ├── 📂s001_split_train_difficulty_mixed_ambience_day # scene 001's data
│ │ ├── 📂database # annotations, grouped by clip
│ │ │ ├── 📂0076fd69_clip_[0000000000000-0000000029529] # a clip's database, please use our devkit to read
│ │ │ └ ...
│ │ └── 📂images # images, grouped by clips
│ │ ├── 📂0076fd69
│ │ └ ...
│ ├── 📂s002_split_train_difficulty_mixed_ambience_day
│ ├── 📂s003_split_train_difficulty_mixed_ambience_day
│ ├── 📂s004_split_train_difficulty_mixed_ambience_day
│ └── 📂night_split_train_difficulty_mixed_ambience_night
│
│
├── 📂validation # validation set
│ ├── 📂s001_split_validation_difficulty_mixed_ambience_day # scene 001's data
│ ├── 📂s002_split_validation_difficulty_mixed_ambience_day
│ ├── 📂s003_split_validation_difficulty_mixed_ambience_day
│ ├── 📂s004_split_validation_difficulty_mixed_ambience_day
│ └── 📂night_split_validation_difficulty_mixed_ambience_night
│
│
└── 📂test # test set
├── 📂NO_GTs005_split_test_difficulty_mixed_ambience_day # scene 005's data
├── 📂NO_GTs006_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs007_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs008_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs009_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs010_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs011_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs012_split_test_difficulty_mixed_ambience_day
├── 📂NO_GTs013_split_test_difficulty_mixed_ambience_day
└── 📂NO_GTs014_split_test_difficulty_mixed_ambience_day
Use PyPI to directly install RoScenes devkit:
pip install roscenesAlso, you can clone this repository and install roscenes manually for developing.
git clone https://github.com/roscenes/RoScenes.git
cd RoScenes
pip install -e .import roscenes as ro
# load the training set
dataset = ro.load('[DATA_ROOT]/train/*')
# number of total frames
print(len(dataset))Then, we can iterate over the dataset, in two ways:
You can use indexing:
# use integer indexing
index = 10
# a Frame instance
print(type(dataset[index]))
for i in range(len(dataset)):
# print num of objects for every frame
print(len(dataset[index].boxes3D))OR, you can directly iterate it:
# a frame instance
for frame in dataset:
print(len(frame.boxes3D))Important
Please refer to frame.py, camera.py for the detailed comments on box format, instrinsic and extrinsic definition, etc.
python -m roscenes.visualizer [DATA_ROOT]/train/s001_split_train_difficulty_mixed_ambience_day 0 vis_result
A example in companion with mmdet3d is given in examples.
Please go to that folder README.md for details.
Note that we use nuScenes detection score (NDS) to evaluate performance. Additionally, we provide an optimized implementation of NDS which could significantly speed-up calculation. It should produce identical result with the official NDS under same inputs. Benchmark is shown below:
| Input frames | Boxes/Frame | Original nuScenes devkit | Ours |
|---|---|---|---|
| 360 | 109 | 1697s | 118s |
Unfortunately, since the code is built from scratch, our evaluation suite can not be an in-place replacement to the original one ---- You need to modify the code. To take evaluation, please refer to the example given in examples/mmdet3d/mmdet3d_plugin/datasets/roscenes_dataset.py and the README file in roscenes/evaluation/README.md.
- Devkit release
- Dataset release
- Example dataset loader based on
MMDetection3D - 3D detection task and evaluation suite
- 3D tracking task and evaluation suite