3D Multi-Object Tracker

This project is developed for tracking multiple objects in 3D scene. The visualization code is from here.

Features

• Fast: currently, the codes can achieve 700 FPS using only CPU (not include detection and data op), can perform tracking on all kitti val sequence in several seconds.
• Support online, near online and global implementation. The overall framework of design is shown below:

Kitti Results

Car/Pedestrian tracking results on the Kitti tracking val seq [1,6,8,10,12,13,14,15,16,18,19] using second-iou, point-rcnn and pv-rcnn detections. We also followed the HOTA metric, and tuned the parameters by first considering the HOTA performance.

Detector	online	near online (latency=2s)	global
point-rcnn	76.653	77.576	78.853
second-iou	77.29/48.011	78.17/50.362	78.799/51.20
pv-rcnn	78.289/49.662	79.48/50.824	80.075/51.753

Online: CA-based KF + greedy matching. Near online: online + rescoring tracks in a temporal window. Global: online + rescoring tracks globally.

Prepare data

You can download the Kitti tracking pose data from here, and you can download the point-rcnn, second-iou and pv-rcnn detections from here.

To run this code, you should organize Kitti tracking dataset as below:

# Kitti Tracking Dataset       
└── kitti_tracking
       ├── testing 
       |      ├──calib
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      ├──image_02
       |      |    ├──0000
       |      |    ├──....
       |      |    └──0028
       |      ├──pose
       |      |    ├──0000
       |      |    |    └──pose.txt
       |      |    ├──....
       |      |    └──0028
       |      |         └──pose.txt
       |      ├──label_02
       |      |    ├──0000.txt
       |      |    ├──....txt
       |      |    └──0028.txt
       |      └──velodyne
       |           ├──0000
       |           ├──....
       |           └──0028      
       └── training # the structure is same as testing set
              ├──calib
              ├──image_02
              ├──pose
              ├──label_02
              └──velodyne 

Detections

└── point-rcnn
       ├── training
       |      ├──0000
       |      |    ├──000001.txt
       |      |    ├──....txt
       |      |    └──000153.txt
       |      ├──...
       |      └──0020
       └──testing 

Requirements

python3
numpy
opencv
yaml

Quick start

• Please modify the dataset path and detections path in the yaml file to your own path.
• Then run python3 kitti_3DMOT.py config/online/pvrcnn_mot.yaml
• The results are automatically saved to evaluation/results/sha_key/data, and evaluated by HOTA metrics.

Notes

The evaluation codes are copied from Kitti.