Question related to RayIoU #5
Description
Hi again,
sorry for the many issues!
I wanted to ask how to properly evaluate the RayIoU metrics (Table A.10 of the supplementary material). I tried running:
bash tools/dist_test_ray.sh configs/gdfusion/alocc_3d_r50_256x704_bevdet_preatrain_gdfusion.py ckpts/alocc_3d_r50_256x704_bevdet_preatrain_gdfusion.pth 1
But the results are different from the ones reported in the paper:
+----------------------+----------+----------+----------+
| Class Names | RayIoU@1 | RayIoU@2 | RayIoU@4 |
+----------------------+----------+----------+----------+
| others | 0.125 | 0.131 | 0.133 |
| barrier | 0.454 | 0.496 | 0.515 |
| bicycle | 0.250 | 0.292 | 0.300 |
| bus | 0.534 | 0.633 | 0.694 |
| car | 0.547 | 0.618 | 0.647 |
| construction_vehicle | 0.257 | 0.342 | 0.376 |
| motorcycle | 0.266 | 0.325 | 0.342 |
| pedestrian | 0.351 | 0.402 | 0.424 |
| traffic_cone | 0.340 | 0.368 | 0.380 |
| trailer | 0.259 | 0.345 | 0.431 |
| truck | 0.487 | 0.572 | 0.612 |
| driveable_surface | 0.274 | 0.429 | 0.631 |
| other_flat | 0.159 | 0.227 | 0.291 |
| sidewalk | 0.172 | 0.235 | 0.316 |
| terrain | 0.131 | 0.211 | 0.322 |
| manmade | 0.413 | 0.506 | 0.571 |
| vegetation | 0.297 | 0.408 | 0.503 |
+----------------------+----------+----------+----------+
| MEAN | 0.313 | 0.385 | 0.440 |
+----------------------+----------+----------+----------+
{'RayIoU': 0.3792673555788003, 'RayIoU@1': 0.3126522095066655, 'RayIoU@2': 0.3847022912096031, 'RayIoU@4': 0.44044756602013213}