Added min confidence threshold to existing trackers

[Fleyderer]

Hello! I've added some fixes to existing trackers according to new detections with conf >= 0.01

1. BotSort - already has this hyperparam, HOTA: 68.504 -> 68.251 (-0.3)
2. ByteTrack - added min_conf instead of existing 0.1 const. HOTA: 66.536 -> 67.619 (+1.1)
3. DeepOcSort - added small bug fix (idk how did it work before). HOTA: 62.913 -> 67.348 (+4.4)
4. HybridSort - didn't change, because it is still broken
5. ImprAssOc - already has min_conf, HOTA 64.096 -> 63.699 (-0.4). With changing this min_conf to higher values, tracker has better results, but I didnt change their default parameter
6. OcSort - added min_conf instead of existing 0.1 const. HOTA: 65.187 -> 66.441 (+1.3)
7. StrongSort - didn't have any threshold, I've added one which gives best results. HOTA: 68.329 -> 67.394 (-1)

Results are a bit strange at the first glance, but are much more correct according to papers with code benchmark. For example, in this table OcSort < StrongSort < DeepOcSort, and our results now look better. Botsort and Boosttrack are the best, the same as in table.

[Fleyderer]

Meh, I don't know why Cannot install fonttools. or build-system.requires and how is this error related to new min_conf values in trackers 🥲

[mikel-brostrom]

DeepOCSort (+4.4) 😮

[mikel-brostrom]

Just some random lib installation conflicts apparently

[mikel-brostrom]

botsort     ✅  68.77   77.176  80.997  46
strongsort  ✅  68.642  76.409  81.242  17
bytetrack   ✅  66.773  76.929  77.874  126
ocsort      ✅  65.469  74.934  75.983  1483
imprassoc   ✅  64.36   76.578  71.903  26
deepocsort  ✅  63.18   74.474  73.496  12
boosttrack  ✅  59.799  63.659  68.256

[mikel-brostrom]

After all the merges I am still here. With a low evaluation score for boosttrack. I am wondering if you are using a different set of embs and dets?

[mikel-brostrom]

I updated the runs.zip file in the pipeline, with the one you provided here: #1853

[Fleyderer]

After all the merges I am still here. With a low evaluation score for boosttrack. I am wondering if you are using a different set of embs and dets?

That's so strange, because I did it now from scratch:

0. Deleted all previous detections
1. Downloaded runs.zip which I have published, unzipped it
2. Ran command for bytetrack:

python3 tracking/val.py --classes 0 --yolo-model yolox_x_ablation.pth --reid-model osnet_x1_0_dukemtmcreid.pt --tracking-method bytetrack --verbose --source ./tracking/val_utils/data/MOT17-50/train

And got:

{"HOTA": 67.619, "MOTA": 78.081, "IDF1": 79.188}

Then, ran command for boosttrack:

python3 tracking/val.py --classes 0 --yolo-model yolox_x_ablation.pth --reid-model osnet_x1_0_dukemtmcreid.pt --tracking-method boosttrack --verbose --source ./tracking/val_utils/data/MOT17-50/train

And got:

{"HOTA": 68.649, "MOTA": 76.042, "IDF1": 81.923}

Please, try to run commands I provided and check if they are the same as in benchmark.yml. I could reproduce my results one to one.

[Fleyderer]

botsort     ✅  68.77   77.176  80.997  46
strongsort  ✅  68.642  76.409  81.242  17
bytetrack   ✅  66.773  76.929  77.874  126
ocsort      ✅  65.469  74.934  75.983  1483
imprassoc   ✅  64.36   76.578  71.903  26
deepocsort  ✅  63.18   74.474  73.496  12
boosttrack  ✅  59.799  63.659  68.256

I see that the table hasn't changed, so it looks like the new detections weren't used. I think you'll first need to reproduce my results one-to-one manually by downloading the runs.zip, unzipping it and running the commands - basically to see the new results and trust that the detections have changed 😄. And second, run the benchmark and see what might be wrong. It looks like the old runs.zip is being used.

[mikel-brostrom]

Ahaaa, now I get it. The folder names inside the runs.zip changed

[mikel-brostrom]

Fixing this

[mikel-brostrom]

And therefore it was generating dets and embs from scratch. Now everything is fixed

[mikel-brostrom]

Updating the README when new results are ready

[mikel-brostrom]

Success!! 🚀

[mikel-brostrom]

Thanks for all your contributions during the past days @Fleyderer! ❤️