[CVPRW 2024] Improving Object Detection to Fisheye Cameras with Open-Vocabulary Pseudo-Label Approach
This repository contains the source code for AI City Challenge 2024 Track 4 (Road Object Detection in Fish-Eye Cameras).
- Team Name: SKKU-AutoLab
- Team ID: 05
The project directory should look like this:
aicity_2024_fisheye8k
|__ data
| |__ aicity
| |__ aicity_2024_fisheye8k
| |__ test
| | |__ <camera_id>
| | | |__ images
| | | |__ labels (Pseudo-labels from YOLO-World)
| | ...
| |__ train
| | |__ <camera_id>
| | | |__ images
| | | |__ labels
| | ...
| |__ train_syn (Synthesis nighttime images)
| | |__ <camera_id>
| | | |__ images
| | | |__ labels
| | ...
| |__ val
| | |__ <camera_id>
| | | |__ images
| | | |__ labels
| | ...
| |__ val_syn ((Synthesis nighttime images))
| | |__ <camera_id>
| | | |__ images
| | | |__ labels
| | ...
| ...
|__ project
| |__ aicity_2024_fisheye8k
| |__ config
| |__ run
| | |__ predict
| | | |__ aicity_2024_fisheye8k
| | | |__ submission
| | | |__ results.json (Final submission results)
| | |__ train
| |__ gen_submission.py
| |__ run_predict.sh
| |__ run_train.sh
|__ src
| |__ cyclegan_pix2pix
| |__ mon
| |__ yolo_world
| |__ yolor
|__ zoo (Pretrained weights)
The code was implemented and tested on Ubuntu 22.04, Python 3.11 and PyTorch (>=v1.13.0) with conda environment.
- The
data
folder can be download here: Google Drive - The
zoo
folder can be download here: Google Drive - Download the repo and install the dependencies:
cd <to_the_repo_directory>
chmod +x install.sh
./install.sh
apt-get install ffmpeg
apt-get install '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev
# Check installation
conda activate mon
python
import mon # Should not raise any error
Enter the Docker container and run the following commands:
conda activate mon
cd aicity_2024_fisheye8k/project/aicity_2024_fisheye8k
./run_predict.sh
The final submission results will be saved in: aicity_2024_fisheye8k/project/aicity_2024_fisheye8k/run/predict/aicity_2024_fisheye8k/submission/results.json
.
Enter the Docker container and run the following commands:
- Install the environment:
cd aicity_2024_fisheye8k
./install_cycle_gan.sh
It can take a while to install all dependencies.
- To train the model, please run as bellow:
conda activate cyclegan
cd aicity_2024_fisheye8k/project/style_transfer
./train.sh
All training models and training results will be stored in: run/train/cyclegan_pix2pix
To view the training results per epoch, please look for the html
file in: run/train/cyclegan_pix2pix
directory.
- To generate synthesis dataset, run the
infer.sh
script:
conda activate cyclegan
cd aicity_2024_fisheye8k/project/style_transfer
./infer.sh
Synthesis images and new synthetic dataset will be generated automatically. All generated images and additional datasets will be stored in run/generate
- Install the environment:
cd aicity_2024_fisheye8k
./install_yolo_world.sh
It can take a while to install all dependencies.
- To train the model, please run as bellow:
conda activate yolo_world
cd aicity_2024_fisheye8k/project/yolo_world
./train.sh
All training models and training results will be stored in: run/train/yolo_world
- For inferencing, run the
predict.sh
script:
conda activate yolo_world
cd aicity_2024_fisheye8k/project/yolo_world
./predict.sh
New annotations and results will be generated and stored in /run/predict/yolo_world
- To train the model, please run as bellow:
conda activate mopn
cd aicity_2024_fisheye8k/project/aicity_2024_fisheye8k
./run_train_1280.sh
./run_train_1536.sh
./run_train_1920.sh
All training models and training results will be stored in: run/train/...
If you have any questions, feel free to contact Long H. Pham
(longpham3105@gmail.com or phlong@skku.edu)