This repo contains the 5th place solution for the TAU Vehicle Type Recognition Competition on Kaggle which was organized by Tampere University in 2019. The competition was a mandatory part of Pattern Recognition and Machine Learning course.
- Anna Iashina (LinkedIn)
- Einary Vaaras (LinkedIn)
- Vladimir Iashin (LinkedIn)
- Maral Zadehdarrehshoorian (LinkedIn)
For the two submissions for the private leaderboard, we decided to use the best model on the local validation and the best on the public leaderboard.
The best model on local validation was a finetuned ResNeXt 101 WSL (32x16d) pre-trained on weak labels from Instagram images. We finetuned it on 90 % of the dataset with mild augmentation. This model solely gave us 92.75 on local validation (10 % of the dataset) and 90.8 on public lb (92.1 private (5th place, the same)). We decided not to use this one for the final prediction as we were afraid of the "lb shake-up" as well as this 90/10 validation scheme wasn't tested properly and we didn't want to have an overfitted model in our final set of submissions for private lb since 10 % was still a quite small portion of data even though the split was stratified. So, instead, we used the same model but which was trained on 75/25 split (local: 92.3, public: 91.0, private: 91.45).
The final submission (91.88 private, 92.22 public) which you see on the private leaderboard is the majority vote of many models from previous submissions. We believe, the models we chose for the blend were orthogonal enough to each other. Specifically, it was a blend of three: another result of ResNeXt 101 WSL (32x16d) (local: 92.65); DenseNet (local: 91.45); and fused prediction of other 5 models (sklearn classifiers on ResNet features, Inception v3, Resnext 101 WSL, Efficientnet, Resnet 101).
Validation: stratified split 75/25 (we didn't do K-Fold CV to save time but you can add one loop to the code easily).
What didn't work:
- Weighted sampler nor loss
- Two-stage training: when the model weights are frozen and only the last layer is trained and, after, the whole model is finetuned with a lower lr
- AdamW optimizer (the same results)
You can find the details on how to use the code to replicate the solution in each folder of this repo. This is a brief description of what they contain.
The folder contains the initial Tensorflow (Keras) solutions including the ResNet101 + sklearn models which we used for the final blend and a bunch of pre-trained models with ImageNet weights. All the solutions reach up to 0.88 accuracy on validation (0.86 on public and private LB).
Here we provide a code (PyTorch) for finetuning a ResNeXt 101 WSL model which we found to achieve the best performance among all tested models. The best model was ResNeXt 101 WSL (32x16d) which gave 0.9275 on local validation (0.908 on public and 0.921 on private LB).
This part contains the code for blending the predictions from several models. Our best model on private LB (final submission) was a majority vote of the second-best model on local validation (ResNeXt 101 WSL), DenseNet, and the best model on a public LB at that time.