ResNet strikes back: An improved training procedure in timm

[mingqiJ]

Hi @rwightman

I saw in ResNet strikes back: An improved training procedure in timm, authors test for ResNet50 with A2 training procedure when considering 100 distinct seeds.

Does anyone know where we can find the pre-trained weight of these ResNet50(using the same training procedure with different seeds). I was recently testing models with different seeds, and at the time our GPU resources did not support training so many models.

It would be great if some kind person could provide some models that have already been trained (one model using the same training procedure with different seeds). Any models are okay, especially for ResNet50, Swin-T, LeViT-256, DeiT-S, etc.... Hopefully, they're all trained on ImageNet-1K.

I would be very grateful if anyone would like to help.

[utkuozbulak]

Hey, were you able to find weights of the any resnet-50 models benchmarked in the paper?

[hankyul2]

Hi @utkuozbulak

Here is the weight link for the Resnet Strikes Back paper.

Thank you

hankyul

[utkuozbulak]

Thanks a lot!

[abdksyed]

Any idea what all these different weights correspond to?

I tried searching for the meaning, but couldn't get anything.

[hankyul2]

Hi @abdksyed

The file names pattern indicates for [ModelName]_[TrainProcedure]-[hash], which means the ModelName is trained in TrainProcedure.
For example, the weight of resent50_c2-d01e05b2.pth means the resent50 model trained in c2 training procedures.
The details of each training procedure of b, c, and d, could be found in the appendix of the ResNet Strikes Back paper.

Thank you.

Hankyul

[rwightman]

This might help a bit https://huggingface.co/timm/seresnextaa101d_32x8d.sw_in12k_ft_in1k_288#model-comparison

All of the ResNet weights have been pushed to the hub, I spent some time disentangling weight sources as I setup pretrained tags for them. There is a method in the madness. I still need to document better but there is some high level info if you click into the different models via that comparison table... as @hankyul2 the suffixes in the original weight files relates to the pretrain recipe for timm trained models. That's now moved to the 'tags' where models with multi-weight support use model_arch.tag as the identifier once converted (backwards compat to the first tag ocurrence when you just use model_arch).