Skip to content

Latest commit

 

History

History
103 lines (73 loc) · 3.6 KB

README.md

File metadata and controls

103 lines (73 loc) · 3.6 KB

SelfSup

Collections of self-supervised methods (MoCo series, SimCLR, SiMo, BYOL, SimSiam, SwAV, PointContrast, etc.).

Get Started

Install cvpods following the instructions.

Install cvpods from https://github.com/Megvii-BaseDetection/cvpods.git .

Prepare Datasets

cd cvpods
ln -s /path/to/your/ImageNet datasets/imagenet

Train your own models

cd /path/to/your/SelfSup/examples/simclr/simclr.res50.scratch.imagenet.224size.256bs.200e
# pre-train
pods_train --num-gpus 8
# convert to weights
python convert.py simclr.res50.scratch.imagenet.224size.256bs.200e/log/model_final.pth weights.pkl
# downstream evaluation
cd /path/to/your/simclr.res50.scratch.imagenet.224size.256bs.200e.lin_cls
pods_train --num-gpus 8 MODEL.WEIGHTS /path/to/your/weights.pkl

Model Zoo

Supervised Classification

ImageNet

Methods Training Schedule Top 1 Acc
Res50 100e 76.4

CIFAR 10

Methods Training Schedule Top 1 Acc
Res50 200e 95.4

STL 10

Methods Training Schedule Top 1 Acc
Res50 150e 86.1

Self-Supervised Learning - Classification

All results in the below table are trained using resnet-50 and reported on the ILSVRC2012 dataset.

Methods Training Schedule Batch Size Our Acc@1 Official Acc@1
MoCo 200e 256 60.5 60.5
MoCov2 200e 256 67.6 67.5
SimCLR 200e 256 63.2 61.9
SimCLR* 200e 256 67.3 Ours
SiMo 200e 256 68.1 Ours
SimSiam 100e 256 67.6 67.7
SwAV 200e 256 73.0 72.7
BYOL 100e 2048 69.8 66.5 (bs4096 from SimSiam paper)
BarlowTwins 300e 1024 Comming Soon 71.7

Self-Supervised Learning - Detection (2D)

All the results reported below are trained on ILSVRC2012 and evaluated on MS COCO using Faster-RCNN-FPN and resnet-50.

Methods Training Schedule Batch Size Box AP
SCRL 200 4096 39.9 ( official: 40.5 with bs 8192)
DetCon 200 256 Comming Soon.

Self-Supervised Learning - 3D Scene Understanding

Methods Training Schedule Downstream task
PointContrast - Comming Soon.

Citation

SelfSup is a part of cvpods, so if you find this repo useful in your research, or if you want to refer the implementations in this repo, please consider cite:

@article{zhu2020eqco,
  title={EqCo: Equivalent Rules for Self-supervised Contrastive Learning},
  author={Zhu, Benjin and Huang, Junqiang and Li, Zeming and Zhang, Xiangyu and Sun, Jian},
  journal={arXiv preprint arXiv:2010.01929},
  year={2020}
}

@misc{zhu2020cvpods,
  title={cvpods: All-in-one Toolbox for Computer Vision Research},
  author={Zhu*, Benjin and Wang*, Feng and Wang, Jianfeng and Yang, Siwei and Chen, Jianhu and Li, Zeming},
  year={2020}
}