Skip to content

Experiments (Katya)

Jump to bottom
Ekaterina Glazkova edited this page May 14, 2020 · 1 revision

DANN: Dropout usage in heads with AlexNet and ResNet backbones

Experiments setting:

  • BATCH_SIZE = 32, N_EPOCHS = 200, STEPS_PER_EPOCH = 20, DATASET = "office-31", SOURCE_DOMAIN = "amazon", TARGET_DOMAIN = "webcam"
EXP name model backbone domain head adaptation block usage domain dropout class dropout Source best acc Target best acc Source final acc Target final acc
Alexnet_vanilla AlexNet vanilla-dann true false false 1.0 0.59375 0.9989 0.48698
Alexnet_domain_dropout AlexNet dropout_dann true true false 1.0 0.61719 1.0 0.58203
Alexnet_domain_and_class_dropout AlexNet dropout_dann true true true 1.0 0.62370 1.0 0.59375
Resnet_vanilla ResNet50 vanilla-dann false false false 0.99751 0.78255 0.99503 0.72786
Resnet_domain_dropout ResNet50 dropout_dann false true false 1.0 0.85286 1.0 0.80339
Resnet_domain_and_class_dropout ResNet50 dropout_dann false true true 1.0 0.85156 1.0. 0.84766

AlexNet loss and accuracy plot: Alexnet_dropout

ResNet loss and accuracy plot: resnet_dropout

Conclusion: Dropout helps to avoid overfitting (see target accuracy plots), the final metrics with dropout are better.

Strange thing: in AlexNet dropout helps better (all metrics of Alexnet_domain_and_class_dropout are better than other AlexNet options, but with ResNet nest accuracy of Resnet_domain_and_class_dropout is worse than Resnet_domain_dropout).

Why and what to do to investigate:

  1. Is that because extra domain adaptation module usage in AlexNet? ToDo: Add domain adaptation block to ResNet
  2. Is that just some training unstability? ToDo: Repeat the same experiments several times and check.
  3. Because different layers ratio is freezed? ToDo: Check that the same ratio of layers is freezed

DANN: preprocessing for ResNet backbone

In original ResNet paper resize and crop is used as preprocessing. Check if it helps to improve training.

  • BATCH_SIZE = 32, MODEL_BACKBONE = "resnet50", DOMAIN_HEAD = "dropout-dann", class dropout = True, N_EPOCHS = 200, STEPS_PER_EPOCH = 20, DATASET = "office-31", SOURCE_DOMAIN = "amazon", TARGET_DOMAIN = "webcam", no adaptation block is used
EXP name Source best acc Target best acc Source final acc Target final acc
resnet50_no_crop 1.0 0.85156 1.0 0.84766
resnet50_with_crop 1.0 0.83984 1.0 0.82161

Loss and accuracy plots: resnet_crop

Conclusion: crop does not help.

Clone this wiki locally