-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathHCT_MSRVTT_1kB.txt
2444 lines (2444 loc) · 141 KB
/
HCT_MSRVTT_1kB.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Experiment directory: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB
Preparing the dataloaders ...
Loading dataset MSRVTT_miech_trainval in ram ...
Finish loading dataset MSRVTT_miech_trainval in ram, taking 379.74728417396545 s.
Loading dataset MSRVTT_miech_test in ram ...
Finish loading dataset MSRVTT_miech_test in ram, taking 66.88046956062317 s.
Loading dataset MSRVTT_miech_test in ram ...
Finish loading dataset MSRVTT_miech_test in ram, taking 47.25347709655762 s.
Training ...
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch0.pth ...
Done in 1.574s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch0.pth ...
Done in 3.097s
epoch : 0
loss : 0
learning_rate : 5e-05
n_samples : 0
n_steps : 0
MSRVTT_miech_test/t2v_metrics/R1: 0.2
MSRVTT_miech_test/t2v_metrics/R5: 0.6
MSRVTT_miech_test/t2v_metrics/R10: 0.9
MSRVTT_miech_test/t2v_metrics/R50: 5.2
MSRVTT_miech_test/t2v_metrics/MedR: 514.5
MSRVTT_miech_test/t2v_metrics/MeanR: 503.203
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 0.47622031559045985
MSRVTT_miech_test/v2t_metrics/R1: 0.1
MSRVTT_miech_test/v2t_metrics/R5: 0.3
MSRVTT_miech_test/v2t_metrics/R10: 1.1
MSRVTT_miech_test/v2t_metrics/R50: 4.9
MSRVTT_miech_test/v2t_metrics/MedR: 496.5
MSRVTT_miech_test/v2t_metrics/MeanR: 503.3395
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 0.32075343299958264
mnt_best : 0.47622031559045985
not_improved_count: 0
Train Epoch: 1 [1/250 128/32000 (0%)] Loss: 9.81301 batch_time=20.68054
Train Epoch: 1 [12/250 1536/32000 (5%)] Loss: 8.51052 batch_time=0.35663
Train Epoch: 1 [23/250 2944/32000 (9%)] Loss: 7.10561 batch_time=0.49005
Train Epoch: 1 [34/250 4352/32000 (14%)] Loss: 6.79951 batch_time=0.34142
Train Epoch: 1 [45/250 5760/32000 (18%)] Loss: 6.68773 batch_time=0.35577
Train Epoch: 1 [56/250 7168/32000 (22%)] Loss: 6.07005 batch_time=0.38790
Train Epoch: 1 [67/250 8576/32000 (27%)] Loss: 5.62990 batch_time=0.37594
Train Epoch: 1 [78/250 9984/32000 (31%)] Loss: 5.77235 batch_time=0.36710
Train Epoch: 1 [89/250 11392/32000 (36%)] Loss: 5.97230 batch_time=0.35833
Train Epoch: 1 [100/250 12800/32000 (40%)] Loss: 5.23799 batch_time=0.37515
Train Epoch: 1 [111/250 14208/32000 (44%)] Loss: 5.34634 batch_time=0.38769
Train Epoch: 1 [122/250 15616/32000 (49%)] Loss: 5.04708 batch_time=0.39643
Train Epoch: 1 [133/250 17024/32000 (53%)] Loss: 4.75130 batch_time=0.41639
Train Epoch: 1 [144/250 18432/32000 (58%)] Loss: 4.62108 batch_time=0.37549
Train Epoch: 1 [155/250 19840/32000 (62%)] Loss: 4.55384 batch_time=0.36827
Train Epoch: 1 [166/250 21248/32000 (66%)] Loss: 4.65518 batch_time=0.36450
Train Epoch: 1 [177/250 22656/32000 (71%)] Loss: 4.51432 batch_time=0.34794
Train Epoch: 1 [188/250 24064/32000 (75%)] Loss: 5.37140 batch_time=0.35830
Train Epoch: 1 [199/250 25472/32000 (80%)] Loss: 4.36436 batch_time=0.35655
Train Epoch: 1 [210/250 26880/32000 (84%)] Loss: 3.68632 batch_time=0.36921
Train Epoch: 1 [221/250 28288/32000 (88%)] Loss: 3.82443 batch_time=0.37674
Train Epoch: 1 [232/250 29696/32000 (93%)] Loss: 4.69323 batch_time=0.35044
Train Epoch: 1 [243/250 31104/32000 (97%)] Loss: 3.83129 batch_time=0.38771
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch1.pth ...
Done in 4.005s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch1.pth ...
Done in 7.866s
epoch : 1
loss : 5.336124703407288
learning_rate : 5e-05
n_samples : 32000
n_steps : 250
MSRVTT_miech_test/t2v_metrics/R1: 13.3
MSRVTT_miech_test/t2v_metrics/R5: 37.4
MSRVTT_miech_test/t2v_metrics/R10: 51.2
MSRVTT_miech_test/t2v_metrics/R50: 83.1
MSRVTT_miech_test/t2v_metrics/MedR: 10.0
MSRVTT_miech_test/t2v_metrics/MeanR: 41.338
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 29.42147227418173
MSRVTT_miech_test/v2t_metrics/R1: 14.0
MSRVTT_miech_test/v2t_metrics/R5: 38.3
MSRVTT_miech_test/v2t_metrics/R10: 51.5
MSRVTT_miech_test/v2t_metrics/R50: 82.5
MSRVTT_miech_test/v2t_metrics/MedR: 10.0
MSRVTT_miech_test/v2t_metrics/MeanR: 40.6855
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 30.225814513962185
mnt_best : 29.42147227418173
not_improved_count: 0
Train Epoch: 2 [1/250 128/32000 (0%)] Loss: 3.61214 batch_time=27.56297
Train Epoch: 2 [12/250 1536/32000 (5%)] Loss: 4.35404 batch_time=0.41324
Train Epoch: 2 [23/250 2944/32000 (9%)] Loss: 3.78257 batch_time=0.38241
Train Epoch: 2 [34/250 4352/32000 (14%)] Loss: 4.13511 batch_time=0.37122
Train Epoch: 2 [45/250 5760/32000 (18%)] Loss: 3.75864 batch_time=0.35476
Train Epoch: 2 [56/250 7168/32000 (22%)] Loss: 3.49039 batch_time=0.35366
Train Epoch: 2 [67/250 8576/32000 (27%)] Loss: 3.84900 batch_time=0.34348
Train Epoch: 2 [78/250 9984/32000 (31%)] Loss: 3.93844 batch_time=0.35036
Train Epoch: 2 [89/250 11392/32000 (36%)] Loss: 3.67693 batch_time=0.35193
Train Epoch: 2 [100/250 12800/32000 (40%)] Loss: 3.48052 batch_time=0.36386
Train Epoch: 2 [111/250 14208/32000 (44%)] Loss: 3.45724 batch_time=0.36282
Train Epoch: 2 [122/250 15616/32000 (49%)] Loss: 3.97946 batch_time=0.35633
Train Epoch: 2 [133/250 17024/32000 (53%)] Loss: 3.21649 batch_time=5.28163
Train Epoch: 2 [144/250 18432/32000 (58%)] Loss: 3.55057 batch_time=0.34521
Train Epoch: 2 [155/250 19840/32000 (62%)] Loss: 2.97193 batch_time=0.38915
Train Epoch: 2 [166/250 21248/32000 (66%)] Loss: 3.47757 batch_time=0.35260
Train Epoch: 2 [177/250 22656/32000 (71%)] Loss: 3.28672 batch_time=0.35835
Train Epoch: 2 [188/250 24064/32000 (75%)] Loss: 4.07319 batch_time=0.35577
Train Epoch: 2 [199/250 25472/32000 (80%)] Loss: 3.22883 batch_time=0.35478
Train Epoch: 2 [210/250 26880/32000 (84%)] Loss: 3.57827 batch_time=0.35424
Train Epoch: 2 [221/250 28288/32000 (88%)] Loss: 3.22640 batch_time=0.58992
Train Epoch: 2 [232/250 29696/32000 (93%)] Loss: 3.72927 batch_time=0.35080
Train Epoch: 2 [243/250 31104/32000 (97%)] Loss: 3.50107 batch_time=0.35036
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch2.pth ...
Done in 3.855s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch2.pth ...
Done in 7.591s
removing stale ckpt [epoch 1] [took 0.03s]
removing stale ckpt [epoch 0] [took 0.02s]
epoch : 2
loss : 3.668350088119507
learning_rate : 4.75e-05
n_samples : 64000
n_steps : 500
MSRVTT_miech_test/t2v_metrics/R1: 16.8
MSRVTT_miech_test/t2v_metrics/R5: 42.7
MSRVTT_miech_test/t2v_metrics/R10: 58.0
MSRVTT_miech_test/t2v_metrics/R50: 85.0
MSRVTT_miech_test/t2v_metrics/MedR: 8.0
MSRVTT_miech_test/t2v_metrics/MeanR: 35.319
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 34.65147427662521
MSRVTT_miech_test/v2t_metrics/R1: 15.7
MSRVTT_miech_test/v2t_metrics/R5: 43.6
MSRVTT_miech_test/v2t_metrics/R10: 58.6
MSRVTT_miech_test/v2t_metrics/R50: 86.0
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 33.3435
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 34.2316567911028
mnt_best : 34.65147427662521
not_improved_count: 0
Train Epoch: 3 [1/250 128/32000 (0%)] Loss: 3.85604 batch_time=25.80228
Train Epoch: 3 [12/250 1536/32000 (5%)] Loss: 3.32972 batch_time=0.34142
Train Epoch: 3 [23/250 2944/32000 (9%)] Loss: 3.33273 batch_time=0.33935
Train Epoch: 3 [34/250 4352/32000 (14%)] Loss: 3.12736 batch_time=0.38966
Train Epoch: 3 [45/250 5760/32000 (18%)] Loss: 3.37505 batch_time=0.63811
Train Epoch: 3 [56/250 7168/32000 (22%)] Loss: 3.24445 batch_time=0.35001
Train Epoch: 3 [67/250 8576/32000 (27%)] Loss: 3.35771 batch_time=2.83152
Train Epoch: 3 [78/250 9984/32000 (31%)] Loss: 3.31402 batch_time=0.36971
Train Epoch: 3 [89/250 11392/32000 (36%)] Loss: 3.14861 batch_time=0.39184
Train Epoch: 3 [100/250 12800/32000 (40%)] Loss: 2.98134 batch_time=0.39607
Train Epoch: 3 [111/250 14208/32000 (44%)] Loss: 3.41648 batch_time=0.38271
Train Epoch: 3 [122/250 15616/32000 (49%)] Loss: 3.08556 batch_time=0.53938
Train Epoch: 3 [133/250 17024/32000 (53%)] Loss: 3.49060 batch_time=0.36224
Train Epoch: 3 [144/250 18432/32000 (58%)] Loss: 3.48905 batch_time=0.37703
Train Epoch: 3 [155/250 19840/32000 (62%)] Loss: 3.09768 batch_time=0.37647
Train Epoch: 3 [166/250 21248/32000 (66%)] Loss: 2.67791 batch_time=0.35881
Train Epoch: 3 [177/250 22656/32000 (71%)] Loss: 2.97187 batch_time=0.39742
Train Epoch: 3 [188/250 24064/32000 (75%)] Loss: 2.57511 batch_time=0.40423
Train Epoch: 3 [199/250 25472/32000 (80%)] Loss: 3.10334 batch_time=0.38667
Train Epoch: 3 [210/250 26880/32000 (84%)] Loss: 2.51827 batch_time=0.34492
Train Epoch: 3 [221/250 28288/32000 (88%)] Loss: 2.76552 batch_time=0.36238
Train Epoch: 3 [232/250 29696/32000 (93%)] Loss: 3.54261 batch_time=0.35045
Train Epoch: 3 [243/250 31104/32000 (97%)] Loss: 3.60128 batch_time=0.34462
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch3.pth ...
Done in 3.915s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch3.pth ...
Done in 7.680s
removing stale ckpt [epoch 2] [took 0.01s]
epoch : 3
loss : 3.067719319343567
learning_rate : 4.5125e-05
n_samples : 96000
n_steps : 750
MSRVTT_miech_test/t2v_metrics/R1: 19.2
MSRVTT_miech_test/t2v_metrics/R5: 45.7
MSRVTT_miech_test/t2v_metrics/R10: 58.5
MSRVTT_miech_test/t2v_metrics/R50: 86.5
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 34.046
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 37.164169453788205
MSRVTT_miech_test/v2t_metrics/R1: 18.4
MSRVTT_miech_test/v2t_metrics/R5: 47.4
MSRVTT_miech_test/v2t_metrics/R10: 59.7
MSRVTT_miech_test/v2t_metrics/R50: 86.9
MSRVTT_miech_test/v2t_metrics/MedR: 6.0
MSRVTT_miech_test/v2t_metrics/MeanR: 31.7985
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 37.34136292707899
mnt_best : 37.164169453788205
not_improved_count: 0
Train Epoch: 4 [1/250 128/32000 (0%)] Loss: 2.90370 batch_time=25.20949
Train Epoch: 4 [12/250 1536/32000 (5%)] Loss: 2.74206 batch_time=0.57948
Train Epoch: 4 [23/250 2944/32000 (9%)] Loss: 2.97002 batch_time=0.36488
Train Epoch: 4 [34/250 4352/32000 (14%)] Loss: 3.05718 batch_time=0.35592
Train Epoch: 4 [45/250 5760/32000 (18%)] Loss: 2.89057 batch_time=0.41745
Train Epoch: 4 [56/250 7168/32000 (22%)] Loss: 3.38286 batch_time=0.38073
Train Epoch: 4 [67/250 8576/32000 (27%)] Loss: 3.02573 batch_time=0.42150
Train Epoch: 4 [78/250 9984/32000 (31%)] Loss: 3.04091 batch_time=0.38127
Train Epoch: 4 [89/250 11392/32000 (36%)] Loss: 2.82917 batch_time=0.36843
Train Epoch: 4 [100/250 12800/32000 (40%)] Loss: 2.80626 batch_time=0.35677
Train Epoch: 4 [111/250 14208/32000 (44%)] Loss: 3.08599 batch_time=0.36959
Train Epoch: 4 [122/250 15616/32000 (49%)] Loss: 2.68519 batch_time=0.37094
Train Epoch: 4 [133/250 17024/32000 (53%)] Loss: 2.41285 batch_time=0.35928
Train Epoch: 4 [144/250 18432/32000 (58%)] Loss: 2.94896 batch_time=0.38713
Train Epoch: 4 [155/250 19840/32000 (62%)] Loss: 2.71765 batch_time=0.36409
Train Epoch: 4 [166/250 21248/32000 (66%)] Loss: 2.73569 batch_time=0.34302
Train Epoch: 4 [177/250 22656/32000 (71%)] Loss: 2.26930 batch_time=0.36711
Train Epoch: 4 [188/250 24064/32000 (75%)] Loss: 2.87302 batch_time=0.35586
Train Epoch: 4 [199/250 25472/32000 (80%)] Loss: 2.87631 batch_time=0.40082
Train Epoch: 4 [210/250 26880/32000 (84%)] Loss: 2.89010 batch_time=0.74819
Train Epoch: 4 [221/250 28288/32000 (88%)] Loss: 2.84118 batch_time=1.26823
Train Epoch: 4 [232/250 29696/32000 (93%)] Loss: 3.11151 batch_time=0.38761
Train Epoch: 4 [243/250 31104/32000 (97%)] Loss: 2.72281 batch_time=0.35587
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch4.pth ...
Done in 3.764s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch4.pth ...
Done in 7.905s
removing stale ckpt [epoch 3] [took 0.01s]
epoch : 4
loss : 2.715348472595215
learning_rate : 4.2868749999999995e-05
n_samples : 128000
n_steps : 1000
MSRVTT_miech_test/t2v_metrics/R1: 18.8
MSRVTT_miech_test/t2v_metrics/R5: 46.1
MSRVTT_miech_test/t2v_metrics/R10: 60.7
MSRVTT_miech_test/t2v_metrics/R50: 87.6
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 32.179
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 37.469896076938866
MSRVTT_miech_test/v2t_metrics/R1: 19.6
MSRVTT_miech_test/v2t_metrics/R5: 46.4
MSRVTT_miech_test/v2t_metrics/R10: 61.6
MSRVTT_miech_test/v2t_metrics/R50: 86.6
MSRVTT_miech_test/v2t_metrics/MedR: 6.0
MSRVTT_miech_test/v2t_metrics/MeanR: 30.426
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 38.26352013258008
mnt_best : 37.469896076938866
not_improved_count: 0
Train Epoch: 5 [1/250 128/32000 (0%)] Loss: 2.76511 batch_time=23.83909
Train Epoch: 5 [12/250 1536/32000 (5%)] Loss: 2.59770 batch_time=0.36498
Train Epoch: 5 [23/250 2944/32000 (9%)] Loss: 2.59699 batch_time=0.35230
Train Epoch: 5 [34/250 4352/32000 (14%)] Loss: 2.36201 batch_time=0.39964
Train Epoch: 5 [45/250 5760/32000 (18%)] Loss: 2.10042 batch_time=0.71424
Train Epoch: 5 [56/250 7168/32000 (22%)] Loss: 2.46977 batch_time=0.35665
Train Epoch: 5 [67/250 8576/32000 (27%)] Loss: 2.29154 batch_time=1.44292
Train Epoch: 5 [78/250 9984/32000 (31%)] Loss: 2.30084 batch_time=0.35144
Train Epoch: 5 [89/250 11392/32000 (36%)] Loss: 2.53296 batch_time=0.35741
Train Epoch: 5 [100/250 12800/32000 (40%)] Loss: 1.98317 batch_time=0.34547
Train Epoch: 5 [111/250 14208/32000 (44%)] Loss: 2.25765 batch_time=0.41091
Train Epoch: 5 [122/250 15616/32000 (49%)] Loss: 2.39943 batch_time=0.39978
Train Epoch: 5 [133/250 17024/32000 (53%)] Loss: 2.39108 batch_time=0.34812
Train Epoch: 5 [144/250 18432/32000 (58%)] Loss: 2.54602 batch_time=0.56155
Train Epoch: 5 [155/250 19840/32000 (62%)] Loss: 2.66904 batch_time=0.36861
Train Epoch: 5 [166/250 21248/32000 (66%)] Loss: 2.36406 batch_time=0.35526
Train Epoch: 5 [177/250 22656/32000 (71%)] Loss: 2.49855 batch_time=0.38978
Train Epoch: 5 [188/250 24064/32000 (75%)] Loss: 2.55291 batch_time=0.41336
Train Epoch: 5 [199/250 25472/32000 (80%)] Loss: 2.43786 batch_time=0.34735
Train Epoch: 5 [210/250 26880/32000 (84%)] Loss: 2.16695 batch_time=0.41915
Train Epoch: 5 [221/250 28288/32000 (88%)] Loss: 1.94490 batch_time=0.35436
Train Epoch: 5 [232/250 29696/32000 (93%)] Loss: 2.47068 batch_time=0.34302
Train Epoch: 5 [243/250 31104/32000 (97%)] Loss: 2.36113 batch_time=0.37014
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch5.pth ...
Done in 3.520s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch5.pth ...
Done in 7.048s
removing stale ckpt [epoch 4] [took 0.03s]
epoch : 5
loss : 2.4457504491806032
learning_rate : 4.072531249999999e-05
n_samples : 160000
n_steps : 1250
MSRVTT_miech_test/t2v_metrics/R1: 18.7
MSRVTT_miech_test/t2v_metrics/R5: 48.0
MSRVTT_miech_test/t2v_metrics/R10: 61.1
MSRVTT_miech_test/t2v_metrics/R50: 87.8
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.1
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 37.99338758442951
MSRVTT_miech_test/v2t_metrics/R1: 20.0
MSRVTT_miech_test/v2t_metrics/R5: 48.9
MSRVTT_miech_test/v2t_metrics/R10: 62.6
MSRVTT_miech_test/v2t_metrics/R50: 89.1
MSRVTT_miech_test/v2t_metrics/MedR: 6.0
MSRVTT_miech_test/v2t_metrics/MeanR: 28.39
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 39.41283991493113
mnt_best : 37.99338758442951
not_improved_count: 0
Train Epoch: 6 [1/250 128/32000 (0%)] Loss: 2.24500 batch_time=25.84644
Train Epoch: 6 [12/250 1536/32000 (5%)] Loss: 2.34721 batch_time=0.34611
Train Epoch: 6 [23/250 2944/32000 (9%)] Loss: 2.59372 batch_time=0.36045
Train Epoch: 6 [34/250 4352/32000 (14%)] Loss: 2.36703 batch_time=0.34155
Train Epoch: 6 [45/250 5760/32000 (18%)] Loss: 2.43131 batch_time=0.36063
Train Epoch: 6 [56/250 7168/32000 (22%)] Loss: 2.35656 batch_time=0.34924
Train Epoch: 6 [67/250 8576/32000 (27%)] Loss: 2.71568 batch_time=3.75456
Train Epoch: 6 [78/250 9984/32000 (31%)] Loss: 2.24728 batch_time=0.33973
Train Epoch: 6 [89/250 11392/32000 (36%)] Loss: 2.45869 batch_time=0.35651
Train Epoch: 6 [100/250 12800/32000 (40%)] Loss: 2.53544 batch_time=0.35322
Train Epoch: 6 [111/250 14208/32000 (44%)] Loss: 2.29201 batch_time=0.35302
Train Epoch: 6 [122/250 15616/32000 (49%)] Loss: 1.78285 batch_time=0.35075
Train Epoch: 6 [133/250 17024/32000 (53%)] Loss: 2.97119 batch_time=0.38638
Train Epoch: 6 [144/250 18432/32000 (58%)] Loss: 2.25519 batch_time=0.41799
Train Epoch: 6 [155/250 19840/32000 (62%)] Loss: 1.84983 batch_time=0.36999
Train Epoch: 6 [166/250 21248/32000 (66%)] Loss: 1.87494 batch_time=0.35663
Train Epoch: 6 [177/250 22656/32000 (71%)] Loss: 2.06321 batch_time=0.48792
Train Epoch: 6 [188/250 24064/32000 (75%)] Loss: 2.23616 batch_time=0.36447
Train Epoch: 6 [199/250 25472/32000 (80%)] Loss: 2.61769 batch_time=0.60031
Train Epoch: 6 [210/250 26880/32000 (84%)] Loss: 2.19221 batch_time=0.36128
Train Epoch: 6 [221/250 28288/32000 (88%)] Loss: 1.87513 batch_time=0.34885
Train Epoch: 6 [232/250 29696/32000 (93%)] Loss: 2.26768 batch_time=0.35480
Train Epoch: 6 [243/250 31104/32000 (97%)] Loss: 1.91976 batch_time=0.37817
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch6.pth ...
Done in 4.024s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch6.pth ...
Done in 7.921s
removing stale ckpt [epoch 5] [took 0.01s]
epoch : 6
loss : 2.2521901683807375
learning_rate : 3.868904687499999e-05
n_samples : 192000
n_steps : 1500
MSRVTT_miech_test/t2v_metrics/R1: 20.0
MSRVTT_miech_test/t2v_metrics/R5: 49.5
MSRVTT_miech_test/t2v_metrics/R10: 64.2
MSRVTT_miech_test/t2v_metrics/R50: 87.5
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.086
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 39.907703866056416
MSRVTT_miech_test/v2t_metrics/R1: 20.3
MSRVTT_miech_test/v2t_metrics/R5: 51.3
MSRVTT_miech_test/v2t_metrics/R10: 64.8
MSRVTT_miech_test/v2t_metrics/R50: 89.3
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 28.3025
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 40.71265918314143
mnt_best : 39.907703866056416
not_improved_count: 0
Train Epoch: 7 [1/250 128/32000 (0%)] Loss: 2.54986 batch_time=25.59043
Train Epoch: 7 [12/250 1536/32000 (5%)] Loss: 2.28502 batch_time=0.79748
Train Epoch: 7 [23/250 2944/32000 (9%)] Loss: 2.29780 batch_time=0.38928
Train Epoch: 7 [34/250 4352/32000 (14%)] Loss: 2.35498 batch_time=0.37872
Train Epoch: 7 [45/250 5760/32000 (18%)] Loss: 2.86906 batch_time=0.40055
Train Epoch: 7 [56/250 7168/32000 (22%)] Loss: 2.41225 batch_time=0.34520
Train Epoch: 7 [67/250 8576/32000 (27%)] Loss: 2.03384 batch_time=0.46874
Train Epoch: 7 [78/250 9984/32000 (31%)] Loss: 2.22494 batch_time=0.33941
Train Epoch: 7 [89/250 11392/32000 (36%)] Loss: 1.62496 batch_time=0.36469
Train Epoch: 7 [100/250 12800/32000 (40%)] Loss: 2.28709 batch_time=0.35659
Train Epoch: 7 [111/250 14208/32000 (44%)] Loss: 2.45548 batch_time=0.38084
Train Epoch: 7 [122/250 15616/32000 (49%)] Loss: 2.01827 batch_time=0.37023
Train Epoch: 7 [133/250 17024/32000 (53%)] Loss: 1.74968 batch_time=0.39923
Train Epoch: 7 [144/250 18432/32000 (58%)] Loss: 2.40829 batch_time=0.39514
Train Epoch: 7 [155/250 19840/32000 (62%)] Loss: 2.20810 batch_time=0.34429
Train Epoch: 7 [166/250 21248/32000 (66%)] Loss: 2.32850 batch_time=0.40551
Train Epoch: 7 [177/250 22656/32000 (71%)] Loss: 1.87997 batch_time=0.36995
Train Epoch: 7 [188/250 24064/32000 (75%)] Loss: 2.09166 batch_time=0.35532
Train Epoch: 7 [199/250 25472/32000 (80%)] Loss: 1.66215 batch_time=0.35591
Train Epoch: 7 [210/250 26880/32000 (84%)] Loss: 1.97489 batch_time=1.41443
Train Epoch: 7 [221/250 28288/32000 (88%)] Loss: 2.09368 batch_time=0.36149
Train Epoch: 7 [232/250 29696/32000 (93%)] Loss: 1.95802 batch_time=0.35101
Train Epoch: 7 [243/250 31104/32000 (97%)] Loss: 1.85928 batch_time=0.34752
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch7.pth ...
Done in 3.700s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch7.pth ...
Done in 7.597s
removing stale ckpt [epoch 6] [took 0.01s]
epoch : 7
loss : 2.1153629417419433
learning_rate : 3.675459453124999e-05
n_samples : 224000
n_steps : 1750
MSRVTT_miech_test/t2v_metrics/R1: 21.1
MSRVTT_miech_test/t2v_metrics/R5: 49.6
MSRVTT_miech_test/t2v_metrics/R10: 62.9
MSRVTT_miech_test/t2v_metrics/R50: 87.9
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.258
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 40.37739152636484
MSRVTT_miech_test/v2t_metrics/R1: 20.1
MSRVTT_miech_test/v2t_metrics/R5: 52.3
MSRVTT_miech_test/v2t_metrics/R10: 65.3
MSRVTT_miech_test/v2t_metrics/R50: 89.3
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.8125
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 40.94526087851158
mnt_best : 40.37739152636484
not_improved_count: 0
Train Epoch: 8 [1/250 128/32000 (0%)] Loss: 1.84155 batch_time=30.90837
Train Epoch: 8 [12/250 1536/32000 (5%)] Loss: 1.77638 batch_time=0.44871
Train Epoch: 8 [23/250 2944/32000 (9%)] Loss: 2.67367 batch_time=0.36634
Train Epoch: 8 [34/250 4352/32000 (14%)] Loss: 1.74566 batch_time=0.36206
Train Epoch: 8 [45/250 5760/32000 (18%)] Loss: 1.70479 batch_time=0.35004
Train Epoch: 8 [56/250 7168/32000 (22%)] Loss: 1.88148 batch_time=0.36151
Train Epoch: 8 [67/250 8576/32000 (27%)] Loss: 2.06548 batch_time=0.35078
Train Epoch: 8 [78/250 9984/32000 (31%)] Loss: 2.30791 batch_time=0.36576
Train Epoch: 8 [89/250 11392/32000 (36%)] Loss: 2.20003 batch_time=0.40387
Train Epoch: 8 [100/250 12800/32000 (40%)] Loss: 1.66321 batch_time=0.37135
Train Epoch: 8 [111/250 14208/32000 (44%)] Loss: 1.64435 batch_time=0.38223
Train Epoch: 8 [122/250 15616/32000 (49%)] Loss: 2.07288 batch_time=0.37424
Train Epoch: 8 [133/250 17024/32000 (53%)] Loss: 2.03468 batch_time=0.35324
Train Epoch: 8 [144/250 18432/32000 (58%)] Loss: 1.75662 batch_time=0.42129
Train Epoch: 8 [155/250 19840/32000 (62%)] Loss: 1.95899 batch_time=0.39876
Train Epoch: 8 [166/250 21248/32000 (66%)] Loss: 2.01601 batch_time=0.40502
Train Epoch: 8 [177/250 22656/32000 (71%)] Loss: 1.56369 batch_time=0.43747
Train Epoch: 8 [188/250 24064/32000 (75%)] Loss: 1.68275 batch_time=0.36494
Train Epoch: 8 [199/250 25472/32000 (80%)] Loss: 1.76722 batch_time=0.35294
Train Epoch: 8 [210/250 26880/32000 (84%)] Loss: 1.80824 batch_time=1.98193
Train Epoch: 8 [221/250 28288/32000 (88%)] Loss: 1.86952 batch_time=0.36627
Train Epoch: 8 [232/250 29696/32000 (93%)] Loss: 1.97603 batch_time=0.35454
Train Epoch: 8 [243/250 31104/32000 (97%)] Loss: 2.23706 batch_time=0.40867
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch8.pth ...
Done in 3.909s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch8.pth ...
Done in 7.490s
removing stale ckpt [epoch 7] [took 0.01s]
epoch : 8
loss : 1.9578331189155578
learning_rate : 3.4916864804687486e-05
n_samples : 256000
n_steps : 2000
MSRVTT_miech_test/t2v_metrics/R1: 21.9
MSRVTT_miech_test/t2v_metrics/R5: 49.5
MSRVTT_miech_test/t2v_metrics/R10: 63.8
MSRVTT_miech_test/t2v_metrics/R50: 87.8
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.207
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 41.04781057525799
MSRVTT_miech_test/v2t_metrics/R1: 21.5
MSRVTT_miech_test/v2t_metrics/R5: 52.5
MSRVTT_miech_test/v2t_metrics/R10: 65.1
MSRVTT_miech_test/v2t_metrics/R50: 89.1
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 27.739
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 41.88510263413009
mnt_best : 41.04781057525799
not_improved_count: 0
Train Epoch: 9 [1/250 128/32000 (0%)] Loss: 1.97703 batch_time=32.41846
Train Epoch: 9 [12/250 1536/32000 (5%)] Loss: 1.71466 batch_time=0.36277
Train Epoch: 9 [23/250 2944/32000 (9%)] Loss: 2.53815 batch_time=0.34792
Train Epoch: 9 [34/250 4352/32000 (14%)] Loss: 2.19617 batch_time=0.35620
Train Epoch: 9 [45/250 5760/32000 (18%)] Loss: 1.72506 batch_time=0.35117
Train Epoch: 9 [56/250 7168/32000 (22%)] Loss: 1.88625 batch_time=0.34325
Train Epoch: 9 [67/250 8576/32000 (27%)] Loss: 1.80783 batch_time=0.34028
Train Epoch: 9 [78/250 9984/32000 (31%)] Loss: 1.65418 batch_time=0.34302
Train Epoch: 9 [89/250 11392/32000 (36%)] Loss: 1.65969 batch_time=0.34160
Train Epoch: 9 [100/250 12800/32000 (40%)] Loss: 2.04401 batch_time=0.35243
Train Epoch: 9 [111/250 14208/32000 (44%)] Loss: 1.73167 batch_time=0.34526
Train Epoch: 9 [122/250 15616/32000 (49%)] Loss: 1.49793 batch_time=0.34860
Train Epoch: 9 [133/250 17024/32000 (53%)] Loss: 1.65518 batch_time=0.34149
Train Epoch: 9 [144/250 18432/32000 (58%)] Loss: 1.77520 batch_time=1.33533
Train Epoch: 9 [155/250 19840/32000 (62%)] Loss: 2.34728 batch_time=0.35103
Train Epoch: 9 [166/250 21248/32000 (66%)] Loss: 1.67099 batch_time=0.48517
Train Epoch: 9 [177/250 22656/32000 (71%)] Loss: 1.55370 batch_time=0.34308
Train Epoch: 9 [188/250 24064/32000 (75%)] Loss: 1.68859 batch_time=0.34277
Train Epoch: 9 [199/250 25472/32000 (80%)] Loss: 1.95727 batch_time=0.36295
Train Epoch: 9 [210/250 26880/32000 (84%)] Loss: 1.84181 batch_time=0.40015
Train Epoch: 9 [221/250 28288/32000 (88%)] Loss: 1.65784 batch_time=0.35776
Train Epoch: 9 [232/250 29696/32000 (93%)] Loss: 1.94232 batch_time=0.34747
Train Epoch: 9 [243/250 31104/32000 (97%)] Loss: 1.58709 batch_time=0.36713
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch9.pth ...
Done in 20.296s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch9.pth ...
Done in 23.994s
removing stale ckpt [epoch 8] [took 0.01s]
epoch : 9
loss : 1.8345636019706726
learning_rate : 3.317102156445311e-05
n_samples : 288000
n_steps : 2250
MSRVTT_miech_test/t2v_metrics/R1: 21.9
MSRVTT_miech_test/t2v_metrics/R5: 50.1
MSRVTT_miech_test/t2v_metrics/R10: 64.8
MSRVTT_miech_test/t2v_metrics/R50: 88.4
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.597
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 41.427203293248546
MSRVTT_miech_test/v2t_metrics/R1: 22.9
MSRVTT_miech_test/v2t_metrics/R5: 52.1
MSRVTT_miech_test/v2t_metrics/R10: 66.7
MSRVTT_miech_test/v2t_metrics/R50: 89.0
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 27.353
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.01299463039693
mnt_best : 41.427203293248546
not_improved_count: 0
Train Epoch: 10 [1/250 128/32000 (0%)] Loss: 1.87531 batch_time=28.08419
Train Epoch: 10 [12/250 1536/32000 (5%)] Loss: 2.03991 batch_time=0.36252
Train Epoch: 10 [23/250 2944/32000 (9%)] Loss: 2.17499 batch_time=0.39014
Train Epoch: 10 [34/250 4352/32000 (14%)] Loss: 2.20833 batch_time=0.35930
Train Epoch: 10 [45/250 5760/32000 (18%)] Loss: 1.55775 batch_time=0.37012
Train Epoch: 10 [56/250 7168/32000 (22%)] Loss: 1.84384 batch_time=0.36344
Train Epoch: 10 [67/250 8576/32000 (27%)] Loss: 1.95941 batch_time=0.35642
Train Epoch: 10 [78/250 9984/32000 (31%)] Loss: 2.34298 batch_time=0.42453
Train Epoch: 10 [89/250 11392/32000 (36%)] Loss: 1.41549 batch_time=0.36027
Train Epoch: 10 [100/250 12800/32000 (40%)] Loss: 1.62825 batch_time=0.36236
Train Epoch: 10 [111/250 14208/32000 (44%)] Loss: 1.97509 batch_time=0.37230
Train Epoch: 10 [122/250 15616/32000 (49%)] Loss: 1.47225 batch_time=0.35345
Train Epoch: 10 [133/250 17024/32000 (53%)] Loss: 1.91047 batch_time=4.62403
Train Epoch: 10 [144/250 18432/32000 (58%)] Loss: 1.78640 batch_time=0.38028
Train Epoch: 10 [155/250 19840/32000 (62%)] Loss: 1.64833 batch_time=0.55703
Train Epoch: 10 [166/250 21248/32000 (66%)] Loss: 1.34891 batch_time=0.36011
Train Epoch: 10 [177/250 22656/32000 (71%)] Loss: 1.44978 batch_time=0.37490
Train Epoch: 10 [188/250 24064/32000 (75%)] Loss: 1.55432 batch_time=0.38889
Train Epoch: 10 [199/250 25472/32000 (80%)] Loss: 1.58583 batch_time=0.34896
Train Epoch: 10 [210/250 26880/32000 (84%)] Loss: 1.55773 batch_time=0.35650
Train Epoch: 10 [221/250 28288/32000 (88%)] Loss: 1.59405 batch_time=0.35293
Train Epoch: 10 [232/250 29696/32000 (93%)] Loss: 2.00770 batch_time=0.35613
Train Epoch: 10 [243/250 31104/32000 (97%)] Loss: 1.64896 batch_time=0.35205
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch10.pth ...
Done in 3.798s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch10.pth ...
Done in 7.614s
removing stale ckpt [epoch 9] [took 0.00s]
epoch : 10
loss : 1.7579918656349183
learning_rate : 3.151247048623045e-05
n_samples : 320000
n_steps : 2500
MSRVTT_miech_test/t2v_metrics/R1: 23.0
MSRVTT_miech_test/t2v_metrics/R5: 49.4
MSRVTT_miech_test/t2v_metrics/R10: 63.5
MSRVTT_miech_test/t2v_metrics/R50: 87.9
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.358
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 41.63029643997359
MSRVTT_miech_test/v2t_metrics/R1: 21.9
MSRVTT_miech_test/v2t_metrics/R5: 52.0
MSRVTT_miech_test/v2t_metrics/R10: 65.3
MSRVTT_miech_test/v2t_metrics/R50: 89.7
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.7845
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 42.05202170648426
mnt_best : 41.63029643997359
not_improved_count: 0
Train Epoch: 11 [1/250 128/32000 (0%)] Loss: 1.47363 batch_time=26.51306
Train Epoch: 11 [12/250 1536/32000 (5%)] Loss: 1.58359 batch_time=0.34328
Train Epoch: 11 [23/250 2944/32000 (9%)] Loss: 1.79510 batch_time=0.34684
Train Epoch: 11 [34/250 4352/32000 (14%)] Loss: 1.66708 batch_time=0.36131
Train Epoch: 11 [45/250 5760/32000 (18%)] Loss: 1.63964 batch_time=1.08028
Train Epoch: 11 [56/250 7168/32000 (22%)] Loss: 1.54998 batch_time=0.35720
Train Epoch: 11 [67/250 8576/32000 (27%)] Loss: 1.49087 batch_time=0.36549
Train Epoch: 11 [78/250 9984/32000 (31%)] Loss: 1.70937 batch_time=0.40833
Train Epoch: 11 [89/250 11392/32000 (36%)] Loss: 1.29518 batch_time=0.37169
Train Epoch: 11 [100/250 12800/32000 (40%)] Loss: 1.84490 batch_time=0.34760
Train Epoch: 11 [111/250 14208/32000 (44%)] Loss: 2.07655 batch_time=0.35080
Train Epoch: 11 [122/250 15616/32000 (49%)] Loss: 1.89554 batch_time=0.34669
Train Epoch: 11 [133/250 17024/32000 (53%)] Loss: 1.35216 batch_time=0.43813
Train Epoch: 11 [144/250 18432/32000 (58%)] Loss: 1.85272 batch_time=0.38206
Train Epoch: 11 [155/250 19840/32000 (62%)] Loss: 1.95058 batch_time=0.37729
Train Epoch: 11 [166/250 21248/32000 (66%)] Loss: 1.51932 batch_time=0.40281
Train Epoch: 11 [177/250 22656/32000 (71%)] Loss: 1.99375 batch_time=0.35767
Train Epoch: 11 [188/250 24064/32000 (75%)] Loss: 1.46431 batch_time=0.35537
Train Epoch: 11 [199/250 25472/32000 (80%)] Loss: 1.56833 batch_time=0.35468
Train Epoch: 11 [210/250 26880/32000 (84%)] Loss: 1.52077 batch_time=0.36173
Train Epoch: 11 [221/250 28288/32000 (88%)] Loss: 1.75650 batch_time=0.34373
Train Epoch: 11 [232/250 29696/32000 (93%)] Loss: 1.43236 batch_time=0.36995
Train Epoch: 11 [243/250 31104/32000 (97%)] Loss: 1.53416 batch_time=0.37401
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch11.pth ...
Done in 3.868s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch11.pth ...
Done in 7.835s
removing stale ckpt [epoch 10] [took 0.00s]
epoch : 11
loss : 1.6620072050094605
learning_rate : 2.993684696191893e-05
n_samples : 352000
n_steps : 2750
MSRVTT_miech_test/t2v_metrics/R1: 22.6
MSRVTT_miech_test/t2v_metrics/R5: 52.0
MSRVTT_miech_test/t2v_metrics/R10: 63.9
MSRVTT_miech_test/t2v_metrics/R50: 87.9
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 32.224000000000004
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 42.189483989000266
MSRVTT_miech_test/v2t_metrics/R1: 22.1
MSRVTT_miech_test/v2t_metrics/R5: 54.9
MSRVTT_miech_test/v2t_metrics/R10: 66.7
MSRVTT_miech_test/v2t_metrics/R50: 89.0
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 27.598
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.25438591404339
mnt_best : 42.189483989000266
not_improved_count: 0
Train Epoch: 12 [1/250 128/32000 (0%)] Loss: 1.75555 batch_time=27.49348
Train Epoch: 12 [12/250 1536/32000 (5%)] Loss: 1.83401 batch_time=0.35666
Train Epoch: 12 [23/250 2944/32000 (9%)] Loss: 2.03526 batch_time=0.38647
Train Epoch: 12 [34/250 4352/32000 (14%)] Loss: 1.40293 batch_time=0.35424
Train Epoch: 12 [45/250 5760/32000 (18%)] Loss: 2.06760 batch_time=0.33770
Train Epoch: 12 [56/250 7168/32000 (22%)] Loss: 1.74652 batch_time=0.36247
Train Epoch: 12 [67/250 8576/32000 (27%)] Loss: 1.45977 batch_time=1.96002
Train Epoch: 12 [78/250 9984/32000 (31%)] Loss: 1.70281 batch_time=0.49851
Train Epoch: 12 [89/250 11392/32000 (36%)] Loss: 1.94596 batch_time=0.35556
Train Epoch: 12 [100/250 12800/32000 (40%)] Loss: 1.90159 batch_time=0.40255
Train Epoch: 12 [111/250 14208/32000 (44%)] Loss: 1.36056 batch_time=0.37257
Train Epoch: 12 [122/250 15616/32000 (49%)] Loss: 2.02544 batch_time=0.35579
Train Epoch: 12 [133/250 17024/32000 (53%)] Loss: 1.41979 batch_time=0.44028
Train Epoch: 12 [144/250 18432/32000 (58%)] Loss: 1.33884 batch_time=0.36793
Train Epoch: 12 [155/250 19840/32000 (62%)] Loss: 1.83579 batch_time=0.35509
Train Epoch: 12 [166/250 21248/32000 (66%)] Loss: 1.54935 batch_time=0.34290
Train Epoch: 12 [177/250 22656/32000 (71%)] Loss: 1.66817 batch_time=0.35185
Train Epoch: 12 [188/250 24064/32000 (75%)] Loss: 1.50979 batch_time=0.34786
Train Epoch: 12 [199/250 25472/32000 (80%)] Loss: 1.53627 batch_time=0.34838
Train Epoch: 12 [210/250 26880/32000 (84%)] Loss: 1.32392 batch_time=0.38502
Train Epoch: 12 [221/250 28288/32000 (88%)] Loss: 1.58680 batch_time=0.34277
Train Epoch: 12 [232/250 29696/32000 (93%)] Loss: 1.46130 batch_time=0.67088
Train Epoch: 12 [243/250 31104/32000 (97%)] Loss: 1.51918 batch_time=0.35283
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch12.pth ...
Done in 3.865s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch12.pth ...
Done in 7.653s
removing stale ckpt [epoch 11] [took 0.01s]
epoch : 12
loss : 1.573335865497589
learning_rate : 2.844000461382298e-05
n_samples : 384000
n_steps : 3000
MSRVTT_miech_test/t2v_metrics/R1: 23.2
MSRVTT_miech_test/t2v_metrics/R5: 51.5
MSRVTT_miech_test/t2v_metrics/R10: 65.8
MSRVTT_miech_test/t2v_metrics/R50: 88.8
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 29.354
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 42.83910305309571
MSRVTT_miech_test/v2t_metrics/R1: 22.5
MSRVTT_miech_test/v2t_metrics/R5: 55.2
MSRVTT_miech_test/v2t_metrics/R10: 67.4
MSRVTT_miech_test/v2t_metrics/R50: 90.0
MSRVTT_miech_test/v2t_metrics/MedR: 4.0
MSRVTT_miech_test/v2t_metrics/MeanR: 25.699
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.74487341289988
mnt_best : 42.83910305309571
not_improved_count: 0
Train Epoch: 13 [1/250 128/32000 (0%)] Loss: 1.16406 batch_time=27.59269
Train Epoch: 13 [12/250 1536/32000 (5%)] Loss: 1.49269 batch_time=0.35150
Train Epoch: 13 [23/250 2944/32000 (9%)] Loss: 1.67586 batch_time=0.41032
Train Epoch: 13 [34/250 4352/32000 (14%)] Loss: 1.87999 batch_time=0.37467
Train Epoch: 13 [45/250 5760/32000 (18%)] Loss: 1.73075 batch_time=0.36037
Train Epoch: 13 [56/250 7168/32000 (22%)] Loss: 1.12939 batch_time=0.36913
Train Epoch: 13 [67/250 8576/32000 (27%)] Loss: 1.39588 batch_time=0.38788
Train Epoch: 13 [78/250 9984/32000 (31%)] Loss: 1.56129 batch_time=3.31118
Train Epoch: 13 [89/250 11392/32000 (36%)] Loss: 1.39197 batch_time=0.34279
Train Epoch: 13 [100/250 12800/32000 (40%)] Loss: 1.85482 batch_time=0.34391
Train Epoch: 13 [111/250 14208/32000 (44%)] Loss: 1.45414 batch_time=0.33906
Train Epoch: 13 [122/250 15616/32000 (49%)] Loss: 1.65983 batch_time=0.39916
Train Epoch: 13 [133/250 17024/32000 (53%)] Loss: 1.34475 batch_time=0.35683
Train Epoch: 13 [144/250 18432/32000 (58%)] Loss: 1.43357 batch_time=0.34888
Train Epoch: 13 [155/250 19840/32000 (62%)] Loss: 1.64406 batch_time=0.36204
Train Epoch: 13 [166/250 21248/32000 (66%)] Loss: 1.54350 batch_time=0.52148
Train Epoch: 13 [177/250 22656/32000 (71%)] Loss: 1.78459 batch_time=0.35901
Train Epoch: 13 [188/250 24064/32000 (75%)] Loss: 1.66198 batch_time=0.36621
Train Epoch: 13 [199/250 25472/32000 (80%)] Loss: 1.52257 batch_time=0.34238
Train Epoch: 13 [210/250 26880/32000 (84%)] Loss: 1.79538 batch_time=0.34694
Train Epoch: 13 [221/250 28288/32000 (88%)] Loss: 1.69186 batch_time=0.44923
Train Epoch: 13 [232/250 29696/32000 (93%)] Loss: 1.10738 batch_time=0.34996
Train Epoch: 13 [243/250 31104/32000 (97%)] Loss: 1.46056 batch_time=0.35589
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch13.pth ...
Done in 3.731s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch13.pth ...
Done in 7.396s
removing stale ckpt [epoch 12] [took 0.01s]
epoch : 13
loss : 1.5342960953712463
learning_rate : 2.7018004383131832e-05
n_samples : 416000
n_steps : 3250
MSRVTT_miech_test/t2v_metrics/R1: 23.4
MSRVTT_miech_test/t2v_metrics/R5: 52.8
MSRVTT_miech_test/t2v_metrics/R10: 66.0
MSRVTT_miech_test/t2v_metrics/R50: 88.3
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 29.723
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 43.364189988293056
MSRVTT_miech_test/v2t_metrics/R1: 23.1
MSRVTT_miech_test/v2t_metrics/R5: 53.9
MSRVTT_miech_test/v2t_metrics/R10: 66.2
MSRVTT_miech_test/v2t_metrics/R50: 90.1
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 25.157
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.51973526491117
mnt_best : 43.364189988293056
not_improved_count: 0
Train Epoch: 14 [1/250 128/32000 (0%)] Loss: 1.41987 batch_time=28.05829
Train Epoch: 14 [12/250 1536/32000 (5%)] Loss: 1.36072 batch_time=0.35653
Train Epoch: 14 [23/250 2944/32000 (9%)] Loss: 1.54980 batch_time=0.36658
Train Epoch: 14 [34/250 4352/32000 (14%)] Loss: 1.25283 batch_time=0.40324
Train Epoch: 14 [45/250 5760/32000 (18%)] Loss: 1.17606 batch_time=0.34522
Train Epoch: 14 [56/250 7168/32000 (22%)] Loss: 1.79735 batch_time=0.36243
Train Epoch: 14 [67/250 8576/32000 (27%)] Loss: 1.35880 batch_time=0.84489
Train Epoch: 14 [78/250 9984/32000 (31%)] Loss: 1.63399 batch_time=0.40147
Train Epoch: 14 [89/250 11392/32000 (36%)] Loss: 1.37703 batch_time=0.41057
Train Epoch: 14 [100/250 12800/32000 (40%)] Loss: 1.30396 batch_time=0.36931
Train Epoch: 14 [111/250 14208/32000 (44%)] Loss: 1.50300 batch_time=0.35886
Train Epoch: 14 [122/250 15616/32000 (49%)] Loss: 1.47987 batch_time=0.36044
Train Epoch: 14 [133/250 17024/32000 (53%)] Loss: 1.60873 batch_time=4.61287
Train Epoch: 14 [144/250 18432/32000 (58%)] Loss: 1.62592 batch_time=0.34895
Train Epoch: 14 [155/250 19840/32000 (62%)] Loss: 1.97823 batch_time=0.35715
Train Epoch: 14 [166/250 21248/32000 (66%)] Loss: 1.53617 batch_time=0.35508
Train Epoch: 14 [177/250 22656/32000 (71%)] Loss: 1.51306 batch_time=0.36658
Train Epoch: 14 [188/250 24064/32000 (75%)] Loss: 1.02641 batch_time=0.38550
Train Epoch: 14 [199/250 25472/32000 (80%)] Loss: 1.33779 batch_time=0.38131
Train Epoch: 14 [210/250 26880/32000 (84%)] Loss: 1.52214 batch_time=0.35052
Train Epoch: 14 [221/250 28288/32000 (88%)] Loss: 1.46985 batch_time=0.36939
Train Epoch: 14 [232/250 29696/32000 (93%)] Loss: 1.41774 batch_time=0.36569
Train Epoch: 14 [243/250 31104/32000 (97%)] Loss: 1.34712 batch_time=0.35074
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch14.pth ...
Done in 3.957s
removing stale ckpt [epoch 13] [took 0.00s]
epoch : 14
loss : 1.43575910115242
learning_rate : 2.566710416397524e-05
n_samples : 448000
n_steps : 3500
MSRVTT_miech_test/t2v_metrics/R1: 23.2
MSRVTT_miech_test/t2v_metrics/R5: 52.2
MSRVTT_miech_test/t2v_metrics/R10: 66.1
MSRVTT_miech_test/t2v_metrics/R50: 88.2
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.932
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 43.09762280014443
MSRVTT_miech_test/v2t_metrics/R1: 22.6
MSRVTT_miech_test/v2t_metrics/R5: 56.0
MSRVTT_miech_test/v2t_metrics/R10: 68.6
MSRVTT_miech_test/v2t_metrics/R50: 89.3
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.968
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 44.27992336945139
mnt_best : 43.364189988293056
not_improved_count: 1
Train Epoch: 15 [1/250 128/32000 (0%)] Loss: 1.60090 batch_time=31.74711
Train Epoch: 15 [12/250 1536/32000 (5%)] Loss: 1.57451 batch_time=0.40771
Train Epoch: 15 [23/250 2944/32000 (9%)] Loss: 1.43439 batch_time=0.35856
Train Epoch: 15 [34/250 4352/32000 (14%)] Loss: 1.44726 batch_time=0.35515
Train Epoch: 15 [45/250 5760/32000 (18%)] Loss: 1.09729 batch_time=0.34964
Train Epoch: 15 [56/250 7168/32000 (22%)] Loss: 1.65165 batch_time=0.34166
Train Epoch: 15 [67/250 8576/32000 (27%)] Loss: 1.51159 batch_time=0.38619
Train Epoch: 15 [78/250 9984/32000 (31%)] Loss: 1.30225 batch_time=0.39679
Train Epoch: 15 [89/250 11392/32000 (36%)] Loss: 1.06621 batch_time=0.37361
Train Epoch: 15 [100/250 12800/32000 (40%)] Loss: 1.03385 batch_time=0.34848
Train Epoch: 15 [111/250 14208/32000 (44%)] Loss: 1.45477 batch_time=0.37420
Train Epoch: 15 [122/250 15616/32000 (49%)] Loss: 1.16658 batch_time=0.38926
Train Epoch: 15 [133/250 17024/32000 (53%)] Loss: 1.34381 batch_time=0.36424
Train Epoch: 15 [144/250 18432/32000 (58%)] Loss: 1.42346 batch_time=0.34216
Train Epoch: 15 [155/250 19840/32000 (62%)] Loss: 1.21149 batch_time=0.34535
Train Epoch: 15 [166/250 21248/32000 (66%)] Loss: 1.39654 batch_time=0.36309
Train Epoch: 15 [177/250 22656/32000 (71%)] Loss: 1.44432 batch_time=0.38454
Train Epoch: 15 [188/250 24064/32000 (75%)] Loss: 1.35245 batch_time=0.36862
Train Epoch: 15 [199/250 25472/32000 (80%)] Loss: 1.32302 batch_time=0.42640
Train Epoch: 15 [210/250 26880/32000 (84%)] Loss: 1.63492 batch_time=0.36292
Train Epoch: 15 [221/250 28288/32000 (88%)] Loss: 1.00457 batch_time=0.35088
Train Epoch: 15 [232/250 29696/32000 (93%)] Loss: 1.51420 batch_time=0.35739
Train Epoch: 15 [243/250 31104/32000 (97%)] Loss: 1.33207 batch_time=0.34942
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch15.pth ...
Done in 9.895s
removing stale ckpt [epoch 14] [took 0.10s]
epoch : 15
loss : 1.381222734451294
learning_rate : 2.4383748955776477e-05
n_samples : 480000
n_steps : 3750
MSRVTT_miech_test/t2v_metrics/R1: 23.7
MSRVTT_miech_test/t2v_metrics/R5: 51.1
MSRVTT_miech_test/t2v_metrics/R10: 65.0
MSRVTT_miech_test/t2v_metrics/R50: 87.7
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.401
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 42.857569122744366
MSRVTT_miech_test/v2t_metrics/R1: 23.2
MSRVTT_miech_test/v2t_metrics/R5: 53.5
MSRVTT_miech_test/v2t_metrics/R10: 67.1
MSRVTT_miech_test/v2t_metrics/R50: 88.8
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 25.9305
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.670493096175946
mnt_best : 43.364189988293056
not_improved_count: 2
Train Epoch: 16 [1/250 128/32000 (0%)] Loss: 1.46118 batch_time=28.54155
Train Epoch: 16 [12/250 1536/32000 (5%)] Loss: 1.12036 batch_time=0.35323
Train Epoch: 16 [23/250 2944/32000 (9%)] Loss: 1.05996 batch_time=3.68280
Train Epoch: 16 [34/250 4352/32000 (14%)] Loss: 1.06063 batch_time=0.42353
Train Epoch: 16 [45/250 5760/32000 (18%)] Loss: 1.38461 batch_time=0.39053
Train Epoch: 16 [56/250 7168/32000 (22%)] Loss: 1.32176 batch_time=0.34582
Train Epoch: 16 [67/250 8576/32000 (27%)] Loss: 1.61301 batch_time=0.35260
Train Epoch: 16 [78/250 9984/32000 (31%)] Loss: 1.43234 batch_time=0.36770
Train Epoch: 16 [89/250 11392/32000 (36%)] Loss: 1.23902 batch_time=0.36202
Train Epoch: 16 [100/250 12800/32000 (40%)] Loss: 1.31571 batch_time=0.34632
Train Epoch: 16 [111/250 14208/32000 (44%)] Loss: 1.28370 batch_time=0.47078
Train Epoch: 16 [122/250 15616/32000 (49%)] Loss: 1.11165 batch_time=0.35438
Train Epoch: 16 [133/250 17024/32000 (53%)] Loss: 1.14607 batch_time=0.35299
Train Epoch: 16 [144/250 18432/32000 (58%)] Loss: 1.40557 batch_time=0.98279
Train Epoch: 16 [155/250 19840/32000 (62%)] Loss: 1.07216 batch_time=0.35281
Train Epoch: 16 [166/250 21248/32000 (66%)] Loss: 1.49114 batch_time=0.35999
Train Epoch: 16 [177/250 22656/32000 (71%)] Loss: 1.36805 batch_time=0.35135
Train Epoch: 16 [188/250 24064/32000 (75%)] Loss: 1.36150 batch_time=0.36250
Train Epoch: 16 [199/250 25472/32000 (80%)] Loss: 1.33868 batch_time=1.34001
Train Epoch: 16 [210/250 26880/32000 (84%)] Loss: 1.55424 batch_time=0.34940
Train Epoch: 16 [221/250 28288/32000 (88%)] Loss: 1.48288 batch_time=0.35149
Train Epoch: 16 [232/250 29696/32000 (93%)] Loss: 1.39617 batch_time=0.35562
Train Epoch: 16 [243/250 31104/32000 (97%)] Loss: 1.26422 batch_time=0.36416
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch16.pth ...
Done in 4.340s
removing stale ckpt [epoch 15] [took 0.01s]
epoch : 16
loss : 1.3506860921382904
learning_rate : 2.3164561507987653e-05
n_samples : 512000
n_steps : 4000
MSRVTT_miech_test/t2v_metrics/R1: 22.9
MSRVTT_miech_test/t2v_metrics/R5: 52.4
MSRVTT_miech_test/t2v_metrics/R10: 66.1
MSRVTT_miech_test/t2v_metrics/R50: 88.5
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.821
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 42.96578421077108
MSRVTT_miech_test/v2t_metrics/R1: 23.3
MSRVTT_miech_test/v2t_metrics/R5: 53.6
MSRVTT_miech_test/v2t_metrics/R10: 67.1
MSRVTT_miech_test/v2t_metrics/R50: 88.9
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.4645
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 43.76037922995852
mnt_best : 43.364189988293056
not_improved_count: 3
Train Epoch: 17 [1/250 128/32000 (0%)] Loss: 1.31874 batch_time=28.85642
Train Epoch: 17 [12/250 1536/32000 (5%)] Loss: 1.35951 batch_time=0.36108
Train Epoch: 17 [23/250 2944/32000 (9%)] Loss: 1.47159 batch_time=0.34663
Train Epoch: 17 [34/250 4352/32000 (14%)] Loss: 1.40127 batch_time=0.39587
Train Epoch: 17 [45/250 5760/32000 (18%)] Loss: 1.24886 batch_time=0.59783
Train Epoch: 17 [56/250 7168/32000 (22%)] Loss: 1.73453 batch_time=0.35612
Train Epoch: 17 [67/250 8576/32000 (27%)] Loss: 1.01472 batch_time=4.57280
Train Epoch: 17 [78/250 9984/32000 (31%)] Loss: 1.32960 batch_time=1.58048
Train Epoch: 17 [89/250 11392/32000 (36%)] Loss: 1.24484 batch_time=0.49120
Train Epoch: 17 [100/250 12800/32000 (40%)] Loss: 1.04786 batch_time=1.76755
Train Epoch: 17 [111/250 14208/32000 (44%)] Loss: 1.32878 batch_time=0.34245
Train Epoch: 17 [122/250 15616/32000 (49%)] Loss: 1.56339 batch_time=0.34983
Train Epoch: 17 [133/250 17024/32000 (53%)] Loss: 1.39114 batch_time=0.34928
Train Epoch: 17 [144/250 18432/32000 (58%)] Loss: 1.19721 batch_time=0.36810
Train Epoch: 17 [155/250 19840/32000 (62%)] Loss: 1.31059 batch_time=0.37397
Train Epoch: 17 [166/250 21248/32000 (66%)] Loss: 1.41799 batch_time=0.34693
Train Epoch: 17 [177/250 22656/32000 (71%)] Loss: 1.31240 batch_time=0.35285
Train Epoch: 17 [188/250 24064/32000 (75%)] Loss: 1.30655 batch_time=0.36612
Train Epoch: 17 [199/250 25472/32000 (80%)] Loss: 1.32822 batch_time=0.34266
Train Epoch: 17 [210/250 26880/32000 (84%)] Loss: 1.27101 batch_time=0.36747
Train Epoch: 17 [221/250 28288/32000 (88%)] Loss: 1.30083 batch_time=0.34685
Train Epoch: 17 [232/250 29696/32000 (93%)] Loss: 1.30629 batch_time=0.35701
Train Epoch: 17 [243/250 31104/32000 (97%)] Loss: 1.51228 batch_time=0.36067
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch17.pth ...
Done in 5.444s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch17.pth ...
Done in 10.190s
removing stale ckpt [epoch 16] [took 0.04s]
epoch : 17
loss : 1.309631707906723
learning_rate : 2.2006333432588268e-05
n_samples : 544000
n_steps : 4250
MSRVTT_miech_test/t2v_metrics/R1: 23.5
MSRVTT_miech_test/t2v_metrics/R5: 52.5
MSRVTT_miech_test/t2v_metrics/R10: 66.3
MSRVTT_miech_test/t2v_metrics/R50: 87.8
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 30.527
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 43.409044990577506
MSRVTT_miech_test/v2t_metrics/R1: 24.3
MSRVTT_miech_test/v2t_metrics/R5: 54.6
MSRVTT_miech_test/v2t_metrics/R10: 67.2
MSRVTT_miech_test/v2t_metrics/R50: 88.5
MSRVTT_miech_test/v2t_metrics/MedR: 5.0
MSRVTT_miech_test/v2t_metrics/MeanR: 25.8665
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 44.674125830341104
mnt_best : 43.409044990577506
not_improved_count: 0
Train Epoch: 18 [1/250 128/32000 (0%)] Loss: 1.32018 batch_time=22.21709
Train Epoch: 18 [12/250 1536/32000 (5%)] Loss: 1.23068 batch_time=2.28849
Train Epoch: 18 [23/250 2944/32000 (9%)] Loss: 1.34335 batch_time=0.36724
Train Epoch: 18 [34/250 4352/32000 (14%)] Loss: 1.31106 batch_time=0.35845
Train Epoch: 18 [45/250 5760/32000 (18%)] Loss: 1.22076 batch_time=0.35934
Train Epoch: 18 [56/250 7168/32000 (22%)] Loss: 1.39904 batch_time=0.37893
Train Epoch: 18 [67/250 8576/32000 (27%)] Loss: 1.03904 batch_time=2.60490
Train Epoch: 18 [78/250 9984/32000 (31%)] Loss: 1.35513 batch_time=0.42063
Train Epoch: 18 [89/250 11392/32000 (36%)] Loss: 1.09037 batch_time=0.37075
Train Epoch: 18 [100/250 12800/32000 (40%)] Loss: 1.16705 batch_time=0.34956
Train Epoch: 18 [111/250 14208/32000 (44%)] Loss: 1.07854 batch_time=0.36590
Train Epoch: 18 [122/250 15616/32000 (49%)] Loss: 1.21500 batch_time=0.36072
Train Epoch: 18 [133/250 17024/32000 (53%)] Loss: 1.23186 batch_time=0.35223
Train Epoch: 18 [144/250 18432/32000 (58%)] Loss: 1.27840 batch_time=0.35620
Train Epoch: 18 [155/250 19840/32000 (62%)] Loss: 1.11765 batch_time=0.36024
Train Epoch: 18 [166/250 21248/32000 (66%)] Loss: 1.74810 batch_time=0.38143
Train Epoch: 18 [177/250 22656/32000 (71%)] Loss: 1.00492 batch_time=0.35210
Train Epoch: 18 [188/250 24064/32000 (75%)] Loss: 1.53199 batch_time=0.35120
Train Epoch: 18 [199/250 25472/32000 (80%)] Loss: 1.33770 batch_time=0.34680
Train Epoch: 18 [210/250 26880/32000 (84%)] Loss: 1.31344 batch_time=0.36086
Train Epoch: 18 [221/250 28288/32000 (88%)] Loss: 1.34606 batch_time=0.35815
Train Epoch: 18 [232/250 29696/32000 (93%)] Loss: 1.20225 batch_time=0.34735
Train Epoch: 18 [243/250 31104/32000 (97%)] Loss: 0.97979 batch_time=0.36025
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch18.pth ...
Done in 5.941s
removing stale ckpt [epoch 17] [took 0.00s]
epoch : 18
loss : 1.2556621260643006
learning_rate : 2.0906016760958855e-05
n_samples : 576000
n_steps : 4500
MSRVTT_miech_test/t2v_metrics/R1: 23.7
MSRVTT_miech_test/t2v_metrics/R5: 52.4
MSRVTT_miech_test/t2v_metrics/R10: 65.3
MSRVTT_miech_test/t2v_metrics/R50: 87.7
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.749
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 43.28435378498381
MSRVTT_miech_test/v2t_metrics/R1: 23.7
MSRVTT_miech_test/v2t_metrics/R5: 54.9
MSRVTT_miech_test/v2t_metrics/R10: 67.7
MSRVTT_miech_test/v2t_metrics/R50: 89.1
MSRVTT_miech_test/v2t_metrics/MedR: 4.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.6125
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 44.49417101616151
mnt_best : 43.409044990577506
not_improved_count: 1
Train Epoch: 19 [1/250 128/32000 (0%)] Loss: 1.39931 batch_time=23.89570
Train Epoch: 19 [12/250 1536/32000 (5%)] Loss: 1.36793 batch_time=0.34040
Train Epoch: 19 [23/250 2944/32000 (9%)] Loss: 1.10841 batch_time=0.34013
Train Epoch: 19 [34/250 4352/32000 (14%)] Loss: 1.19366 batch_time=0.34242
Train Epoch: 19 [45/250 5760/32000 (18%)] Loss: 1.15596 batch_time=0.33727
Train Epoch: 19 [56/250 7168/32000 (22%)] Loss: 1.51505 batch_time=0.34927
Train Epoch: 19 [67/250 8576/32000 (27%)] Loss: 1.33787 batch_time=2.77195
Train Epoch: 19 [78/250 9984/32000 (31%)] Loss: 1.37172 batch_time=0.34343
Train Epoch: 19 [89/250 11392/32000 (36%)] Loss: 1.12388 batch_time=0.34775
Train Epoch: 19 [100/250 12800/32000 (40%)] Loss: 1.10573 batch_time=0.37473
Train Epoch: 19 [111/250 14208/32000 (44%)] Loss: 1.15684 batch_time=0.34938
Train Epoch: 19 [122/250 15616/32000 (49%)] Loss: 1.07767 batch_time=0.35757
Train Epoch: 19 [133/250 17024/32000 (53%)] Loss: 0.96112 batch_time=0.37657
Train Epoch: 19 [144/250 18432/32000 (58%)] Loss: 1.37308 batch_time=0.35264
Train Epoch: 19 [155/250 19840/32000 (62%)] Loss: 1.48699 batch_time=0.34412
Train Epoch: 19 [166/250 21248/32000 (66%)] Loss: 1.83772 batch_time=0.37304
Train Epoch: 19 [177/250 22656/32000 (71%)] Loss: 1.33748 batch_time=0.35745
Train Epoch: 19 [188/250 24064/32000 (75%)] Loss: 1.10016 batch_time=0.36328
Train Epoch: 19 [199/250 25472/32000 (80%)] Loss: 1.25450 batch_time=0.36991
Train Epoch: 19 [210/250 26880/32000 (84%)] Loss: 1.20271 batch_time=0.35719
Train Epoch: 19 [221/250 28288/32000 (88%)] Loss: 1.21903 batch_time=0.34512
Train Epoch: 19 [232/250 29696/32000 (93%)] Loss: 1.35166 batch_time=0.36507
Train Epoch: 19 [243/250 31104/32000 (97%)] Loss: 1.37708 batch_time=0.36521
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch19.pth ...
Done in 5.120s
removing stale ckpt [epoch 18] [took 0.01s]
epoch : 19
loss : 1.222164200782776
learning_rate : 1.986071592291091e-05
n_samples : 608000
n_steps : 4750
MSRVTT_miech_test/t2v_metrics/R1: 23.3
MSRVTT_miech_test/t2v_metrics/R5: 52.7
MSRVTT_miech_test/t2v_metrics/R10: 65.3
MSRVTT_miech_test/t2v_metrics/R50: 87.8
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.328
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 43.12143840081546
MSRVTT_miech_test/v2t_metrics/R1: 23.5
MSRVTT_miech_test/v2t_metrics/R5: 56.7
MSRVTT_miech_test/v2t_metrics/R10: 67.7
MSRVTT_miech_test/v2t_metrics/R50: 89.5
MSRVTT_miech_test/v2t_metrics/MedR: 4.0
MSRVTT_miech_test/v2t_metrics/MeanR: 26.4
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 44.84835622212356
mnt_best : 43.409044990577506
not_improved_count: 2
Train Epoch: 20 [1/250 128/32000 (0%)] Loss: 1.10748 batch_time=22.64185
Train Epoch: 20 [12/250 1536/32000 (5%)] Loss: 1.29006 batch_time=0.58126
Train Epoch: 20 [23/250 2944/32000 (9%)] Loss: 1.30621 batch_time=0.34788
Train Epoch: 20 [34/250 4352/32000 (14%)] Loss: 1.04994 batch_time=0.35453
Train Epoch: 20 [45/250 5760/32000 (18%)] Loss: 1.19601 batch_time=0.34839
Train Epoch: 20 [56/250 7168/32000 (22%)] Loss: 1.15970 batch_time=0.34307
Train Epoch: 20 [67/250 8576/32000 (27%)] Loss: 1.23060 batch_time=0.37352
Train Epoch: 20 [78/250 9984/32000 (31%)] Loss: 1.24667 batch_time=0.35375
Train Epoch: 20 [89/250 11392/32000 (36%)] Loss: 1.38678 batch_time=0.35409
Train Epoch: 20 [100/250 12800/32000 (40%)] Loss: 1.36055 batch_time=0.39722
Train Epoch: 20 [111/250 14208/32000 (44%)] Loss: 1.07742 batch_time=0.37678
Train Epoch: 20 [122/250 15616/32000 (49%)] Loss: 1.00168 batch_time=0.35753
Train Epoch: 20 [133/250 17024/32000 (53%)] Loss: 1.68373 batch_time=0.35813
Train Epoch: 20 [144/250 18432/32000 (58%)] Loss: 1.21054 batch_time=0.36665
Train Epoch: 20 [155/250 19840/32000 (62%)] Loss: 1.28025 batch_time=0.35815
Train Epoch: 20 [166/250 21248/32000 (66%)] Loss: 1.11267 batch_time=0.34426
Train Epoch: 20 [177/250 22656/32000 (71%)] Loss: 1.16337 batch_time=0.34322
Train Epoch: 20 [188/250 24064/32000 (75%)] Loss: 1.10453 batch_time=0.35100
Train Epoch: 20 [199/250 25472/32000 (80%)] Loss: 0.95041 batch_time=0.34079
Train Epoch: 20 [210/250 26880/32000 (84%)] Loss: 1.34897 batch_time=0.45127
Train Epoch: 20 [221/250 28288/32000 (88%)] Loss: 1.12416 batch_time=0.40301
Train Epoch: 20 [232/250 29696/32000 (93%)] Loss: 1.10923 batch_time=0.36791
Train Epoch: 20 [243/250 31104/32000 (97%)] Loss: 1.05857 batch_time=0.35208
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch20.pth ...
Done in 5.353s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCT_MSRVTT_1kB/checkpoint-epoch20.pth ...
Done in 10.015s
removing stale ckpt [epoch 19] [took 0.05s]
epoch : 20
loss : 1.1909048681259156
learning_rate : 1.8867680126765363e-05
n_samples : 640000
n_steps : 5000
MSRVTT_miech_test/t2v_metrics/R1: 24.7
MSRVTT_miech_test/t2v_metrics/R5: 52.6
MSRVTT_miech_test/t2v_metrics/R10: 66.2
MSRVTT_miech_test/t2v_metrics/R50: 88.1
MSRVTT_miech_test/t2v_metrics/MedR: 5.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.088
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 44.14148053685193
MSRVTT_miech_test/v2t_metrics/R1: 23.8
MSRVTT_miech_test/v2t_metrics/R5: 56.5
MSRVTT_miech_test/v2t_metrics/R10: 67.9
MSRVTT_miech_test/v2t_metrics/R50: 89.7
MSRVTT_miech_test/v2t_metrics/MedR: 4.0