-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathHCQ_MSRVTT_1kB_bs64.txt
4597 lines (4597 loc) · 421 KB
/
HCQ_MSRVTT_1kB_bs64.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Experiment directory: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64
Preparing the dataloaders ...
Loading dataset MSRVTT_miech_trainval in ram ...
Finish loading dataset MSRVTT_miech_trainval in ram, taking 603.3520195484161 s.
Loading dataset MSRVTT_miech_test in ram ...
Finish loading dataset MSRVTT_miech_test in ram, taking 91.26318216323853 s.
Loading dataset MSRVTT_miech_test in ram ...
Finish loading dataset MSRVTT_miech_test in ram, taking 46.25491547584534 s.
Training ...
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch0.pth ...
Done in 2.260s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch0.pth ...
Done in 4.333s
epoch : 0
loss : 0
learning_rate : 5e-05
n_samples : 0
n_steps : 0
MSRVTT_miech_test/t2v_metrics/R1: 0.1
MSRVTT_miech_test/t2v_metrics/R5: 0.6
MSRVTT_miech_test/t2v_metrics/R10: 1.0
MSRVTT_miech_test/t2v_metrics/R50: 5.0
MSRVTT_miech_test/t2v_metrics/MedR: 503.0
MSRVTT_miech_test/t2v_metrics/MeanR: 505.179
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 0.3914867641168864
MSRVTT_miech_test/v2t_metrics/R1: 0.1
MSRVTT_miech_test/v2t_metrics/R5: 0.4
MSRVTT_miech_test/v2t_metrics/R10: 1.0
MSRVTT_miech_test/v2t_metrics/R50: 5.4
MSRVTT_miech_test/v2t_metrics/MedR: 511.5
MSRVTT_miech_test/v2t_metrics/MeanR: 499.893
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 0.3419951893353394
mnt_best : 0.3914867641168864
not_improved_count: 0
Train Epoch: 1 [1/500 64/32000 (0%)] Loss: 8.43814 (QuantReg: 22.47124) QuantErr: 22.47124 batch_time=23.61913
Train Epoch: 1 [9/500 576/32000 (2%)] Loss: 8.07294 (QuantReg: 22.43234) QuantErr: 22.43234 batch_time=0.42483
Train Epoch: 1 [17/500 1088/32000 (3%)] Loss: 6.92733 (QuantReg: 22.50832) QuantErr: 22.50832 batch_time=0.42285
Train Epoch: 1 [25/500 1600/32000 (5%)] Loss: 6.19994 (QuantReg: 22.49500) QuantErr: 22.49500 batch_time=0.56379
Train Epoch: 1 [33/500 2112/32000 (7%)] Loss: 6.23030 (QuantReg: 22.50699) QuantErr: 22.50699 batch_time=0.51311
Train Epoch: 1 [41/500 2624/32000 (8%)] Loss: 6.19945 (QuantReg: 22.61296) QuantErr: 22.61296 batch_time=0.43473
Train Epoch: 1 [49/500 3136/32000 (10%)] Loss: 4.92869 (QuantReg: 22.56521) QuantErr: 22.56521 batch_time=0.48633
Train Epoch: 1 [57/500 3648/32000 (11%)] Loss: 5.11551 (QuantReg: 22.60281) QuantErr: 22.60281 batch_time=0.46246
Train Epoch: 1 [65/500 4160/32000 (13%)] Loss: 4.76426 (QuantReg: 22.58537) QuantErr: 22.58537 batch_time=0.55391
Train Epoch: 1 [73/500 4672/32000 (15%)] Loss: 5.52754 (QuantReg: 22.56047) QuantErr: 22.56047 batch_time=0.48567
Train Epoch: 1 [81/500 5184/32000 (16%)] Loss: 5.04083 (QuantReg: 22.61699) QuantErr: 22.61699 batch_time=0.47121
Train Epoch: 1 [89/500 5696/32000 (18%)] Loss: 4.18837 (QuantReg: 22.62926) QuantErr: 22.62926 batch_time=0.41079
Train Epoch: 1 [97/500 6208/32000 (19%)] Loss: 4.79785 (QuantReg: 22.60438) QuantErr: 22.60438 batch_time=0.42248
Train Epoch: 1 [105/500 6720/32000 (21%)] Loss: 4.74828 (QuantReg: 22.61328) QuantErr: 22.61328 batch_time=0.47407
Train Epoch: 1 [113/500 7232/32000 (23%)] Loss: 4.74092 (QuantReg: 22.52281) QuantErr: 22.52281 batch_time=0.45388
Train Epoch: 1 [121/500 7744/32000 (24%)] Loss: 4.77593 (QuantReg: 22.55562) QuantErr: 22.55562 batch_time=0.41391
Train Epoch: 1 [129/500 8256/32000 (26%)] Loss: 4.39460 (QuantReg: 22.60296) QuantErr: 22.60296 batch_time=0.47530
Train Epoch: 1 [137/500 8768/32000 (27%)] Loss: 4.47458 (QuantReg: 22.57417) QuantErr: 22.57417 batch_time=0.48999
Train Epoch: 1 [145/500 9280/32000 (29%)] Loss: 4.24649 (QuantReg: 22.63674) QuantErr: 22.63674 batch_time=0.44756
Train Epoch: 1 [153/500 9792/32000 (31%)] Loss: 4.77954 (QuantReg: 22.60246) QuantErr: 22.60246 batch_time=0.41413
Train Epoch: 1 [161/500 10304/32000 (32%)] Loss: 3.95147 (QuantReg: 22.63867) QuantErr: 22.63867 batch_time=0.39644
Train Epoch: 1 [169/500 10816/32000 (34%)] Loss: 3.60523 (QuantReg: 22.63819) QuantErr: 22.63819 batch_time=0.44636
Train Epoch: 1 [177/500 11328/32000 (35%)] Loss: 3.81490 (QuantReg: 22.64024) QuantErr: 22.64024 batch_time=0.48835
Train Epoch: 1 [185/500 11840/32000 (37%)] Loss: 3.79137 (QuantReg: 22.57215) QuantErr: 22.57215 batch_time=0.45684
Train Epoch: 1 [193/500 12352/32000 (39%)] Loss: 4.49014 (QuantReg: 22.61815) QuantErr: 22.61815 batch_time=0.52773
Train Epoch: 1 [201/500 12864/32000 (40%)] Loss: 3.40249 (QuantReg: 22.59383) QuantErr: 22.59383 batch_time=0.41370
Train Epoch: 1 [209/500 13376/32000 (42%)] Loss: 3.84287 (QuantReg: 22.64277) QuantErr: 22.64277 batch_time=0.46871
Train Epoch: 1 [217/500 13888/32000 (43%)] Loss: 3.58844 (QuantReg: 22.63635) QuantErr: 22.63635 batch_time=0.43487
Train Epoch: 1 [225/500 14400/32000 (45%)] Loss: 3.16189 (QuantReg: 22.70997) QuantErr: 22.70997 batch_time=0.52648
Train Epoch: 1 [233/500 14912/32000 (47%)] Loss: 3.27441 (QuantReg: 22.63987) QuantErr: 22.63987 batch_time=0.51021
Train Epoch: 1 [241/500 15424/32000 (48%)] Loss: 3.54390 (QuantReg: 22.65001) QuantErr: 22.65001 batch_time=0.39796
Train Epoch: 1 [249/500 15936/32000 (50%)] Loss: 3.20771 (QuantReg: 22.64298) QuantErr: 22.64298 batch_time=0.46186
Train Epoch: 1 [257/500 16448/32000 (51%)] Loss: 4.03505 (QuantReg: 22.68443) QuantErr: 22.68443 batch_time=0.51778
Train Epoch: 1 [265/500 16960/32000 (53%)] Loss: 3.58372 (QuantReg: 22.69003) QuantErr: 22.69003 batch_time=0.46286
Train Epoch: 1 [273/500 17472/32000 (55%)] Loss: 3.34735 (QuantReg: 22.62654) QuantErr: 22.62654 batch_time=0.44499
Train Epoch: 1 [281/500 17984/32000 (56%)] Loss: 3.53118 (QuantReg: 22.67187) QuantErr: 22.67187 batch_time=0.43744
Train Epoch: 1 [289/500 18496/32000 (58%)] Loss: 3.76725 (QuantReg: 22.66370) QuantErr: 22.66370 batch_time=0.41000
Train Epoch: 1 [297/500 19008/32000 (59%)] Loss: 2.86490 (QuantReg: 22.65937) QuantErr: 22.65937 batch_time=0.44484
Train Epoch: 1 [305/500 19520/32000 (61%)] Loss: 3.57860 (QuantReg: 22.71281) QuantErr: 22.71281 batch_time=0.48664
Train Epoch: 1 [313/500 20032/32000 (63%)] Loss: 3.84359 (QuantReg: 22.68862) QuantErr: 22.68862 batch_time=0.47469
Train Epoch: 1 [321/500 20544/32000 (64%)] Loss: 2.88834 (QuantReg: 22.67020) QuantErr: 22.67020 batch_time=0.63129
Train Epoch: 1 [329/500 21056/32000 (66%)] Loss: 3.57420 (QuantReg: 22.69869) QuantErr: 22.69869 batch_time=0.53389
Train Epoch: 1 [337/500 21568/32000 (67%)] Loss: 3.79530 (QuantReg: 22.71920) QuantErr: 22.71920 batch_time=0.47161
Train Epoch: 1 [345/500 22080/32000 (69%)] Loss: 3.73685 (QuantReg: 22.71589) QuantErr: 22.71589 batch_time=0.42942
Train Epoch: 1 [353/500 22592/32000 (71%)] Loss: 3.14037 (QuantReg: 22.68034) QuantErr: 22.68034 batch_time=0.46035
Train Epoch: 1 [361/500 23104/32000 (72%)] Loss: 3.90545 (QuantReg: 22.69583) QuantErr: 22.69583 batch_time=0.50070
Train Epoch: 1 [369/500 23616/32000 (74%)] Loss: 3.15793 (QuantReg: 22.64673) QuantErr: 22.64673 batch_time=0.50159
Train Epoch: 1 [377/500 24128/32000 (75%)] Loss: 3.25415 (QuantReg: 22.64655) QuantErr: 22.64655 batch_time=0.49335
Train Epoch: 1 [385/500 24640/32000 (77%)] Loss: 3.10594 (QuantReg: 22.67885) QuantErr: 22.67885 batch_time=0.58397
Train Epoch: 1 [393/500 25152/32000 (79%)] Loss: 3.41504 (QuantReg: 22.68661) QuantErr: 22.68661 batch_time=0.41897
Train Epoch: 1 [401/500 25664/32000 (80%)] Loss: 4.05284 (QuantReg: 22.69730) QuantErr: 22.69730 batch_time=0.45376
Train Epoch: 1 [409/500 26176/32000 (82%)] Loss: 2.82250 (QuantReg: 22.65941) QuantErr: 22.65941 batch_time=0.51459
Train Epoch: 1 [417/500 26688/32000 (83%)] Loss: 3.50118 (QuantReg: 22.70216) QuantErr: 22.70216 batch_time=0.49859
Train Epoch: 1 [425/500 27200/32000 (85%)] Loss: 2.85913 (QuantReg: 22.68780) QuantErr: 22.68780 batch_time=0.48100
Train Epoch: 1 [433/500 27712/32000 (87%)] Loss: 3.22243 (QuantReg: 22.71450) QuantErr: 22.71450 batch_time=0.49737
Train Epoch: 1 [441/500 28224/32000 (88%)] Loss: 3.82724 (QuantReg: 22.69940) QuantErr: 22.69940 batch_time=0.48921
Train Epoch: 1 [449/500 28736/32000 (90%)] Loss: 3.13193 (QuantReg: 22.67572) QuantErr: 22.67572 batch_time=0.60816
Train Epoch: 1 [457/500 29248/32000 (91%)] Loss: 2.70542 (QuantReg: 22.64740) QuantErr: 22.64740 batch_time=0.43575
Train Epoch: 1 [465/500 29760/32000 (93%)] Loss: 2.77277 (QuantReg: 22.65420) QuantErr: 22.65420 batch_time=0.47388
Train Epoch: 1 [473/500 30272/32000 (95%)] Loss: 3.08022 (QuantReg: 22.69294) QuantErr: 22.69294 batch_time=0.47355
Train Epoch: 1 [481/500 30784/32000 (96%)] Loss: 2.87616 (QuantReg: 22.66028) QuantErr: 22.66028 batch_time=0.54771
Train Epoch: 1 [489/500 31296/32000 (98%)] Loss: 2.76735 (QuantReg: 22.66916) QuantErr: 22.66916 batch_time=0.50221
Train Epoch: 1 [497/500 31808/32000 (99%)] Loss: 2.39162 (QuantReg: 22.68764) QuantErr: 22.68764 batch_time=0.49712
Train Epoch: 1 codebook_update_time=3.28826
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch1.pth ...
Done in 12.086s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch1.pth ...
Done in 28.456s
epoch : 1
loss : 3.980151246547699
quant_reg : 22.637884368896483
quant_err : 22.637884368896483
learning_rate : 5e-05
n_samples : 32000
n_steps : 500
MSRVTT_miech_test/t2v_metrics/R1: 11.1
MSRVTT_miech_test/t2v_metrics/R5: 32.9
MSRVTT_miech_test/t2v_metrics/R10: 46.5
MSRVTT_miech_test/t2v_metrics/R50: 79.9
MSRVTT_miech_test/t2v_metrics/MedR: 12.5
MSRVTT_miech_test/t2v_metrics/MeanR: 44.989
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 25.7034020740142
MSRVTT_miech_test/v2t_metrics/R1: 11.2
MSRVTT_miech_test/v2t_metrics/R5: 33.3
MSRVTT_miech_test/v2t_metrics/R10: 46.1
MSRVTT_miech_test/v2t_metrics/R50: 78.5
MSRVTT_miech_test/v2t_metrics/MedR: 12.0
MSRVTT_miech_test/v2t_metrics/MeanR: 43.741
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 25.809983516865874
mnt_best : 25.7034020740142
not_improved_count: 0
Train Epoch: 2 [1/500 64/32000 (0%)] Loss: 2.66995 (QuantReg: 11.55571) QuantErr: 11.55571 batch_time=26.83050
Train Epoch: 2 [9/500 576/32000 (2%)] Loss: 2.40881 (QuantReg: 11.62492) QuantErr: 11.62492 batch_time=0.42582
Train Epoch: 2 [17/500 1088/32000 (3%)] Loss: 3.17483 (QuantReg: 11.77660) QuantErr: 11.77660 batch_time=0.44437
Train Epoch: 2 [25/500 1600/32000 (5%)] Loss: 2.65678 (QuantReg: 11.74711) QuantErr: 11.74711 batch_time=0.43581
Train Epoch: 2 [33/500 2112/32000 (7%)] Loss: 2.56971 (QuantReg: 12.14401) QuantErr: 12.14401 batch_time=0.43113
Train Epoch: 2 [41/500 2624/32000 (8%)] Loss: 3.12301 (QuantReg: 11.91295) QuantErr: 11.91295 batch_time=0.41209
Train Epoch: 2 [49/500 3136/32000 (10%)] Loss: 2.96589 (QuantReg: 12.40466) QuantErr: 12.40466 batch_time=0.44555
Train Epoch: 2 [57/500 3648/32000 (11%)] Loss: 2.79995 (QuantReg: 12.56196) QuantErr: 12.56196 batch_time=0.44725
Train Epoch: 2 [65/500 4160/32000 (13%)] Loss: 2.07827 (QuantReg: 12.54146) QuantErr: 12.54146 batch_time=0.83357
Train Epoch: 2 [73/500 4672/32000 (15%)] Loss: 2.06579 (QuantReg: 12.12918) QuantErr: 12.12918 batch_time=0.54152
Train Epoch: 2 [81/500 5184/32000 (16%)] Loss: 2.28785 (QuantReg: 12.56490) QuantErr: 12.56490 batch_time=0.45306
Train Epoch: 2 [89/500 5696/32000 (18%)] Loss: 2.52368 (QuantReg: 12.51778) QuantErr: 12.51778 batch_time=0.44593
Train Epoch: 2 [97/500 6208/32000 (19%)] Loss: 2.42394 (QuantReg: 12.59426) QuantErr: 12.59426 batch_time=0.41848
Train Epoch: 2 [105/500 6720/32000 (21%)] Loss: 2.34612 (QuantReg: 12.67617) QuantErr: 12.67617 batch_time=0.48786
Train Epoch: 2 [113/500 7232/32000 (23%)] Loss: 2.41511 (QuantReg: 12.36383) QuantErr: 12.36383 batch_time=0.48893
Train Epoch: 2 [121/500 7744/32000 (24%)] Loss: 3.12779 (QuantReg: 12.49131) QuantErr: 12.49131 batch_time=0.47809
Train Epoch: 2 [129/500 8256/32000 (26%)] Loss: 2.46386 (QuantReg: 12.30723) QuantErr: 12.30723 batch_time=0.90786
Train Epoch: 2 [137/500 8768/32000 (27%)] Loss: 2.99875 (QuantReg: 12.92780) QuantErr: 12.92780 batch_time=0.45929
Train Epoch: 2 [145/500 9280/32000 (29%)] Loss: 2.33295 (QuantReg: 12.68370) QuantErr: 12.68370 batch_time=0.41837
Train Epoch: 2 [153/500 9792/32000 (31%)] Loss: 3.16634 (QuantReg: 12.92526) QuantErr: 12.92526 batch_time=0.45036
Train Epoch: 2 [161/500 10304/32000 (32%)] Loss: 2.83237 (QuantReg: 12.87321) QuantErr: 12.87321 batch_time=0.44175
Train Epoch: 2 [169/500 10816/32000 (34%)] Loss: 2.50749 (QuantReg: 12.73324) QuantErr: 12.73324 batch_time=0.44705
Train Epoch: 2 [177/500 11328/32000 (35%)] Loss: 2.46704 (QuantReg: 12.79665) QuantErr: 12.79665 batch_time=0.43157
Train Epoch: 2 [185/500 11840/32000 (37%)] Loss: 2.79930 (QuantReg: 12.73149) QuantErr: 12.73149 batch_time=0.49979
Train Epoch: 2 [193/500 12352/32000 (39%)] Loss: 2.47420 (QuantReg: 12.73126) QuantErr: 12.73126 batch_time=0.96094
Train Epoch: 2 [201/500 12864/32000 (40%)] Loss: 2.26586 (QuantReg: 12.87775) QuantErr: 12.87775 batch_time=0.44500
Train Epoch: 2 [209/500 13376/32000 (42%)] Loss: 3.46138 (QuantReg: 13.07528) QuantErr: 13.07528 batch_time=0.46282
Train Epoch: 2 [217/500 13888/32000 (43%)] Loss: 2.15036 (QuantReg: 13.34962) QuantErr: 13.34962 batch_time=0.42728
Train Epoch: 2 [225/500 14400/32000 (45%)] Loss: 2.88753 (QuantReg: 13.19498) QuantErr: 13.19498 batch_time=0.44054
Train Epoch: 2 [233/500 14912/32000 (47%)] Loss: 2.42430 (QuantReg: 13.25283) QuantErr: 13.25283 batch_time=0.44627
Train Epoch: 2 [241/500 15424/32000 (48%)] Loss: 2.67366 (QuantReg: 13.28589) QuantErr: 13.28589 batch_time=0.48545
Train Epoch: 2 [249/500 15936/32000 (50%)] Loss: 3.04484 (QuantReg: 13.29069) QuantErr: 13.29069 batch_time=0.46336
Train Epoch: 2 [257/500 16448/32000 (51%)] Loss: 2.73075 (QuantReg: 13.05627) QuantErr: 13.05627 batch_time=0.90437
Train Epoch: 2 [265/500 16960/32000 (53%)] Loss: 2.47153 (QuantReg: 13.72828) QuantErr: 13.72828 batch_time=0.47332
Train Epoch: 2 [273/500 17472/32000 (55%)] Loss: 3.63847 (QuantReg: 13.24408) QuantErr: 13.24408 batch_time=0.41175
Train Epoch: 2 [281/500 17984/32000 (56%)] Loss: 2.27098 (QuantReg: 12.93545) QuantErr: 12.93545 batch_time=0.46006
Train Epoch: 2 [289/500 18496/32000 (58%)] Loss: 2.44629 (QuantReg: 13.40579) QuantErr: 13.40579 batch_time=0.44720
Train Epoch: 2 [297/500 19008/32000 (59%)] Loss: 3.05747 (QuantReg: 13.54981) QuantErr: 13.54981 batch_time=0.49605
Train Epoch: 2 [305/500 19520/32000 (61%)] Loss: 2.59397 (QuantReg: 13.66596) QuantErr: 13.66596 batch_time=0.45154
Train Epoch: 2 [313/500 20032/32000 (63%)] Loss: 3.07523 (QuantReg: 13.67981) QuantErr: 13.67981 batch_time=0.45021
Train Epoch: 2 [321/500 20544/32000 (64%)] Loss: 2.87630 (QuantReg: 13.89714) QuantErr: 13.89714 batch_time=0.98594
Train Epoch: 2 [329/500 21056/32000 (66%)] Loss: 1.81512 (QuantReg: 13.85850) QuantErr: 13.85850 batch_time=0.44815
Train Epoch: 2 [337/500 21568/32000 (67%)] Loss: 1.92257 (QuantReg: 13.81502) QuantErr: 13.81502 batch_time=0.46580
Train Epoch: 2 [345/500 22080/32000 (69%)] Loss: 2.89609 (QuantReg: 13.77586) QuantErr: 13.77586 batch_time=0.45155
Train Epoch: 2 [353/500 22592/32000 (71%)] Loss: 2.56913 (QuantReg: 13.82472) QuantErr: 13.82472 batch_time=0.47432
Train Epoch: 2 [361/500 23104/32000 (72%)] Loss: 2.42609 (QuantReg: 13.84679) QuantErr: 13.84679 batch_time=0.45069
Train Epoch: 2 [369/500 23616/32000 (74%)] Loss: 2.75916 (QuantReg: 13.63808) QuantErr: 13.63808 batch_time=0.45060
Train Epoch: 2 [377/500 24128/32000 (75%)] Loss: 2.87178 (QuantReg: 13.43409) QuantErr: 13.43409 batch_time=0.50236
Train Epoch: 2 [385/500 24640/32000 (77%)] Loss: 1.93589 (QuantReg: 14.05791) QuantErr: 14.05791 batch_time=0.97736
Train Epoch: 2 [393/500 25152/32000 (79%)] Loss: 2.80028 (QuantReg: 13.79401) QuantErr: 13.79401 batch_time=0.45209
Train Epoch: 2 [401/500 25664/32000 (80%)] Loss: 2.24260 (QuantReg: 13.61337) QuantErr: 13.61337 batch_time=0.46196
Train Epoch: 2 [409/500 26176/32000 (82%)] Loss: 1.99017 (QuantReg: 14.35624) QuantErr: 14.35624 batch_time=0.51352
Train Epoch: 2 [417/500 26688/32000 (83%)] Loss: 2.50725 (QuantReg: 13.69607) QuantErr: 13.69607 batch_time=0.49081
Train Epoch: 2 [425/500 27200/32000 (85%)] Loss: 2.28266 (QuantReg: 13.82867) QuantErr: 13.82867 batch_time=0.44456
Train Epoch: 2 [433/500 27712/32000 (87%)] Loss: 2.74086 (QuantReg: 14.00528) QuantErr: 14.00528 batch_time=0.41262
Train Epoch: 2 [441/500 28224/32000 (88%)] Loss: 2.83342 (QuantReg: 13.86763) QuantErr: 13.86763 batch_time=0.43291
Train Epoch: 2 [449/500 28736/32000 (90%)] Loss: 1.92974 (QuantReg: 13.90391) QuantErr: 13.90391 batch_time=0.97424
Train Epoch: 2 [457/500 29248/32000 (91%)] Loss: 2.30065 (QuantReg: 14.34379) QuantErr: 14.34379 batch_time=0.43963
Train Epoch: 2 [465/500 29760/32000 (93%)] Loss: 2.32413 (QuantReg: 13.79689) QuantErr: 13.79689 batch_time=0.50842
Train Epoch: 2 [473/500 30272/32000 (95%)] Loss: 2.88024 (QuantReg: 14.18608) QuantErr: 14.18608 batch_time=0.49813
Train Epoch: 2 [481/500 30784/32000 (96%)] Loss: 2.40285 (QuantReg: 14.30106) QuantErr: 14.30106 batch_time=0.43867
Train Epoch: 2 [489/500 31296/32000 (98%)] Loss: 2.18404 (QuantReg: 14.25579) QuantErr: 14.25579 batch_time=0.43897
Train Epoch: 2 [497/500 31808/32000 (99%)] Loss: 2.17708 (QuantReg: 14.26103) QuantErr: 14.26103 batch_time=0.50595
Train Epoch: 2 codebook_update_time=2.89804
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch2.pth ...
Done in 4.849s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch2.pth ...
Done in 10.258s
removing stale ckpt [epoch 1] [took 0.01s]
removing stale ckpt [epoch 0] [took 0.01s]
epoch : 2
loss : 2.6039750611782075
quant_reg : 13.16470530128479
quant_err : 13.16470530128479
learning_rate : 4.75e-05
n_samples : 64000
n_steps : 1000
MSRVTT_miech_test/t2v_metrics/R1: 14.3
MSRVTT_miech_test/t2v_metrics/R5: 36.9
MSRVTT_miech_test/t2v_metrics/R10: 54.6
MSRVTT_miech_test/t2v_metrics/R50: 83.5
MSRVTT_miech_test/t2v_metrics/MedR: 9.0
MSRVTT_miech_test/t2v_metrics/MeanR: 39.201
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 30.65620197974316
MSRVTT_miech_test/v2t_metrics/R1: 14.9
MSRVTT_miech_test/v2t_metrics/R5: 39.9
MSRVTT_miech_test/v2t_metrics/R10: 53.9
MSRVTT_miech_test/v2t_metrics/R50: 84.0
MSRVTT_miech_test/v2t_metrics/MedR: 9.0
MSRVTT_miech_test/v2t_metrics/MeanR: 37.628
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 31.762594957558992
mnt_best : 30.65620197974316
not_improved_count: 0
Train Epoch: 3 [1/500 64/32000 (0%)] Loss: 2.18030 (QuantReg: 11.80213) QuantErr: 11.80213 batch_time=36.10986
Train Epoch: 3 [9/500 576/32000 (2%)] Loss: 2.10700 (QuantReg: 11.79395) QuantErr: 11.79395 batch_time=0.52540
Train Epoch: 3 [17/500 1088/32000 (3%)] Loss: 1.99392 (QuantReg: 11.43971) QuantErr: 11.43971 batch_time=0.48002
Train Epoch: 3 [25/500 1600/32000 (5%)] Loss: 2.55489 (QuantReg: 11.87774) QuantErr: 11.87774 batch_time=0.48931
Train Epoch: 3 [33/500 2112/32000 (7%)] Loss: 2.30214 (QuantReg: 12.18901) QuantErr: 12.18901 batch_time=0.55480
Train Epoch: 3 [41/500 2624/32000 (8%)] Loss: 2.67283 (QuantReg: 11.72664) QuantErr: 11.72664 batch_time=0.49767
Train Epoch: 3 [49/500 3136/32000 (10%)] Loss: 2.67267 (QuantReg: 11.72926) QuantErr: 11.72926 batch_time=0.42271
Train Epoch: 3 [57/500 3648/32000 (11%)] Loss: 2.61369 (QuantReg: 11.22631) QuantErr: 11.22631 batch_time=0.40284
Train Epoch: 3 [65/500 4160/32000 (13%)] Loss: 2.42297 (QuantReg: 11.57442) QuantErr: 11.57442 batch_time=2.30504
Train Epoch: 3 [73/500 4672/32000 (15%)] Loss: 1.95861 (QuantReg: 11.78656) QuantErr: 11.78656 batch_time=0.42070
Train Epoch: 3 [81/500 5184/32000 (16%)] Loss: 2.52319 (QuantReg: 12.32985) QuantErr: 12.32985 batch_time=0.39801
Train Epoch: 3 [89/500 5696/32000 (18%)] Loss: 2.02544 (QuantReg: 12.19769) QuantErr: 12.19769 batch_time=0.43538
Train Epoch: 3 [97/500 6208/32000 (19%)] Loss: 2.80291 (QuantReg: 11.82364) QuantErr: 11.82364 batch_time=0.48269
Train Epoch: 3 [105/500 6720/32000 (21%)] Loss: 2.49499 (QuantReg: 12.33361) QuantErr: 12.33361 batch_time=0.41027
Train Epoch: 3 [113/500 7232/32000 (23%)] Loss: 1.83366 (QuantReg: 12.14582) QuantErr: 12.14582 batch_time=0.58142
Train Epoch: 3 [121/500 7744/32000 (24%)] Loss: 1.97037 (QuantReg: 12.46274) QuantErr: 12.46274 batch_time=0.43835
Train Epoch: 3 [129/500 8256/32000 (26%)] Loss: 2.25651 (QuantReg: 12.83375) QuantErr: 12.83375 batch_time=2.53212
Train Epoch: 3 [137/500 8768/32000 (27%)] Loss: 1.89999 (QuantReg: 12.49448) QuantErr: 12.49448 batch_time=0.41047
Train Epoch: 3 [145/500 9280/32000 (29%)] Loss: 2.01861 (QuantReg: 12.33278) QuantErr: 12.33278 batch_time=0.45622
Train Epoch: 3 [153/500 9792/32000 (31%)] Loss: 2.99276 (QuantReg: 12.47216) QuantErr: 12.47216 batch_time=0.43462
Train Epoch: 3 [161/500 10304/32000 (32%)] Loss: 2.01008 (QuantReg: 12.56074) QuantErr: 12.56074 batch_time=0.46984
Train Epoch: 3 [169/500 10816/32000 (34%)] Loss: 2.23035 (QuantReg: 12.04363) QuantErr: 12.04363 batch_time=0.43167
Train Epoch: 3 [177/500 11328/32000 (35%)] Loss: 2.33641 (QuantReg: 12.29197) QuantErr: 12.29197 batch_time=0.49025
Train Epoch: 3 [185/500 11840/32000 (37%)] Loss: 2.56330 (QuantReg: 12.35677) QuantErr: 12.35677 batch_time=0.48074
Train Epoch: 3 [193/500 12352/32000 (39%)] Loss: 2.58356 (QuantReg: 12.61258) QuantErr: 12.61258 batch_time=2.67454
Train Epoch: 3 [201/500 12864/32000 (40%)] Loss: 2.14092 (QuantReg: 12.24232) QuantErr: 12.24232 batch_time=0.47277
Train Epoch: 3 [209/500 13376/32000 (42%)] Loss: 1.94027 (QuantReg: 12.41714) QuantErr: 12.41714 batch_time=0.60404
Train Epoch: 3 [217/500 13888/32000 (43%)] Loss: 2.25864 (QuantReg: 12.56966) QuantErr: 12.56966 batch_time=0.46298
Train Epoch: 3 [225/500 14400/32000 (45%)] Loss: 2.07370 (QuantReg: 12.42604) QuantErr: 12.42604 batch_time=0.52091
Train Epoch: 3 [233/500 14912/32000 (47%)] Loss: 2.23376 (QuantReg: 12.78770) QuantErr: 12.78770 batch_time=0.46056
Train Epoch: 3 [241/500 15424/32000 (48%)] Loss: 2.14863 (QuantReg: 12.94302) QuantErr: 12.94302 batch_time=0.50759
Train Epoch: 3 [249/500 15936/32000 (50%)] Loss: 2.20012 (QuantReg: 12.37574) QuantErr: 12.37574 batch_time=0.47140
Train Epoch: 3 [257/500 16448/32000 (51%)] Loss: 1.65446 (QuantReg: 12.53732) QuantErr: 12.53732 batch_time=2.73439
Train Epoch: 3 [265/500 16960/32000 (53%)] Loss: 3.07125 (QuantReg: 12.88729) QuantErr: 12.88729 batch_time=0.40687
Train Epoch: 3 [273/500 17472/32000 (55%)] Loss: 2.06571 (QuantReg: 12.53778) QuantErr: 12.53778 batch_time=0.40406
Train Epoch: 3 [281/500 17984/32000 (56%)] Loss: 2.07359 (QuantReg: 12.58630) QuantErr: 12.58630 batch_time=0.39971
Train Epoch: 3 [289/500 18496/32000 (58%)] Loss: 2.07516 (QuantReg: 12.48426) QuantErr: 12.48426 batch_time=0.49957
Train Epoch: 3 [297/500 19008/32000 (59%)] Loss: 1.85982 (QuantReg: 12.10082) QuantErr: 12.10082 batch_time=0.40229
Train Epoch: 3 [305/500 19520/32000 (61%)] Loss: 2.01447 (QuantReg: 12.41943) QuantErr: 12.41943 batch_time=0.40840
Train Epoch: 3 [313/500 20032/32000 (63%)] Loss: 2.26052 (QuantReg: 12.88373) QuantErr: 12.88373 batch_time=0.44886
Train Epoch: 3 [321/500 20544/32000 (64%)] Loss: 2.82974 (QuantReg: 12.47017) QuantErr: 12.47017 batch_time=2.63279
Train Epoch: 3 [329/500 21056/32000 (66%)] Loss: 1.99766 (QuantReg: 12.90596) QuantErr: 12.90596 batch_time=0.44874
Train Epoch: 3 [337/500 21568/32000 (67%)] Loss: 1.67316 (QuantReg: 12.75562) QuantErr: 12.75562 batch_time=0.43531
Train Epoch: 3 [345/500 22080/32000 (69%)] Loss: 2.19226 (QuantReg: 13.08959) QuantErr: 13.08959 batch_time=0.41990
Train Epoch: 3 [353/500 22592/32000 (71%)] Loss: 2.25768 (QuantReg: 12.66493) QuantErr: 12.66493 batch_time=0.46517
Train Epoch: 3 [361/500 23104/32000 (72%)] Loss: 2.63788 (QuantReg: 12.72449) QuantErr: 12.72449 batch_time=0.47353
Train Epoch: 3 [369/500 23616/32000 (74%)] Loss: 2.30565 (QuantReg: 12.65969) QuantErr: 12.65969 batch_time=0.40789
Train Epoch: 3 [377/500 24128/32000 (75%)] Loss: 1.75709 (QuantReg: 12.61146) QuantErr: 12.61146 batch_time=0.39690
Train Epoch: 3 [385/500 24640/32000 (77%)] Loss: 1.98541 (QuantReg: 12.97916) QuantErr: 12.97916 batch_time=3.20633
Train Epoch: 3 [393/500 25152/32000 (79%)] Loss: 1.93858 (QuantReg: 12.64315) QuantErr: 12.64315 batch_time=0.46780
Train Epoch: 3 [401/500 25664/32000 (80%)] Loss: 1.82696 (QuantReg: 12.81874) QuantErr: 12.81874 batch_time=0.45033
Train Epoch: 3 [409/500 26176/32000 (82%)] Loss: 2.37164 (QuantReg: 12.55046) QuantErr: 12.55046 batch_time=0.41812
Train Epoch: 3 [417/500 26688/32000 (83%)] Loss: 1.74740 (QuantReg: 13.29182) QuantErr: 13.29182 batch_time=0.43832
Train Epoch: 3 [425/500 27200/32000 (85%)] Loss: 2.22175 (QuantReg: 12.79917) QuantErr: 12.79917 batch_time=0.46188
Train Epoch: 3 [433/500 27712/32000 (87%)] Loss: 1.94500 (QuantReg: 13.55361) QuantErr: 13.55361 batch_time=0.46997
Train Epoch: 3 [441/500 28224/32000 (88%)] Loss: 1.78179 (QuantReg: 12.98005) QuantErr: 12.98005 batch_time=0.46422
Train Epoch: 3 [449/500 28736/32000 (90%)] Loss: 1.90222 (QuantReg: 12.48379) QuantErr: 12.48379 batch_time=2.67340
Train Epoch: 3 [457/500 29248/32000 (91%)] Loss: 2.06475 (QuantReg: 12.83208) QuantErr: 12.83208 batch_time=0.50311
Train Epoch: 3 [465/500 29760/32000 (93%)] Loss: 1.80476 (QuantReg: 12.71628) QuantErr: 12.71628 batch_time=0.49811
Train Epoch: 3 [473/500 30272/32000 (95%)] Loss: 2.49712 (QuantReg: 12.93897) QuantErr: 12.93897 batch_time=0.55178
Train Epoch: 3 [481/500 30784/32000 (96%)] Loss: 1.48182 (QuantReg: 13.04478) QuantErr: 13.04478 batch_time=0.51242
Train Epoch: 3 [489/500 31296/32000 (98%)] Loss: 2.08169 (QuantReg: 12.88307) QuantErr: 12.88307 batch_time=0.40999
Train Epoch: 3 [497/500 31808/32000 (99%)] Loss: 2.13722 (QuantReg: 12.75874) QuantErr: 12.75874 batch_time=0.45935
Train Epoch: 3 codebook_update_time=2.26390
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch3.pth ...
Done in 4.973s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch3.pth ...
Done in 10.013s
removing stale ckpt [epoch 2] [took 0.01s]
epoch : 3
loss : 2.184857651472092
quant_reg : 12.460440084457398
quant_err : 12.460440084457398
learning_rate : 4.5125e-05
n_samples : 96000
n_steps : 1500
MSRVTT_miech_test/t2v_metrics/R1: 14.0
MSRVTT_miech_test/t2v_metrics/R5: 40.5
MSRVTT_miech_test/t2v_metrics/R10: 53.9
MSRVTT_miech_test/t2v_metrics/R50: 85.2
MSRVTT_miech_test/t2v_metrics/MedR: 9.0
MSRVTT_miech_test/t2v_metrics/MeanR: 35.019
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 31.264916612030362
MSRVTT_miech_test/v2t_metrics/R1: 15.6
MSRVTT_miech_test/v2t_metrics/R5: 40.5
MSRVTT_miech_test/v2t_metrics/R10: 55.1
MSRVTT_miech_test/v2t_metrics/R50: 85.5
MSRVTT_miech_test/v2t_metrics/MedR: 8.0
MSRVTT_miech_test/v2t_metrics/MeanR: 33.6445
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 32.65204653739984
mnt_best : 31.264916612030362
not_improved_count: 0
Train Epoch: 4 [1/500 64/32000 (0%)] Loss: 1.86584 (QuantReg: 11.76178) QuantErr: 11.76178 batch_time=30.29378
Train Epoch: 4 [9/500 576/32000 (2%)] Loss: 1.88952 (QuantReg: 12.23775) QuantErr: 12.23775 batch_time=0.47551
Train Epoch: 4 [17/500 1088/32000 (3%)] Loss: 2.63090 (QuantReg: 11.62045) QuantErr: 11.62045 batch_time=2.07593
Train Epoch: 4 [25/500 1600/32000 (5%)] Loss: 1.63397 (QuantReg: 12.13112) QuantErr: 12.13112 batch_time=0.41854
Train Epoch: 4 [33/500 2112/32000 (7%)] Loss: 1.89244 (QuantReg: 12.46095) QuantErr: 12.46095 batch_time=0.52430
Train Epoch: 4 [41/500 2624/32000 (8%)] Loss: 2.49700 (QuantReg: 11.57185) QuantErr: 11.57185 batch_time=0.44880
Train Epoch: 4 [49/500 3136/32000 (10%)] Loss: 1.73788 (QuantReg: 12.20028) QuantErr: 12.20028 batch_time=0.44950
Train Epoch: 4 [57/500 3648/32000 (11%)] Loss: 2.00902 (QuantReg: 11.74708) QuantErr: 11.74708 batch_time=0.46049
Train Epoch: 4 [65/500 4160/32000 (13%)] Loss: 2.30735 (QuantReg: 11.49040) QuantErr: 11.49040 batch_time=1.22625
Train Epoch: 4 [73/500 4672/32000 (15%)] Loss: 1.75649 (QuantReg: 12.41536) QuantErr: 12.41536 batch_time=0.56900
Train Epoch: 4 [81/500 5184/32000 (16%)] Loss: 1.94976 (QuantReg: 12.04239) QuantErr: 12.04239 batch_time=1.57968
Train Epoch: 4 [89/500 5696/32000 (18%)] Loss: 2.09430 (QuantReg: 12.31594) QuantErr: 12.31594 batch_time=0.39509
Train Epoch: 4 [97/500 6208/32000 (19%)] Loss: 1.99608 (QuantReg: 12.38807) QuantErr: 12.38807 batch_time=0.46186
Train Epoch: 4 [105/500 6720/32000 (21%)] Loss: 1.89851 (QuantReg: 11.89798) QuantErr: 11.89798 batch_time=0.42675
Train Epoch: 4 [113/500 7232/32000 (23%)] Loss: 2.16836 (QuantReg: 12.21844) QuantErr: 12.21844 batch_time=0.47317
Train Epoch: 4 [121/500 7744/32000 (24%)] Loss: 1.56834 (QuantReg: 11.55428) QuantErr: 11.55428 batch_time=0.47400
Train Epoch: 4 [129/500 8256/32000 (26%)] Loss: 1.63069 (QuantReg: 12.02200) QuantErr: 12.02200 batch_time=1.24508
Train Epoch: 4 [137/500 8768/32000 (27%)] Loss: 2.10442 (QuantReg: 11.92870) QuantErr: 11.92870 batch_time=0.47405
Train Epoch: 4 [145/500 9280/32000 (29%)] Loss: 1.63423 (QuantReg: 12.76367) QuantErr: 12.76367 batch_time=1.63136
Train Epoch: 4 [153/500 9792/32000 (31%)] Loss: 1.43390 (QuantReg: 12.38485) QuantErr: 12.38485 batch_time=0.44703
Train Epoch: 4 [161/500 10304/32000 (32%)] Loss: 2.02552 (QuantReg: 12.35194) QuantErr: 12.35194 batch_time=0.45841
Train Epoch: 4 [169/500 10816/32000 (34%)] Loss: 1.74882 (QuantReg: 13.03497) QuantErr: 13.03497 batch_time=0.49160
Train Epoch: 4 [177/500 11328/32000 (35%)] Loss: 2.05428 (QuantReg: 12.49060) QuantErr: 12.49060 batch_time=0.45284
Train Epoch: 4 [185/500 11840/32000 (37%)] Loss: 1.47528 (QuantReg: 11.78984) QuantErr: 11.78984 batch_time=0.42459
Train Epoch: 4 [193/500 12352/32000 (39%)] Loss: 1.83878 (QuantReg: 12.22366) QuantErr: 12.22366 batch_time=1.24129
Train Epoch: 4 [201/500 12864/32000 (40%)] Loss: 1.89844 (QuantReg: 12.43092) QuantErr: 12.43092 batch_time=0.41933
Train Epoch: 4 [209/500 13376/32000 (42%)] Loss: 1.52845 (QuantReg: 12.04218) QuantErr: 12.04218 batch_time=1.58456
Train Epoch: 4 [217/500 13888/32000 (43%)] Loss: 1.97620 (QuantReg: 12.48612) QuantErr: 12.48612 batch_time=0.49803
Train Epoch: 4 [225/500 14400/32000 (45%)] Loss: 2.37783 (QuantReg: 11.84620) QuantErr: 11.84620 batch_time=0.49161
Train Epoch: 4 [233/500 14912/32000 (47%)] Loss: 1.96215 (QuantReg: 11.95651) QuantErr: 11.95651 batch_time=0.51404
Train Epoch: 4 [241/500 15424/32000 (48%)] Loss: 2.15777 (QuantReg: 12.33015) QuantErr: 12.33015 batch_time=0.47570
Train Epoch: 4 [249/500 15936/32000 (50%)] Loss: 1.80609 (QuantReg: 12.71112) QuantErr: 12.71112 batch_time=0.47505
Train Epoch: 4 [257/500 16448/32000 (51%)] Loss: 1.57029 (QuantReg: 12.61343) QuantErr: 12.61343 batch_time=1.29386
Train Epoch: 4 [265/500 16960/32000 (53%)] Loss: 1.95411 (QuantReg: 12.72124) QuantErr: 12.72124 batch_time=0.43622
Train Epoch: 4 [273/500 17472/32000 (55%)] Loss: 2.36210 (QuantReg: 12.52159) QuantErr: 12.52159 batch_time=1.52009
Train Epoch: 4 [281/500 17984/32000 (56%)] Loss: 1.63589 (QuantReg: 12.68240) QuantErr: 12.68240 batch_time=0.44307
Train Epoch: 4 [289/500 18496/32000 (58%)] Loss: 1.27261 (QuantReg: 12.55503) QuantErr: 12.55503 batch_time=0.45454
Train Epoch: 4 [297/500 19008/32000 (59%)] Loss: 1.43424 (QuantReg: 12.50386) QuantErr: 12.50386 batch_time=0.48864
Train Epoch: 4 [305/500 19520/32000 (61%)] Loss: 1.58699 (QuantReg: 12.27264) QuantErr: 12.27264 batch_time=0.46939
Train Epoch: 4 [313/500 20032/32000 (63%)] Loss: 1.50383 (QuantReg: 12.28135) QuantErr: 12.28135 batch_time=0.47338
Train Epoch: 4 [321/500 20544/32000 (64%)] Loss: 1.87073 (QuantReg: 12.41397) QuantErr: 12.41397 batch_time=1.24737
Train Epoch: 4 [329/500 21056/32000 (66%)] Loss: 1.38509 (QuantReg: 12.30655) QuantErr: 12.30655 batch_time=0.44180
Train Epoch: 4 [337/500 21568/32000 (67%)] Loss: 1.49479 (QuantReg: 12.58740) QuantErr: 12.58740 batch_time=1.68239
Train Epoch: 4 [345/500 22080/32000 (69%)] Loss: 1.58346 (QuantReg: 12.79800) QuantErr: 12.79800 batch_time=0.43218
Train Epoch: 4 [353/500 22592/32000 (71%)] Loss: 1.83337 (QuantReg: 13.18153) QuantErr: 13.18153 batch_time=0.52009
Train Epoch: 4 [361/500 23104/32000 (72%)] Loss: 1.86937 (QuantReg: 12.70987) QuantErr: 12.70987 batch_time=0.44501
Train Epoch: 4 [369/500 23616/32000 (74%)] Loss: 1.99156 (QuantReg: 12.85652) QuantErr: 12.85652 batch_time=0.42973
Train Epoch: 4 [377/500 24128/32000 (75%)] Loss: 1.57703 (QuantReg: 12.74110) QuantErr: 12.74110 batch_time=0.42272
Train Epoch: 4 [385/500 24640/32000 (77%)] Loss: 2.10272 (QuantReg: 12.73159) QuantErr: 12.73159 batch_time=1.22171
Train Epoch: 4 [393/500 25152/32000 (79%)] Loss: 1.47591 (QuantReg: 12.98123) QuantErr: 12.98123 batch_time=0.40978
Train Epoch: 4 [401/500 25664/32000 (80%)] Loss: 1.72391 (QuantReg: 12.77159) QuantErr: 12.77159 batch_time=1.50124
Train Epoch: 4 [409/500 26176/32000 (82%)] Loss: 1.81793 (QuantReg: 13.20210) QuantErr: 13.20210 batch_time=0.52696
Train Epoch: 4 [417/500 26688/32000 (83%)] Loss: 2.04119 (QuantReg: 12.32269) QuantErr: 12.32269 batch_time=0.54574
Train Epoch: 4 [425/500 27200/32000 (85%)] Loss: 1.70492 (QuantReg: 12.88768) QuantErr: 12.88768 batch_time=0.43555
Train Epoch: 4 [433/500 27712/32000 (87%)] Loss: 1.67503 (QuantReg: 12.64933) QuantErr: 12.64933 batch_time=0.41528
Train Epoch: 4 [441/500 28224/32000 (88%)] Loss: 1.32707 (QuantReg: 12.91637) QuantErr: 12.91637 batch_time=0.43660
Train Epoch: 4 [449/500 28736/32000 (90%)] Loss: 1.98884 (QuantReg: 12.85951) QuantErr: 12.85951 batch_time=1.18093
Train Epoch: 4 [457/500 29248/32000 (91%)] Loss: 1.62309 (QuantReg: 13.17012) QuantErr: 13.17012 batch_time=0.47114
Train Epoch: 4 [465/500 29760/32000 (93%)] Loss: 1.75556 (QuantReg: 12.66971) QuantErr: 12.66971 batch_time=1.60788
Train Epoch: 4 [473/500 30272/32000 (95%)] Loss: 1.63953 (QuantReg: 13.18196) QuantErr: 13.18196 batch_time=0.47337
Train Epoch: 4 [481/500 30784/32000 (96%)] Loss: 1.95383 (QuantReg: 12.70278) QuantErr: 12.70278 batch_time=0.49608
Train Epoch: 4 [489/500 31296/32000 (98%)] Loss: 1.65431 (QuantReg: 12.94776) QuantErr: 12.94776 batch_time=0.41680
Train Epoch: 4 [497/500 31808/32000 (99%)] Loss: 1.86081 (QuantReg: 13.57233) QuantErr: 13.57233 batch_time=0.40584
Train Epoch: 4 codebook_update_time=2.20275
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch4.pth ...
Done in 5.350s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch4.pth ...
Done in 10.271s
removing stale ckpt [epoch 3] [took 0.01s]
epoch : 4
loss : 1.8901634414196014
quant_reg : 12.394084913253785
quant_err : 12.394084913253785
learning_rate : 4.2868749999999995e-05
n_samples : 128000
n_steps : 2000
MSRVTT_miech_test/t2v_metrics/R1: 16.6
MSRVTT_miech_test/t2v_metrics/R5: 43.5
MSRVTT_miech_test/t2v_metrics/R10: 57.4
MSRVTT_miech_test/t2v_metrics/R50: 86.7
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.294
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 34.60746160926859
MSRVTT_miech_test/v2t_metrics/R1: 16.8
MSRVTT_miech_test/v2t_metrics/R5: 43.4
MSRVTT_miech_test/v2t_metrics/R10: 58.6
MSRVTT_miech_test/v2t_metrics/R50: 87.1
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 30.553
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 34.959526548509025
mnt_best : 34.60746160926859
not_improved_count: 0
Train Epoch: 5 [1/500 64/32000 (0%)] Loss: 1.99368 (QuantReg: 12.17865) QuantErr: 12.17865 batch_time=32.30713
Train Epoch: 5 [9/500 576/32000 (2%)] Loss: 2.32232 (QuantReg: 11.75036) QuantErr: 11.75036 batch_time=0.43179
Train Epoch: 5 [17/500 1088/32000 (3%)] Loss: 1.73079 (QuantReg: 12.12050) QuantErr: 12.12050 batch_time=0.41441
Train Epoch: 5 [25/500 1600/32000 (5%)] Loss: 1.35359 (QuantReg: 12.31075) QuantErr: 12.31075 batch_time=0.41369
Train Epoch: 5 [33/500 2112/32000 (7%)] Loss: 2.32086 (QuantReg: 12.55247) QuantErr: 12.55247 batch_time=0.43794
Train Epoch: 5 [41/500 2624/32000 (8%)] Loss: 1.60859 (QuantReg: 11.95379) QuantErr: 11.95379 batch_time=0.43884
Train Epoch: 5 [49/500 3136/32000 (10%)] Loss: 1.26397 (QuantReg: 12.06956) QuantErr: 12.06956 batch_time=0.41759
Train Epoch: 5 [57/500 3648/32000 (11%)] Loss: 1.66659 (QuantReg: 12.59511) QuantErr: 12.59511 batch_time=0.43995
Train Epoch: 5 [65/500 4160/32000 (13%)] Loss: 1.74646 (QuantReg: 12.52991) QuantErr: 12.52991 batch_time=2.68180
Train Epoch: 5 [73/500 4672/32000 (15%)] Loss: 1.83375 (QuantReg: 12.31959) QuantErr: 12.31959 batch_time=0.43906
Train Epoch: 5 [81/500 5184/32000 (16%)] Loss: 1.78811 (QuantReg: 12.68943) QuantErr: 12.68943 batch_time=0.43194
Train Epoch: 5 [89/500 5696/32000 (18%)] Loss: 2.22426 (QuantReg: 12.35760) QuantErr: 12.35760 batch_time=0.44899
Train Epoch: 5 [97/500 6208/32000 (19%)] Loss: 1.77296 (QuantReg: 12.39280) QuantErr: 12.39280 batch_time=0.41020
Train Epoch: 5 [105/500 6720/32000 (21%)] Loss: 1.70716 (QuantReg: 12.53420) QuantErr: 12.53420 batch_time=0.55884
Train Epoch: 5 [113/500 7232/32000 (23%)] Loss: 2.02518 (QuantReg: 12.39881) QuantErr: 12.39881 batch_time=0.44290
Train Epoch: 5 [121/500 7744/32000 (24%)] Loss: 1.85536 (QuantReg: 12.30526) QuantErr: 12.30526 batch_time=0.46215
Train Epoch: 5 [129/500 8256/32000 (26%)] Loss: 1.59791 (QuantReg: 12.42286) QuantErr: 12.42286 batch_time=3.10540
Train Epoch: 5 [137/500 8768/32000 (27%)] Loss: 1.74654 (QuantReg: 11.98316) QuantErr: 11.98316 batch_time=0.46474
Train Epoch: 5 [145/500 9280/32000 (29%)] Loss: 1.77824 (QuantReg: 12.55652) QuantErr: 12.55652 batch_time=0.49738
Train Epoch: 5 [153/500 9792/32000 (31%)] Loss: 1.61779 (QuantReg: 12.39413) QuantErr: 12.39413 batch_time=0.48256
Train Epoch: 5 [161/500 10304/32000 (32%)] Loss: 1.36432 (QuantReg: 12.12751) QuantErr: 12.12751 batch_time=0.50880
Train Epoch: 5 [169/500 10816/32000 (34%)] Loss: 1.83999 (QuantReg: 12.54591) QuantErr: 12.54591 batch_time=0.48266
Train Epoch: 5 [177/500 11328/32000 (35%)] Loss: 1.74575 (QuantReg: 12.04678) QuantErr: 12.04678 batch_time=0.46421
Train Epoch: 5 [185/500 11840/32000 (37%)] Loss: 1.78347 (QuantReg: 12.50024) QuantErr: 12.50024 batch_time=0.46817
Train Epoch: 5 [193/500 12352/32000 (39%)] Loss: 2.04298 (QuantReg: 12.59157) QuantErr: 12.59157 batch_time=2.77731
Train Epoch: 5 [201/500 12864/32000 (40%)] Loss: 2.17398 (QuantReg: 12.19317) QuantErr: 12.19317 batch_time=0.42790
Train Epoch: 5 [209/500 13376/32000 (42%)] Loss: 1.60835 (QuantReg: 12.74063) QuantErr: 12.74063 batch_time=0.45238
Train Epoch: 5 [217/500 13888/32000 (43%)] Loss: 1.78107 (QuantReg: 12.44697) QuantErr: 12.44697 batch_time=0.46429
Train Epoch: 5 [225/500 14400/32000 (45%)] Loss: 2.01369 (QuantReg: 12.23605) QuantErr: 12.23605 batch_time=0.40904
Train Epoch: 5 [233/500 14912/32000 (47%)] Loss: 1.67027 (QuantReg: 12.52401) QuantErr: 12.52401 batch_time=0.39865
Train Epoch: 5 [241/500 15424/32000 (48%)] Loss: 1.91463 (QuantReg: 12.40037) QuantErr: 12.40037 batch_time=0.47188
Train Epoch: 5 [249/500 15936/32000 (50%)] Loss: 2.32583 (QuantReg: 12.45362) QuantErr: 12.45362 batch_time=0.42985
Train Epoch: 5 [257/500 16448/32000 (51%)] Loss: 2.08368 (QuantReg: 12.12322) QuantErr: 12.12322 batch_time=2.81410
Train Epoch: 5 [265/500 16960/32000 (53%)] Loss: 2.04825 (QuantReg: 12.76051) QuantErr: 12.76051 batch_time=0.44393
Train Epoch: 5 [273/500 17472/32000 (55%)] Loss: 1.45047 (QuantReg: 12.85554) QuantErr: 12.85554 batch_time=0.43518
Train Epoch: 5 [281/500 17984/32000 (56%)] Loss: 1.82128 (QuantReg: 12.77593) QuantErr: 12.77593 batch_time=0.53529
Train Epoch: 5 [289/500 18496/32000 (58%)] Loss: 1.61425 (QuantReg: 12.66634) QuantErr: 12.66634 batch_time=0.43264
Train Epoch: 5 [297/500 19008/32000 (59%)] Loss: 1.95562 (QuantReg: 12.85970) QuantErr: 12.85970 batch_time=0.45887
Train Epoch: 5 [305/500 19520/32000 (61%)] Loss: 1.41433 (QuantReg: 12.77836) QuantErr: 12.77836 batch_time=0.46652
Train Epoch: 5 [313/500 20032/32000 (63%)] Loss: 1.67431 (QuantReg: 12.38724) QuantErr: 12.38724 batch_time=0.44699
Train Epoch: 5 [321/500 20544/32000 (64%)] Loss: 1.89668 (QuantReg: 12.28319) QuantErr: 12.28319 batch_time=2.81426
Train Epoch: 5 [329/500 21056/32000 (66%)] Loss: 1.91038 (QuantReg: 11.94040) QuantErr: 11.94040 batch_time=0.41989
Train Epoch: 5 [337/500 21568/32000 (67%)] Loss: 1.67415 (QuantReg: 12.55987) QuantErr: 12.55987 batch_time=0.43652
Train Epoch: 5 [345/500 22080/32000 (69%)] Loss: 1.78444 (QuantReg: 12.32649) QuantErr: 12.32649 batch_time=0.47666
Train Epoch: 5 [353/500 22592/32000 (71%)] Loss: 1.65810 (QuantReg: 12.75825) QuantErr: 12.75825 batch_time=0.52411
Train Epoch: 5 [361/500 23104/32000 (72%)] Loss: 1.61765 (QuantReg: 12.77230) QuantErr: 12.77230 batch_time=0.41849
Train Epoch: 5 [369/500 23616/32000 (74%)] Loss: 1.70292 (QuantReg: 12.11658) QuantErr: 12.11658 batch_time=0.43351
Train Epoch: 5 [377/500 24128/32000 (75%)] Loss: 1.41619 (QuantReg: 12.54065) QuantErr: 12.54065 batch_time=0.54449
Train Epoch: 5 [385/500 24640/32000 (77%)] Loss: 1.54994 (QuantReg: 12.93802) QuantErr: 12.93802 batch_time=2.70139
Train Epoch: 5 [393/500 25152/32000 (79%)] Loss: 1.66609 (QuantReg: 13.23559) QuantErr: 13.23559 batch_time=0.41241
Train Epoch: 5 [401/500 25664/32000 (80%)] Loss: 2.42002 (QuantReg: 12.51219) QuantErr: 12.51219 batch_time=0.39997
Train Epoch: 5 [409/500 26176/32000 (82%)] Loss: 1.26322 (QuantReg: 12.69090) QuantErr: 12.69090 batch_time=0.46918
Train Epoch: 5 [417/500 26688/32000 (83%)] Loss: 1.80410 (QuantReg: 12.61777) QuantErr: 12.61777 batch_time=0.44721
Train Epoch: 5 [425/500 27200/32000 (85%)] Loss: 1.88740 (QuantReg: 12.66142) QuantErr: 12.66142 batch_time=0.47094
Train Epoch: 5 [433/500 27712/32000 (87%)] Loss: 1.24539 (QuantReg: 12.95887) QuantErr: 12.95887 batch_time=0.47901
Train Epoch: 5 [441/500 28224/32000 (88%)] Loss: 1.60295 (QuantReg: 12.71714) QuantErr: 12.71714 batch_time=0.47854
Train Epoch: 5 [449/500 28736/32000 (90%)] Loss: 2.00912 (QuantReg: 12.76694) QuantErr: 12.76694 batch_time=2.70566
Train Epoch: 5 [457/500 29248/32000 (91%)] Loss: 1.37917 (QuantReg: 13.07587) QuantErr: 13.07587 batch_time=0.44471
Train Epoch: 5 [465/500 29760/32000 (93%)] Loss: 1.56469 (QuantReg: 13.31014) QuantErr: 13.31014 batch_time=0.51873
Train Epoch: 5 [473/500 30272/32000 (95%)] Loss: 1.64040 (QuantReg: 13.19507) QuantErr: 13.19507 batch_time=0.50773
Train Epoch: 5 [481/500 30784/32000 (96%)] Loss: 1.60965 (QuantReg: 12.87023) QuantErr: 12.87023 batch_time=0.45586
Train Epoch: 5 [489/500 31296/32000 (98%)] Loss: 1.98184 (QuantReg: 12.49057) QuantErr: 12.49057 batch_time=0.46319
Train Epoch: 5 [497/500 31808/32000 (99%)] Loss: 1.72644 (QuantReg: 12.99829) QuantErr: 12.99829 batch_time=0.50652
Train Epoch: 5 codebook_update_time=2.00601
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch5.pth ...
Done in 5.116s
removing stale ckpt [epoch 4] [took 0.00s]
epoch : 5
loss : 1.6862954934835435
quant_reg : 12.525859945297242
quant_err : 12.525859945297242
learning_rate : 4.072531249999999e-05
n_samples : 160000
n_steps : 2500
MSRVTT_miech_test/t2v_metrics/R1: 15.5
MSRVTT_miech_test/t2v_metrics/R5: 43.4
MSRVTT_miech_test/t2v_metrics/R10: 57.9
MSRVTT_miech_test/t2v_metrics/R50: 86.5
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 31.751
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 33.897421510966666
MSRVTT_miech_test/v2t_metrics/R1: 16.4
MSRVTT_miech_test/v2t_metrics/R5: 45.0
MSRVTT_miech_test/v2t_metrics/R10: 59.0
MSRVTT_miech_test/v2t_metrics/R50: 86.1
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 30.629
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 35.180563477623394
mnt_best : 34.60746160926859
not_improved_count: 1
Train Epoch: 6 [1/500 64/32000 (0%)] Loss: 1.41491 (QuantReg: 12.01762) QuantErr: 12.01762 batch_time=27.96566
Train Epoch: 6 [9/500 576/32000 (2%)] Loss: 1.10708 (QuantReg: 12.07107) QuantErr: 12.07107 batch_time=0.45005
Train Epoch: 6 [17/500 1088/32000 (3%)] Loss: 1.37160 (QuantReg: 11.99766) QuantErr: 11.99766 batch_time=0.52998
Train Epoch: 6 [25/500 1600/32000 (5%)] Loss: 1.56911 (QuantReg: 11.78032) QuantErr: 11.78032 batch_time=0.41786
Train Epoch: 6 [33/500 2112/32000 (7%)] Loss: 1.51327 (QuantReg: 12.12950) QuantErr: 12.12950 batch_time=0.45965
Train Epoch: 6 [41/500 2624/32000 (8%)] Loss: 1.35228 (QuantReg: 12.39129) QuantErr: 12.39129 batch_time=0.44090
Train Epoch: 6 [49/500 3136/32000 (10%)] Loss: 1.50194 (QuantReg: 12.74267) QuantErr: 12.74267 batch_time=0.49241
Train Epoch: 6 [57/500 3648/32000 (11%)] Loss: 1.79286 (QuantReg: 12.73287) QuantErr: 12.73287 batch_time=0.47922
Train Epoch: 6 [65/500 4160/32000 (13%)] Loss: 1.52155 (QuantReg: 13.20605) QuantErr: 13.20605 batch_time=0.48542
Train Epoch: 6 [73/500 4672/32000 (15%)] Loss: 0.97426 (QuantReg: 11.81585) QuantErr: 11.81585 batch_time=0.45049
Train Epoch: 6 [81/500 5184/32000 (16%)] Loss: 1.34691 (QuantReg: 11.88713) QuantErr: 11.88713 batch_time=0.50583
Train Epoch: 6 [89/500 5696/32000 (18%)] Loss: 1.62898 (QuantReg: 11.76647) QuantErr: 11.76647 batch_time=0.48023
Train Epoch: 6 [97/500 6208/32000 (19%)] Loss: 1.30553 (QuantReg: 12.31618) QuantErr: 12.31618 batch_time=0.49071
Train Epoch: 6 [105/500 6720/32000 (21%)] Loss: 1.91576 (QuantReg: 12.46734) QuantErr: 12.46734 batch_time=0.45048
Train Epoch: 6 [113/500 7232/32000 (23%)] Loss: 1.62162 (QuantReg: 12.54294) QuantErr: 12.54294 batch_time=0.46812
Train Epoch: 6 [121/500 7744/32000 (24%)] Loss: 1.37731 (QuantReg: 12.69439) QuantErr: 12.69439 batch_time=0.40731
Train Epoch: 6 [129/500 8256/32000 (26%)] Loss: 1.45499 (QuantReg: 13.13073) QuantErr: 13.13073 batch_time=0.46200
Train Epoch: 6 [137/500 8768/32000 (27%)] Loss: 1.66541 (QuantReg: 12.25709) QuantErr: 12.25709 batch_time=0.51222
Train Epoch: 6 [145/500 9280/32000 (29%)] Loss: 1.55348 (QuantReg: 13.02937) QuantErr: 13.02937 batch_time=0.39272
Train Epoch: 6 [153/500 9792/32000 (31%)] Loss: 1.42355 (QuantReg: 12.25845) QuantErr: 12.25845 batch_time=0.48361
Train Epoch: 6 [161/500 10304/32000 (32%)] Loss: 1.21097 (QuantReg: 12.55017) QuantErr: 12.55017 batch_time=0.42726
Train Epoch: 6 [169/500 10816/32000 (34%)] Loss: 2.30217 (QuantReg: 12.30526) QuantErr: 12.30526 batch_time=0.40697
Train Epoch: 6 [177/500 11328/32000 (35%)] Loss: 1.39703 (QuantReg: 12.17224) QuantErr: 12.17224 batch_time=0.49766
Train Epoch: 6 [185/500 11840/32000 (37%)] Loss: 1.22261 (QuantReg: 12.75738) QuantErr: 12.75738 batch_time=0.45405
Train Epoch: 6 [193/500 12352/32000 (39%)] Loss: 1.03570 (QuantReg: 12.36463) QuantErr: 12.36463 batch_time=0.39287
Train Epoch: 6 [201/500 12864/32000 (40%)] Loss: 1.50764 (QuantReg: 12.60363) QuantErr: 12.60363 batch_time=0.46481
Train Epoch: 6 [209/500 13376/32000 (42%)] Loss: 1.72636 (QuantReg: 12.70906) QuantErr: 12.70906 batch_time=0.44973
Train Epoch: 6 [217/500 13888/32000 (43%)] Loss: 2.32504 (QuantReg: 12.16002) QuantErr: 12.16002 batch_time=0.45646
Train Epoch: 6 [225/500 14400/32000 (45%)] Loss: 1.30522 (QuantReg: 12.65286) QuantErr: 12.65286 batch_time=0.43194
Train Epoch: 6 [233/500 14912/32000 (47%)] Loss: 1.38569 (QuantReg: 12.37758) QuantErr: 12.37758 batch_time=0.46986
Train Epoch: 6 [241/500 15424/32000 (48%)] Loss: 1.47624 (QuantReg: 12.70391) QuantErr: 12.70391 batch_time=0.47627
Train Epoch: 6 [249/500 15936/32000 (50%)] Loss: 1.23195 (QuantReg: 12.57061) QuantErr: 12.57061 batch_time=0.42167
Train Epoch: 6 [257/500 16448/32000 (51%)] Loss: 1.83306 (QuantReg: 12.20422) QuantErr: 12.20422 batch_time=0.44920
Train Epoch: 6 [265/500 16960/32000 (53%)] Loss: 2.06241 (QuantReg: 12.50087) QuantErr: 12.50087 batch_time=0.43858
Train Epoch: 6 [273/500 17472/32000 (55%)] Loss: 1.23566 (QuantReg: 12.59134) QuantErr: 12.59134 batch_time=0.44537
Train Epoch: 6 [281/500 17984/32000 (56%)] Loss: 1.12484 (QuantReg: 12.84098) QuantErr: 12.84098 batch_time=0.42478
Train Epoch: 6 [289/500 18496/32000 (58%)] Loss: 1.21851 (QuantReg: 12.56406) QuantErr: 12.56406 batch_time=0.41588
Train Epoch: 6 [297/500 19008/32000 (59%)] Loss: 1.47606 (QuantReg: 12.62532) QuantErr: 12.62532 batch_time=0.45338
Train Epoch: 6 [305/500 19520/32000 (61%)] Loss: 1.38306 (QuantReg: 12.02523) QuantErr: 12.02523 batch_time=0.52726
Train Epoch: 6 [313/500 20032/32000 (63%)] Loss: 1.59348 (QuantReg: 13.10393) QuantErr: 13.10393 batch_time=0.44647
Train Epoch: 6 [321/500 20544/32000 (64%)] Loss: 1.74086 (QuantReg: 12.71668) QuantErr: 12.71668 batch_time=0.41108
Train Epoch: 6 [329/500 21056/32000 (66%)] Loss: 1.70399 (QuantReg: 12.78121) QuantErr: 12.78121 batch_time=0.54405
Train Epoch: 6 [337/500 21568/32000 (67%)] Loss: 1.28540 (QuantReg: 12.89859) QuantErr: 12.89859 batch_time=0.47120
Train Epoch: 6 [345/500 22080/32000 (69%)] Loss: 1.15952 (QuantReg: 12.56947) QuantErr: 12.56947 batch_time=0.51518
Train Epoch: 6 [353/500 22592/32000 (71%)] Loss: 1.39990 (QuantReg: 12.69602) QuantErr: 12.69602 batch_time=0.51566
Train Epoch: 6 [361/500 23104/32000 (72%)] Loss: 1.22565 (QuantReg: 13.13126) QuantErr: 13.13126 batch_time=0.48039
Train Epoch: 6 [369/500 23616/32000 (74%)] Loss: 0.77195 (QuantReg: 12.88772) QuantErr: 12.88772 batch_time=0.56890
Train Epoch: 6 [377/500 24128/32000 (75%)] Loss: 1.34675 (QuantReg: 13.15860) QuantErr: 13.15860 batch_time=0.41767
Train Epoch: 6 [385/500 24640/32000 (77%)] Loss: 1.55290 (QuantReg: 12.73950) QuantErr: 12.73950 batch_time=0.47182
Train Epoch: 6 [393/500 25152/32000 (79%)] Loss: 1.65529 (QuantReg: 12.90111) QuantErr: 12.90111 batch_time=0.47808
Train Epoch: 6 [401/500 25664/32000 (80%)] Loss: 1.39843 (QuantReg: 13.08328) QuantErr: 13.08328 batch_time=0.48557
Train Epoch: 6 [409/500 26176/32000 (82%)] Loss: 1.17474 (QuantReg: 12.69515) QuantErr: 12.69515 batch_time=0.46850
Train Epoch: 6 [417/500 26688/32000 (83%)] Loss: 1.35976 (QuantReg: 12.71531) QuantErr: 12.71531 batch_time=0.46516
Train Epoch: 6 [425/500 27200/32000 (85%)] Loss: 1.02517 (QuantReg: 13.38121) QuantErr: 13.38121 batch_time=0.50119
Train Epoch: 6 [433/500 27712/32000 (87%)] Loss: 1.20106 (QuantReg: 12.62858) QuantErr: 12.62858 batch_time=0.57569
Train Epoch: 6 [441/500 28224/32000 (88%)] Loss: 1.22644 (QuantReg: 13.29923) QuantErr: 13.29923 batch_time=0.48009
Train Epoch: 6 [449/500 28736/32000 (90%)] Loss: 0.80408 (QuantReg: 12.93303) QuantErr: 12.93303 batch_time=0.47712
Train Epoch: 6 [457/500 29248/32000 (91%)] Loss: 1.59753 (QuantReg: 12.61256) QuantErr: 12.61256 batch_time=0.42350
Train Epoch: 6 [465/500 29760/32000 (93%)] Loss: 1.88005 (QuantReg: 12.88532) QuantErr: 12.88532 batch_time=0.46254
Train Epoch: 6 [473/500 30272/32000 (95%)] Loss: 1.31445 (QuantReg: 13.11265) QuantErr: 13.11265 batch_time=0.47220
Train Epoch: 6 [481/500 30784/32000 (96%)] Loss: 1.86834 (QuantReg: 12.63626) QuantErr: 12.63626 batch_time=0.44321
Train Epoch: 6 [489/500 31296/32000 (98%)] Loss: 1.85550 (QuantReg: 13.25086) QuantErr: 13.25086 batch_time=0.49524
Train Epoch: 6 [497/500 31808/32000 (99%)] Loss: 1.74819 (QuantReg: 13.07341) QuantErr: 13.07341 batch_time=0.57702
Train Epoch: 6 codebook_update_time=2.17075
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch6.pth ...
Done in 5.323s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch6.pth ...
Done in 10.455s
removing stale ckpt [epoch 5] [took 0.07s]
epoch : 6
loss : 1.5361189658641816
quant_reg : 12.581753173828124
quant_err : 12.581753173828124
learning_rate : 3.868904687499999e-05
n_samples : 192000
n_steps : 3000
MSRVTT_miech_test/t2v_metrics/R1: 15.8
MSRVTT_miech_test/t2v_metrics/R5: 45.1
MSRVTT_miech_test/t2v_metrics/R10: 59.5
MSRVTT_miech_test/t2v_metrics/R50: 87.0
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 32.959
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 34.86985955590018
MSRVTT_miech_test/v2t_metrics/R1: 17.5
MSRVTT_miech_test/v2t_metrics/R5: 44.4
MSRVTT_miech_test/v2t_metrics/R10: 59.5
MSRVTT_miech_test/v2t_metrics/R50: 86.7
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 30.4555
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 35.89048508582687
mnt_best : 34.86985955590018
not_improved_count: 0
Train Epoch: 7 [1/500 64/32000 (0%)] Loss: 1.84174 (QuantReg: 12.22212) QuantErr: 12.22212 batch_time=28.92164
Train Epoch: 7 [9/500 576/32000 (2%)] Loss: 1.50923 (QuantReg: 11.94333) QuantErr: 11.94333 batch_time=0.41042
Train Epoch: 7 [17/500 1088/32000 (3%)] Loss: 1.70424 (QuantReg: 12.44126) QuantErr: 12.44126 batch_time=0.44596
Train Epoch: 7 [25/500 1600/32000 (5%)] Loss: 1.03340 (QuantReg: 12.25613) QuantErr: 12.25613 batch_time=0.42272
Train Epoch: 7 [33/500 2112/32000 (7%)] Loss: 1.65071 (QuantReg: 12.79188) QuantErr: 12.79188 batch_time=0.40773
Train Epoch: 7 [41/500 2624/32000 (8%)] Loss: 1.29423 (QuantReg: 12.77496) QuantErr: 12.77496 batch_time=0.44099
Train Epoch: 7 [49/500 3136/32000 (10%)] Loss: 1.40518 (QuantReg: 12.48885) QuantErr: 12.48885 batch_time=0.50287
Train Epoch: 7 [57/500 3648/32000 (11%)] Loss: 1.47360 (QuantReg: 12.79632) QuantErr: 12.79632 batch_time=0.48715
Train Epoch: 7 [65/500 4160/32000 (13%)] Loss: 1.34499 (QuantReg: 12.26890) QuantErr: 12.26890 batch_time=0.61057
Train Epoch: 7 [73/500 4672/32000 (15%)] Loss: 1.85565 (QuantReg: 12.79506) QuantErr: 12.79506 batch_time=0.44466
Train Epoch: 7 [81/500 5184/32000 (16%)] Loss: 1.67964 (QuantReg: 12.70883) QuantErr: 12.70883 batch_time=0.40309
Train Epoch: 7 [89/500 5696/32000 (18%)] Loss: 1.24285 (QuantReg: 12.91287) QuantErr: 12.91287 batch_time=0.42767
Train Epoch: 7 [97/500 6208/32000 (19%)] Loss: 1.33697 (QuantReg: 12.64962) QuantErr: 12.64962 batch_time=0.45423
Train Epoch: 7 [105/500 6720/32000 (21%)] Loss: 1.18356 (QuantReg: 12.79130) QuantErr: 12.79130 batch_time=0.79074
Train Epoch: 7 [113/500 7232/32000 (23%)] Loss: 1.28268 (QuantReg: 12.66916) QuantErr: 12.66916 batch_time=0.45826
Train Epoch: 7 [121/500 7744/32000 (24%)] Loss: 1.92865 (QuantReg: 12.36657) QuantErr: 12.36657 batch_time=0.47017
Train Epoch: 7 [129/500 8256/32000 (26%)] Loss: 1.23014 (QuantReg: 12.84466) QuantErr: 12.84466 batch_time=0.51090
Train Epoch: 7 [137/500 8768/32000 (27%)] Loss: 1.44451 (QuantReg: 11.93519) QuantErr: 11.93519 batch_time=0.54503
Train Epoch: 7 [145/500 9280/32000 (29%)] Loss: 1.35956 (QuantReg: 12.96636) QuantErr: 12.96636 batch_time=0.43185
Train Epoch: 7 [153/500 9792/32000 (31%)] Loss: 1.21024 (QuantReg: 12.88296) QuantErr: 12.88296 batch_time=0.42858
Train Epoch: 7 [161/500 10304/32000 (32%)] Loss: 0.82195 (QuantReg: 12.63825) QuantErr: 12.63825 batch_time=0.41167
Train Epoch: 7 [169/500 10816/32000 (34%)] Loss: 1.57454 (QuantReg: 12.39977) QuantErr: 12.39977 batch_time=0.41096
Train Epoch: 7 [177/500 11328/32000 (35%)] Loss: 1.61834 (QuantReg: 12.51348) QuantErr: 12.51348 batch_time=0.47555
Train Epoch: 7 [185/500 11840/32000 (37%)] Loss: 1.46407 (QuantReg: 12.36407) QuantErr: 12.36407 batch_time=0.54754
Train Epoch: 7 [193/500 12352/32000 (39%)] Loss: 1.72499 (QuantReg: 12.57921) QuantErr: 12.57921 batch_time=0.58233
Train Epoch: 7 [201/500 12864/32000 (40%)] Loss: 1.56986 (QuantReg: 12.68047) QuantErr: 12.68047 batch_time=0.40738
Train Epoch: 7 [209/500 13376/32000 (42%)] Loss: 1.25366 (QuantReg: 12.93775) QuantErr: 12.93775 batch_time=0.44609
Train Epoch: 7 [217/500 13888/32000 (43%)] Loss: 1.24100 (QuantReg: 12.42547) QuantErr: 12.42547 batch_time=0.48896
Train Epoch: 7 [225/500 14400/32000 (45%)] Loss: 1.70333 (QuantReg: 13.00524) QuantErr: 13.00524 batch_time=0.49467
Train Epoch: 7 [233/500 14912/32000 (47%)] Loss: 1.03447 (QuantReg: 12.83596) QuantErr: 12.83596 batch_time=0.47803
Train Epoch: 7 [241/500 15424/32000 (48%)] Loss: 2.20592 (QuantReg: 12.88484) QuantErr: 12.88484 batch_time=0.50560
Train Epoch: 7 [249/500 15936/32000 (50%)] Loss: 1.32521 (QuantReg: 12.52742) QuantErr: 12.52742 batch_time=0.48977
Train Epoch: 7 [257/500 16448/32000 (51%)] Loss: 1.29671 (QuantReg: 12.75150) QuantErr: 12.75150 batch_time=0.56117
Train Epoch: 7 [265/500 16960/32000 (53%)] Loss: 1.42787 (QuantReg: 12.87114) QuantErr: 12.87114 batch_time=0.48289
Train Epoch: 7 [273/500 17472/32000 (55%)] Loss: 2.03603 (QuantReg: 12.18441) QuantErr: 12.18441 batch_time=0.54128
Train Epoch: 7 [281/500 17984/32000 (56%)] Loss: 1.71855 (QuantReg: 13.09393) QuantErr: 13.09393 batch_time=0.40970
Train Epoch: 7 [289/500 18496/32000 (58%)] Loss: 1.41151 (QuantReg: 12.75095) QuantErr: 12.75095 batch_time=0.42288
Train Epoch: 7 [297/500 19008/32000 (59%)] Loss: 2.02663 (QuantReg: 12.57560) QuantErr: 12.57560 batch_time=0.43870
Train Epoch: 7 [305/500 19520/32000 (61%)] Loss: 1.17769 (QuantReg: 12.55620) QuantErr: 12.55620 batch_time=0.42499
Train Epoch: 7 [313/500 20032/32000 (63%)] Loss: 1.52243 (QuantReg: 12.50719) QuantErr: 12.50719 batch_time=0.45033
Train Epoch: 7 [321/500 20544/32000 (64%)] Loss: 1.27078 (QuantReg: 12.95559) QuantErr: 12.95559 batch_time=0.59275
Train Epoch: 7 [329/500 21056/32000 (66%)] Loss: 1.36434 (QuantReg: 12.75642) QuantErr: 12.75642 batch_time=0.40432
Train Epoch: 7 [337/500 21568/32000 (67%)] Loss: 1.69557 (QuantReg: 12.88067) QuantErr: 12.88067 batch_time=0.48328
Train Epoch: 7 [345/500 22080/32000 (69%)] Loss: 1.64005 (QuantReg: 13.27255) QuantErr: 13.27255 batch_time=0.49841
Train Epoch: 7 [353/500 22592/32000 (71%)] Loss: 1.79260 (QuantReg: 12.92255) QuantErr: 12.92255 batch_time=0.45933
Train Epoch: 7 [361/500 23104/32000 (72%)] Loss: 1.71961 (QuantReg: 13.06326) QuantErr: 13.06326 batch_time=0.49284
Train Epoch: 7 [369/500 23616/32000 (74%)] Loss: 1.12470 (QuantReg: 13.18389) QuantErr: 13.18389 batch_time=0.46082
Train Epoch: 7 [377/500 24128/32000 (75%)] Loss: 1.29541 (QuantReg: 12.95416) QuantErr: 12.95416 batch_time=0.42109
Train Epoch: 7 [385/500 24640/32000 (77%)] Loss: 1.32380 (QuantReg: 12.87965) QuantErr: 12.87965 batch_time=0.51867
Train Epoch: 7 [393/500 25152/32000 (79%)] Loss: 1.08095 (QuantReg: 13.08936) QuantErr: 13.08936 batch_time=0.41924
Train Epoch: 7 [401/500 25664/32000 (80%)] Loss: 1.31627 (QuantReg: 12.70989) QuantErr: 12.70989 batch_time=0.39551
Train Epoch: 7 [409/500 26176/32000 (82%)] Loss: 1.39618 (QuantReg: 12.82360) QuantErr: 12.82360 batch_time=0.49688
Train Epoch: 7 [417/500 26688/32000 (83%)] Loss: 1.43596 (QuantReg: 13.09965) QuantErr: 13.09965 batch_time=0.47929
Train Epoch: 7 [425/500 27200/32000 (85%)] Loss: 1.27969 (QuantReg: 12.65185) QuantErr: 12.65185 batch_time=0.47893
Train Epoch: 7 [433/500 27712/32000 (87%)] Loss: 1.53662 (QuantReg: 12.92426) QuantErr: 12.92426 batch_time=0.43099
Train Epoch: 7 [441/500 28224/32000 (88%)] Loss: 1.32542 (QuantReg: 12.91008) QuantErr: 12.91008 batch_time=0.39961
Train Epoch: 7 [449/500 28736/32000 (90%)] Loss: 1.04841 (QuantReg: 12.93786) QuantErr: 12.93786 batch_time=0.47377
Train Epoch: 7 [457/500 29248/32000 (91%)] Loss: 0.92652 (QuantReg: 12.37111) QuantErr: 12.37111 batch_time=0.50152
Train Epoch: 7 [465/500 29760/32000 (93%)] Loss: 1.17636 (QuantReg: 13.32080) QuantErr: 13.32080 batch_time=0.46729
Train Epoch: 7 [473/500 30272/32000 (95%)] Loss: 1.25637 (QuantReg: 12.86776) QuantErr: 12.86776 batch_time=0.45400
Train Epoch: 7 [481/500 30784/32000 (96%)] Loss: 1.41441 (QuantReg: 13.37978) QuantErr: 13.37978 batch_time=0.42054
Train Epoch: 7 [489/500 31296/32000 (98%)] Loss: 1.34066 (QuantReg: 12.34151) QuantErr: 12.34151 batch_time=0.52410
Train Epoch: 7 [497/500 31808/32000 (99%)] Loss: 1.00677 (QuantReg: 12.88277) QuantErr: 12.88277 batch_time=0.49080
Train Epoch: 7 codebook_update_time=2.19061
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch7.pth ...
Done in 4.448s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch7.pth ...
Done in 9.528s
removing stale ckpt [epoch 6] [took 0.00s]
epoch : 7
loss : 1.4146579413414002
quant_reg : 12.6609709815979
quant_err : 12.6609709815979
learning_rate : 3.675459453124999e-05
n_samples : 224000
n_steps : 3500
MSRVTT_miech_test/t2v_metrics/R1: 16.1
MSRVTT_miech_test/t2v_metrics/R5: 44.8
MSRVTT_miech_test/t2v_metrics/R10: 59.7
MSRVTT_miech_test/t2v_metrics/R50: 86.9
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 33.881
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 35.05038077788584
MSRVTT_miech_test/v2t_metrics/R1: 17.2
MSRVTT_miech_test/v2t_metrics/R5: 46.0
MSRVTT_miech_test/v2t_metrics/R10: 60.3
MSRVTT_miech_test/v2t_metrics/R50: 87.1
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 31.159
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 36.268912208373244
mnt_best : 35.05038077788584
not_improved_count: 0
Train Epoch: 8 [1/500 64/32000 (0%)] Loss: 1.46265 (QuantReg: 12.50960) QuantErr: 12.50960 batch_time=29.82033
Train Epoch: 8 [9/500 576/32000 (2%)] Loss: 1.42504 (QuantReg: 12.45017) QuantErr: 12.45017 batch_time=0.40276
Train Epoch: 8 [17/500 1088/32000 (3%)] Loss: 1.13264 (QuantReg: 12.49066) QuantErr: 12.49066 batch_time=0.47727
Train Epoch: 8 [25/500 1600/32000 (5%)] Loss: 0.92281 (QuantReg: 12.87460) QuantErr: 12.87460 batch_time=0.43093
Train Epoch: 8 [33/500 2112/32000 (7%)] Loss: 1.73664 (QuantReg: 12.70941) QuantErr: 12.70941 batch_time=0.48393
Train Epoch: 8 [41/500 2624/32000 (8%)] Loss: 1.31048 (QuantReg: 12.28110) QuantErr: 12.28110 batch_time=0.57011
Train Epoch: 8 [49/500 3136/32000 (10%)] Loss: 1.41507 (QuantReg: 13.10825) QuantErr: 13.10825 batch_time=0.46146
Train Epoch: 8 [57/500 3648/32000 (11%)] Loss: 1.28306 (QuantReg: 12.87189) QuantErr: 12.87189 batch_time=0.47694
Train Epoch: 8 [65/500 4160/32000 (13%)] Loss: 1.04445 (QuantReg: 12.79856) QuantErr: 12.79856 batch_time=1.16682
Train Epoch: 8 [73/500 4672/32000 (15%)] Loss: 1.52607 (QuantReg: 12.77103) QuantErr: 12.77103 batch_time=0.49567
Train Epoch: 8 [81/500 5184/32000 (16%)] Loss: 1.32268 (QuantReg: 12.82366) QuantErr: 12.82366 batch_time=0.40479
Train Epoch: 8 [89/500 5696/32000 (18%)] Loss: 0.78805 (QuantReg: 13.08159) QuantErr: 13.08159 batch_time=0.45932
Train Epoch: 8 [97/500 6208/32000 (19%)] Loss: 1.38602 (QuantReg: 12.94444) QuantErr: 12.94444 batch_time=0.42659
Train Epoch: 8 [105/500 6720/32000 (21%)] Loss: 1.38292 (QuantReg: 12.34405) QuantErr: 12.34405 batch_time=0.56514
Train Epoch: 8 [113/500 7232/32000 (23%)] Loss: 1.02288 (QuantReg: 12.36835) QuantErr: 12.36835 batch_time=0.44705
Train Epoch: 8 [121/500 7744/32000 (24%)] Loss: 1.14406 (QuantReg: 12.75089) QuantErr: 12.75089 batch_time=0.46011
Train Epoch: 8 [129/500 8256/32000 (26%)] Loss: 1.28593 (QuantReg: 13.10087) QuantErr: 13.10087 batch_time=1.57934
Train Epoch: 8 [137/500 8768/32000 (27%)] Loss: 1.48266 (QuantReg: 12.66252) QuantErr: 12.66252 batch_time=0.45555
Train Epoch: 8 [145/500 9280/32000 (29%)] Loss: 1.53715 (QuantReg: 12.63384) QuantErr: 12.63384 batch_time=0.41479
Train Epoch: 8 [153/500 9792/32000 (31%)] Loss: 1.24826 (QuantReg: 12.86876) QuantErr: 12.86876 batch_time=0.41632
Train Epoch: 8 [161/500 10304/32000 (32%)] Loss: 1.66939 (QuantReg: 12.19662) QuantErr: 12.19662 batch_time=0.44409
Train Epoch: 8 [169/500 10816/32000 (34%)] Loss: 1.17392 (QuantReg: 12.93545) QuantErr: 12.93545 batch_time=0.61230
Train Epoch: 8 [177/500 11328/32000 (35%)] Loss: 1.73348 (QuantReg: 12.39663) QuantErr: 12.39663 batch_time=0.45806
Train Epoch: 8 [185/500 11840/32000 (37%)] Loss: 1.55151 (QuantReg: 12.48335) QuantErr: 12.48335 batch_time=0.47496
Train Epoch: 8 [193/500 12352/32000 (39%)] Loss: 1.48218 (QuantReg: 12.58381) QuantErr: 12.58381 batch_time=1.23871
Train Epoch: 8 [201/500 12864/32000 (40%)] Loss: 1.46909 (QuantReg: 12.24983) QuantErr: 12.24983 batch_time=0.45363
Train Epoch: 8 [209/500 13376/32000 (42%)] Loss: 1.46537 (QuantReg: 12.61434) QuantErr: 12.61434 batch_time=0.45791
Train Epoch: 8 [217/500 13888/32000 (43%)] Loss: 1.45370 (QuantReg: 12.95172) QuantErr: 12.95172 batch_time=0.45096
Train Epoch: 8 [225/500 14400/32000 (45%)] Loss: 1.15622 (QuantReg: 12.44202) QuantErr: 12.44202 batch_time=0.47436
Train Epoch: 8 [233/500 14912/32000 (47%)] Loss: 1.10169 (QuantReg: 12.80095) QuantErr: 12.80095 batch_time=0.58655
Train Epoch: 8 [241/500 15424/32000 (48%)] Loss: 1.35681 (QuantReg: 12.74862) QuantErr: 12.74862 batch_time=0.43692
Train Epoch: 8 [249/500 15936/32000 (50%)] Loss: 0.96481 (QuantReg: 12.58073) QuantErr: 12.58073 batch_time=0.52861
Train Epoch: 8 [257/500 16448/32000 (51%)] Loss: 0.90969 (QuantReg: 12.68723) QuantErr: 12.68723 batch_time=1.35414
Train Epoch: 8 [265/500 16960/32000 (53%)] Loss: 1.09381 (QuantReg: 12.87638) QuantErr: 12.87638 batch_time=0.41703
Train Epoch: 8 [273/500 17472/32000 (55%)] Loss: 1.35458 (QuantReg: 12.35079) QuantErr: 12.35079 batch_time=0.46253
Train Epoch: 8 [281/500 17984/32000 (56%)] Loss: 1.16714 (QuantReg: 13.01597) QuantErr: 13.01597 batch_time=0.47566
Train Epoch: 8 [289/500 18496/32000 (58%)] Loss: 1.38788 (QuantReg: 12.64725) QuantErr: 12.64725 batch_time=0.40647
Train Epoch: 8 [297/500 19008/32000 (59%)] Loss: 1.67470 (QuantReg: 12.35454) QuantErr: 12.35454 batch_time=0.57407
Train Epoch: 8 [305/500 19520/32000 (61%)] Loss: 0.97308 (QuantReg: 13.10870) QuantErr: 13.10870 batch_time=0.47481
Train Epoch: 8 [313/500 20032/32000 (63%)] Loss: 1.22634 (QuantReg: 12.65942) QuantErr: 12.65942 batch_time=0.44489
Train Epoch: 8 [321/500 20544/32000 (64%)] Loss: 1.14302 (QuantReg: 12.84880) QuantErr: 12.84880 batch_time=1.26971
Train Epoch: 8 [329/500 21056/32000 (66%)] Loss: 1.03649 (QuantReg: 12.81328) QuantErr: 12.81328 batch_time=0.51073
Train Epoch: 8 [337/500 21568/32000 (67%)] Loss: 1.03090 (QuantReg: 12.85404) QuantErr: 12.85404 batch_time=0.44624
Train Epoch: 8 [345/500 22080/32000 (69%)] Loss: 1.12595 (QuantReg: 12.89426) QuantErr: 12.89426 batch_time=0.51494
Train Epoch: 8 [353/500 22592/32000 (71%)] Loss: 0.92864 (QuantReg: 13.05548) QuantErr: 13.05548 batch_time=0.40778
Train Epoch: 8 [361/500 23104/32000 (72%)] Loss: 0.97847 (QuantReg: 12.76788) QuantErr: 12.76788 batch_time=0.56356
Train Epoch: 8 [369/500 23616/32000 (74%)] Loss: 1.17733 (QuantReg: 12.64118) QuantErr: 12.64118 batch_time=0.51670
Train Epoch: 8 [377/500 24128/32000 (75%)] Loss: 1.25237 (QuantReg: 13.17153) QuantErr: 13.17153 batch_time=0.39948
Train Epoch: 8 [385/500 24640/32000 (77%)] Loss: 1.00481 (QuantReg: 12.83464) QuantErr: 12.83464 batch_time=1.14324
Train Epoch: 8 [393/500 25152/32000 (79%)] Loss: 1.27557 (QuantReg: 13.20222) QuantErr: 13.20222 batch_time=0.45522
Train Epoch: 8 [401/500 25664/32000 (80%)] Loss: 1.31552 (QuantReg: 13.02286) QuantErr: 13.02286 batch_time=0.53962
Train Epoch: 8 [409/500 26176/32000 (82%)] Loss: 1.18111 (QuantReg: 13.25836) QuantErr: 13.25836 batch_time=0.44204
Train Epoch: 8 [417/500 26688/32000 (83%)] Loss: 1.72153 (QuantReg: 12.82008) QuantErr: 12.82008 batch_time=0.50065
Train Epoch: 8 [425/500 27200/32000 (85%)] Loss: 1.32765 (QuantReg: 13.09061) QuantErr: 13.09061 batch_time=0.62485
Train Epoch: 8 [433/500 27712/32000 (87%)] Loss: 1.64968 (QuantReg: 12.61699) QuantErr: 12.61699 batch_time=0.52873
Train Epoch: 8 [441/500 28224/32000 (88%)] Loss: 1.07906 (QuantReg: 12.49747) QuantErr: 12.49747 batch_time=0.39073
Train Epoch: 8 [449/500 28736/32000 (90%)] Loss: 1.22741 (QuantReg: 12.85380) QuantErr: 12.85380 batch_time=1.15526
Train Epoch: 8 [457/500 29248/32000 (91%)] Loss: 1.11324 (QuantReg: 13.28852) QuantErr: 13.28852 batch_time=0.43687
Train Epoch: 8 [465/500 29760/32000 (93%)] Loss: 1.25603 (QuantReg: 12.72138) QuantErr: 12.72138 batch_time=0.43707
Train Epoch: 8 [473/500 30272/32000 (95%)] Loss: 1.32612 (QuantReg: 12.80540) QuantErr: 12.80540 batch_time=0.42675
Train Epoch: 8 [481/500 30784/32000 (96%)] Loss: 1.07021 (QuantReg: 12.72726) QuantErr: 12.72726 batch_time=0.47683
Train Epoch: 8 [489/500 31296/32000 (98%)] Loss: 0.76001 (QuantReg: 12.78538) QuantErr: 12.78538 batch_time=0.55677
Train Epoch: 8 [497/500 31808/32000 (99%)] Loss: 0.98679 (QuantReg: 13.00345) QuantErr: 13.00345 batch_time=0.47854
Train Epoch: 8 codebook_update_time=2.99445
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch8.pth ...
Done in 5.349s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch8.pth ...
Done in 10.562s
removing stale ckpt [epoch 7] [took 0.01s]
epoch : 8
loss : 1.3069131968021392
quant_reg : 12.75444584274292
quant_err : 12.75444584274292
learning_rate : 3.4916864804687486e-05
n_samples : 256000
n_steps : 4000
MSRVTT_miech_test/t2v_metrics/R1: 16.8
MSRVTT_miech_test/t2v_metrics/R5: 44.8
MSRVTT_miech_test/t2v_metrics/R10: 60.9
MSRVTT_miech_test/t2v_metrics/R50: 87.6
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 33.358
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 35.78778855903613
MSRVTT_miech_test/v2t_metrics/R1: 17.0
MSRVTT_miech_test/v2t_metrics/R5: 45.4
MSRVTT_miech_test/v2t_metrics/R10: 60.5
MSRVTT_miech_test/v2t_metrics/R50: 86.8
MSRVTT_miech_test/v2t_metrics/MedR: 7.0
MSRVTT_miech_test/v2t_metrics/MeanR: 28.729
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 36.0097453040668
mnt_best : 35.78778855903613
not_improved_count: 0
Train Epoch: 9 [1/500 64/32000 (0%)] Loss: 1.73989 (QuantReg: 12.00029) QuantErr: 12.00029 batch_time=28.61478
Train Epoch: 9 [9/500 576/32000 (2%)] Loss: 1.10469 (QuantReg: 12.62416) QuantErr: 12.62416 batch_time=0.42006
Train Epoch: 9 [17/500 1088/32000 (3%)] Loss: 0.67825 (QuantReg: 12.93050) QuantErr: 12.93050 batch_time=0.60426
Train Epoch: 9 [25/500 1600/32000 (5%)] Loss: 1.12579 (QuantReg: 12.73076) QuantErr: 12.73076 batch_time=0.48670
Train Epoch: 9 [33/500 2112/32000 (7%)] Loss: 1.62132 (QuantReg: 12.51073) QuantErr: 12.51073 batch_time=0.44526
Train Epoch: 9 [41/500 2624/32000 (8%)] Loss: 1.13172 (QuantReg: 12.99414) QuantErr: 12.99414 batch_time=0.54601
Train Epoch: 9 [49/500 3136/32000 (10%)] Loss: 1.54660 (QuantReg: 12.65222) QuantErr: 12.65222 batch_time=0.53942
Train Epoch: 9 [57/500 3648/32000 (11%)] Loss: 1.02708 (QuantReg: 12.44658) QuantErr: 12.44658 batch_time=0.49246
Train Epoch: 9 [65/500 4160/32000 (13%)] Loss: 1.05674 (QuantReg: 12.19165) QuantErr: 12.19165 batch_time=0.79063
Train Epoch: 9 [73/500 4672/32000 (15%)] Loss: 1.09752 (QuantReg: 12.71539) QuantErr: 12.71539 batch_time=0.48916
Train Epoch: 9 [81/500 5184/32000 (16%)] Loss: 1.10397 (QuantReg: 12.71904) QuantErr: 12.71904 batch_time=0.53280
Train Epoch: 9 [89/500 5696/32000 (18%)] Loss: 1.35187 (QuantReg: 12.61161) QuantErr: 12.61161 batch_time=0.49316
Train Epoch: 9 [97/500 6208/32000 (19%)] Loss: 1.24342 (QuantReg: 12.42277) QuantErr: 12.42277 batch_time=0.50165
Train Epoch: 9 [105/500 6720/32000 (21%)] Loss: 1.06921 (QuantReg: 13.21412) QuantErr: 13.21412 batch_time=0.52919
Train Epoch: 9 [113/500 7232/32000 (23%)] Loss: 1.49903 (QuantReg: 12.51637) QuantErr: 12.51637 batch_time=0.45237
Train Epoch: 9 [121/500 7744/32000 (24%)] Loss: 1.06987 (QuantReg: 12.42058) QuantErr: 12.42058 batch_time=0.45112
Train Epoch: 9 [129/500 8256/32000 (26%)] Loss: 1.12568 (QuantReg: 12.39576) QuantErr: 12.39576 batch_time=0.73970
Train Epoch: 9 [137/500 8768/32000 (27%)] Loss: 1.09008 (QuantReg: 12.35178) QuantErr: 12.35178 batch_time=0.48934
Train Epoch: 9 [145/500 9280/32000 (29%)] Loss: 1.29231 (QuantReg: 12.77280) QuantErr: 12.77280 batch_time=0.53065
Train Epoch: 9 [153/500 9792/32000 (31%)] Loss: 0.79700 (QuantReg: 12.88759) QuantErr: 12.88759 batch_time=0.52790
Train Epoch: 9 [161/500 10304/32000 (32%)] Loss: 1.31297 (QuantReg: 13.27272) QuantErr: 13.27272 batch_time=0.47464
Train Epoch: 9 [169/500 10816/32000 (34%)] Loss: 1.10108 (QuantReg: 12.38882) QuantErr: 12.38882 batch_time=0.50997
Train Epoch: 9 [177/500 11328/32000 (35%)] Loss: 1.12527 (QuantReg: 12.52718) QuantErr: 12.52718 batch_time=0.56803
Train Epoch: 9 [185/500 11840/32000 (37%)] Loss: 0.91916 (QuantReg: 13.06045) QuantErr: 13.06045 batch_time=0.46490
Train Epoch: 9 [193/500 12352/32000 (39%)] Loss: 0.84320 (QuantReg: 13.09797) QuantErr: 13.09797 batch_time=0.78684
Train Epoch: 9 [201/500 12864/32000 (40%)] Loss: 1.16535 (QuantReg: 13.13492) QuantErr: 13.13492 batch_time=0.49857
Train Epoch: 9 [209/500 13376/32000 (42%)] Loss: 1.73442 (QuantReg: 12.39948) QuantErr: 12.39948 batch_time=0.42474
Train Epoch: 9 [217/500 13888/32000 (43%)] Loss: 1.08697 (QuantReg: 12.88025) QuantErr: 12.88025 batch_time=0.47821
Train Epoch: 9 [225/500 14400/32000 (45%)] Loss: 1.00041 (QuantReg: 12.77651) QuantErr: 12.77651 batch_time=0.47124
Train Epoch: 9 [233/500 14912/32000 (47%)] Loss: 1.08141 (QuantReg: 12.73603) QuantErr: 12.73603 batch_time=0.44112
Train Epoch: 9 [241/500 15424/32000 (48%)] Loss: 1.30000 (QuantReg: 12.85983) QuantErr: 12.85983 batch_time=0.42288
Train Epoch: 9 [249/500 15936/32000 (50%)] Loss: 1.46060 (QuantReg: 12.58682) QuantErr: 12.58682 batch_time=0.46921
Train Epoch: 9 [257/500 16448/32000 (51%)] Loss: 1.29269 (QuantReg: 12.99455) QuantErr: 12.99455 batch_time=0.74779
Train Epoch: 9 [265/500 16960/32000 (53%)] Loss: 1.35084 (QuantReg: 12.92393) QuantErr: 12.92393 batch_time=0.44663
Train Epoch: 9 [273/500 17472/32000 (55%)] Loss: 1.05659 (QuantReg: 13.64208) QuantErr: 13.64208 batch_time=0.47832
Train Epoch: 9 [281/500 17984/32000 (56%)] Loss: 1.26570 (QuantReg: 12.97506) QuantErr: 12.97506 batch_time=0.46765
Train Epoch: 9 [289/500 18496/32000 (58%)] Loss: 1.00566 (QuantReg: 12.52721) QuantErr: 12.52721 batch_time=0.50190
Train Epoch: 9 [297/500 19008/32000 (59%)] Loss: 1.63458 (QuantReg: 12.62184) QuantErr: 12.62184 batch_time=0.46341
Train Epoch: 9 [305/500 19520/32000 (61%)] Loss: 1.20612 (QuantReg: 12.74989) QuantErr: 12.74989 batch_time=0.48713
Train Epoch: 9 [313/500 20032/32000 (63%)] Loss: 1.26984 (QuantReg: 12.97959) QuantErr: 12.97959 batch_time=0.46958
Train Epoch: 9 [321/500 20544/32000 (64%)] Loss: 1.71112 (QuantReg: 13.08664) QuantErr: 13.08664 batch_time=0.80422
Train Epoch: 9 [329/500 21056/32000 (66%)] Loss: 1.23190 (QuantReg: 12.57864) QuantErr: 12.57864 batch_time=0.47850
Train Epoch: 9 [337/500 21568/32000 (67%)] Loss: 1.86541 (QuantReg: 12.61082) QuantErr: 12.61082 batch_time=0.48256
Train Epoch: 9 [345/500 22080/32000 (69%)] Loss: 1.07864 (QuantReg: 13.27171) QuantErr: 13.27171 batch_time=0.43799
Train Epoch: 9 [353/500 22592/32000 (71%)] Loss: 1.00452 (QuantReg: 12.97715) QuantErr: 12.97715 batch_time=0.41169
Train Epoch: 9 [361/500 23104/32000 (72%)] Loss: 0.83664 (QuantReg: 12.93672) QuantErr: 12.93672 batch_time=0.55113
Train Epoch: 9 [369/500 23616/32000 (74%)] Loss: 1.63726 (QuantReg: 12.67108) QuantErr: 12.67108 batch_time=0.47311
Train Epoch: 9 [377/500 24128/32000 (75%)] Loss: 1.54560 (QuantReg: 13.13623) QuantErr: 13.13623 batch_time=0.44195
Train Epoch: 9 [385/500 24640/32000 (77%)] Loss: 1.24819 (QuantReg: 13.02949) QuantErr: 13.02949 batch_time=0.85228
Train Epoch: 9 [393/500 25152/32000 (79%)] Loss: 1.27654 (QuantReg: 12.89377) QuantErr: 12.89377 batch_time=0.49282
Train Epoch: 9 [401/500 25664/32000 (80%)] Loss: 1.30709 (QuantReg: 13.52410) QuantErr: 13.52410 batch_time=0.47019
Train Epoch: 9 [409/500 26176/32000 (82%)] Loss: 1.19235 (QuantReg: 12.81387) QuantErr: 12.81387 batch_time=0.56629
Train Epoch: 9 [417/500 26688/32000 (83%)] Loss: 1.21030 (QuantReg: 12.64149) QuantErr: 12.64149 batch_time=0.46756
Train Epoch: 9 [425/500 27200/32000 (85%)] Loss: 1.46531 (QuantReg: 12.99912) QuantErr: 12.99912 batch_time=0.47737
Train Epoch: 9 [433/500 27712/32000 (87%)] Loss: 1.59437 (QuantReg: 13.50458) QuantErr: 13.50458 batch_time=0.49735
Train Epoch: 9 [441/500 28224/32000 (88%)] Loss: 1.15819 (QuantReg: 12.90855) QuantErr: 12.90855 batch_time=0.46161
Train Epoch: 9 [449/500 28736/32000 (90%)] Loss: 1.39257 (QuantReg: 13.28507) QuantErr: 13.28507 batch_time=0.87359
Train Epoch: 9 [457/500 29248/32000 (91%)] Loss: 1.32424 (QuantReg: 12.91557) QuantErr: 12.91557 batch_time=0.41697
Train Epoch: 9 [465/500 29760/32000 (93%)] Loss: 1.26922 (QuantReg: 12.96212) QuantErr: 12.96212 batch_time=0.44336
Train Epoch: 9 [473/500 30272/32000 (95%)] Loss: 1.99016 (QuantReg: 12.55994) QuantErr: 12.55994 batch_time=0.48766
Train Epoch: 9 [481/500 30784/32000 (96%)] Loss: 0.93038 (QuantReg: 13.17656) QuantErr: 13.17656 batch_time=0.46382
Train Epoch: 9 [489/500 31296/32000 (98%)] Loss: 0.82096 (QuantReg: 13.23530) QuantErr: 13.23530 batch_time=0.44893
Train Epoch: 9 [497/500 31808/32000 (99%)] Loss: 1.13659 (QuantReg: 12.92505) QuantErr: 12.92505 batch_time=0.47295
Train Epoch: 9 codebook_update_time=2.59755
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch9.pth ...
Done in 6.709s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch9.pth ...
Done in 12.442s
removing stale ckpt [epoch 8] [took 0.01s]
epoch : 9
loss : 1.2265772086381912
quant_reg : 12.8299009475708
quant_err : 12.8299009475708
learning_rate : 3.317102156445311e-05
n_samples : 288000
n_steps : 4500
MSRVTT_miech_test/t2v_metrics/R1: 16.5
MSRVTT_miech_test/t2v_metrics/R5: 45.9
MSRVTT_miech_test/t2v_metrics/R10: 60.6
MSRVTT_miech_test/t2v_metrics/R50: 87.5
MSRVTT_miech_test/t2v_metrics/MedR: 7.0
MSRVTT_miech_test/t2v_metrics/MeanR: 32.472
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 35.8033022348432
MSRVTT_miech_test/v2t_metrics/R1: 18.9
MSRVTT_miech_test/v2t_metrics/R5: 47.8
MSRVTT_miech_test/v2t_metrics/R10: 61.7
MSRVTT_miech_test/v2t_metrics/R50: 88.2
MSRVTT_miech_test/v2t_metrics/MedR: 6.0
MSRVTT_miech_test/v2t_metrics/MeanR: 28.41
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 38.19955364371588
mnt_best : 35.8033022348432
not_improved_count: 0
Train Epoch: 10 [1/500 64/32000 (0%)] Loss: 1.60630 (QuantReg: 12.41196) QuantErr: 12.41196 batch_time=28.18276
Train Epoch: 10 [9/500 576/32000 (2%)] Loss: 1.13399 (QuantReg: 12.67072) QuantErr: 12.67072 batch_time=2.44042
Train Epoch: 10 [17/500 1088/32000 (3%)] Loss: 0.88370 (QuantReg: 13.02428) QuantErr: 13.02428 batch_time=0.43629
Train Epoch: 10 [25/500 1600/32000 (5%)] Loss: 0.76213 (QuantReg: 12.85063) QuantErr: 12.85063 batch_time=0.43911
Train Epoch: 10 [33/500 2112/32000 (7%)] Loss: 1.47959 (QuantReg: 12.79949) QuantErr: 12.79949 batch_time=0.47826
Train Epoch: 10 [41/500 2624/32000 (8%)] Loss: 0.79894 (QuantReg: 12.98528) QuantErr: 12.98528 batch_time=0.48188
Train Epoch: 10 [49/500 3136/32000 (10%)] Loss: 1.19100 (QuantReg: 12.72130) QuantErr: 12.72130 batch_time=0.42822
Train Epoch: 10 [57/500 3648/32000 (11%)] Loss: 1.08630 (QuantReg: 12.24261) QuantErr: 12.24261 batch_time=0.40117
Train Epoch: 10 [65/500 4160/32000 (13%)] Loss: 1.16720 (QuantReg: 12.74789) QuantErr: 12.74789 batch_time=0.45166
Train Epoch: 10 [73/500 4672/32000 (15%)] Loss: 0.89070 (QuantReg: 13.05535) QuantErr: 13.05535 batch_time=1.99346
Train Epoch: 10 [81/500 5184/32000 (16%)] Loss: 1.50137 (QuantReg: 12.77930) QuantErr: 12.77930 batch_time=0.44093
Train Epoch: 10 [89/500 5696/32000 (18%)] Loss: 0.88854 (QuantReg: 12.82435) QuantErr: 12.82435 batch_time=0.39034
Train Epoch: 10 [97/500 6208/32000 (19%)] Loss: 0.85108 (QuantReg: 12.73342) QuantErr: 12.73342 batch_time=0.52830
Train Epoch: 10 [105/500 6720/32000 (21%)] Loss: 1.02140 (QuantReg: 12.43796) QuantErr: 12.43796 batch_time=0.47960
Train Epoch: 10 [113/500 7232/32000 (23%)] Loss: 1.22220 (QuantReg: 12.57121) QuantErr: 12.57121 batch_time=0.45079
Train Epoch: 10 [121/500 7744/32000 (24%)] Loss: 1.12316 (QuantReg: 12.88692) QuantErr: 12.88692 batch_time=0.45017
Train Epoch: 10 [129/500 8256/32000 (26%)] Loss: 1.97229 (QuantReg: 12.32979) QuantErr: 12.32979 batch_time=0.44959
Train Epoch: 10 [137/500 8768/32000 (27%)] Loss: 0.96638 (QuantReg: 13.42104) QuantErr: 13.42104 batch_time=1.97287
Train Epoch: 10 [145/500 9280/32000 (29%)] Loss: 1.37695 (QuantReg: 12.38757) QuantErr: 12.38757 batch_time=0.42063
Train Epoch: 10 [153/500 9792/32000 (31%)] Loss: 1.00211 (QuantReg: 13.00617) QuantErr: 13.00617 batch_time=0.47993
Train Epoch: 10 [161/500 10304/32000 (32%)] Loss: 0.79739 (QuantReg: 13.06471) QuantErr: 13.06471 batch_time=0.41271
Train Epoch: 10 [169/500 10816/32000 (34%)] Loss: 0.96452 (QuantReg: 12.72461) QuantErr: 12.72461 batch_time=0.44240
Train Epoch: 10 [177/500 11328/32000 (35%)] Loss: 1.14208 (QuantReg: 12.59007) QuantErr: 12.59007 batch_time=0.46979
Train Epoch: 10 [185/500 11840/32000 (37%)] Loss: 1.02417 (QuantReg: 12.23666) QuantErr: 12.23666 batch_time=0.45022
Train Epoch: 10 [193/500 12352/32000 (39%)] Loss: 0.86885 (QuantReg: 13.22657) QuantErr: 13.22657 batch_time=0.47656
Train Epoch: 10 [201/500 12864/32000 (40%)] Loss: 1.24488 (QuantReg: 12.40308) QuantErr: 12.40308 batch_time=1.91235
Train Epoch: 10 [209/500 13376/32000 (42%)] Loss: 1.19388 (QuantReg: 13.22376) QuantErr: 13.22376 batch_time=0.60138
Train Epoch: 10 [217/500 13888/32000 (43%)] Loss: 1.49593 (QuantReg: 13.02934) QuantErr: 13.02934 batch_time=0.43985
Train Epoch: 10 [225/500 14400/32000 (45%)] Loss: 1.15115 (QuantReg: 12.52764) QuantErr: 12.52764 batch_time=0.46901
Train Epoch: 10 [233/500 14912/32000 (47%)] Loss: 1.17078 (QuantReg: 12.78234) QuantErr: 12.78234 batch_time=0.42358
Train Epoch: 10 [241/500 15424/32000 (48%)] Loss: 1.12853 (QuantReg: 12.96617) QuantErr: 12.96617 batch_time=0.45155
Train Epoch: 10 [249/500 15936/32000 (50%)] Loss: 0.64248 (QuantReg: 13.21779) QuantErr: 13.21779 batch_time=0.45248
Train Epoch: 10 [257/500 16448/32000 (51%)] Loss: 1.18192 (QuantReg: 13.02199) QuantErr: 13.02199 batch_time=0.45520
Train Epoch: 10 [265/500 16960/32000 (53%)] Loss: 1.27388 (QuantReg: 13.19846) QuantErr: 13.19846 batch_time=1.91122
Train Epoch: 10 [273/500 17472/32000 (55%)] Loss: 0.81130 (QuantReg: 12.98177) QuantErr: 12.98177 batch_time=0.51897
Train Epoch: 10 [281/500 17984/32000 (56%)] Loss: 1.39044 (QuantReg: 12.95994) QuantErr: 12.95994 batch_time=0.51733
Train Epoch: 10 [289/500 18496/32000 (58%)] Loss: 1.86589 (QuantReg: 12.91997) QuantErr: 12.91997 batch_time=0.50740
Train Epoch: 10 [297/500 19008/32000 (59%)] Loss: 1.29507 (QuantReg: 12.81686) QuantErr: 12.81686 batch_time=0.46352
Train Epoch: 10 [305/500 19520/32000 (61%)] Loss: 1.48617 (QuantReg: 12.89285) QuantErr: 12.89285 batch_time=0.42399
Train Epoch: 10 [313/500 20032/32000 (63%)] Loss: 1.46487 (QuantReg: 12.98212) QuantErr: 12.98212 batch_time=0.43289
Train Epoch: 10 [321/500 20544/32000 (64%)] Loss: 1.29570 (QuantReg: 12.92833) QuantErr: 12.92833 batch_time=0.42364
Train Epoch: 10 [329/500 21056/32000 (66%)] Loss: 0.89882 (QuantReg: 12.80382) QuantErr: 12.80382 batch_time=1.77419
Train Epoch: 10 [337/500 21568/32000 (67%)] Loss: 1.05701 (QuantReg: 12.64648) QuantErr: 12.64648 batch_time=0.39952
Train Epoch: 10 [345/500 22080/32000 (69%)] Loss: 1.06273 (QuantReg: 12.69319) QuantErr: 12.69319 batch_time=0.43247
Train Epoch: 10 [353/500 22592/32000 (71%)] Loss: 1.25381 (QuantReg: 12.47947) QuantErr: 12.47947 batch_time=0.46269
Train Epoch: 10 [361/500 23104/32000 (72%)] Loss: 0.90111 (QuantReg: 13.38481) QuantErr: 13.38481 batch_time=0.40718
Train Epoch: 10 [369/500 23616/32000 (74%)] Loss: 0.87566 (QuantReg: 12.85132) QuantErr: 12.85132 batch_time=0.43459
Train Epoch: 10 [377/500 24128/32000 (75%)] Loss: 1.30428 (QuantReg: 13.06152) QuantErr: 13.06152 batch_time=0.45306
Train Epoch: 10 [385/500 24640/32000 (77%)] Loss: 1.40174 (QuantReg: 12.97473) QuantErr: 12.97473 batch_time=0.42016
Train Epoch: 10 [393/500 25152/32000 (79%)] Loss: 1.32369 (QuantReg: 12.48600) QuantErr: 12.48600 batch_time=1.96187
Train Epoch: 10 [401/500 25664/32000 (80%)] Loss: 1.03992 (QuantReg: 13.11001) QuantErr: 13.11001 batch_time=0.54278
Train Epoch: 10 [409/500 26176/32000 (82%)] Loss: 0.99865 (QuantReg: 13.13980) QuantErr: 13.13980 batch_time=0.47400
Train Epoch: 10 [417/500 26688/32000 (83%)] Loss: 1.05910 (QuantReg: 13.12758) QuantErr: 13.12758 batch_time=0.48309
Train Epoch: 10 [425/500 27200/32000 (85%)] Loss: 0.93752 (QuantReg: 13.12325) QuantErr: 13.12325 batch_time=0.51325
Train Epoch: 10 [433/500 27712/32000 (87%)] Loss: 0.85934 (QuantReg: 12.99709) QuantErr: 12.99709 batch_time=0.45832
Train Epoch: 10 [441/500 28224/32000 (88%)] Loss: 1.18547 (QuantReg: 13.39048) QuantErr: 13.39048 batch_time=0.53189
Train Epoch: 10 [449/500 28736/32000 (90%)] Loss: 1.07809 (QuantReg: 12.77365) QuantErr: 12.77365 batch_time=0.53805
Train Epoch: 10 [457/500 29248/32000 (91%)] Loss: 0.77969 (QuantReg: 13.07327) QuantErr: 13.07327 batch_time=1.90789
Train Epoch: 10 [465/500 29760/32000 (93%)] Loss: 1.28313 (QuantReg: 12.44077) QuantErr: 12.44077 batch_time=0.45804
Train Epoch: 10 [473/500 30272/32000 (95%)] Loss: 0.62572 (QuantReg: 13.51319) QuantErr: 13.51319 batch_time=0.48537
Train Epoch: 10 [481/500 30784/32000 (96%)] Loss: 1.11259 (QuantReg: 12.81067) QuantErr: 12.81067 batch_time=0.47454
Train Epoch: 10 [489/500 31296/32000 (98%)] Loss: 0.78369 (QuantReg: 13.41077) QuantErr: 13.41077 batch_time=0.46758
Train Epoch: 10 [497/500 31808/32000 (99%)] Loss: 1.01867 (QuantReg: 12.59945) QuantErr: 12.59945 batch_time=0.41519
Train Epoch: 10 codebook_update_time=2.24088
Saving checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch10.pth ...
Done in 6.061s
Updating 'best' checkpoint: /apdcephfs/share_47076/gimwang/HCQ/exps/HCQ_MSRVTT_1kB_bs64/checkpoint-epoch10.pth ...
Done in 13.861s
removing stale ckpt [epoch 9] [took 0.05s]
epoch : 10
loss : 1.1500453941822053
quant_reg : 12.85799836730957
quant_err : 12.85799836730957
learning_rate : 3.151247048623045e-05
n_samples : 320000
n_steps : 5000
MSRVTT_miech_test/t2v_metrics/R1: 18.0
MSRVTT_miech_test/t2v_metrics/R5: 45.7
MSRVTT_miech_test/t2v_metrics/R10: 59.5
MSRVTT_miech_test/t2v_metrics/R50: 87.3
MSRVTT_miech_test/t2v_metrics/MedR: 6.0
MSRVTT_miech_test/t2v_metrics/MeanR: 33.161
MSRVTT_miech_test/t2v_metrics/geometric_mean_R1-R5-R10: 36.57928595878166
MSRVTT_miech_test/v2t_metrics/R1: 18.0
MSRVTT_miech_test/v2t_metrics/R5: 47.9
MSRVTT_miech_test/v2t_metrics/R10: 61.7
MSRVTT_miech_test/v2t_metrics/R50: 87.4
MSRVTT_miech_test/v2t_metrics/MedR: 6.0
MSRVTT_miech_test/v2t_metrics/MeanR: 28.771
MSRVTT_miech_test/v2t_metrics/geometric_mean_R1-R5-R10: 37.6095145304656
mnt_best : 36.57928595878166
not_improved_count: 0
Train Epoch: 11 [1/500 64/32000 (0%)] Loss: 1.28619 (QuantReg: 13.09749) QuantErr: 13.09749 batch_time=28.51529
Train Epoch: 11 [9/500 576/32000 (2%)] Loss: 1.07991 (QuantReg: 12.31726) QuantErr: 12.31726 batch_time=0.45342
Train Epoch: 11 [17/500 1088/32000 (3%)] Loss: 1.42388 (QuantReg: 12.29814) QuantErr: 12.29814 batch_time=0.47245
Train Epoch: 11 [25/500 1600/32000 (5%)] Loss: 1.07878 (QuantReg: 13.19767) QuantErr: 13.19767 batch_time=1.07484
Train Epoch: 11 [33/500 2112/32000 (7%)] Loss: 1.59925 (QuantReg: 12.64885) QuantErr: 12.64885 batch_time=0.46330
Train Epoch: 11 [41/500 2624/32000 (8%)] Loss: 1.05409 (QuantReg: 12.36735) QuantErr: 12.36735 batch_time=0.46893
Train Epoch: 11 [49/500 3136/32000 (10%)] Loss: 1.34143 (QuantReg: 12.75987) QuantErr: 12.75987 batch_time=0.42930
Train Epoch: 11 [57/500 3648/32000 (11%)] Loss: 1.55401 (QuantReg: 12.53874) QuantErr: 12.53874 batch_time=0.45360
Train Epoch: 11 [65/500 4160/32000 (13%)] Loss: 1.65510 (QuantReg: 12.89469) QuantErr: 12.89469 batch_time=0.46322
Train Epoch: 11 [73/500 4672/32000 (15%)] Loss: 1.15992 (QuantReg: 12.73244) QuantErr: 12.73244 batch_time=0.44743
Train Epoch: 11 [81/500 5184/32000 (16%)] Loss: 1.19693 (QuantReg: 12.35847) QuantErr: 12.35847 batch_time=0.56409
Train Epoch: 11 [89/500 5696/32000 (18%)] Loss: 0.85517 (QuantReg: 12.87137) QuantErr: 12.87137 batch_time=1.04123
Train Epoch: 11 [97/500 6208/32000 (19%)] Loss: 1.16097 (QuantReg: 12.43211) QuantErr: 12.43211 batch_time=0.44812
Train Epoch: 11 [105/500 6720/32000 (21%)] Loss: 1.25122 (QuantReg: 12.47066) QuantErr: 12.47066 batch_time=0.45274
Train Epoch: 11 [113/500 7232/32000 (23%)] Loss: 1.12341 (QuantReg: 12.23228) QuantErr: 12.23228 batch_time=0.41768
Train Epoch: 11 [121/500 7744/32000 (24%)] Loss: 0.75510 (QuantReg: 12.54999) QuantErr: 12.54999 batch_time=0.44395
Train Epoch: 11 [129/500 8256/32000 (26%)] Loss: 0.98455 (QuantReg: 12.96374) QuantErr: 12.96374 batch_time=0.46652
Train Epoch: 11 [137/500 8768/32000 (27%)] Loss: 1.20713 (QuantReg: 12.88494) QuantErr: 12.88494 batch_time=0.54908
Train Epoch: 11 [145/500 9280/32000 (29%)] Loss: 1.50936 (QuantReg: 12.91210) QuantErr: 12.91210 batch_time=0.51132
Train Epoch: 11 [153/500 9792/32000 (31%)] Loss: 1.17898 (QuantReg: 12.87853) QuantErr: 12.87853 batch_time=0.89281
Train Epoch: 11 [161/500 10304/32000 (32%)] Loss: 1.27087 (QuantReg: 12.87614) QuantErr: 12.87614 batch_time=0.44433
Train Epoch: 11 [169/500 10816/32000 (34%)] Loss: 1.69138 (QuantReg: 12.64311) QuantErr: 12.64311 batch_time=0.41738
Train Epoch: 11 [177/500 11328/32000 (35%)] Loss: 1.31639 (QuantReg: 13.00309) QuantErr: 13.00309 batch_time=0.43673
Train Epoch: 11 [185/500 11840/32000 (37%)] Loss: 1.25071 (QuantReg: 13.32796) QuantErr: 13.32796 batch_time=0.44411
Train Epoch: 11 [193/500 12352/32000 (39%)] Loss: 1.30558 (QuantReg: 12.25272) QuantErr: 12.25272 batch_time=0.43649
Train Epoch: 11 [201/500 12864/32000 (40%)] Loss: 0.75123 (QuantReg: 12.95832) QuantErr: 12.95832 batch_time=0.46602
Train Epoch: 11 [209/500 13376/32000 (42%)] Loss: 1.30700 (QuantReg: 12.74451) QuantErr: 12.74451 batch_time=0.51310
Train Epoch: 11 [217/500 13888/32000 (43%)] Loss: 0.82211 (QuantReg: 12.57428) QuantErr: 12.57428 batch_time=0.93148
Train Epoch: 11 [225/500 14400/32000 (45%)] Loss: 0.86084 (QuantReg: 13.16245) QuantErr: 13.16245 batch_time=0.42304
Train Epoch: 11 [233/500 14912/32000 (47%)] Loss: 1.00617 (QuantReg: 12.72408) QuantErr: 12.72408 batch_time=0.42199
Train Epoch: 11 [241/500 15424/32000 (48%)] Loss: 1.27930 (QuantReg: 12.78797) QuantErr: 12.78797 batch_time=0.43166
Train Epoch: 11 [249/500 15936/32000 (50%)] Loss: 0.97629 (QuantReg: 12.66047) QuantErr: 12.66047 batch_time=0.41405
Train Epoch: 11 [257/500 16448/32000 (51%)] Loss: 1.13012 (QuantReg: 12.93499) QuantErr: 12.93499 batch_time=0.42134
Train Epoch: 11 [265/500 16960/32000 (53%)] Loss: 1.09737 (QuantReg: 13.07603) QuantErr: 13.07603 batch_time=0.43896
Train Epoch: 11 [273/500 17472/32000 (55%)] Loss: 1.04557 (QuantReg: 13.41565) QuantErr: 13.41565 batch_time=0.43499
Train Epoch: 11 [281/500 17984/32000 (56%)] Loss: 1.08659 (QuantReg: 12.82671) QuantErr: 12.82671 batch_time=0.99436
Train Epoch: 11 [289/500 18496/32000 (58%)] Loss: 1.03933 (QuantReg: 13.25818) QuantErr: 13.25818 batch_time=0.51292
Train Epoch: 11 [297/500 19008/32000 (59%)] Loss: 0.95554 (QuantReg: 13.11172) QuantErr: 13.11172 batch_time=0.44982
Train Epoch: 11 [305/500 19520/32000 (61%)] Loss: 1.37873 (QuantReg: 13.10417) QuantErr: 13.10417 batch_time=0.44753
Train Epoch: 11 [313/500 20032/32000 (63%)] Loss: 1.35598 (QuantReg: 13.13944) QuantErr: 13.13944 batch_time=0.42179
Train Epoch: 11 [321/500 20544/32000 (64%)] Loss: 0.94170 (QuantReg: 12.72989) QuantErr: 12.72989 batch_time=0.50396
Train Epoch: 11 [329/500 21056/32000 (66%)] Loss: 0.84422 (QuantReg: 13.61640) QuantErr: 13.61640 batch_time=0.39554
Train Epoch: 11 [337/500 21568/32000 (67%)] Loss: 0.98709 (QuantReg: 12.78237) QuantErr: 12.78237 batch_time=0.44011
Train Epoch: 11 [345/500 22080/32000 (69%)] Loss: 1.08060 (QuantReg: 13.07524) QuantErr: 13.07524 batch_time=1.01755
Train Epoch: 11 [353/500 22592/32000 (71%)] Loss: 1.44050 (QuantReg: 12.62722) QuantErr: 12.62722 batch_time=0.51546
Train Epoch: 11 [361/500 23104/32000 (72%)] Loss: 1.18608 (QuantReg: 13.07934) QuantErr: 13.07934 batch_time=0.46843
Train Epoch: 11 [369/500 23616/32000 (74%)] Loss: 1.03668 (QuantReg: 12.89354) QuantErr: 12.89354 batch_time=0.41325
Train Epoch: 11 [377/500 24128/32000 (75%)] Loss: 0.93328 (QuantReg: 13.27618) QuantErr: 13.27618 batch_time=0.40860