Skip to content

Latest commit

 

History

History
171 lines (152 loc) · 16.1 KB

reproduced_results.md

File metadata and controls

171 lines (152 loc) · 16.1 KB
Raw

Reproduced results

QVHighlights (Moment retrieval & highlight detection)

Val set scores are reported.

ResNet152+GloVe

Models R1@0.5 R1@0.7 HD mAP HIT@1 checkpoint
Moment DETR 41.5 25.2 29.1 41.4 ckpt
QD-DETR 53.2 37.5 34.1 52.1 ckpt
EaTR 54.9 36.0 35.1 54.7 ckpt
TR-DETR 48.3 32.9 34.9 51.4 ckpt
UVCOM 53.7 39.7 34.9 53.0 ckpt
TaskWeave 51.7 37.4 32.6 45.2 mr2hd/hd2mr
CG-DETR 51.9 39.0 34.1 53.2 ckpt

CLIP

Models R1@0.5 R1@0.7 HD mAP HIT@1 checkpoint
Moment DETR 53.5 34.1 35.3 54.0 ckpt
QD-DETR 59.7 42.3 38.0 59.2 ckpt
EaTR 54.9 36.0 35.1 54.7 ckpt
TR-DETR 63.6 43.9 40.1 63.2 ckpt
UVCOM 64.8 48.0 38.7 62.2 ckpt
TaskWeave 60.1 38.7 38.0 58.9 mr2hd/hd2mr
CG-DETR 66.6 49.9 39.9 64.3 ckpt

CLIP+Slowfast

Models R1@0.5 R1@0.7 HD mAP HIT@1 checkpoint
Moment DETR 54.2 36.1 35.9 56.7 ckpt
QD-DETR 63.0 46.4 39.1 61.3 ckpt
EaTR 59.6 40.3 36.6 57.9 ckpt
TR-DETR 66.5 48.8 40.8 66.2 ckpt
UVCOM 64.0 49.4 39.7 64.3 ckpt
TaskWeave 64.2 49.2 39.0 62.7 mr2hd/hd2mr
CG-DETR 65.6 52.1 40.7 67.0 ckpt

CLIP+Slowfast+PANNs

Models R1@0.5 R1@0.7 HD mAP HIT@1 checkpoint
Moment DETR 54.6 37.0 35.8 55.4 ckpt
QD-DETR 62.0 45.8 39.3 62.8 ckpt
EaTR 57.7 41.7 36.3 57.0 ckpt
TR-DETR 66.9 50.1 40.7 64.7 ckpt
UVCOM 63.7 49.9 40.0 64.3 ckpt
TaskWeave 63.0 48.3 38.4 61.0 mr2hd/hd2mr
CG-DETR 65.0 49.8 40.3 65.4 ckpt

ActivityNet Captions (Moment retrieval)

Val_1 scores are reported.

ResNet152+GloVe

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 34.2 19.5 46.3 24.4 ckpt
QD-DETR 35.4 20.3 47.4 24.9 ckpt
EaTR 32.4 18.2 44.3 21.9 ckpt
UVCOM 34.4 19.9 46.1 24.4 ckpt
TaskWeave 33.3 19.5 44.7 24.5 ckpt
CG-DETR 37.0 21.2 48.6 26.5 ckpt

CLIP

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 36.1 20.4 48.2 25.7 ckpt
QD-DETR 36.9 21.4 48.4 26.3 ckpt
EaTR 34.6 19.7 45.1 23.1 ckpt
UVCOM 37.0 21.5 48.3 25.7 ckpt
TaskWeave 36.8 21.7 47.1 27.1 ckpt
CG-DETR 38.8 22.6 50.6 27.5 ckpt

CLIP+Slowfast

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 36.5 21.1 48.4 26.0 ckpt
QD-DETR 37.5 22.1 48.9 26.4 ckpt
EaTR 34.6 19.3 45.2 22.3 ckpt
UVCOM 37.3 21.6 48.9 25.7 ckpt
TaskWeave 35.9 21.2 47.5 25.9 ckpt
CG-DETR 40.0 23.2 51.0 27.7 ckpt

Charades-STA (Moment retrieval)

Test set scores are reported.

ResNet152+GloVe

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 38.4 22.9 52.4 22.2 ckpt
QD-DETR 42.1 24.0 56.7 24.5 ckpt
EaTR 37.6 20.1 53.5 23.6 ckpt
UVCOM 38.1 18.2 54.4 21.1 ckpt
TaskWeave 28.5 12.8 39.6 14.0 ckpt
CG-DETR 39.7 19.4 56.9 23.2 ckpt

CLIP

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 47.9 26.7 61.0 28.8 ckpt
QD-DETR 52.0 31.7 63.6 29.4 ckpt
EaTR 48.4 27.5 59.9 26.9 ckpt
UVCOM 48.4 27.1 60.9 27.9 ckpt
TaskWeave 50.7 28.1 60.8 26.1 ckpt
CG-DETR 54.4 31.8 65.5 30.5 ckpt

CLIP+Slowfast

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 53.4 30.7 62.0 29.1 ckpt
QD-DETR 59.4 37.9 66.6 33.8 ckpt
EaTR 55.2 33.1 65.4 30.4 ckpt
UVCOM 56.9 35.9 65.6 33.6 ckpt
TaskWeave 56.8 35.2 65.7 32.4 ckpt
CG-DETR 57.6 35.1 65.9 30.9 ckpt

TaCoS (Moment retrieval)

Test set scores are reported.

ResNet152+GloVe

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 20.0 8.6 24.2 6.9 ckpt
QD-DETR 30.6 15.1 35.1 12.3 ckpt
EaTR 22.5 9.2 26.3 7.9 ckpt
UVCOM 24.1 10.7 28.1 8.6 ckpt
TaskWeave 30.9 15.8 35.2 13.2 ckpt
CG-DETR 34.2 17.4 39.7 14.6 ckpt

CLIP

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 18.0 7.9 21.3 6.7 ckpt
QD-DETR 32.3 17.2 36.0 14.1 ckpt
EaTR 24.7 10.0 28.8 8.7 ckpt
UVCOM 36.8 20.0 41.5 16.3 ckpt
TaskWeave 34.0 17.7 37.6 14.1 ckpt
CG-DETR 34.3 19.8 38.6 15.8 ckpt

CLIP+Slowfast

Models R1@0.5 R1@0.7 mAP@0.5 mAP@0.75 checkpoint
Moment DETR 25.5 12.9 29.1 10.3 ckpt
QD-DETR 38.7 22.1 42.9 16.7 ckpt
EaTR 31.7 15.6 37.4 14.0 ckpt
UVCOM 40.2 23.3 43.5 19.1 ckpt
TaskWeave 38.3 21.5 42.3 17.6 ckpt
CG-DETR 39.8 25.1 44.2 19.6 ckpt

TVSum (Highlight detection)

Avgerage top-5 mAP across 10 domains is reported.

Models ResNet152+GloVe CLIP CLIP+Slowfast I3D+CLIP
Moment DETR 85.9 89.1 86.7 86.7
QD-DETR 87.2 88.4 87.1 87.1
EaTR 86.2 86.7 85.0 85.0
UVCOM 87.6 87.7 87.9 87.9
TaskWeave 83.6 82.7 84.2 83.5
CG-DETR 87.1 88.1 87.9 87.9

YouTube Highlight (Highlight detection)

Avgerage top-5 mAP across 10 domains is reported.

Models CLIP CLIP+Slowfast
Moment DETR 66.2 67.7
QD-DETR 70.9 73.4
EaTR 66.7 67.4
UVCOM 75.6 73.1
TaskWeave 71.5 69.1
CG-DETR 74.3 74.1

Due to the file size, we do not distribute the individual weights of TVSum and YouTube Highlight (Highlight detection). If you want them, download from here.