Skip to content

Commit 9e80432

Browse files
chenhaomingbobchenhaomingbob
committed
Add
0 parents  commit 9e80432

File tree

190 files changed

+22933
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

190 files changed

+22933
-0
lines changed

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
/demo/input/
2+
/demo/output/
3+
.idea

‎DCPose_requirement.txt

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
absl-py==0.10.0
2+
attrs==20.2.0
3+
cachetools==4.1.1
4+
certifi==2020.6.20
5+
chardet==3.0.4
6+
click==7.1.2
7+
cycler==0.10.0
8+
Cython==0.29.21
9+
flake8==3.8.3
10+
flake8-import-order==0.18.1
11+
future==0.18.2
12+
google-auth==1.22.1
13+
google-auth-oauthlib==0.4.1
14+
grpcio==1.32.0
15+
idna==2.10
16+
importlib-metadata==2.0.0
17+
iniconfig==1.0.1
18+
kiwisolver==1.2.0
19+
Markdown==3.3
20+
matplotlib==3.3.2
21+
mccabe==0.6.1
22+
motmetrics==1.2.0
23+
numpy==1.19.2
24+
oauthlib==3.1.0
25+
opencv-python==4.4.0.44
26+
packaging==20.4
27+
pandas==1.1.2
28+
Pillow==7.2.0
29+
pluggy==0.13.1
30+
protobuf==3.13.0
31+
py==1.9.0
32+
py-cpuinfo==7.0.0
33+
pyasn1==0.4.8
34+
pyasn1-modules==0.2.8
35+
pycocotools==2.0.2
36+
pycodestyle==2.6.0
37+
pyflakes==2.2.0
38+
pyparsing==2.4.7
39+
pytest==6.1.0
40+
pytest-benchmark==3.2.3
41+
python-dateutil==2.8.1
42+
pytz==2020.1
43+
PyYAML==5.3.1
44+
requests==2.24.0
45+
requests-oauthlib==1.3.0
46+
rsa==4.6
47+
scipy==1.5.2
48+
Shapely==1.7.1
49+
sheen==0.1.2
50+
six==1.15.0
51+
tabulate==0.8.7
52+
tensorboard==2.3.0
53+
tensorboard-plugin-wit==1.7.0
54+
tensorboardX==2.1
55+
termcolor==1.1.0
56+
toml==0.10.1
57+
torch==1.4.0
58+
torchvision==0.5.0
59+
tqdm==4.49.0
60+
urllib3==1.25.10
61+
Werkzeug==1.0.1
62+
xmltodict==0.12.0
63+
yacs==0.1.8
64+
zipp==3.2.0

‎README.md

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Deep Dual Consecutive Network for Human Pose Estimation （CVPR2021）
2+
3+
## Introduction
4+
5+
This is the official code of **Deep Dual Consecutive Network for Human Pose Estimation.**
6+
7+
Multi-frame human pose estimation in complicated situations is challenging. Although state-of-the-art human joints detectors have demonstrated remarkable results for static images, their performances come short when we apply these models to video sequences. Prevalent shortcomings include the failure to handle motion blur, video defocus, or pose occlusions, arising from the inability in capturing the temporal dependency among video frames. On the other hand, directly employing conventional recurrent neural networks incurs empirical difficulties in modeling spatial contexts, especially for dealing with pose occlusions. In this paper, we propose a novel multi-frame human pose estimation framework, leveraging abundant temporal cues between video frames to facilitate keypoint detection. Three modular components are designed in our framework. A Pose Temporal Merger encodes keypoint spatiotemporal context to generate effective searching scopes while a Pose Residual Fusion module computes weighted pose residuals in dual directions. These are then processed via our Pose Correction Network for efficient refining of pose estimations. Our method ranks No.1 in the Multi-frame Person Pose Estimation Challenge on the large-scale benchmark datasets PoseTrack2017 and PoseTrack2018. We have released our code, hoping to inspire future research.
8+
9+
10+
11+
![](docs/images/Pipeline.png)
12+
13+
14+
15+
## Visual Results
16+
17+
<video id="video" controls="" preload="none"> <source id="mp4" src="docs/DCPoseShow.mp4" type="video/mp4"> </video>
18+
19+
#### On PoseTrack
20+
21+
22+
<p align='center'>
23+
<img src="./docs/gifs/val_1.gif" style="zoom:100%;" />
24+
<img src="./docs/gifs/val_2.gif" style="zoom:100%;" />
25+
</p>
26+
<p align='center'>
27+
<img src="./docs/gifs/val_3.gif" style="zoom:100%;" />
28+
<img src="./docs/gifs/val_4.gif" style="zoom:100%;" />
29+
</p>
30+
<p align='center'>
31+
<img src="./docs/gifs/val_5.gif" style="zoom:100%;" />
32+
<img src="./docs/gifs/val_6.gif" style="zoom:100%;" />
33+
</p>
34+
35+
<p align='center'>
36+
<img src="./docs/gifs/val_7.gif" style="zoom:100%;" />
37+
<img src="./docs/gifs/val_8.gif" style="zoom:100%;" />
38+
</p>
39+
40+
41+
#### Comparison with SOTA method
42+
43+
<img src="./docs/gifs/con_1.gif" style="zoom:120%;" />
44+
45+
<img src="./docs/gifs/con_2.gif" style="zoom:144%;" />
46+
47+
48+
49+
50+
51+
52+
53+
## Experiments
54+
55+
56+
57+
#### Results on PoseTrack 2017 validation set
58+
59+
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mean |
60+
| ----------------------------- | ---- | -------- | ----- | ----- | ---- | ---- | ----- | ---- |
61+
| **PoseFlow** | 66.7 | 73.3 | 68.3 | 61.1 | 67.5 | 67.0 | 61.3 | 66.5 |
62+
| **JointFlow** | - | - | - | - | - | - | - | 69.3 |
63+
| **FastPose** | 80.0 | 80.3 | 69.5 | 59.1 | 71.4 | 67.5 | 59.4 | 70.3 |
64+
| **SimpleBaseline**(2018 ECCV) | 81.7 | 83.4 | 80.0 | 72.4 | 75.3 | 74.8 | 67.1 | 76.7 |
65+
| **STEmbedding** | 83.8 | 81.6 | 77.1 | 70.0 | 77.4 | 74.5 | 70.8 | 77.0 |
66+
| **HRNet**(2019 CVPR) | 82.1 | 83.6 | 80.4 | 73.3 | 75.5 | 75.3 | 68.5 | 77.3 |
67+
| **MDPN** | 85.2 | 88.8 | 83.9 | 77.5 | 79.0 | 77.0 | 71.4 | 80.7 |
68+
| **PoseWarper**(2019 NIPS) | 81.4 | 88.3 | 83.9 | 78.0 | 82.4 | 80.5 | 73.6 | 81.2 |
69+
| **DCPose** | 88.0 | 88.7 | 84.1 | 78.4 | 83.0 | 81.4 | 74.2 | 82.8 |
70+
71+
72+
73+
#### Results on PoseTrack 2017 test set(https://posetrack.net/leaderboard.php)
74+
75+
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Total |
76+
| -------------- | ---- | -------- | ----- | ----- | ---- | ---- | ----- | ----- |
77+
| PoseFlow | 64.9 | 67.5 | 65.0 | 59.0 | 62.5 | 62.8 | 57.9 | 63.0 |
78+
| JointFlow | - | - | - | 53.1 | - | - | 50.4 | 63.4 |
79+
| KeyTrack | - | - | - | 71.9 | - | - | 65.0 | 74.0 |
80+
| DetTrack | - | - | - | 69.8 | - | - | 65.9 | 74.1 |
81+
| SimpleBaseline | 80.1 | 80.2 | 76.9 | 71.5 | 72.5 | 72.4 | 65.7 | 74.6 |
82+
| HRNet | 80.0 | 80.2 | 76.9 | 72.0 | 73.4 | 72.5 | 67.0 | 74.9 |
83+
| PoseWarper | 79.5 | 84.3 | 80.1 | 75.8 | 77.6 | 76.8 | 70.8 | 77.9 |
84+
| DCPose | 84.3 | 84.9 | 80.5 | 76.1 | 77.9 | 77.1 | 71.2 | 79.2 |
85+
86+
87+
88+
#### Results on PoseTrack 2018 validation set
89+
90+
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mean |
91+
| ---------- | ---- | -------- | ----- | ----- | ---- | ---- | ----- | ---- |
92+
| AlphaPose | 63.9 | 78.7 | 77.4 | 71.0 | 73.7 | 73.0 | 69.7 | 71.9 |
93+
| MDPN | 75.4 | 81.2 | 79.0 | 74.1 | 72.4 | 73.0 | 69.9 | 75.0 |
94+
| PoseWarper | 79.9 | 86.3 | 82.4 | 77.5 | 79.8 | 78.8 | 73.2 | 79.7 |
95+
| DCPose | 84.0 | 86.6 | 82.7 | 78.0 | 80.4 | 79.3 | 73.8 | 80.9 |
96+
97+
98+
99+
#### Results on PoseTrack 2018 test set
100+
101+
| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mean |
102+
| ----------- | ---- | -------- | ----- | ----- | ---- | ---- | ----- | ---- |
103+
| AlphaPose++ | - | - | - | 66.2 | - | - | 65.0 | 67.6 |
104+
| DetTrack | - | - | - | 69.8 | - | - | 67.1 | 73.5 |
105+
| MDPN | - | - | - | 74.5 | - | - | 69.0 | 76.4 |
106+
| PoseWarper | 78.9 | 84.4 | 80.9 | 76.8 | 75.6 | 77.5 | 71.8 | 78.0 |
107+
| DCPose | 82.8 | 84.0 | 80.8 | 77.2 | 76.1 | 77.6 | 72.3 | 79.0 |
108+
109+
110+
111+
## Installation & Quick Start
112+
113+
**Check** [docs/installation.md](docs/Installation.md) for instructions on how to build DCPose from source.
114+
115+

‎configs/posetimation/DcPose/Base_PoseTrack17.yaml

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
CUDNN:
2+
BENCHMARK: false
3+
DETERMINISTIC: false
4+
ENABLED: true
5+
6+
WORKERS: 4
7+
PRINT_FREQ: 100
8+
DISTANCE: 2
9+
10+
DATASET:
11+
NAME: "PoseTrack"
12+
JSON_DIR: "./DcPose_supp_files/posetrack17_json_files"
13+
IMG_DIR: "./DataSet/PoseTrack2017/images_renamed"
14+
TEST_IMG_DIR: "./DataSet/PoseTrack2017/images"
15+
IS_POSETRACK18: false
16+
COLOR_RGB: true
17+
DATASET: 'posetrack'
18+
ROOT: ''
19+
TEST_SET: 'val2017'
20+
TRAIN_SET: 'train2017'
21+
# INPUT_TYPE in ['single_frame','spatiotemporal_window']
22+
INPUT_TYPE: 'spatiotemporal_window'
23+
BBOX_ENLARGE_FACTOR: 1.25
24+
25+
26+
MODEL:
27+
FREEZE_HRNET_WEIGHTS: true
28+
EVALUATE: true
29+
INIT_WEIGHTS: true
30+
31+
NUM_JOINTS: 17
32+
PRETRAINED: './DcPose_supp_files/pretrained_models/pretrained_coco_model.pth'
33+
TARGET_TYPE: gaussian
34+
IMAGE_SIZE:
35+
- 288
36+
- 384
37+
HEATMAP_SIZE:
38+
- 72
39+
- 96
40+
SIGMA: 3
41+
USE_RECTIFIER: true
42+
43+
EXTRA:
44+
PRETRAINED_LAYERS:
45+
- '*'
46+
FINAL_CONV_KERNEL: 1
47+
STAGE2:
48+
NUM_MODULES: 1
49+
NUM_BRANCHES: 2
50+
BLOCK: BASIC
51+
NUM_BLOCKS:
52+
- 4
53+
- 4
54+
NUM_CHANNELS:
55+
- 48
56+
- 96
57+
FUSE_METHOD: SUM
58+
STAGE3:
59+
NUM_MODULES: 4
60+
NUM_BRANCHES: 3
61+
BLOCK: BASIC
62+
NUM_BLOCKS:
63+
- 4
64+
- 4
65+
- 4
66+
NUM_CHANNELS:
67+
- 48
68+
- 96
69+
- 192
70+
FUSE_METHOD: SUM
71+
STAGE4:
72+
NUM_MODULES: 3
73+
NUM_BRANCHES: 4
74+
BLOCK: BASIC
75+
NUM_BLOCKS:
76+
- 4
77+
- 4
78+
- 4
79+
- 4
80+
NUM_CHANNELS:
81+
- 48
82+
- 96
83+
- 192
84+
- 384
85+
FUSE_METHOD: SUM
86+
DECONV:
87+
NUM_DECONVS: 1
88+
NUM_CHANNELS:
89+
- 48
90+
KERNEL_SIZE:
91+
- 4
92+
NUM_BASIC_BLOCKS: 4
93+
CAT_OUTPUT:
94+
- True
95+
96+
TRAIN:
97+
FLIP: true
98+
NUM_JOINTS_HALF_BODY: 8
99+
PROB_HALF_BODY: 0.3
100+
ROT_FACTOR: 45
101+
SCALE_FACTOR: [ 0.35,0.35 ]
102+
SHUFFLE: true
103+
BEGIN_EPOCH: 0
104+
END_EPOCH: 20
105+
OPTIMIZER: adam
106+
MILESTONES:
107+
- 4
108+
- 8
109+
- 12
110+
- 16
111+
GAMMA: 0.1
112+
113+
114+
MOMENTUM: 0.9
115+
NESTEROV: false
116+
117+
VAL:
118+
ANNOT_DIR: './DcPose_supp_files/posetrack17_annotation_dirs/jsons/val/'
119+
COCO_BBOX_FILE: './DcPose_supp_files/posetrack17_precomputed_boxes/val_boxes.json'
120+
USE_GT_BBOX: false
121+
BBOX_THRE: 1.0
122+
IMAGE_THRE: 0.2
123+
IN_VIS_THRE: 0.2
124+
NMS_THRE: 1.0
125+
OKS_THRE: 0.9
126+
FLIP_VAL: false
127+
POST_PROCESS: true
128+
129+
TEST:
130+
ANNOT_DIR: './DcPose_supp_files/posetrack17_annotation_dirs/jsons/test'
131+
COCO_BBOX_FILE: './DcPose_supp_files/posetrack17_precomputed_boxes/test_boxes.json'
132+
USE_GT_BBOX: false
133+
BBOX_THRE: 1.0
134+
IMAGE_THRE: 0.2
135+
IN_VIS_THRE: 0.2
136+
NMS_THRE: 1.0
137+
OKS_THRE: 0.9
138+
FLIP_TEST: false
139+
POST_PROCESS: true

0 commit comments

Comments
 (0)