Skip to content

Latest commit

 

History

History
56 lines (41 loc) · 8.07 KB

README.md

File metadata and controls

56 lines (41 loc) · 8.07 KB
Raw

STGCN++

Introduction

STGCN++ is a variant of STGCN we developed in PYSKL with some modifications in the architecture of the spatial module and the temporal module. We provide STGCN++ trained on NTURGB+D with 2D skeletons (HRNet) and 3D skeletons with PYSKL training setting. We provide checkpoints for four modalities: Joint, Bone, Joint Motion, and Bone Motion. The architecture of STGCN++ is described in PYSKL tech report.

Citation

@inproceedings{duan2022pyskl,
  title={Pyskl: Towards good practices for skeleton action recognition},
  author={Duan, Haodong and Wang, Jiaqi and Chen, Kai and Lin, Dahua},
  booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
  pages={7351--7354},
  year={2022}
}

Model Zoo

We release numerous checkpoints trained with various modalities, annotations on NTURGB+D and NTURGB+D 120. The accuracy of each modality links to the weight file.

Dataset Annotation GPUs Joint Top1 Bone Top1 Joint Motion Top1 Bone-Motion Top1 Two-Stream Top1 Four Stream Top1
NTURGB+D XSub Official 3D Skeleton 8 joint_config: 89.3 bone_config: 90.1 joint_motion_config: 87.5 bone_motion_config: 87.3 91.4 92.1
NTURGB+D XSub HRNet 2D Skeleton 8 joint_config: 89.3 bone_config: 92.3 joint_motion_config: 84.0 bone_motion_config: 88.8 92.8 93.2
NTURGB+D XView Official 3D Skeleton 8 joint_config: 95.6 bone_config: 95.5 joint_motion_config: 94.3 bone_motion_config: 93.8 96.7 97.0
NTURGB+D XView HRNet 2D Skeleton 8 joint_config: 97.4 bone_config: 97.2 joint_motion_config: 93.4 bone_motion_config: 95.4 98.4 98.5
NTURGB+D 120 XSub Official 3D Skeleton 8 joint_config: 83.2 bone_config: 85.6 joint_motion_config: 80.4 bone_motion_config: 81.5 87.0 87.5
NTURGB+D 120 XSub HRNet 2D Skeleton 8 joint_config: 84.4 bone_config: 84.8 joint_motion_config: 76.4 bone_motion_config: 81.1 86.4 86.4
NTURGB+D 120 XSet Official 3D Skeleton 8 joint_config: 85.6 bone_config: 87.5 joint_motion_config: 84.3 bone_motion_config: 83.0 89.1 89.8
NTURGB+D 120 XSet HRNet 2D Skeleton 8 joint_config: 88.1 bone_config: 88.5 joint_motion_config: 82.6 bone_motion_config: 84.1 90.0 90.3

Note

  1. We use the linear-scaling learning rate (Initial LR ∝ Batch Size). If you change the training batch size, remember to change the initial LR proportionally.
  2. For Two-Stream results, we adopt the 1 (Joint):1 (Bone) fusion. For Four-Stream results, we adopt the 2 (Joint):2 (Bone):1 (Joint Motion):1 (Bone Motion) fusion.

Training & Testing

You can use the following command to train a model.

bash tools/dist_train.sh ${CONFIG_FILE} ${NUM_GPUS} [optional arguments]
# For example: train STGCN++ on NTURGB+D XSub (3D skeleton, Joint Modality) with 8 GPUs, with validation, and test the last and the best (with best validation metric) checkpoint.
bash tools/dist_train.sh configs/stgcn++/stgcn++_ntu60_xsub_3dkp/j.py 8 --validate --test-last --test-best

You can use the following command to test a model.

bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${NUM_GPUS} [optional arguments]
# For example: test STGCN++ on NTURGB+D XSub (3D skeleton, Joint Modality) with metrics `top_k_accuracy`, and dump the result to `result.pkl`.
bash tools/dist_test.sh configs/stgcn++/stgcn++_ntu60_xsub_3dkp/j.py checkpoints/SOME_CHECKPOINT.pth 8 --eval top_k_accuracy --out result.pkl