This repository holds the Pytorch implementation of Joint-Partition Group Attention for Skeleton-Based Action Recognition by Hu Cui, Tessai Hayama.
We propose Joint-Partition Group Attention (JPGA) to simultaneously capture correlations between joints and body parts of different granularity sizes for skeleton-based action recognition. Specifically, JPGA integrates the joint tokens according to the joint’s human body partition attributes and produces different body parts tokens (partition-tokens) with different granularities. Then the attention map of JPGA is computed from joint-token and partition-token of different granularity sizes to represent the relationship between joints and body parts. Based on JPGA, we construct our Joint-Partition Former (JPFormer), and conduct extensive experiments on NTU-RGB+D, NTU-RRB+D 120, and Northwestern UCLA datasets and achieve state-of-the-art results, which highlights the effectiveness of our design and practice
This package has the following requirements:
Python 3.8
Pytorch 1.12
-
SHREC’17 Track dataset from http://www-rech.telecom-lille.fr/shrec2017-hand/
-
First, extract all files to
/data/shrec/shrec17_dataset
-
Then, run
python gen_traindataset.py
andpython gen_testdataset.py
-
-
DHG-14/28 dataset from http://www-rech.telecom-lille.fr/DHGdataset/
-
First, extract all files to
./data/DHG14-28/DHG14-28_dataset
-
Then, run
python python gen_dhgdataset.py
-
-
NTU RGB+D 60 Skeleton dataset from https://rose1.ntu.edu.sg/dataset/actionRecognition/
-
First, extract all skeleton files to
./data/ntu/nturgbd_raw
-
Then, run
python get_raw_skes_data.py
,python get_raw_denoised_data.py
andpython seq_transformation.py
in sequence -
put the .npz file to ntu or ntu120 fold.
-
-
NW-UCLA dataset from Download NW-UCLA dataset
-
Put downloaded data into the following directory structure:
- data/ - shrec/ - shrec17_dataset/ - HandGestureDataset_SHREC2017/ - gesture_1 ... - DHG14-28/ - DHG14-28_dataset/ - gesture_1 ... - NW-UCLA/ - all_sqe ... - ntu/ - NTU60_Xsub.npz - ... - ntu120/ - NTU120_XSet.npz - ...
You can change the configuration in the yaml file and in the main function.
Run python main.py --device 0 --config ./config/shrec/joint14.yaml
On cross-view, run python main.py --device 0 --config ./config/nturgbd-cross-view/joint.yaml
On cross-subject, run python main.py --device 0 --config ./config/nturgbd-cross-subject/joint.yaml
On cross-view, run python main.py --device 0 --config ./config/nturgbd120-cross-view/joint.yaml
On cross-subject, run python main.py --device 0 --config ./config/nturgbd120-cross-subject/joint.yaml
Run python main.py --device 0 --config ./config/ucla/joint.yaml
Run python main.py --device 0 --config ./config/ntu_test/bm.yaml
You need to specify the path value according to your own weights file.
Acknowledgements
This repo is based on