Awesome Action Quality Assessment (AQA)

Pull requests are welcome if you find any interesting paper is missing.

• 🔥🔥🔥 denotes the highly recommended papers for the novel insights and new directions.
• We have created a Wechat group for our colleagues to exchange and discuss the latest progress in the field of AQA, contributing to the development of this field. If it expires, please scan my personal account.

Table of Contents: Papers and Datasets

1. Papers
   • Sports
   • Medical Care
   • Skill Assessment
2. Datasets

Papers

Currently, AQA is widely applied in sports, medical care (e.g., rehabilitation), skill assessment, etc.

Highly Recommended Survey and Benchmark

• [arXiv 2024] A Comprehensive Survey of Action Quality Assessment: Method and Benchmark. PDF | Github | $\color{orange}\textsf{Survey}$

Sports

2024

• [IJCV 2024] Procedure-Aware Action Quality Assessment: Datasets and Performance Evaluation. PDF | Github
• [TIP 2024] Self-supervised subaction Parsing Network for Semi-supervised Action Quality Assessment. PDF | DOI
• [TIP 2024] Multimodal Action Quality Assessment. PDF | Github | $\color{yellow}\textsf{Video + Audio}\color{yellow}\textsf{ + Flow}$
• [TCSVT 2024] Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling. PDF | Github | 🧠 $\color{pink}\textsf{Continual Learning}$
• [TCSVT 2024] Visual-semantic Alignment Temporal Parsing for Action Quality Assessment. DOI | $\color{cyan}\textsf{Video + Text}$
• [Information Sciences 2024] Two-path target-aware contrastive regression for action quality assessment. DOI | Github
• [TIM 2024] Learning Sparse Temporal Video Mapping for Action Quality Assessment in Floor Gymnastics. PDF | $\color{lightgreen}\textsf{Video + Skeleton}$
• [TCE 2024] ResFNN: Residual Structure-Based Feedforward Neural Network for Action Quality Assessment in Sports Consumer Electronics. DOI
• [Applied Intelligence 2024] Assessing action quality with semantic-sequence performance regression and densely distributed sample weighting. DOI
• [Neurocomputing 2024] Dual-referenced assistive network for action quality assessment. DOI
• [ESWA 2024] Vision-based human action quality assessment: A systematic review. DOI | $\color{orange}\textsf{Survey}$

---

• [NeurIPS 2024 Spotlight] GAIA: Rethinking Action Quality Assessment for AI-Generated Videos. PDF | Github |🔥🔥🔥
• [NeurIPS 2024] LucidAction: A Hierarchical and Multi-model Dataset for Comprehensive Action Quality Assessment. NeurIPS 2024 Website | $\color{lightgreen}\textsf{Video + Skeleton}$
• [CVPR 2024 Oral] FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment. PDF | Github
• [CVPR 2024] Narrative Action Evaluation with Prompt-Guided Multimodal Interaction. PDF | Github | $\color{magenta}\textsf{Interpretable Feedback}$, $\color{cyan}\textsf{Video + Text}$
• [ECCV 2024 Oral] MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment. PDF | Github | 🧠 $\color{pink}\textsf{Continual Learning}$
• [ECCV 2024] RICA^2: Rubric-Informed, Calibrated Assessment of Actions. PDF | Github | $\color{cyan}\textsf{Video + Text}$
• [ECCV 2024] Semi-Supervised Teacher-Reference-Student Architecture for Action Quality Assessment. PDF
• [ECCV 2024] Vision-Language Action Knowledge Learning for Semantic-Aware Action Quality Assessment. PDF | $\color{cyan}\textsf{Video + Text}$
• [IJCAI 2024] CoFInAl: Enhancing Action Quality Assessment with Coarse-to-Fine Instruction Alignment. PDF | Github
• [ACM MM 2024] 2M-AF: A Strong Multi-Modality Framework For Human Action Quality Assessment with Self-supervised Representation Learning. PDF
• [BMVC 2024] Interpretable Long-term Action Quality Assessment. PDF | Github
• [WACV 2024 Oral] PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment. PDF | Github | 🧠 $\color{pink}\textsf{Continual Learning}$
• [ICASSP 2024] Multi-Stage Contrastive Regression for Action Quality Assessment. PDF | Github
• [CVPRW 2024] Hierarchical NeuroSymbolic Approach for Comprehensive and Explainable Action Quality Assessment. PDF | Github | $\color{magenta}\textsf{Interpretable Feedback}$

---

• [arXiv 2024] Learning to Score Sign Language with Two-stage Method. PDF
• [arXiv 2024] TechCoach: Towards Technical Keypoint-Aware Descriptive Action Coaching. PDF | $\color{magenta}\textsf{Interpretable Feedback}$
• [arXiv 2024] ExpertAF: Expert Actionable Feedback from Video. PDF | $\color{magenta}\textsf{Interpretable Feedback}$

2023

• [IJCV 2023] Automatic Modelling for Interactive Action Assessment. DOI
• [TIP 2023] Fine-grained Spatio-temporal Parsing Network for Action Quality Assessment. DOI
• [TCSVT 2023] Hierarchical Graph Convolutional Networks for Action Quality Assessment. PDF | DOI | Github
• [TMM 2023] Learning Semantics-Guided Representations for Scoring Figure Skating. DOI | $\color{cyan}\textsf{Video + Text}$
• [NCAA 2023] Auto-encoding score distribution regression for action quality assessment. PDF | DOI | Github
• [Applied Intelligence 2023] Multi-skeleton structures graph convolutional network for action quality assessment in long videos. DOI
• [Applied Intelligence 2023] Label-reconstruction-based pseudo-subscore learning for action quality assessment in sporting events. DOI
• [Applied Intelligence 2023] Improving action quality assessment with across-staged temporal reasoning on imbalanced data. DOI

---

• [CVPR 2023] LOGO: A Long-Form Video Dataset for Group Action Quality Assessment. PDF | Github
• [AAAI 2023] Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs. PDF | Github | $\color{yellow}\text{Video + Audio}$
• [ACM MM 2023] A Figure Skating Jumping Dataset for Replay-Guided Action Quality Assessment. DOI
• [ACM MM 2023] Localization-assisted Uncertainty Score Disentanglement Network for Action Quality Assessment. DOI | Github - FineFS dataset avail, code not avail
• [IUI 2023] IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment. PDF | Github
• [ICASSP 2023] Contrastive Self-Supervised Learning for Automated Multi-Modal Dance Performance Assessment. DOI

2022

• [TCSVT 2022] Semi-Supervised Action Quality Assessment With Self-Supervised Segment Feature Recovery. DOI
• [JVCIR 2022] Skeleton-based deep pose feature learning for action quality assessment on figure skating videos. DOI

---

• [CVPR 2022 Oral] FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment. PDF | Github
• [CVPR 2022] Likert Scoring with Grade Decoupling for Long-term Action Assessment. PDF | Github
• [ECCV 2022] Action Quality Assessment with Temporal Parsing Transformer. PDF | Github
• [ECCV 2022] Pairwise Contrastive Learning Network for Action Quality Assessment. PDF

2021

• [TPAMI 2021] Adaptive Action Assessment. DOI
• [TOMM 2021] Will You Ever Become Popular? Learning to Predict Virality of Dance Clips. PDF

---

• [CVPR 2021] AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training. PDF | $\color{magenta}\textsf{Interpretable Feedback}$
• [ICCV 2021] Group-aware Contrastive Regression for Action Quality Assessment. PDF | Github
• [ACM MM 2021] TSA-Net: Tube Self-Attention Network for Action Quality Assessment. PDF | Github
• [WACV 2021] EAGLE-Eye: Extreme-pose Action Grader using detaiL bird’s-Eye view. PDF
• [ICIP 2021] Action quality assessment with ignoring scene context. DOI
• [PETRA 2021] Towards Improved and Interpretable Action Quality Assessment with Self-Supervised Alignment. PDF
• [INSAI 2021] A survey of video-based action quality assessment. PDF | $\color{orange}\textsf{Survey}$

2020

• [CVPR 2020] Uncertainty-aware Score Distribution Learning for Action Quality Assessment. PDF | Github
• [ECCV 2020] An Asymmetric Modeling for Action Assessment. PDF

2019 and Before

• [CVPR 2019] What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment. PDF | Github
• [ICCV 2019] Action Assessment by Joint Relation Graphs. PDF
• [WACV 2019] Action Quality Assessment Across Multiple Actions. PDF
• [PCM 2018] End-to-end learning for action quality assessment. DOI
• [ICIP 2018] S3d: Stacking segmental p3d for action quality assessment. DOI
• [ACCV 2018] Scoringnet: Learning key fragment for action quality assessment with ranking loss in skilled sports. DOI
• [ICCV 2017] Am I a Baller? Basketball Performance Assessment from First-Person Videos. PDF
• [BMVC 2015] Dynamical Regularity for Action Analysis. PDF
• [ECCV 2014] Assessing the Quality of Actions. PDF | Project

(back to top)

Medical Care

2024

• [TPAMI 2024] EGCN++: A New Fusion Strategy for Ensemble Learning in Skeleton-based Rehabilitation Exercise Assessment. PDF | GitHub - EGCN++ not avail. EHE dataset avail.
• [TNSRE 2024] Automatic Assessment of Upper Extremity Function and Mobile Application for Self-Administered Stroke Rehabilitation. DOI
• [TNSRE 2024] An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises Assessment. DOI

---

• [CVPRW 2024] FineRehab: A Multi-modality and Multi-task Dataset for Rehabilitation Analysis. PDF | Project

2023

• [TVCG 2023] A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis. PDF | DOI | Video | $\color{magenta}\textsf{Interpretable Feedback}$
• [TNSRE 2023] A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises. PDF
• [TNSRE 2023] A Skeleton-Based Rehabilitation Exercise Assessment System With Rotation Invariance. DOI | Github

---

• [CHASE 2023] Short: Deep Learning Approach to Skeletal Performance Evaluation of Physical Therapy Exercises. DOI
• [ICCVW 2023] Personalized Monitoring in Home Healthcare: An Assistive System for Post Hip Replacement Rehabilitation. PDF

2022

• [TNSRE 2022] Graph convolutional networks for assessment of physical rehabilitation exercises. DOI

---

• [IJCAI 2022] EGCN: An Ensemble-based Learning Framework for Exploring Effective Skeleton-based Rehabilitation Exercise Assessment. PDF | Github

2021

• [PR 2021] Skeleton-based human action evaluation using graph convolutional network for monitoring Alzheimer’s progression. PDF | DOI

2020

• [TNSRE 2020] A Deep Learning Framework for Assessing Physical Rehabilitation Exercises. PDF | DOI
• [Sensors 2020] VI-Net: View-Invariant Quality of Human Movement Assessment. PDF

2019 and Before

• [TNSRE 2019] The KIMORE Dataset: KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation. PDF | DOI
• [Data 2018] A Data Set of Human Body Movements for Physical Rehabilitation Exercises. PDF

(back to top)

Skill Assessment

2024

• [TCSVT 2024] Rhythmer: Ranking-based Skill Assessment with Rhythm-aware Transformer. DOI
• [TMM 2024] Adaptive Stage-Aware Assessment Skill Transfer for Skill Determination. DOI

---

• [ICASSP 2024] Which is the Better Teacher Action? A New Ranking Model and Dataset. DOI | Github

2023

• [RA-L 2023] Keep Your Eye on the Best: Contrastive Regression Transformer for Skill Assessment in Robotic Surgery. PDF | Github

---

• [MICCAI 2023] Sedskill: Surgical events driven method for skill assessment from thoracoscopic surgical videos. PDF | Github

2022

• [ECCV 2022] Domain knowledge-informed self-supervised representations for workout form assessment. PDF | Github
• [MICCAI 2022] Surgical skill assessment via video semantic aggregation. PDF | Github
• [MICCAI 2022] Video-based surgical skills assessment using long term tool tracking. PDF

2021

• [CVPR 2021] Towards unified surgical skill assessment. PDF

• [MMSP 2021] Piano-Skills-Assessment. DOI | Github

2020

• [MICCAI 2020] Towards accurate and interpretable surgical skill assessment: A video-based method incorporating recognized surgical gestures and skill levels. PDF

2019 and Before

• [CVPR 2019] The pros and cons: Rank-aware temporal attention for skill determination in long videos. PDF
• [ICCVW 2019] Manipulation-skill assessment from videos with spatial attention network. PDF
• [CVPR 2018] Who's better? who's best? pairwise deep ranking for skill determination. PDF

(back to top)

Datasets

Dataset	Year	Modality	Domains	# Classes	# Samples	# Avg. Frames	Annotations	URL
MIT Olympic	2014	Video, 2D Skeleton	Sports	2	309	Dive: 150, Figure Skate: 4200	Score	Link
UNLV Olympic	2017	Video	Sports	3	717	Dive: 150, Figure Skate: 4200, Vault: 75	Score	Link
AQA-7	2019	Video	Sports	7	1189	Dive: 97-105-156, Vault: 87, Big Air: 122-132, Trampoline: 634	Score	Link
MTL-AQA	2019	Video	Sports	16	1412	96	Score	Link
Fis-V	2019	Video	Sports	1	500	4300	TES PCS	Link
RG	2020	Video	Sports	1	250	2375	Difficulty, Execution, Total	Link
FineDiving	2022	Video	Sports	52	3000	105	Step, Score	Link
FS1000	2023	Video, Audio	Sports	7	1604	5000	TES PCS, detailed PCS	Link
FineFS	2023	Video, 2D/3D Skeleton	Sports	4	1167	5000	detailed Score, Subaction Class, Segmentation	Link
LOGO	2023	Video	Sports	12	200	5100	Action Class, Formation, Score	Link
GAIA	2024	Video (AI Gen.)	All	510	9180	70	Subject, Completeness, Interaction	Link
JIGSAWS	2014	Video	Skill Assessment	3	103	--	Surgemes Class, Rating	Link
EPIC-Skills	2018	Video	Skill Assessment	7	216	--	Relative Rank	Link
BEST	2019	Video	Skill Assessment	5	500	6400	Relative Rank	Link
PISA	2021	Video, Audio	Skill Assessment	1	992	160	Skill Level, Difficulty	Link
TAQR	2024	Video	Skill Assessment	4	300	488	Relative Rank	Link
UI-PRMD	2018	3D Skeleton, Joint Pos. & Ori.	Medical Care	10	1326	--	Binary Class	Link
KIMORE	2019	Video, 3D Skeleton, Joint Pos. & Ori.	Medical Care	5	1560	--	Score	Link
EHE	2021	3D Skeleton, Joint Pos. & Ori.	Medical Care	6	869	--	Binary Class	Link
FineRehab	2024	Video, 3D Skeleton, Joint Pos. & Ori.	Medical Care	16	4215	--	Score	Link

(back to top)