Skip to content

maryjis/multimodal_DL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

70 Commits
 
 

Repository files navigation

Multimodal deep learning in neuroimaging

The articles related to Multimodal deep learning in neuroimaging are represented in the table. There are collection of articles related to classification task in psychiatry and neurology based on DL methods. In each article some neuroimaging data (sMRI, fMRI, PET, DTI, EEG, MEG, MultiOmics) are fused with each other for improving model predictions. The source code of proposed method is attached if it available. TO BE UPDATED

Article Disease Data/Preprocessing Method Fusion technique Result Code
Multimodal deep learning models for early detection of Alzheimer’s disease stage Alzheimer’s disease classification Data: ADNI Count: 220 patients Modality: sMRI,Genetic, Clinical Preprocessing: ART toolbox, extract 3D areas of 21 brain regions (associated with Alzheimer’s disease),Features including the brain volumes, voxel intensities, and texture based features. We extract features such as energy, entropy, and 13 Haralick texture features 3D CNN + Sparse Denoising autoencoders + Dense network Early,Intermediate, Late 0.8% +- 0.03% acc (Control vs MCI)
Multimodal deep learning for Alzheimer’s disease dementia assessment Alzheimer’s disease classification Data: ADNI Modality: sMRI, Clinical, demographics, past medical history, neuropsychological testing, functional assessments Preprocessing FMRIB Software Library v6.0 (FSL) for MRI 3D CNN + CatBoost Late 0.804 ± 0.011 acc (Control vs MCI) code
Combining Neuroimaging and Omics Datasets for Disease Classification Using Graph Neural Networks Parkinson's disease (PD) classification Data: PPMI Modality: fMRI, DTI, multi-omics (RNA Expression, Single Nucleotide Polymorphism (SNP), DNA Methylation and non-coding RNA) Preprocessing: fmriPrep, simple noise removal and Wilcoxon signed rank test for omics data CycleGAN on structural and functional connectomes to generate missing imaging modalities, JOIN-GCLA (Joining Omics and Imaging Networks via Graph Convolutional Layers and Attention) Early - -
Interpretable Graph Convolutional Network of multi-modality brain imaging for Alzheimer’s disease diagnosis Alzheimer’s disease classification Data: ADNI Count: 182 HC subjects, 476MCI subjects, and 97 AD subjects Modality: sMRI, FDG-PET,(AV45-PET) Preprocessing: SPM software Graph Convolutional Neural Network Early 0.818 ±.031 (MCI vs HC vs Alzheimer) -
Multi-View Imputation and Cross-Attention Network Based on Incomplete Longitudinal and Multi-Modal Data for Alzheimer’s Disease Prediction Alzheimer’s disease classification, Alzheimer’s longitudinal classification Data: ADNI , OASIS Count: 1387 Modality: sMRI, FDG-PET Preprocessing 90-dimensional region-of interest (ROI) features were separately extracted from the MRI and PET data for each subject. RNN-based network with cross-attention block (MCNet) Intermediate 0.830 ± 0.019 ADNI-1 code
Deep Learning-Based Feature Representation for AD/MCI Classification Alzheimer’s disease classification Data: ADNI Count: 99 MCI,52 HC Modality:MRI, PET, and CSF Preprocessing: 93 region-of-interest-based volumetric features from MRI and 93 mean intensity from PET, 3 CSF biomarkers of Aβ42, t-tau, and p-tau. Stacked AutoEncoder +SK-SVM Stacked AutoEncoder+ multi-kernel SVM Intermediate MK-SVM 0.850±0.012 acc (MCI vs Control)
Multi-Center and Multi-Channel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network Alzheimer’s disease classification Data: ADNI Count 163 NC, 44 SMC, 86 early MCI (EMCI), and 166 late MCI (LMCI) Modality: fMRI, DTI Preprocessing: GRETNA toolbox for fMRI data, PANDA toolbox for DTI data -> AAL template with 90 ROIs for each modality Graph Convolutional Neural Network Early NC vs. SMC 93.23 acc, NC vs. EMCI 91.16 acc, NC vs. LMCI 94.22 acc code
M-GCN: A Multimodal Graph Convolutional Network to Integrate Functional and Structural Connectomics Data to Predict Multidimensional Phenotypic Characterizations Predict Phenotypic measures Data: Human Connectome Project, Kennedy Krieger Institute dataset Modality: fMRI, DTI Preprocessing CompCorr and HCP pipeline for rs-fMRI, connectivity matrices with Automatic Anatomical Labeling (AAL) atlas; FSL and standard Neurodata MR Graphs package for DTI: Graph Convolutional Neural Network Intermediate CFIS (Cognitive Fluid Intelligence Score) - 12.87 ± 9.65 MAE , ADOS - 2.71 ± 2.15 MAE, SRS -16.50 ± 9.44 MAE code

Releases

No releases published

Packages

No packages published