Skip to main content
placeholder image

Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer’s Disease Prediction

Conference Paper


Abstract


  • Multimodal neuroimage can provide complementary information about the dementia, but small size of complete multimodal data limits the ability in representation learning. Moreover, the data distribution inconsistency from different modalities may lead to ineffective fusion, which fails to sufficiently explore the intra-modal and inter-modal interactions and compromises the disease diagnosis performance. To solve these problems, we proposed a novel multimodal representation learning and adversarial hypergraph fusion (MRL-AHF) framework for Alzheimer’s disease diagnosis using complete trimodal images. First, adversarial strategy and pre-trained model are incorporated into the MRL to extract latent representations from multimodal data. Then two hypergraphs are constructed from the latent representations and the adversarial network based on graph convolution is employed to narrow the distribution difference of hyperedge features. Finally, the hyperedge-invariant features are fused for disease prediction by hyperedge convolution. Experiments on the public Alzheimer’s Disease Neuroimaging Initiative(ADNI) database demonstrate that our model achieves superior performance on Alzheimer’s disease detection compared with other related models and provides a possible way to understand the underlying mechanisms of disorder’s progression by analyzing the abnormal brain connections.

Publication Date


  • 2021

Citation


  • Zuo, Q., Lei, B., Shen, Y., Liu, Y., Feng, Z., & Wang, S. (2021). Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer’s Disease Prediction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Vol. 13021 LNCS (pp. 479-490). doi:10.1007/978-3-030-88010-1_40

Scopus Eid


  • 2-s2.0-85118201032

Web Of Science Accession Number


Start Page


  • 479

End Page


  • 490

Volume


  • 13021 LNCS

Abstract


  • Multimodal neuroimage can provide complementary information about the dementia, but small size of complete multimodal data limits the ability in representation learning. Moreover, the data distribution inconsistency from different modalities may lead to ineffective fusion, which fails to sufficiently explore the intra-modal and inter-modal interactions and compromises the disease diagnosis performance. To solve these problems, we proposed a novel multimodal representation learning and adversarial hypergraph fusion (MRL-AHF) framework for Alzheimer’s disease diagnosis using complete trimodal images. First, adversarial strategy and pre-trained model are incorporated into the MRL to extract latent representations from multimodal data. Then two hypergraphs are constructed from the latent representations and the adversarial network based on graph convolution is employed to narrow the distribution difference of hyperedge features. Finally, the hyperedge-invariant features are fused for disease prediction by hyperedge convolution. Experiments on the public Alzheimer’s Disease Neuroimaging Initiative(ADNI) database demonstrate that our model achieves superior performance on Alzheimer’s disease detection compared with other related models and provides a possible way to understand the underlying mechanisms of disorder’s progression by analyzing the abnormal brain connections.

Publication Date


  • 2021

Citation


  • Zuo, Q., Lei, B., Shen, Y., Liu, Y., Feng, Z., & Wang, S. (2021). Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer’s Disease Prediction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Vol. 13021 LNCS (pp. 479-490). doi:10.1007/978-3-030-88010-1_40

Scopus Eid


  • 2-s2.0-85118201032

Web Of Science Accession Number


Start Page


  • 479

End Page


  • 490

Volume


  • 13021 LNCS