Skip to main content

Advertisement

Springer Nature Link
Log in

Convolutional Cross-Modal Autoencoder-Based Few-Shot Learning for Data Augmentation with Application to Alzheimer Dementia Diagnosis

  • Research
  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

This paper presents a novel deep few-shot learning method for magnetic resonance images-based Alzheimer’s dementia (AD) diagnosis. The proposed method consists of two main phases namely data augmentation and data classification. With regard to data augmentation and, to deal with data scarcity issues, we designed a convolutional cross-modal autoencoder (CCMAE) model for data generation. This model, which consists of two encoders and one decoder, receives two image modalities namely longitudinal and cross-section MRI, and generates a new cross-section image. We opt for a convolutional version of the autoencoder to capture the spatial information more effectively and reduce the number of trainable parameters. Moreover, to make the model able to perform deep analysis of input image, we establish a skip connection strategy between the first encoder and the decoder similar to the UNet mechanism. With regard to classification, we design a convolutional neural network-based model in which both textual and visual features are fused to strengthen the network performance and produce more reliable decisions. A comprehensive experiment on a publicly available dataset has been conducted to demonstrate the effectiveness of the proposed method compared to some related works. The code is publicly available at: https://github.com/Bazine-Othmane/scientific-paper-code.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data that support the findings of this study are openly available in the Open Access Series of Imaging Studies (OASIS-1) at www.oasis-brains.org.information

Notes

  1. www.oasis-brains.org

References

  1. Aiadi O, Khaldi B, Saadeddine C. MDFNet: an unsupervised lightweight network for ear print recognition. J. Ambient. Int. Humanized Comput. 2022;1–14

  2. Aiadi O, Khaldi B. A fast lightweight network for the discrimination of COVID-19 and pulmonary diseases. Biomed Signal Process Control. 2022;78.

  3. Lu L, Zheng Y, Carneiro G, Yang L. Deep learning and convolutional neural networks for medical image computing. Adv Comput Vis Pattern Recognit. 2017;10:978–3.

    MATH  Google Scholar 

  4. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH. Deep learning for identifying metastatic breast cancer 2016. arXiv preprint arXiv:1606.05718

  5. Sarraf S, DeSouza DD, Anderson J, Tofighi G, Initiativ ADN. DeepAD: Alzheimer’s disease classification via deep convolutional neural networks using MRI and fMRI. BioRxiv, 2016;070441

  6. Payan A, Montana G. Predicting Alzheimer’s disease: a neuroimaging study with 3d convolutional neural networks 2015. arXiv preprint arXiv:1502.02506.

  7. Suk H-I, Lee S-W, Shen D, Initiative ADN. Latent feature representation with stacked auto-encoder for ad/mci diagnosis. Brain Struct Funct. 2015;220:841–59.

    Article  MATH  Google Scholar 

  8. Kumar S, Payne P, Sotiras A. Normative modeling using multimodal variational autoencoders to identify abnormal brain structural patterns in Alzheimer disease 2021. arXiv preprint arXiv:2110.04903

  9. Ferri R, Babiloni C, Karami V, Triggiani AI, Carducci F, Noce G, Lizio R, Pascarelli MT, Soricelli A, Amenta F, et al. Stacked autoencoders as new models for an accurate Alzheimer’s disease classification support using resting-state EEG and MRI measurements. Clin Neurophysiol. 2021;132(1):232–45.

    Article  Google Scholar 

  10. Xing X, Liang G, Zhang Y, Khanal S, Lin A-L, Jacobs N. Advit: Vision transformer on multi-modality pet images for alzheimer disease diagnosis. In: IEEE 19th International symposium on biomedical imaging (ISBI). IEEE. 2022;2022:1–4.

  11. Tummala S, Kadry S, Bukhari SAC, Rauf HT. Classification of brain tumor from magnetic resonance imaging using vision transformers ensembling. Curr Oncol. 2022;29(10):7498–511.

    Article  Google Scholar 

  12. Carcagnì P, Leo M, Del Coco M, Distante C, De Salve A. Convolution neural networks and self-attention learners for Alzheimer dementia diagnosis from brain MRI. Sensors. 2023;23(3):1694.

    Article  Google Scholar 

  13. Dai Z, Yi J, Yan L, Xu Q, Hu L, Zhang Q, Li J, Wang G. PFEMed: few-shot medical image classification using prior guided feature enhancement. Pattern Recogn. 2023;134.

  14. Wang X, Yuan Y, Guo D, Huang X, Cui Y, Xia M, Wang Z, Bai C, Chen S. SSA-Net: spatial self-attention network for COVID-19 pneumonia infection segmentation with semi-supervised few-shot learning. Med Image Anal. 2022;79.

  15. Tong T, Wolz R, Gao Q, Guerrero R, Hajnal JV, Rueckert D, Initiative ADN, et al. Multiple instance learning for classification of dementia in brain MRI. Med Image Anal. 2014;18(5):808–18.

    Article  Google Scholar 

  16. Atnafu SW, Diciotti S. Development of an interpretable deep learning system for the identification of patients with Alzheimer’s disease. 2022

  17. Liu S, Masurkar AV, Rusinek H, Chen J, Zhang B, Zhu W, Fernandez-Granda C, Razavian N. Generalizable deep learning model for early Alzheimer’s disease detection from structural MRIs. Sci Rep. 2022;12(1):17106.

    Article  Google Scholar 

  18. Noella RN, Priyadarshini J. Diagnosis of Alzheimer’s, Parkinson’s disease and frontotemporal dementia using a generative adversarial deep convolutional neural network. Neural Comput Appl. 2023;35(3):2845–54.

    Article  MATH  Google Scholar 

  19. Yadav BK, Hashmi MF .: An attention-based CNN architecture for Alzheimer’s classification and detection. In: 2023 IEEE IAS Global Conference on Emerging Technologies (GlobConET), IEEE, 2023; 1–5

  20. Shi B, Chen Y, Zhang P, Smith CD, Liu J, Initiative ADN, et al. Nonlinear feature transformation and deep fusion for Alzheimer’s disease staging analysis. Pattern Recogn. 2017;63:487–98.

    Article  Google Scholar 

  21. Venugopalan J, Tong L, Hassanzadeh HR, Wang MD. Multimodal deep learning models for early detection of Alzheimer’s disease stage. Sci Rep. 2021;11(1):3254.

    Article  MATH  Google Scholar 

  22. Allioui H, Sadgal M, Elfazziki A. Deep MRI segmentation: a convolutional method applied to Alzheimer disease detection. Int. J. Adv. Comput. Sci. Appl. 10;11

  23. Mishra D, Singh SK, Singh RK. Lossy medical image compression using residual learning-based dual autoencoder model. In: IEEE 7th Uttar Pradesh section international conference on electrical, electronics and computer engineering (UPCON). IEEE. 2020;2020:1–5.

  24. Farshad A, Yeganeh Y, Gehlbach P, Navab N. Y-net: a spatiospectral dual-encoder network for medical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, 2022;582–592

  25. Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017;1125–1134

  26. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks 2015. arXiv preprint arXiv:1511.06434

  27. Mehmood A, Maqsood M, Bashir M, Shuyuan Y. A deep Siamese convolution neural network for multi-class classification of Alzheimer disease. Brain Sci. 2020;10(2):84.

    Article  Google Scholar 

  28. Cui R, Liu M, Initiative ADN, et al. RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Comput Med Imaging Graph. 2019;73:1–10.

    Article  MATH  Google Scholar 

  29. Hong X, Lin R, Yang C, Zeng N, Cai C, Gou J, Yang J. Predicting Alzheimer’s disease using LSTM. Ieee Access. 2019;7:80893–901.

    Article  Google Scholar 

  30. Faturrahman M, Wasito I, Hanifah N, Mufidah R. Structural mri classification for alzheimer’s disease detection using deep belief network. In: 11th International conference on information & communication technology and system (ICTS). IEEE. 2017;2017:37–42.

  31. Yagis E, Atnafu SW, Herrera A, Marzi C, Scheda R, Giannelli M, Tessa C, Citi L, Diciotti S. Effect of data leakage in brain MRI classification using 2d convolutional neural networks. Sci Rep. 2021;11(1):22544.

    Article  Google Scholar 

  32. Biswas R, Gini RJ. Multi-class classification of Alzheimer’s disease detection from 3d MRI image using ml techniques and its performance analysis. Multimed Tool Appl. 2024;83(11):33527–54.

    Article  MATH  Google Scholar 

  33. Ávila-Jiménez JL, et al. A deep learning model for Alzheimer’s disease diagnosis based on patient clinical records. Comput Biol Med. 2024;169.

  34. Mercaldo F, et al. Triad: a deep ensemble network for Alzheimer classification and localisation. IEEE Access. 2023

  35. Shin H, et al. Vision transformer approach for classification of Alzheimer’s disease using 18F-Florbetaben brain images. Appl Sci. 2023;13(6):3453.

    Article  MATH  Google Scholar 

  36. Lee M-W, et al. A multimodal machine learning model for predicting dementia conversion in Alzheimer’s disease. Sci Rep. 2024;14(1):12276.

    Article  MATH  Google Scholar 

  37. Nguyen H, Dang HB, Dao PN. On-policy and off-policy q-learning strategies for spacecraft systems: an approach for time-varying discrete-time without controllability assumption of augmented system. Aerosp Sci Technol. 2024;146.

  38. Wang J, Liu J, Zheng Y, Zhang D. Data-based l2 gain optimal control for discrete-time system with unknown dynamics. J Franklin Inst. 2023;360(6):4354–77.

    Article  MathSciNet  MATH  Google Scholar 

  39. Raj A, Mirzaei G. Reinforcement-learning-based localization of hippocampus for Alzheimer’s disease detection. Diagnostics. 2023;13(21):3292.

    Article  MATH  Google Scholar 

  40. Dhanusha C, Kumar AS. Deep recurrent q reinforcement learning model to predict the Alzheimer disease using smart home sensor data. In: IOP Conference series: materials science and engineering. IOP Publishing, 2021

  41. Schwartz E, Karlinsky L, Shtok J, Harary S, Marder M, Kumar A, Feris R, Giryes R, Bronstein A. \(\delta \)-encoder: an effective sample synthesis method for few-shot object recognition. In: Annual conference on neural information processing systems. Neural Inform. Process. Syst. Found. 2018

  42. Gao H, Shou Z, Zareian A, Zhang H, Chang S-F. Low-shot learning via covariance-preserving adversarial augmentation networks. Adv. Neural Inform. Process. Syst. 2018;31

  43. Ravi S, Larochelle H. Optimization as a model for few-shot learning. In: International conference on learning representations. 2017

  44. He K, Zhang X, Ren S, Sun J .: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016;770–778

  45. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, Springer, 2015;234–241

  46. Qian L, Zhou X, Li Y, Hu Z. Unet#: a unet-like redesigning skip connections for medical image segmentation 2022. arXiv preprint arXiv:2205.11759

  47. Liu Y, Wang H, Chen Z, Huangliang K, Zhang H. Transunet+: redesigning the skip connection to enhance features in medical image segmentation. Knowl-Based Syst. 2022;256.

  48. Sokolova M, Japkowicz N, Szpakowicz S. Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation. In: Australasian joint conference on artificial intelligence, Springer, 2006;1015–1021

Download references

Acknowledgements

This work was partially supported by the National Key Research and Development Program of China under Grant No. 2018AAA0100400, HY Project under Grant No. LZY2022033004, the Natural Science Foundation of Shandong Province under Grants No. ZR2020MF131 and No. ZR2021ZD19, the Science and Technology Program of Qingdao under Grant No. 21-1-4-ny-19-nsh, and Project of Associative Training of Ocean University of China under Grant No. 202265007.

Author information

Authors and Affiliations

  1. Computer Science, University of Kasdi Merbah, 30000, Ouargla, Algeria

    Othmane Bazine, Omar Rai, Oussama Aiadi & Belal Khaldi

  2. Artificial Intelligence and Information Technology Laboratory (LINATI), 30000, Ouargla, Algeria

    Oussama Aiadi & Belal Khaldi

  3. Computer Science Department, Bishop’s University, Sherbrooke, Quebec, Canada

    Rachid Hedjam

  4. Computer Science and Technology, Ocean University of China, 238 Songling Road, Qingdao, 266100, Laoshan District, China

    Guoqiang Zhong

Authors
  1. Othmane Bazine
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Omar Rai
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Oussama Aiadi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Rachid Hedjam
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Belal Khaldi
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Guoqiang Zhong
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

A.B. wrote the main manuscript C.D. E: supervise the work and revise the manuscript and F. manuscript Reviewing. All authors reviewed the manuscript.

Corresponding author

Correspondence to Oussama Aiadi.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bazine, O., Rai, O., Aiadi, O. et al. Convolutional Cross-Modal Autoencoder-Based Few-Shot Learning for Data Augmentation with Application to Alzheimer Dementia Diagnosis. Cogn Comput 17, 34 (2025). https://doi.org/10.1007/s12559-024-10390-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10390-1

Keywords