Skip to main content
Springer Nature Link
Log in

Classification and quantification of glomerular spike-like projections via deep residual multiple instance learning with multi-scale annotation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The primary pathological feature of Membranous Nephropathy (MN) is the presence of tiny spike-like projections on the glomerular basement membrane. Early detection and efficient treatment of spike-like projections are essential in halting disease progression. Renal pathology biopsy stands as the gold standard for diagnosing MN, and the accurate identification of glomerular spike-like projections plays a vital role in aiding diagnosis. Nevertheless, the tiny spike-like projection lesions and constraints in data quantity pose considerable challenges for supervised learning-based glomerular classification and quantification. We develop a Multi-Scale Annotation-based Multiple Instance Learning (MSA-MIL) model to address the issues. MSA-MIL utilizes image labels and box-level labels to jointly enhance the classification performance of the MIL model. Specifically, we first employ U-Net for glomerular image edge segmentation and subsequently train the MIL model on the dataset with image-level labels. Then, to overcome the limitations arising from the scarcity of positive instances and the relatively small size of spike-like projection features, we manually augment the number of instances with spike-like projections via using box-level annotation to further enhance the MIL model's classification performance. The designed MSA-MIL model enables the classification, visualization, and quantitative analysis of glomeruli with spike-like projections in renal pathology images. We validated and evaluated the designed MSA-MIL model. The model performed exceptionally well, achieving a high accuracy of 0.9847 and demonstrating a high recall rate, effectively preventing misdiagnosis. Additionally, we utilized heatmaps to visualize the locations of spike-like projections within glomeruli, enhancing the model's interpretability. Furthermore, through an analysis of the correlation between the stages of membranous nephropathy and the proportion of spike-like projections, we observed that as the disease advances, the proportion of spike-like projections increases. This finding serves to further validate the results obtained by the model. The MSA-MIL model is the first one specifically designed for classifying glomerular spike-like projections. It not only enhances classification performance but also proves to be more suitable for categorizing minute lesions compared to conventional Convolutional Neural Network (CNN) models. The visualization of glomerular lesions and the proportion of spike-like projections provide doctors with insights into the model's inference process, offering intuitive assistance for accurate diagnoses. This model brings significant hope for advancing research and diagnosis in the field of kidney diseases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

References

  1. Xu X, Wang G, Chen N et al (2016) Long-Term Exposure to Air Pollution and Increased Risk of Membranous Nephropathy in China. J Am Soc Nephrol 27:3739. https://doi.org/10.1681/ASN.2016010093

    Article  Google Scholar 

  2. Jiang Z, Cai M, Dong B et al (2018) Clinicopathological features of atypical membranous nephropathy with unknown etiology in adult Chinese patients. Medicine (Baltimore) 97:e11608. https://doi.org/10.1097/MD.0000000000011608

    Article  Google Scholar 

  3. Scheidat S, Stahl RAK (2003) Membranous glomerulonephritis. Internist (Berl) 44:1120–1130. https://doi.org/10.1007/s00108-003-1022-5

    Article  Google Scholar 

  4. Nazareth TA, Kariburyo F, Kirkemo A et al (2019) Patients with Idiopathic Membranous Nephropathy: A Real-World Clinical and Economic Analysis of U.S. Claims Data. J Manag Care Spec Pharm 25. https://doi.org/10.18553/jmcp.2019.18456

  5. Couser WG (2017) Primary Membranous Nephropathy. Clin J Am Soc Nephrol 12:983–997. https://doi.org/10.2215/CJN.11761116

    Article  Google Scholar 

  6. Jiang Y, Chen L, Zhang H, Xiao X (2019) Breast cancer histopathological image classification using convolutional neural networks with small SE-ResNet module. PLoS ONE 14:e0214587. https://doi.org/10.1371/journal.pone.0214587

    Article  Google Scholar 

  7. Litjens G, Sánchez CI, Timofeeva N et al (2016) Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 6:26286. https://doi.org/10.1038/srep26286

    Article  Google Scholar 

  8. van der Laak J, Litjens G, Ciompi F (2021) Deep learning in histopathology: the path to the clinic. Nat Med 27:775–784. https://doi.org/10.1038/s41591-021-01343-4

    Article  Google Scholar 

  9. Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71. https://doi.org/10.1016/S0004-3702(96)00034-3

    Article  Google Scholar 

  10. Astorino A, Fuduli A, Gaudioso M (2019) A Lagrangian Relaxation Approach for Binary Multiple Instance Classification. IEEE Trans Neural Netw Learning Syst 30:2662–2671. https://doi.org/10.1109/TNNLS.2018.2885852

    Article  MathSciNet  Google Scholar 

  11. Bergeron C, Moore G, Zaretzki J et al (2012) Fast Bundle Algorithm for Multiple-Instance Learning. IEEE Trans Pattern Anal Mach Intell 34:1068–1079. https://doi.org/10.1109/TPAMI.2011.194

    Article  Google Scholar 

  12. Astorino A, Fuduli A, Veltri P, Vocaturo E (2020) Melanoma Detection by Means of Multiple Instance Learning. Interdiscip Sci Comput Life Sci 12:24–31. https://doi.org/10.1007/s12539-019-00341-y

    Article  Google Scholar 

  13. Andrews S, Tsochantaridis I, Hofmann T (2002) Support vector machines for multiple-instance learning. In: Proceedings of the 15th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, USA, pp 577–584

  14. Carbonneau M-A, Cheplygina V, Granger E, Gagnon G (2018) Multiple instance learning: a survey of problem characteristics and applications. Patt Recognit 77:329–353. https://doi.org/10.1016/j.patcog.2017.10.009

    Article  Google Scholar 

  15. Xu Y-Y (2016) Multiple-instance learning based decision neural networks for image retrieval and classification. Neurocomputing 171:826–836. https://doi.org/10.1016/j.neucom.2015.07.024

    Article  Google Scholar 

  16. Campanella G, Hanna MG, Geneslaw L et al (2019) Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med 25:1301–1309. https://doi.org/10.1038/s41591-019-0508-1

    Article  Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  18. Liu S, Deng W (2015) Very deep convolutional neural network based image classification using small training sample size. In: 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR). IEEE, Kuala Lumpur, Malaysia, pp 730–734

  19. Szegedy C, Wei Liu, Yangqing Jia et al (2015) Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Boston, MA, USA, pp 1–9

  20. He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Las Vegas, NV, USA, pp 770–778

  21. Aggarwal K, Mijwil MM, Sonia et al (2022) Has the Future Started? The Current Growth of Artificial Intelligence, Machine Learning, and Deep Learning. Iraqi J Comput Sci Math 3:115–123. https://doi.org/10.52866/ijcsm.2022.01.01.013

    Article  Google Scholar 

  22. Setio AAA, Ciompi F, Litjens G et al (2016) Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks. IEEE Trans Med Imaging 35:1160–1169. https://doi.org/10.1109/TMI.2016.2536809

    Article  Google Scholar 

  23. Yu Q, Shi Y, Sun J et al (2019) Crossbar-Net: A Novel Convolutional Neural Network for Kidney Tumor Segmentation in CT Images. IEEE Trans on Image Process 28:4060–4074. https://doi.org/10.1109/TIP.2019.2905537

    Article  MathSciNet  Google Scholar 

  24. Gulshan V, Peng L, Coram M et al (2016) Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316:2402. https://doi.org/10.1001/jama.2016.17216

    Article  Google Scholar 

  25. Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist–level classification of skin cancer with deep neural networks. Nature 542:115–118. https://doi.org/10.1038/nature21056

    Article  Google Scholar 

  26. Pedraza A, Gallego J, Lopez S et al (2017) Glomerulus Classification with Convolutional Neural Networks. In: Valdés Hernández M, González-Castro V (eds) Medical Image Understanding and Analysis. Springer International Publishing, Cham, pp 839–849

    Chapter  Google Scholar 

  27. Kawazoe Y, Shimamoto K, Yamaguchi R et al (2018) Faster R-CNN-Based Glomerular Detection in Multistained Human Whole Slide Images. J Imaging 4:91. https://doi.org/10.3390/jimaging4070091

    Article  Google Scholar 

  28. Liu R, Wang L, He J, Chen W (2019) Towards staining independent segmentation of glomerulus from histopathological images of kidney. bioRxiv 821181. https://doi.org/10.1101/821181

  29. Bueno G, Fernandez-Carrobles MM, Gonzalez-Lopez L, Deniz O (2020) Glomerulosclerosis identification in whole slide images using semantic segmentation. Comput Methods Programs Biomed 184:105273. https://doi.org/10.1016/j.cmpb.2019.105273

    Article  Google Scholar 

  30. Barros GO, Navarro B, Duarte A, dos-Santos WLC (2017) PathoSpotter-K: A computational tool for the automatic identification of glomerular lesions in histological images of kidneys. Sci Rep 7:46769. https://doi.org/10.1038/srep46769

    Article  Google Scholar 

  31. Marsh JN, Matlock MK, Kudose S et al (2018) Deep Learning Global Glomerulosclerosis in Transplant Kidney Frozen Sections. IEEE Trans Med Imaging 37:2718–2728. https://doi.org/10.1109/TMI.2018.2851150

    Article  Google Scholar 

  32. Kannan S, Morgan LA, Liang B et al (2019) Segmentation of Glomeruli Within Trichrome Images Using Deep Learning. Kidney Int Rep 4:955–962. https://doi.org/10.1016/j.ekir.2019.04.008

    Article  Google Scholar 

  33. Ginley B, Lutnick B, Jen K-Y et al (2019) Computational Segmentation and Classification of Diabetic Glomerulosclerosis. J Am Soc Nephrol 30:1953. https://doi.org/10.1681/ASN.2018121259

    Article  Google Scholar 

  34. Uchino E, Suzuki K, Sato N et al (2020) Classification of glomerular pathological findings using deep learning and nephrologist–AI collective intelligence approach. Int J Med Informatics 141:104231. https://doi.org/10.1016/j.ijmedinf.2020.104231

    Article  Google Scholar 

  35. Chagas P, Souza L, Araújo I et al (2020) Classification of glomerular hypercellularity using convolutional features and support vector machine. Artif Intell Med 103:101808. https://doi.org/10.1016/j.artmed.2020.101808

    Article  Google Scholar 

  36. Purwar S, Tripathi R, Barwad AW, Dinda AK (2020) Detection of Mesangial hypercellularity of MEST-C score in immunoglobulin A-nephropathy using deep convolutional neural network. Multimed Tools Appl 79:27683–27703. https://doi.org/10.1007/s11042-020-09304-8

    Article  Google Scholar 

  37. Maron O, Ratan AL (1998) Multiple-instance learning for natural scene classification. In: Proceedings of the Fifteenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp 341–349

  38. Yang C, Lozano-Perez T (2000) Image database retrieval with multiple-instance learning techniques. In: Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073). pp 233–243

  39. Xu Y, Mo T, Feng Q et al (2014) Deep learning of feature representation with multiple instance learning for medical image analysis. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp 1626–1630

  40. Hou L, Samaras D, Kurc TM, et al (2016) Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2424–2433. https://doi.org/10.1109/CVPR.2016.266

  41. Liu X, Li M, Hao F et al (2020) GLO-YOLO: A Dynamic Glomerular Detecting and Slicing Model in Whole Slide Images. In: Proceedings of the 2020 Conference on Artificial Intelligence and Healthcare. ACM, Taiyuan China, pp 229–233

  42. Zelinsky A (2009) Learning OpenCV—Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf]. IEEE Robot Autom Mag 16:100–100. https://doi.org/10.1109/MRA.2009.933612

    Article  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge all the authors from the original laboratories who submitted and shared data, on which this study is based.

Funding

This work was supported by the National Natural Science Foundation of China (Grant No. 11472184, No. 11771321, No. 61901292); the National Youth Science Foundation of China (Grant No.11401423); the Shanxi Province Plan Project on Science and Technology of Social Development (Grant No. 201703D321032); and the Natural Science Foundation of Shanxi Province, China (Grant No. 201901D211080).

Author information

Authors and Affiliations

  1. College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, 030024, People’s Republic of China

    Yilin Chen, Xueyu Liu, Fang Hao, Wen Zheng, Ming Li & Yongfei Wu

  2. Department of Nephrology, Shanxi Provincial People’s Hospital, Taiyuan, Shanxi, 030012, People’s Republic of China

    Xiaoshuang Zhou

  3. Department of Pathology, Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, 030001, People’s Republic of China

    Chen Wang

Authors
  1. Yilin Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xueyu Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Fang Hao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Wen Zheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xiaoshuang Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Ming Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Yongfei Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Chen Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Yongfei Wu and Ming Li conceptualized the study. Yilin Chen, Xueyu Liu, Fang Hao, Xiaoshuang Zhou, and Chen Wang curated the data. Yilin Chen and Xueyu Liu conducted data analysis. Yilin Chen drafted the manuscript. Yongfei Wu, Yilin Chen, Xueyu Liu, Fang Hao, Wen Zheng, Xiaoshuang Zhou, and Chen Wang reviewed and edited the manuscript. All authors read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Yongfei Wu or Chen Wang.

Ethics declarations

Conflicts of interests

We declare that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Liu, X., Hao, F. et al. Classification and quantification of glomerular spike-like projections via deep residual multiple instance learning with multi-scale annotation. Multimed Tools Appl 83, 76529–76549 (2024). https://doi.org/10.1007/s11042-024-18536-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-024-18536-x

Keywords