Skip to main content
Springer Nature Link
Log in

A Cross-scale Attention-Based U-Net for Breast Ultrasound Image Segmentation

  • Original Paper
  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Breast cancer remains a significant global health concern and is a leading cause of mortality among women. The accuracy of breast cancer diagnosis can be greatly improved with the assistance of automatic segmentation of breast ultrasound images. Research has demonstrated the effectiveness of convolutional neural networks (CNNs) and transformers in segmenting these images. Some studies combine transformers and CNNs, using the transformer’s ability to exploit long-distance dependencies to address the limitations inherent in convolutional neural networks. Many of these studies face limitations due to the forced integration of transformer blocks into CNN architectures. This approach often leads to inconsistencies in the feature extraction process, ultimately resulting in suboptimal performance for the complex task of medical image segmentation. This paper presents CSAU-Net, a cross-scale attention-guided U-Net, which is a combined CNN-transformer structure that leverages the local detail depiction of CNNs and the ability of transformers to handle long-distance dependencies. To integrate global context data, we propose a cross-scale cross-attention transformer block that is embedded within the skip connections of the U-shaped architectural network. To further enhance the effectiveness of the segmentation process, we incorporated a gated dilated convolution (GDC) module and a lightweight channel self-attention transformer (LCAT) on the encoder side. Extensive experiments conducted on three open-source datasets demonstrate that our CSAU-Net surpasses state-of-the-art techniques in segmenting ultrasound breast lesions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The data supporting this study can be obtained from the corresponding author [Liu] upon reasonable request.

References

  1. Giaquinto, A.N., et al., Breast Cancer Statistics, 2022. 2022. 72(6): p. 524–541.

  2. Cheng, H.-D., et al., Automated breast cancer detection and classification using ultrasound images: A survey. 2010. 43(1): p. 299-317.

  3. Xian, M., et al., Automatic breast ultrasound image segmentation: A survey. 2018. 79: p. 340-355.

  4. Xue, C., et al., Global guidance network for breast lesion segmentation in ultrasound images. 2021. 70: p. 101989.

  5. Shareef, B., et al. Estan: Enhanced small tumor-aware network for breast ultrasound image segmentation. in Healthcare. 2022. MDPI.

  6. Hu, Y., et al., Automatic tumor segmentation in breast ultrasound images using a dilated fully convolutional network combined with an active contour model. 2019. 46(1): p. 215-228.

  7. Yap, M.H., et al., Automated breast ultrasound lesions detection using convolutional neural networks. 2017. 22(4): p. 1218-1226.

  8. Ronneberger, O., P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18. 2015. Springer.

  9. Zhou, Z., et al., Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. 2019. 39(6): p. 1856–1867.

  10. Oktay, O., et al., Attention u-net: Learning where to look for the pancreas. 2018.

  11. Lin, T.-Y., et al. Feature pyramid networks for object detection. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

  12. Li, C., et al., Weakly supervised mitosis detection in breast histopathology images using concentric loss. 2019. 53: p. 165-178.

  13. Liu, x, et al., Transformer and convolutional based dual branch network for retinal vessel segmentation in OCTA images, Biomedical Signal Processing and Control, Volume 83, 104604, May 2023,

  14. Chen, G., et al., C-Net: Cascaded convolutional neural network with global guidance and refinement residuals for breast ultrasound images segmentation. 2022. 225: p. 107086.

  15. Irfan, R., et al., Dilated semantic segmentation for breast ultrasonic lesion detection using parallel feature fusion. 2021. 11(7): p. 1212.

  16. Vaswani, A., et al., Attention is all you need. 2017. 30.

  17. Dosovitskiy, A., et al., An image is worth 16x16 words: Transformers for image recognition at scale. 2020.

  18. Petit, O., et al. U-net transformer: Self and cross attention for medical image segmentation. in Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12. 2021. Springer.

  19. Chen, J., et al., Transunet: Transformers make strong encoders for medical image segmentation. 2021.

  20. Gao, Y., M. Zhou, and D.N. Metaxas. UTNet: a hybrid transformer architecture for medical image segmentation. in Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24. 2021. Springer.

  21. Moraru, L., et al., Optimization of breast lesion segmentation in texture feature space approach. 2014. 36(1): p. 129-135.

  22. Lotfollahi, M., et al., Segmentation of breast ultrasound images based on active contours using neutrosophic theory. 2018. 45: p. 205-212.

  23. Nugroho, A., H.A. Nugroho, and L. Choridah. Active contour bilateral filter for breast lesions segmentation on ultrasound images. in 2015 International Conference on Science in Information Technology (ICSITech). 2015. IEEE.

  24. Jumaat, A.K., et al., Segmentation of masses from breast ultrasound images using parametric active contour algorithm. 2010. 8: p. 640-647.

  25. Gómez, W., et al. Active contours without edges applied to breast lesions on ultrasound. in XII Mediterranean Conference on Medical and Biological Engineering and Computing 2010: May 27–30, 2010 Chalkidiki, Greece. 2010. Springer.

  26. Huang, Y.-L., et al., Level set contouring for breast tumor in sonography. 2007. 20: p. 238-247.

  27. Liu, B., et al., Probability density difference-based active contour for ultrasound image segmentation. 2010. 43(6): p. 2028-2042.

  28. Gao, L., X. Liu, and W.J.j.o.a.M. Chen, Phase-and GVF-based level set segmentation of ultrasonic breast tumors. 2012. 2012.

  29. Zhu, L., et al. A second-order subregion pooling network for breast lesion segmentation in ultrasound. in Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23. 2020. Springer.

  30. Mo, Y., et al., Hover-trans: Anatomy-aware hover-transformer for roi-free breast cancer diagnosis in ultrasound images. 2023.

  31. Cao, H., et al. Swin-unet: Unet-like pure transformer for medical image segmentation. in European conference on computer vision. 2022. Springer.

  32. Zhou, H.-Y., et al., nnformer: Interleaved transformer for volumetric segmentation. 2021.

  33. Abhisheka, B., S.K. Biswas, and B.J.A.o.C.M.i.E. Purkayastha, A comprehensive review on breast cancer detection, classification and segmentation using deep learning. 2023. 30(8): p. 5023–5052.

  34. Radak, M., et al., Machine learning and deep learning techniques for breast cancer diagnosis and classification: a comprehensive review of medical imaging studies. 2023. 149(12): p. 10473-10491.

  35. Ramesh, S., et al., Segmentation and classification of breast cancer using novel deep learning architecture. 2022. 34(19): p. 16533-16545.

  36. Rezaei, Z.J.E.S.w.A., A review on image-based approaches for breast cancer detection, segmentation, and classification. 2021. 182: p. 115204.

  37. Yusoff, M., et al., Accuracy analysis of deep learning methods in breast cancer classification: A structured review. 2023. 13(4): p. 683.

  38. Jiang, T., et al., A hybrid enhanced attention transformer network for medical ultrasound image segmentation. 2023. 86: p. 105329.

  39. Chen, B., et al., Transattunet: Multi-level attention-guided u-net with transformer for medical image segmentation. 2023.

  40. Lin, X., et al., The lighter the better: rethinking transformers in medical image segmentation through adaptive pruning. 2023.

  41. Hu, J., L. Shen, and G. Sun. Squeeze-and-excitation networks. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

  42. Yeung, M., et al., Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. 2022. 95: p. 102026.

  43. Al-Dhabyani, W., et al., Dataset of breast ultrasound images. 2020. 28: p. 104863.

  44. Huang, Q., et al., Segmentation of breast ultrasound image with semantic classification of superpixels. 2020. 61: p. 101657.

  45. Zhuang, Z., et al., An RDAU-NET model for lesion segmentation in breast ultrasound images. 2019. 14(8): p. e0221535.

  46. Chen, L.-C., et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. in Proceedings of the European conference on computer vision (ECCV). 2018.

  47. Byra, M., et al., Breast mass segmentation in ultrasound with selective kernel U-Net convolutional neural network. 2020. 61: p. 102027.

  48. Yan, Y., et al., Accurate segmentation of breast tumors using AE U-net with HDC model in ultrasound images. 2022. 72: p. 103299.

  49. Lee, D.-H. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. in Workshop on challenges in representation learning, ICML. 2013. Atlanta.

  50. Yang, L., et al. St++: Make self-training work better for semi-supervised semantic segmentation. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

  51. Liu, X., Liu, J., Xu, X. et al. A robust detail preserving anisotropic diffusion for speckle reduction in ultrasound images. BMC Genomics 12 (Suppl 5), S14 (2011).

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430081, China

    Teng Wang & Jun Liu

  2. China & Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan, 430065, China

    Teng Wang & Jun Liu

  3. Health Informatics, College of Public Health, George Mason University, Fairfax, VA, 22030, USA

    Jinshan Tang

Authors
  1. Teng Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jun Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jinshan Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Teng Wang. The first draft of the manuscript was written by Teng Wang, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Jun Liu or Jinshan Tang.

Ethics declarations

Ethics Approval

N/A.

Consent to Participate

N/A.

Consent for Publication

N/A.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, T., Liu, J. & Tang, J. A Cross-scale Attention-Based U-Net for Breast Ultrasound Image Segmentation. J Digit Imaging. Inform. med. (2025). https://doi.org/10.1007/s10278-025-01392-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-025-01392-y

Keywords