Skip to main content

Exploring a Transformer Approach for Pigment Signs Segmentation in Fundus Images

  • Conference paper
  • First Online:
Image Analysis and Processing. ICIAP 2022 Workshops (ICIAP 2022)

Abstract

Over the past couple of years, Transformers became increasingly popular within the deep learning community. Initially designed for Natural Language Processing tasks, Transformers were then tailored to fit to the Image Analysis field. The self-attention mechanism behind Transformers immediately appeared a promising, although computationally expensive, learning approach. However, Transformers do not adapt as well to tasks involving large images or small datasets. This propelled the exploration of hybrid CNN-Transformer models, which seemed to overcome those limitations, thus sparkling an increasing interest also in the field of medical imaging. Here, a hybrid approach is investigated for Pigment Signs (PS) segmentation in Fundus Images of patients suffering from Retinitis Pigmentosa, an eye disorder eventually leading to complete blindness. PS segmentation is a challenging task due to the high variability of their size, shape and colors and to the difficulty to distinguish between PS and blood vessels, which often overlap and display similar colors. To address those issues, we use the Group Transformer U-Net, a hybrid CNN-Transformer. We investigate the effects, on the learning process, of using different losses and choosing an appropriate parameter tuning. We compare the obtained performances with the classical U-Net architecture. Interestingly, although the results show margins for a consistent improvement, they do not suggest a clear superiority of the hybrid architecture. This evidence raises several questions, that we address here but also deserve to be further investigated, on how and when Transformers are really the best choice to address medical imaging tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adamw optimizer and cosine learning rate annealing with restarts. https://github.com/mpyrozhok/adamwr

  2. Berman, M., Triki, A.R., Blaschko, M.B.: The Lovasz-Softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, pp. 4413–4421. IEEE, June 2018

    Google Scholar 

  3. Brancati, N., Frucci, M., Riccio, D., Di Perna, L., Simonelli, F.: Segmentation of pigment signs in fundus images for retinitis pigmentosa analysis by using deep learning. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11752, pp. 437–445. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30645-8_40

    Chapter  Google Scholar 

  4. Dosovitskiy, A., et al.: An image is worth \(16\times 16\) words: transformers for image recognition at scale. arXiv:2010.11929 [cs], June 2021

  5. Han, K., et al.: A survey on vision transformer. arXiv:2012.12556 [cs], August 2021

  6. Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., Shah, M.: Transformers in vision: a survey. ACM Comput. Surv. (CSUR) (2021)

    Google Scholar 

  7. Li, Y., et al.: GT U-Net: a U-Net like group transformer network for tooth root segmentation. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds.) MLMI 2021. LNCS, vol. 12966, pp. 386–395. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87589-3_40

    Chapter  Google Scholar 

  8. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  9. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A ConvNet for the 2020s. arXiv:2201.03545 [cs], March 2022

  10. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv:1711.05101 [cs, math], January 2019

  11. Ma, J., et al.: Loss odyssey in medical image segmentation. Med. Image Anal. 71, 102035 (2021)

    Article  Google Scholar 

  12. Park, N., Kim, S.: How do vision transformers work? arXiv:2202.06709 [cs], February 2022

  13. PyTorch learning rate finder. https://github.com/davidtvs/pytorch-lr-finder

  14. The RIPS dataset. https://www.icar.cnr.it/sites-rips-datasetrips/

  15. Sangiovanni, M., Brancati, N., Frucci, M., Di Perna, L., Simonelli, F., Riccio, D.: Segmentation of pigment signs in fundus images with a hybrid approach: a case study. Pattern Recogn. Image Anal. 32(2), 312–321 (2022)

    Article  Google Scholar 

  16. Smith, L.N.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 464–472. IEEE (2017)

    Google Scholar 

  17. Smith, L.N.: A disciplined approach to neural network hyper-parameters: part 1 - learning rate, batch size, momentum, and weight decay. arXiv:1803.09820 [cs, stat], April 2018

  18. Srinivas, A., Lin, T.Y., Parmar, N., Shlens, J., Abbeel, P., Vaswani, A.: Bottleneck transformers for visual recognition. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 16514–16524. IEEE, June 2021

    Google Scholar 

  19. Strudel, R., Garcia, R., Laptev, I., Schmid, C.: Segmenter: transformer for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7262–7272 (2021)

    Google Scholar 

  20. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mara Sangiovanni .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sangiovanni, M., Frucci, M., Riccio, D., Di Perna, L., Simonelli, F., Brancati, N. (2022). Exploring a Transformer Approach for Pigment Signs Segmentation in Fundus Images. In: Mazzeo, P.L., Frontoni, E., Sclaroff, S., Distante, C. (eds) Image Analysis and Processing. ICIAP 2022 Workshops. ICIAP 2022. Lecture Notes in Computer Science, vol 13374. Springer, Cham. https://doi.org/10.1007/978-3-031-13324-4_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13324-4_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13323-7

  • Online ISBN: 978-3-031-13324-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics