Skip to main content
Springer Nature Link
Log in

Real-Time Traffic Sign Detection Based on Weighted Attention and Model Refinement

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

With the rapid development of technology and artificial intelligence in today’s era, people are increasingly inclined to the technology of automatic driving, Real-time detection of traffic signs while driving is required for safety reasons. In this work, we improve the current state-of-the-art detection algorithm YOLOv7 as a basis and propose a lightweight traffic sign detection algorithm. First, we highlight the weights of salient features through a weighted attention module to make the network more focused on the target features. Then, we propose a lightweight convolution-based spatial pyramidal pooling fusion module (DW-SPPF) and replace the conventional convolution of the detection head with a lightweight depth-separable convolution to reduce the complexity of the model while maintaining detection accuracy. Finally, for the bounding box regression, we consider the directional relationship between the groundtruth and predicted boxes and improve the regression loss function to enhance the convergence speed of the loss. The experimental results show that the accuracy of the model in this paper is improved by 6.7% compared to the base network, and the detection speed is accelerated by about 51 frames/s through model refinement, which basically satisfies the real-time accurate detection of traffic signs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Cheng P, Liu W, Zhang Y, Ma H (2018) Loco: local context based faster R-CNN for small traffic sign detection. In: International conference on multimedia modeling. Springer, Berlin, pp 329–341

    Chapter  Google Scholar 

  2. Yang Y, Luo H, Huarong X, Fuchao W (2015) Towards real-time traffic sign detection and classification. IEEE Trans Intell Transp Syst 17(7):2022–2031

    Article  Google Scholar 

  3. Wali SB, Abdullah MA, Hannan MA, Hussain A, Samad SA, Ker PJ, Mansor MB (2019) Vision-based traffic sign detection and recognition systems: current trends and challenges. Sensors 19(9):2093

    Article  Google Scholar 

  4. Kamal KC, Yin Z, Mingyang W, Zhilu W (2019) Depthwise separable convolution architectures for plant disease classification. Comput Electron Agric 165:104948

    Article  Google Scholar 

  5. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  6. Yue J, Mao S, Li M (2016) A deep learning framework for hyperspectral image classification using spatial pyramid pooling. Remote Sens Lett 7(9):875–884

    Article  Google Scholar 

  7. Huang Z, Wang J, Xuesong F, Tao Yu, Guo Y, Wang R (2020) Dc-spp-yolo: dense connection and spatial pyramid pooling based yolo for object detection. Inf Sci 522:241–258

    Article  MathSciNet  Google Scholar 

  8. LeCun, Y et al (2015) Lenet-5, convolutional neural networks. http://yann.lecun.com/exdb/lenet 20(5):14

  9. Al-Qizwini M, Barjasteh I, Al-Qassab H, Radha H (2017) Deep learning algorithm for autonomous driving using googlenet. In: 2017 IEEE intelligent vehicles symposium (IV). IEEE, pp 89–96

  10. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  11. Pant G, Yadav DP, Gaur A (2020) Resnext convolution neural network topology-based deep learning model for identification and classification of pediastrum. Algal Res 48:101932

    Article  Google Scholar 

  12. Fu J, Zheng H, Mei T (2017) Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4438–4446

  13. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  14. Park J, Woo S, Lee J-Y, Kweon IS (2018) Bam: Bottleneck attention module. arXiv:1807.06514

  15. Yang L, Zhang R-Y, Li L, Xie X (2021) Simam: A simple, parameter-free attention module for convolutional neural networks. In: International conference on machine learning, pp 11863–11874. PMLR

  16. Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv:2207.02696

  17. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, 28

  18. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  19. Cai Z, Vasconcelos N (2018) Cascade R-CNN: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162

  20. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: Single shot multibox detector. European conference on computer vision. Springer, Berlin, pp 21–37

    Google Scholar 

  21. Jiang D, Sun B, Shaojing S, Zuo Z, Peng W, Tan X (2020) FASSD: a feature fusion and spatial attention-based single shot detector for small object detection. Electronics 9(9):1536

    Article  Google Scholar 

  22. Rosas-Arias L, Benitez-Garcia G, Portillo-Portillo J, Olivares-Mercado J, Sanchez-Perez G, Yanai K (2021) Fassd-net: fast and accurate real-time semantic segmentation for embedded systems. IEEE Trans Intell Transp Syst

  23. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  24. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  25. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv:1804.02767

  26. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934

  27. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8759–8768

  28. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  29. Gong Y, Yu X, Ding Y, Peng X, Zhao J, Han Z (2021) Effective fusion factor in FPN for tiny object detection. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1160–1168

  30. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and \({<}0.5\) mb model size. arXiv:1602.07360

  31. Gholami A, Kwon K, Wu B, Tai Z, Yue X, Jin P, Zhao S, Keutzer K (2018) Squeezenext: Hardware-aware neural network design. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1638–1647

  32. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6848–6856

  33. Sinha D, El-Sharkawy M (2019) Thin mobilenet: An enhanced mobilenet architecture. In: 2019 IEEE 10th annual ubiquitous computing, electronics & mobile communication conference (UEMCON), pp 0280–0285. IEEE

  34. Biswas D, Hongbo S, Wang C, Stevanovic A, Wang W (2019) An automatic traffic density estimation using single shot detection (SSD) and mobilenet-SSD. Phys Chem Earth Parts A/B/C 110:176–184

    Article  Google Scholar 

  35. Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020) Ghostnet: more features from cheap operations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1580–1589

  36. Wang C-Y, Liao H-YM, Wu Y-H, Chen P-Y, Hsieh J-W, Yeh I-H (2020) Cspnet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 390–391

  37. Jiang P, Ergu D, Liu F, Cai Y, Ma B (2022) A review of yolo algorithm developments. Procedia Comput Sci 199:1066–1073

    Article  Google Scholar 

  38. Lan W, Dang J, Wang Y, Wang S (2018) Pedestrian detection based on yolo network model. In: 2018 IEEE international conference on mechatronics and automation (ICMA). IEEE, pp 1547–1551

  39. Cui L, Ma R, Lv P, Jiang X, Gao Z, Zhou B, Xu M (2018) MDSSD: multi-scale deconvolutional single shot detector for small objects. arXiv:1805.07009

  40. Zhou D, Fang J, Song X, Guan C, Yin J, Dai Y, Yang R (2019) IOU loss for 2D/3D object detection. In: 2019 international conference on 3D vision (3DV). IEEE, pp 85–94

  41. Zheng Z, Wang P, Liu W, Li J, Ye R, Ren D (2020) Distance-IOU loss: faster and better learning for bounding box regression. Proceedings of the AAAI conference on artificial intelligence 34:12993–13000

    Article  Google Scholar 

  42. Del Signore A, Jan HA, Rob LHJ, Leuven Rob SEW, Breure AM (2016) Development and application of the SSD approach in scientific case studies for ecological risk assessment. Environ Toxicol Chem 35(9):2149–2161

    Article  Google Scholar 

  43. Tian Y, Yang G, Wang Z, Wang H, Li E, Liang Z (2019) Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput Electron Agric 157:417–426

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Key R &D Plan and Jiangsu Key Laboratory of Big Data Analysis Technology, and Nanjing University of Information Science and Technology supported by the National Natural Science Joint Fund Key Program (U21B2027).

Author information

Authors and Affiliations

  1. College of Automation, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Zihao Jia & Shengkun Sun

  2. College of Automation, Southeast University, Nanjing, 210096, China

    Guangcan Liu

Authors
  1. Zihao Jia
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shengkun Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Guangcan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

ZJ mainly wrote manuscript, SS helped collect references, and all the authors have reviewed the articles.

Corresponding author

Correspondence to Zihao Jia.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jia, Z., Sun, S. & Liu, G. Real-Time Traffic Sign Detection Based on Weighted Attention and Model Refinement. Neural Process Lett 55, 7511–7527 (2023). https://doi.org/10.1007/s11063-023-11271-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11271-8

Keywords