Skip to main content
Springer Nature Link
Log in

Automatic Lumbar Spine Tracking Based on Siamese Convolutional Network

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Deep learning has demonstrated great success in various computer vision tasks. However, tracking the lumbar spine by digitalized video fluoroscopic imaging (DVFI), which can quantitatively analyze the motion mode of the spine to diagnose lumbar instability, has not yet been well developed due to the lack of steady and robust tracking method. The aim of this work is to automatically track lumbar vertebras with rotated bounding boxes in DVFI sequences. Instead of distinguishing vertebras using annotated lumbar images or sequences, we train a full-convolutional siamese neural network offline to learn generic image features with transfer learning. The siamese network is trained to learn a similarity function that compares the labeled target from the initial frame with the candidate patches from the current frame. The similarity function returns a high score if the two images depict the same object. Once learned, the similarity function is used to track a previously unseen object without any adapting online. Our tracker is performed by evaluating the candidate rotated patches sampled around the previous target’s position and presents rotated bounding boxes to locate the lumbar spine from L1 to L4. Results indicate that the proposed tracking method can track the lumbar vertebra steadily and robustly. The study demonstrates that the lumbar tracker based on siamese convolutional network can be trained successfully without annotated lumbar sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Landi A, Gregori F, Marotta N, Donnarumma P, Delfini R: Hidden spondylolisthesis: unrecognized cause of low back pain? Prospective study about the use of dynamic projections in standing and recumbent position for the individuation of lumbar instability. Neuroradiology 57(6):583–588, 2015

    Article  Google Scholar 

  2. Ahn K, Jhun HJ: New physical examination tests for lumbar spondylolisthesis and instability: low midline sill sign and interspinous gap change during lumbar flexion-extension motion. BMC musculoskeletal disorders 16(1):97–103, 2015

    Article  Google Scholar 

  3. Patriarca L, Letteriello M, Di Cesare E, Barile A, Gallucci M, Splendiani A: Does evaluator experience have an impact on the diagnosis of lumbar spine instability in dynamic MRI Interobserver agreement study. Neuroradiol 28(3):341–346, 2015

    Article  Google Scholar 

  4. Sui F, Zhang D, Lam SCB, Zhao L, Wang D, Bi Z, Hu Y: Auto-tracking system for human lumbar motion analysis. Journal of X-ray Science and Technology 19(2):205–218, 2011

    Article  Google Scholar 

  5. Clarke MJ, Zadnik PL, Groves ML, Sciubba DM, Witham TF, Bydon A, Wolinsky JP: Fusion following lateral mass reconstruction in the cervical spine. Journal of Neurosurgery: Spine 22(2):139–150, 2015

    PubMed  Google Scholar 

  6. Kettler A, Rohlmann F, Ring C, Mack C, Wilke HJ: Do early stages of lumbar intervertebral disc degeneration really cause instability? Evaluation of an in vitro database. European Spine Journal 20(4):578–584, 2011

    Article  Google Scholar 

  7. Miyasaka K, Ohmori K, Suzuki K, Inoue H: Radiographic analysis of lumbar motion in relation to lumbosacral stability: investigation of moderate and maximum motion. SPINE 25(6):732–737, 2000

    Article  CAS  Google Scholar 

  8. Bertinetto L, Valmadre J, Henriques J F, et al. Fully-Convolutional Siamese Networks for Object Tracking. European Conference on Computer Vision – ECCV2016, 2016:850–865.

  9. Kumar VP, Thomas T: Automatic estimation of orientation and position of spine in digitized X-rays using mathematical morphology. Journal of Digital Imaging 18(3):234–241, 2005

    Article  Google Scholar 

  10. Benjelloun M, Mahmoudi S: Spine localization in X-ray images using interest point detection. Journal of Digital Imaging 22(3):309–318, 2009

    Article  Google Scholar 

  11. Liu Y, Sui X, Sun Y, Liu C, Hu Y: Siamese convolutional networks for tracking the spine motion. In Applications of Digital Image Processing XL. International Society for Optics and Photonics 10396(103961Y), 2017

  12. Zhou Y, Liu Y, Chen Q, Gu G, and Sui X. Automatic Lumbar MRI Detection and Identification Based on Deep Learning. Journal of digital imaging, 32:513, 2019, 520

  13. SMMR AA, Knapp K, Slabaugh G: Fully automatic cervical vertebrae segmentation framework for X-ray images. Computer Methods & Programs in Biomedicine 157:95–111, 2018

    Article  Google Scholar 

  14. Wang N, Yeung DY: Learning a deep compact image representation for visual tracking. In Advances in Neural Information Processing Systems:809–817, 2013

  15. Gao J, Ling H, Hu W, and Xing J. Transfer learning based visual tracking with gaussian processes regression. In ECCV. 188–203. (2014).

  16. Liu Y, Sui X, Kuang X, Liu C, Gu G, Chen Q: Object Tracking Based on Vector Convolutional Network and Discriminant Correlation Filters. Sensors 19(8):2019, 1818

    Google Scholar 

  17. Irshad M, Muhammad N, Sharif M, Yasmeen M: Automatic segmentation of the left ventricle in a cardiac MR short axis image using blind morphological operation. The European Physical Journal Plus 133(4):148, 2018

    Article  Google Scholar 

  18. Kim K, Lee S: Vertebrae localization in CT using both local and global symmetry features. Comput Med Imaging Graph 58:45–55, 2017

    Article  Google Scholar 

  19. Han Z, Wei B, Leung S et al.: Automated Pathogenesis-Based Diagnosis of Lumbar Neural Foraminal Stenosis via Deep Multiscale Multitask Learning. Neuro informatics 1:1–13, 2018

    Google Scholar 

  20. Wang J, Fang Z, Lang N, Yuan H, Su MY, Baldi P: A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks. Computers in Biology & Medicine 84(C):137–146, 2017

    Article  Google Scholar 

  21. Forsberg D, Sjöblom E, Sunshine JL: Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data. Journal of Digital Imaging 30(4):1–7, 2017

    Article  Google Scholar 

  22. Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems. Curran Associates Inc 60(2):1097–1105, 2012

    Google Scholar 

  23. Girshick R, Donahue J, Darrell T and Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, (2014).

    Book  Google Scholar 

  24. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A C, Fei-Fei L. ImageNet Large Scale Visual Recognition Challenge. IJCV (2015).

  25. Oktay AB, Akgul YS: Simultaneous Localization of Lumbar Vertebrae and Intervertebral Discs With SVM-Based MRF. IEEE Trans Biomed Eng 60(9):2375–2383, 2013

    Article  Google Scholar 

  26. Vedaldi, A. and Lenc, K., “Matconvnet: Convolutional neural networks for matlab,” Proc. ACM International Conference on Multimedia, (2015).

Download references

Acknowledgments

This research was financially supported by National Natural Science Foundation of China, (grant number 11503010, 11773018), the Fundamental Research Funds for the Central Universities, (grant number 30916015103), and Qing Lan Project and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligence Sense (3091601410405).

Author information

Authors and Affiliations

  1. School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Yuan Liu, Xiubao Sui, Chengwei Liu & Xiaodong Kuang

  2. Department of Orthopaedics and Traumatology, The University of Hong Kong, Pokfulam, Hong Kong, SAR, China

    Yong Hu

Authors
  1. Yuan Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiubao Sui
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chengwei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaodong Kuang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yong Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiubao Sui.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Sui, X., Liu, C. et al. Automatic Lumbar Spine Tracking Based on Siamese Convolutional Network. J Digit Imaging 33, 423–430 (2020). https://doi.org/10.1007/s10278-019-00273-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-019-00273-5

Keywords