Skip to main content
Springer Nature Link
Log in

Guided Depth Enhancement via Anisotropic Diffusion

  • Conference paper
Advances in Multimedia Information Processing – PCM 2013 (PCM 2013)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8294))

Included in the following conference series:

  • 3498 Accesses

Abstract

In this paper, we propose to conduct inpainting and upsampling for defective depth maps when aligned color images are given. These tasks are referred to as the guided depth enhancement problem. We formulate the problem based on the heat diffusion framework. The pixels with known depth values are treated as the heat sources and the depth enhancement is performed via diffusing the depth from these sources to unknown regions. The diffusion conductivity is designed in terms of the guidance color image so that a linear anisotropic diffusion problem is formed. We further cast the steady state problem of this diffusion into the famous random walk model, by which the enhancement is achieved efficiently by solving a sparse linear system. The proposed algorithm is quantitatively evaluated on the Middlebury stereo dataset and is applied to inpaint Kinect data and upsample Lidar’s range data. Comparisons to the commonly used bilateral filter and Markov Random Field based methods are also presented, showing that our algorithm is competent.

This research work was supported in parts by the National Natural Science Foundation of China via grants 61001171, 61071219, 90820306, and Chinese Universities Scientific Fund.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Barash, D.: Bilateral Filtering and Anisotropic Diffusion: Towards a Unified Viewpoint, HP Laboratories Israel, HPL-2000-18(R.1) (2000)

    Google Scholar 

  2. Bertalmio, M., Sapiro, G.: Image Inpainting. SIGGRAPH (2000)

    Google Scholar 

  3. Chan, T., Shen, J.: Mathematical Models for Local Non-texture Inpaintings. UCLA CAM TR 00-11 (2001)

    Google Scholar 

  4. Clarenz, U., Diewald, U., Rumpf, M.: Anisotropic Geometric Diffusion in Surface Processing. In: IEEE Conference on Visualization (2000)

    Google Scholar 

  5. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13(9), 1–13 (2004)

    Article  Google Scholar 

  6. Diebel, J., Thrun, S.: An Application of Markov Random Fields to Range Sensing. In: NIPS (2005)

    Google Scholar 

  7. Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling Range Data in Dynamic Environments. In: CVPR (2010)

    Google Scholar 

  8. Grady, L.: Random Walks for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(11), 1–17 (2006)

    Article  Google Scholar 

  9. He, K., Sun, J., Tang, X.: Guided Image Filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  10. Janoch, A., Karayev, S., Jia, Y., Barron, J.T., Fritz, M., Saenko, K., Darrell, T.: A category-level 3-d object dataset: Putting the kinect to work. In: ICCV Workshops (2011)

    Google Scholar 

  11. Kim, G., Xing, E.P., Fei-Fei, L., Kanade, T.: Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion. In: ICCV (2011)

    Google Scholar 

  12. Kim, S.-Y., Cho, J.-H., Koschan, A., Abidi, M.A.: Spatial and temporal enhancement of depth images captured by a time-of-flight depth sensor. In: ICPR (2010)

    Google Scholar 

  13. Larsson, S., Thomee, V.: Partial Differential Equations with Numerical Methods. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  14. Liu, J., Gong, X., Liu, J.: Guided Inpainting and Filtering for Kinect Depth Maps. In: ICPR (2012)

    Google Scholar 

  15. Gong, X., Liu, J., Zhou, W., Liu, J.: Guided Depth Enhancement via A Fast Marching Method. Image and Vision Computing 31, 695–703 (2013)

    Article  Google Scholar 

  16. Microsoft (2010), http://www.xbox.com/en-US/kinect/

  17. Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. International Journal of Computer Vision 99(3), 24–52 (2009)

    Article  Google Scholar 

  18. Park, J., Kim, H., Tai, Y.-W., Brown, M.S., Kweon, I.: High Quality Depth Map Upsampling for 3DTOF Cameras. In: ICCV (2011)

    Google Scholar 

  19. Parker, R.: Kinect depth inpainting and filtering (2011), http://www.radfordparker.com/papers

  20. Perona, P., Malik, J.: Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)

    Article  Google Scholar 

  21. Qi, F., Han, J., Wang, P., Shi, G., Li, F.: Structure Guided Fusion for Depth Map Inpainting. In: Pattern Recognition Letters (2012) (in press)

    Google Scholar 

  22. Scharstein, D., Pal, C.: On Learning conditional random fields for stereo. International Journal of Computer Vision 99(3), 319–337 (2007)

    MathSciNet  Google Scholar 

  23. Telea, A.: An image inpainting technique based on the fast marching method. Journal of Graphics Tools 9(1), 25–36 (2004)

    Article  Google Scholar 

  24. Tschumperle, D., Deriche, R.: Anisotropic Diffusion PDE’s for Multi-Channel Image Regularization: Framework and Applications. In: Advances in Imaging and Electron Physics, pp. 1–47 (2007)

    Google Scholar 

  25. Velodyne. Velodyne HDL-64E (2012), http://velodynelidar.com/lidar/

  26. Wasza, J., Bauer, S., Hornegger, J.: Real-time Preprocessing for Dense 3D Range Imaging on the GPU: Defect Interpolation, Bilateral Temporal Averaging and Guided Filtering. In: Workshop in ICCV (2011)

    Google Scholar 

  27. Weickert, J.: Anisotropic Diffusion in Image Processing. ECMI Series. Teubner-Verlag (1998)

    Google Scholar 

  28. Wong, U., Garney, B., Whittaker, W., Whittaker, R.: Camera and lidar fusion for mapping of actively illuminated subterranean voids. In: Howard, A., Iagnemma, K., Kelly, A. (eds.) Field and Service Robotics. STAR, vol. 62, pp. 421–430. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  29. Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-Depth Super Resolution for Range Images. In: CVPR (2007)

    Google Scholar 

  30. Zhang, J., Zheng, J., Cai, J.: A Diffusion Approach to Seeded Image Segmentation. In: CVPR (2010)

    Google Scholar 

  31. Zhu, J., Wang, L., Gao, J., Yang, R.: Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 899–909 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, 310027, P.R. China

    Junyi Liu & Xiaojin Gong

Authors
  1. Junyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojin Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. EURECOM, Multimedia Department, Sophia Antipolis, France

    Benoit Huet

  2. Department of Computer Science, City University of Hong Kong, Tat Chee Ave, Kowloon, Hong Kong

    Chong-Wah Ngo

  3. Nanjing University of Science and Technology, 210093, Nanjing, China

    Jinhui Tang

  4. Department of Computer Science and Technology, Nanjing University, Xianlin Avenue No. 163, 210023, Nanjing, China

    Zhi-Hua Zhou

  5. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA

    Alexander G. Hauptmann

  6. Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, 117583, Singapore, Singapore

    Shuicheng Yan

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Liu, J., Gong, X. (2013). Guided Depth Enhancement via Anisotropic Diffusion. In: Huet, B., Ngo, CW., Tang, J., Zhou, ZH., Hauptmann, A.G., Yan, S. (eds) Advances in Multimedia Information Processing – PCM 2013. PCM 2013. Lecture Notes in Computer Science, vol 8294. Springer, Cham. https://doi.org/10.1007/978-3-319-03731-8_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03731-8_38

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03730-1

  • Online ISBN: 978-3-319-03731-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics