Skip to main content
Springer Nature Link
Log in

3D Deformable Super-Resolution for Multi-Camera 3D Face Scanning

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Low-cost and high-accuracy 3D face measurement is becoming increasingly important in many computer vision applications including face recognition, facial animation, games, orthodontics and aesthetic surgery. In most cases fringe projection based systems are used to overcome the relatively uniform appearance of skin. These systems employ a structured light camera/projector device and require explicit user cooperation and controlled lighting conditions. In this paper, we propose a 3D acquisition solution with a 3D space-time non-rigid super-resolution capability, using three calibrated cameras coupled with a non calibrated projector device, which is particularly suited to 3D face scanning, i.e. rapid, easily movable and robust to ambient lighting variation. The proposed solution is a hybrid stereovision and phase-shifting approach, using two shifted patterns and a texture image, which not only takes advantage of stereovision and structured light, but also overcomes their weaknesses. The super-resolution scheme involves a shape+texture 3D non-rigid registration for 3D artifacts correction in the presence of small non-rigid deformations as facial expressions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Adaskevicius, R., Vasiliauskas, A.: Three-dimensional determination of dental occlusion and facial structures using soft tissue cephalometric analysis. J. Electron. Electron. Eng. 5(121), 93–96 (2010)

    Google Scholar 

  2. Aliaga, D.G., Xu, Y.: Photogeometric structured light: a self-calibrating and multi-viewpoint framework for accurate 3D modeling. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)

  3. Aliaga, D.G., Xu, Y.: A self-calibrating method for photogeometric acquisition of 3D objects. IEEE Trans. Pattern Anal. Mach. Intell. 32(4), 747–754 (2010)

    Article  Google Scholar 

  4. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford Univ. Press, Oxford (1995)

    Google Scholar 

  5. Cox, I., Hingorani, S., Rao, S.: A maximum likelihood stereo algorithm. J. Comput. Vis. Image Underst. 63, 542–567 (1996)

    Article  Google Scholar 

  6. Cui, Y., Stricker, D.: In: 3D Body Scanning with One Kinect. Conference on 3D Body Scanning Technologies, Lugano, Switzerland (2011)

    Google Scholar 

  7. Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc., Ser. B Methodol. 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  8. Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. Image Process (2004)

  9. Han, X., Huang, P.: Combined stereovision and phase shifting method: a new approach for 3-D shape measurement. In: Proc. of SPIE Optical Measurement Systems for Industrial Inspection VI, vol. 7389 (2009)

    Google Scholar 

  10. Kil, Y., Mederos, Y., Amenta, N.: Laser scanner super-resolution. In: Eurographics Symposium on Point-Based Graphics (2006)

  11. Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2262–2275 (2010)

    Article  Google Scholar 

  12. Myronenko, A., Song, X., Carreira-Perpinan, M.A.: Non-rigid point set registration: coherent point drift. In: NIPS Conference (2007)

  13. Ouji, K., Ardabilian, M., Chen, L., Ghorbel, F.: In: Pattern Analysis for an Automatic and Low-Cost 3D Face Acquisition Technique. IEEE Advanced Concepts for Intelligent Vision Systems Conference (ACIVS), Bordeaux, France (2009)

    Google Scholar 

  14. Rajagopalan, A., Bhavsar, A., Wallhoff, F., Rigoll, G.: Resolution Enhancement of PMD Range Maps. Lecture Notes in Computer Science, vol. 5096, pp. 304–313. Springer, Berlin (2008)

    Google Scholar 

  15. Salvi, J., Pagès, J., Batlle, J.: Pattern codification strategies in structured light systems. J. Pattern Recognit. 37, 827–849 (2004)

    Article  MATH  Google Scholar 

  16. Schuon, S., Theobalt, C., Davis, J., Thrun, S.: In: LidarBoost: Depth Superresolution for ToF 3D Shape Scanning. CVPR Conference (2009)

    Google Scholar 

  17. Tikhonov, A.N., Arsenin, V.I.: Solutions of Ill-Posed Problems. Winston and Sons, Washington (1977)

    MATH  Google Scholar 

  18. Weise, T., Leibe, B., Van Gool, L.: Fast 3D scanning with automatic motion compensation. In: IEEE Conference on Computer Vision and Pattern Recognition (2007)

  19. Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: IEEE International Conference on Automatic Face and Gesture Recognition, Washington, DC, USA (2006)

    Google Scholar 

  20. Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands (2008)

    Google Scholar 

  21. Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: ICCV Conference (1999)

  22. Zhang, S., Huang, P.S.: High-resolution, real-time three-dimensional shape measurement. J. Opt. Eng. 45, 123601 (2006)

    Article  Google Scholar 

  23. Zhang, S., Yau, S.: Absolute phase-assisted three-dimensional data registration for a dual-camera structured light system. J. Appl. Opt. 47, 3134–3142 (2008)

    Article  Google Scholar 

  24. Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multipass dynamic programming. In: 3DPVT Conference (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LIRIS, Lyon Research Center for Images and Intelligent Information Systems, Ecole Centrale de Lyon, 36, av. Guy de Collongue, 69134, Ecully, France

    Karima Ouji, Mohsen Ardabilian & Liming Chen

  2. University of Manouba, 2010, Manouba, Tunisia

    Faouzi Ghorbel

Authors
  1. Karima Ouji
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mohsen Ardabilian
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Liming Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Faouzi Ghorbel
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Karima Ouji.

Additional information

This research is supported in part by the ANR project FAR3D under the grant ANR-07-SESU-003.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ouji, K., Ardabilian, M., Chen, L. et al. 3D Deformable Super-Resolution for Multi-Camera 3D Face Scanning. J Math Imaging Vis 47, 124–137 (2013). https://doi.org/10.1007/s10851-012-0399-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-012-0399-y

Keywords