Skip to main content
Springer Nature Link
Log in

Sign correlation subspace for face alignment

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Face alignment is an essential task for facial performance capture and expression analysis. Current methods such as random subspace supervised descent method, stage-wise relational dictionary and coarse-to-fine shape searching can ease multi-pose face alignment problem, but no method can deal with the multiple local minima problem directly. In this paper, we propose a sign correlation subspace method for domain partition in only one reduced low-dimensional subspace. Unlike previous methods, we analyze the sign correlation between features and shapes and project both of them into a mutual sign correlation subspace. Each pair of projected shape and feature keeps their signs consistent in each dimension of the subspace, so that each hyper octant holds the condition that one general descent exists. Then a set of general descents are learned from the samples in different hyperoctants. Requiring only the feature projection for domain partition, our proposed method is effective for face alignment. We have validated our approach with the public face datasets which include a range of poses. The validation results show that our method can reveal their latent relationships to poses. The comparison with state-of-the-art methods demonstrates that our method outperforms them, especially in uncontrolled conditions with various poses, while enjoying the comparable speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Trans Pattern Anal Mach Intell 35(12):2930–40

    Article  Google Scholar 

  • Burgosartizzu XP, Perona P, Dollar P (2013) Robust face landmark estimation under occlusion. In: IEEE international conference on computer vision, pp 1513–1520

  • Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. US Patent App. 13/728,584

  • Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  • Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  • Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. BMVC 41:929–938

    MATH  Google Scholar 

  • Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. IEEE 238(6):1078–1085

    Google Scholar 

  • Fan X, Wang H, Luo Z, Li Y, Hu W, Luo D (2015) Fiducial facial point extraction using a novel projective invariant. IEEE Trans Image Process 24(3):1164–77

    Article  MathSciNet  Google Scholar 

  • Feng ZH, Huber P, Kittler J, Christmas W, Wu XJ (2015a) Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process Lett 22(1):76–80

    Article  Google Scholar 

  • Feng ZH, Hu G, Kittler J, Christmas W, Wu XJ (2015b) Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans Image Process 24(11):3425–3440

    Article  MathSciNet  Google Scholar 

  • Gonzalezmora J, Torre FDL, Murthi R, Guil N, Zapata EL (2007) Bilinear active appearance models. In: IEEE international conference on computer vision, pp 1–8

  • Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments

  • Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: Computer vision and pattern recognition, pp 1867–1874

  • Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In: IEEE international conference on computer vision workshops, ICCV 2011 workshops, Barcelona, Spain, Nov 6–13, 2011, pp 2144–2151

  • Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European conference on computer vision, pp 679–692

  • Lee HS, Kim D (2009) Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans Pattern Anal Mach Intell 31(6):1102–16

    Article  Google Scholar 

  • Lee D, Park H, Yoo CD (2015) Face alignment using cascade gaussian process regression trees. In: IEEE conference on computer vision and pattern recognition, pp 4204–4212

  • Lindner C, Bromiley PA, Ionita MC, Cootes TF (2015) Robust and accurate shape model matching using random forest regression-voting. IEEE Trans Pattern Anal Mach Intell 37(9):1862–74

    Article  Google Scholar 

  • Liu L, Hu J, Zhang S, Deng W (2014) Extended supervised descent method for robust face alignment. Springer, Berlin

    Google Scholar 

  • Martinez B, Pantic M (2015a) Facial landmarking for in-the-wild images with local inference based on global appearance. Image Vis Comput 36(C):40–50

    Article  Google Scholar 

  • Martinez B, Valstar MF (2015b) L 2,1-based regression and prediction accumulation across views for robust facial landmark detection. Image Vis Comput 45(4):371–382

    Google Scholar 

  • Ramanan D (2015) Face detection, pose estimation, and landmark localization in the wild. In: IEEE conference on computer vision and pattern recognition, pp 31–37

  • Ren S, Cao X, Wei Y, Sun J (2014) Face alignment at 3000 FPS via regressing local binary features. In: IEEE transactions on image processing, pp 1685–1692

  • Romdhani S (1999) A multi-view nonlinear active shape model using kernel PCA. In: British machine vision conference, pp 483–492

  • Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semi-automatic methodology for facial landmark annotation. In: IEEE conference on computer vision and pattern recognition workshops, pp 896–903

  • Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215

    Article  MathSciNet  MATH  Google Scholar 

  • Snchez-Lozano E, Martinez B, Valstar MF (2016) Cascaded regression with sparsified feature covariance matrix for facial landmark detection. Pattern Recognit Lett 73(C):19–25

    Article  Google Scholar 

  • Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: Conference on computer vision and pattern recognition, pp 3476–3483

  • Tzimiropoulos G (2015) Project-out cascaded regression with an application to face alignment. In: IEEE conference on computer vision and pattern recognition

  • Valstar M, Martinez B, Binefa X, Pantic M (2010) Facial point detection using boosted regression and graph models, pp 2729–2736

  • Xing J, Niu Z, Huang J, Hu W, Yan S (2014) Towards multi-view and partially-occluded face alignment. In: Computer vision and pattern recognition, pp 1829–1836

  • Xiong X, De la Torre F (2013) Supervised descent method and its applications to face alignment. In: IEEE conference on computer vision and pattern recognition, pp 532–539

  • Xiong X, la Torre FD (2015) Global supervised descent method. In: IEEE conference on computer vision and pattern recognition, pp 2664–2673

  • Yang H, Patras I (2014) Fine-tuning regression forests votes for object alignment in the wild. IEEE Trans Image Process A Publ IEEE Signal Process Soc 24(2):619–31

    Article  MathSciNet  Google Scholar 

  • Yang H, He X, Jia X, Patras I (2015a) Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans Image Process 24(8):2393–403

    Article  MathSciNet  Google Scholar 

  • Yang H, Jia X, Patras I, Chan KP (2015b) Random subspace supervised descent method for regression problems in computer vision. IEEE Signal Process Lett 22(10):1816–1820

    Article  Google Scholar 

  • Yan J, Lei Z, Yi D, Li SZ (2013) Learn to combine multiple hypotheses for accurate face alignment. In: IEEE international conference on computer vision workshops, pp 392–396

  • Zhang Z, Zhang W, Ding H, Liu J, Tang X (2014a) Hierarchical facial landmark localization via cascaded random binary patterns. Pattern Recognit 48(4):1277–1288

    Article  Google Scholar 

  • Zhang Z, Luo P, Loy CC, Tang X (2014b) Facial landmark detection by deep multi-task learning. In: European conference on computer vision, pp 94–108

  • Zhang J, Shan S, Kan M, Chen X (2014c) Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Computer vision-ECCV. Springer, pp 1–16

  • Zhou E, Fan H, Cao Z, Jiang Y, Yin Q (2013) Extensive facial landmark localization with coarse-to-fine convolutional network cascade. In: IEEE international conference on computer vision workshops, pp 386–391

  • Zhu S, Li C, Loy CC, Tang X (2015) Face alignment by coarse-to-fine shape searching. In: CVPR, pp 4998–5006

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 61402133)

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, 92 West Dazhi Street, Nan Gang District, Harbin, 150001, People’s Republic of China

    Dansong Cheng, Yongqiang Zhang & Ce Liu

  2. School of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin, People’s Republic of China

    Xiaofang Liu

  3. Faculty of Science and Technology, Bournemouth University, Poole, UK

    Feng Tian

Authors
  1. Dansong Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yongqiang Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Feng Tian
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ce Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xiaofang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dansong Cheng.

Ethics declarations

Conflict of interest

All authors declare that there is no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, D., Zhang, Y., Tian, F. et al. Sign correlation subspace for face alignment. Soft Comput 23, 241–249 (2019). https://doi.org/10.1007/s00500-018-3389-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3389-1

Keywords