Skip to main content
Springer Nature Link
Log in

Component Extraction for Deep Learning Through Progressive Method

  • Conference paper
  • First Online:
Smart Computing and Communication (SmartCom 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13828))

Included in the following conference series:

  • 1075 Accesses

Abstract

Machine learning has shown great impact in a lot of applications. Within all types of tools, deep learning should be one of the most important techniques thanks to its ability to capture the correlation between the input features and output results. However, the relatively long training time and high computation complexity remain a big problem in deep learning. In addition, the impossibility to explain the model makes it harder for us to look for alternatives to fix the bad fitting results. Therefore, this paper aims at improving the deep learning model training result by proposing a principal component extraction algorithm. Compared with the previous Principal Component Analysis (PCA) methods, this algorithm creatively consider not only the original input components but also the computed variables in the first hidden layer in neural network so as to capture more representative components. The experiment shows that compared to previous PCA method, this can better capture the principal components from all input variables.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Zillow’s home value prediction Kaggle competition data. https://www.kaggle.com/c/zillow-prize-1/data

  2. Bennett, K., Mangasarian, O.: Neural network training via linear programming. Technical report, University of Wisconsin-Madison Department of Computer Sciences (1990)

    Google Scholar 

  3. Brahma, P., Wu, D., She, Y.: Why deep learning works: a manifold disentanglement perspective. IEEE Trans. Neural Netw. Learn. Syst. 27(10), 1997–2008 (2015)

    Article  MathSciNet  Google Scholar 

  4. Breymann, W., Dias, A., Embrechts, P.: Dependence structures for multivariate high-frequency data in finance. Quant. Financ. 3(1), 1 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  5. Carreira-Perpinán. M.: A review of dimension reduction techniques. Department of Computer Science. University of Sheffield. Tech. Rep. CS-96-09, 9:1–69 (1997)

    Google Scholar 

  6. Chakraborty, S., et al.: Interpretability of deep learning models: a survey of results. In: 2017 IEEE smartworld, ubiquitous intelligence, pp. 1–6. IEEE (2017)

    Google Scholar 

  7. Frino, A., Gallagher, D.: Tracking s &p 500 index funds. J. Portf. Manag. 28(1), 44–55 (2001)

    Article  Google Scholar 

  8. Gai, K., Qiu, M., Chen, L., Liu, M.: Electronic health record error prevention approach using ontology in big data. In: IEEE 17th HPCC (2015)

    Google Scholar 

  9. Gai, K., Qiu, M., Elnagdy. S.: A novel secure big data cyber incident analytics framework for cloud-based cybersecurity insurance. In: 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity) (2016)

    Google Scholar 

  10. Gao, X., Qiu, M.: Energy-based learning for preventing backdoor attack. In International Conference on Knowledge Science, Engineering and Management, pp. 706–721 (2022)

    Google Scholar 

  11. Gao, X., Qiu, M.: Recommendation system design for social media using reinforcement learning. In: 2022 IEEE 9th International Conference on Cyber Security and Cloud Computing, pp. 6–11 (2022)

    Google Scholar 

  12. Gao, X., Qiu, M., He, Z.: Big data analysis with momentum strategy on data-driven trading. In: IEEE ISPA, pp. 1328–1335 (2021)

    Google Scholar 

  13. Gulli, A., Pal, S.: Deep Learning with Keras. Packt Publishing Ltd., Birmingham (2017)

    Google Scholar 

  14. Hayou, S., Doucet, A., Rousseau, J.: On the selection of initialization and activation function for deep neural networks. arXiv preprint arXiv:1805.08266 (2018)

  15. Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: Neural Networks for Perception, pp. 65–93. Elsevier, Academic Press (1992)

    Google Scholar 

  16. Hu, F., Lakdawala, S., et al.: Low-power, intelligent sensor hardware interface for medical data preprocessing. IEEE Trans. Infor. Tech. Biomed. 13(4), 656–663 (2009)

    Google Scholar 

  17. Hua, J., Tembe, W., Dougherty. E.: Feature selection in the classification of high-dimension data. In: 2008 IEEE International Workshop on Genomic Signal Processing and Statistics, pp. 1–2, IEEE (2008)

    Google Scholar 

  18. Johnson, V., Rogers, L.: Accuracy of neural network approximators in simulation-optimization. J. Water Resour. Plan. Manag. 126(2), 48–56 (2000)

    Article  Google Scholar 

  19. Kung. S.: XNAS: a regressive/progressive NAS for deep learning. ACM Trans. Sensor Netw. (TOSN) (2022)

    Google Scholar 

  20. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521(7553), 436–444 (2015)

    Google Scholar 

  21. Li, J., Ming, Z., et al.: Resource allocation robustness in multi-core embedded systems with inaccurate information. J. Syst. Arch. 57(9), 840–849 (2011)

    Article  Google Scholar 

  22. Li, Y., Gai, K., et al.: Intercrossed access controls for secure financial services on multimedia big data in cloud systems. ACM Trans. Multim. Compu., Commu., Appli, 12(4) (2016)

    Google Scholar 

  23. Liu, B., Wei, Y., Zhang, X., Yang, Q.: Deep neural networks for high dimension, low sample size data. In: IJCAI, pp. 2287–2293 (2017)

    Google Scholar 

  24. Marcus, G.: Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631 (2018)

  25. Martinez, A., Kak, A.: PCA versus LDA. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 228–233 (2001)

    Article  Google Scholar 

  26. Ng, C.: The future of AI in finance. The AI Book: The Artificial Intelligence Handbook for Investors, Entrepreneurs and FinTech Visionaries, pp. 6–8 (2020)

    Google Scholar 

  27. Niu, J., Gao, Y., Qiu, M., Ming, Z.: Selecting proper wireless network interfaces for user experience enhancement with guaranteed probability. J. Parallell Distrib. Comput. 72(12), 1565–1575 (2012)

    Article  Google Scholar 

  28. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    Google Scholar 

  29. Qiu, H., Dong, T., et al.: Adversarial attacks against network intrusion detection in IoT systems. IEEE Internet of Things J. 8(13), 10327–10335 (2020)

    Article  Google Scholar 

  30. Qiu, H., Qiu, M., Lu, R.: Secure V2X communication network based on intelligent PKI and edge computing. IEEE Netw. 34(2), 172–178 (2019)

    Article  Google Scholar 

  31. Qiu, H., Zeng, Y., Guo, S., et al.: Deepsweep: an evaluation framework for mitigating DNN backdoor attacks using data augmentation. In: ACM ASIA Conference on Computer and Communications Security (2021)

    Google Scholar 

  32. Qiu, H., Zheng, Q., et al.: Topological graph convolutional network-based urban traffic flow and density prediction. IEEE Trans. Intell. Trans. Syst. 22, 4560–4569 (2020)

    Google Scholar 

  33. Qiu, M., Chen, Z., Ming, Z., Qin, X., Niu, J.: Energy-aware data allocation with hybrid memory for mobile cloud systems. IEEE Syst. J. 11(2), 813–822 (2014)

    Article  Google Scholar 

  34. Qiu, M., Jia, Z., Xue, C., Shao, Z., Sha, E.: Voltage assignment with guaranteed probability satisfying timing constraint for real-time multiproceesor DSP. J. VLSI Signal Proc. Sys. 46, 55–73 (2007)

    Google Scholar 

  35. Qiu, M., Li, H., Sha, E.: Heterogeneous real-time embedded software optimization considering hardware platform. In Proceedings of the 2009 ACM Symposium on Applied Computing (SAC), Honolulu, Hawaii, USA, 9-12 March 2009, pp. 1637–1641 (2009)

    Google Scholar 

  36. Qiu, M., Xue, C., Shao, Z., Zhuge, Q., Liu, M., Sha, E.H.-M.: Efficent algorithm of energy minimization for heterogeneous wireless sensor network. In: Sha, E., Han, S.-K., Xu, C.-Z., Kim, M.-H., Yang, L.T., Xiao, B. (eds.) EUC 2006. LNCS, vol. 4096, pp. 25–34. Springer, Heidelberg (2006). https://doi.org/10.1007/11802167_5

    Chapter  Google Scholar 

  37. Qiu, M., Xue, C., Shao, Z., Sha. E.: Energy minimization with soft real-time and DVS for uniprocessor and multiprocessor embedded systems. In: 2007 Design, Automation Test in Europe Conference Exhibition, pp. 1–6 (2007)

    Google Scholar 

  38. Qiu, M., Yang, L., Shao, Z., Sha, E.: Dynamic and leakage energy minimization with soft real-time loop scheduling and voltage assignment. IEEE Trans. Very Large Scale Integr. 18(3), 501–504 (2009)

    Article  Google Scholar 

  39. Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017)

    Google Scholar 

  40. Takase, T., Oyama, S., Kurihara, M.: Effective neural network training with adaptive learning rate based on training loss. Neural Netw. 101, 68–78 (2018)

    Article  Google Scholar 

  41. Tao, L., Golikov, S., et al.: A reusable software component for integrated syntax and semantic validation for services computing. In: IEEE Symposium on Service-Oriented System Eng., pp. 127–132 (2015)

    Google Scholar 

  42. Yu, H., Yang, J.: A direct LDA algorithm for high-dimensional data-with application to face recognition. Pattern Recogn. 34(10), 2067–2070 (2001)

    Article  MATH  Google Scholar 

  43. Zhang, K., Kong, J., et al.: Multimedia layout adaptation through grammatical specifications. Multimedia Syst. 10(3), 245–260 (2005)

    Article  Google Scholar 

  44. Zhao, W., Chellappa, R., Nandhakumar, N.: Empirical performance analysis of linear discriminant classifiers. In: Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. 98CB36231), pp. 164–169. IEEE (1998)

    Google Scholar 

  45. Zheng, X., Chen, W., Li, M., Zhang, T., You, Y., Jiang, Y.: Decoding human brain activity with deep learning. Biomed. Signal Process. Control 56, 101730 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. New York University, New York City, NY, USA

    Xiangyu Gao

  2. Dakota State University, Madison, SD, 57042, USA

    Meikang Qiu

  3. Educational Information Technology Lab., Henan University, Kaifeng, 475000, China

    Hui Zhao

Authors
  1. Xiangyu Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meikang Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meikang Qiu .

Editor information

Editors and Affiliations

  1. Dakota State University, Madison, SD, USA

    Meikang Qiu

  2. Fudan University, Shanghai, China

    Zhihui Lu

  3. Ibaraki University, Ibaraki, Japan

    Cheng Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gao, X., Qiu, M., Zhao, H. (2023). Component Extraction for Deep Learning Through Progressive Method. In: Qiu, M., Lu, Z., Zhang, C. (eds) Smart Computing and Communication. SmartCom 2022. Lecture Notes in Computer Science, vol 13828. Springer, Cham. https://doi.org/10.1007/978-3-031-28124-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28124-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28123-5

  • Online ISBN: 978-3-031-28124-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics