Skip to main content
Springer Nature Link
Log in

Transfer Learning Assisted GPHH for Dynamic Multi-Workflow Scheduling in Cloud Computing

  • Conference paper
  • First Online:
AI 2021: Advances in Artificial Intelligence (AI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13151))

Included in the following conference series:

Abstract

Dynamic Multi-Workflow Scheduling (DWS) in cloud is a challenging optimisation due to the dynamic nature of the problem. It requires complex mapping decisions to be made under unknown dynamic events. Genetic Programming Hyper-Heuristic (GPHH) has been successfully applied to generate scheduling heuristics for DWS due to its flexible representation. However, the simulation-based evaluation is computationally expensive due to calculations required by individuals to make decisions in the simulation. To improve efficiency, this paper proposes a novel Transfer Learning (TL) Assisted GPHH (TL-GPHH). Specifically, TL-GPHH is designed and compared with Non-Assisted GPHH to generate effective heuristic rules in less time. The results show that the proposed algorithm generates heuristics that can minimise makespan in more scenarios than Non-Assisted GPHH and other state-of-the-art heuristics. Moreover, the proposed algorithm can reduce the computational costs of GP without sacrificing the performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Arabnejad, H., Barbosa, J.G.: Maximizing the completion rate of concurrent scientific applications under time and budget constraints. J. Comput. Sci. 23, 120–129 (2017)

    Article  MathSciNet  Google Scholar 

  2. Arabnejad, V., Bubendorfer, K., Ng, B.: Dynamic multi-workflow scheduling: a deadline and cost-aware approach for commercial clouds. Future Gener. Comput. Syst. 100, 98–108 (2019)

    Article  Google Scholar 

  3. Ardeh, M.A., Mei, Y., Zhang, M.: Transfer learning in genetic programming hyper-heuristic for solving uncertain capacitated arc routing problem. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 49–56. IEEE (2019)

    Google Scholar 

  4. Ardeh, M.A., Mei, Y., Zhang, M.: A parametric framework for genetic programming with transfer learning for uncertain capacitated arc routing problem. In: Gallagher, M., Moustafa, N., Lakshika, E. (eds.) AI 2020. LNCS (LNAI), vol. 12576, pp. 150–162. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64984-5_12

    Chapter  Google Scholar 

  5. Blythe, J., et al.: Task scheduling strategies for workflow-based applications in grids. In: IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2005, vol. 2, pp. 759–767. IEEE (2005)

    Google Scholar 

  6. Braun, T.D., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 61(6), 810–837 (2001)

    Article  Google Scholar 

  7. Chawla, Y., Bhonsle, M.: A study on scheduling methods in cloud computing. Int. J. Emerg. Trends Technol. Comput. Sci. (IJETTCS) 1(3), 12–17 (2012)

    Google Scholar 

  8. Chen, W., Deelman, E.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp. 1–8. IEEE (2012)

    Google Scholar 

  9. Dinh, T.T.H., Chu, T.H., Nguyen, Q.U.: Transfer learning in genetic programming. In: 2015 IEEE Congress on Evolutionary Computation (CEC), pp. 1145–1151. IEEE (2015)

    Google Scholar 

  10. Escott, K.-R., Ma, H., Chen, G.: Genetic programming based hyper heuristic approach for dynamic workflow scheduling in the cloud. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) DEXA 2020. LNCS, vol. 12392, pp. 76–90. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59051-2_6

    Chapter  Google Scholar 

  11. Escott, K.R., Ma, H., Chen, G.: A genetic programming hyper-heuristic approach to design high-level heuristics for dynamic workflow scheduling in cloud. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 3141–3148. IEEE (2020)

    Google Scholar 

  12. Faragardi, H.R., Sedghpour, M.R.S., Fazliahmadi, S., Fahringer, T., Rasouli, N.: GRP-HEFT: a budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans. Parallel Distrib. Syst. 31, 1239–1254(2019)

    Google Scholar 

  13. Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29(3), 682–692 (2013)

    Article  Google Scholar 

  14. Lin, J., Zhu, L., Gao, K.: A genetic programming hyper-heuristic approach for the multi-skill resource constrained project scheduling problem. Expert Syst. Appl. 140, 112915 (2020)

    Google Scholar 

  15. Masood, A., Mei, Y., Chen, G., Zhang, M.: Many-objective genetic programming for job-shop scheduling. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 209–216. IEEE (2016)

    Google Scholar 

  16. Meng, S., et al.: Security-aware dynamic scheduling for real-time optimization in cloud-based industrial applications. IEEE Trans. Ind. Inform. 17, 4219–4228 (2020)

    Google Scholar 

  17. Muñoz, L., Trujillo, L., Silva, S.: Transfer learning in constructive induction with genetic programming. Genet. Program. Evolvable Mach. 21(4), 529–569 (2020)

    Article  Google Scholar 

  18. Nguyen, S., Mei, Y., Xue, B., Zhang, M.: A hybrid genetic programming algorithm for automated design of dispatching rules. Evolut. Comput. 27(3), 467–496 (2019)

    Article  Google Scholar 

  19. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  20. Topcuoglu, H., Hariri, S., Wu, M.y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)

    Google Scholar 

  21. Xiao, Q.z., Zhong, J., Feng, L., Luo, L., Lv, J.: A cooperative coevolution hyper-heuristic framework for workflow scheduling problem. IEEE Trans. Serv. Comput. (2019)

    Google Scholar 

  22. Xie, J., Mei, Y., Ernst, A.T., Li, X., Song, A.: A genetic programming-based hyper-heuristic approach for storage location assignment problem. In: 2014 IEEE congress on evolutionary computation (CEC), pp. 3000–3007. IEEE (2014)

    Google Scholar 

  23. Yu, Y., Feng, Y., Ma, H., Chen, A., Wang, C.: Achieving flexible scheduling of heterogeneous workflows in cloud through a genetic programming based approach. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 3102–3109. IEEE (2019)

    Google Scholar 

  24. Zhang, F., Mei, Y., Nguyen, S., Zhang, M.: Collaborative multifidelity-based surrogate models for genetic programming in dynamic flexible job shop scheduling. IEEE Trans. Cybern. (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Victoria University of Wellington, 1 Kelburn Parade, Wellington, New Zealand

    Kirita-Rose Escott, Hui Ma & Gang Chen

Authors
  1. Kirita-Rose Escott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kirita-Rose Escott .

Editor information

Editors and Affiliations

  1. University of Technology Sydney, Sydney, NSW, Australia

    Guodong Long

  2. RMIT University, Melbourne, SA, Australia

    Xinghuo Yu

  3. University of Queensland, Brisbane, QLD, Australia

    Sen Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Escott, KR., Ma, H., Chen, G. (2022). Transfer Learning Assisted GPHH for Dynamic Multi-Workflow Scheduling in Cloud Computing. In: Long, G., Yu, X., Wang, S. (eds) AI 2021: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13151. Springer, Cham. https://doi.org/10.1007/978-3-030-97546-3_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-97546-3_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-97545-6

  • Online ISBN: 978-3-030-97546-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics