Abstract
This study focuses on unspecified dynamic seru scheduling problems with resource constraints (UDSS-R) in seru production system (SPS). A mixed integer linear programming model is formulated to minimize the makespan, which is solved sequentially from both allocation and scheduling perspectives by a strip-packing constructive algorithm (SPCA) with deep reinforcement learning (DRL). The training samples are trained by the DRL model, and the reward values obtained are calculated by SPCA to train the network so that the agent can find a better solution. The output of DRL is the scheduling order of jobs in serus, while the solution of UDSS-R is solved by SPCA. Finally, a set of test instances are generated to conduct computational experiments with different instance scales for the DRL-SPCA, and the results confirm the effectiveness of proposed DRL-SPCA in solving UDSS-R with more outstanding performance in terms of solution quality and efficiency, across three data scales (10 serus × 100 jobs, 20 serus × 250 jobs, and 30 serus × 400 jobs), compared with GA and SAA, the Avg. RPD of DRL-SPCA decreased by 9.93% and 7.56%, 13.36% and 10.72%, and 9.09% and 7.08%, respectively. In addition, the Avg. CPU time was reduced by 29.53% and 27.93%, 57.48% and 57.04%, and 61.73% and 61.76%, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
This manuscript has no associated data.
References
Bello I, Pham H, Le QV, Norouzi M, Bengio S (2016) Neural combinatorial optimization with reinforcement learning. https://doi.org/10.48550/arXiv.1611.09940. arXiv:1611.09940
Boffa M, Houidi ZB, Krolikowski J, Rossi D (2022) Neural combinatorial optimization beyond the TSP: existing architectures under-represent graph structure. https://doi.org/10.48550/arXiv.2201.00668. arXiv:2201.00668
Bortolini M (2023) Analytic model to predict productivity in divisional seru production environment. Comput Ind Eng 177:109058. https://doi.org/10.1016/j.cie.2023.109058
Edis EB, Oguz C (2012) Parallel machine scheduling with flexible resources. Comput Ind Eng 63(2):433–447. https://doi.org/10.1016/j.cie.2012.03.018
Fleszar K, Hindi KS (2018) Algorithms for the unrelated parallel machine scheduling problem with a resource constraint. Eur J Oper Res 271(3):839–848. https://doi.org/10.1016/j.ejor.2018.05.056
Fu G, Han C, Yu Y, Sun W, Kaku I (2023) A phased intelligent algorithm for dynamic seru production considering seru formation changes. Appl Intell 53(2):1959–1980. https://doi.org/10.1007/s10489-022-03579-0
Fujita Y, Izui K, Nishiwaki S, Zhang Z, Yin Y (2022) Production planning method for seru production systems under demand uncertainty. Comput Ind Eng 163:107856. https://doi.org/10.1016/j.cie.2021.107856
Hu H, Zhang X, Yan X, Wang L, Xu Y (2017) Solving a new 3d bin packing problem with deep reinforcement learning method. https://doi.org/10.48550/arXiv.1708.05930. arXiv:1708.05930
Jiang Y, Zhang Z, Song X, Yin Y (2021a) Scheduling controllable processing time jobs in seru production system with resource allocation. J Oper Res Soc 73(11):2551–2571. https://doi.org/10.1080/01605682.2021.1999182
Jiang Y, Zhang Z, Gong X, Yin Y (2021b) An exact solution method for solving seru scheduling problems with past-sequence-dependent setup time and learning effect. Comput Ind Eng 158:107354. https://doi.org/10.1016/j.cie.2021.107354
Li X, Yu Y, Huang M (2022) Multi-objective cooperative coevolution algorithm with a master-slave mechanism for seru production. Appl Soft Comput 119:108593. https://doi.org/10.1016/j.asoc.2022.108593
Li X, Yu Y, Sun W, Tang J (2023) Reducing tardy batches by seru production: model, exact solution, cooperative coevolution solution, and insights. Comput Oper Res 160:106048. https://doi.org/10.1016/j.cor.2022.106048
Li X, Zhang Z, Sun W, Liu Y, Tang J (2024) Parallel dynamic NSGA-II with multi-population search for rescheduling of seru production considering schedule changes under different dynamic events. Expert Syst Appl 238:121993. https://doi.org/10.1016/j.eswa.2023.121993
Lian J, Liu C, Li W, Yin Y (2018) A multi-skilled worker assignment problem in seru production systems considering the worker heterogeneity. Comput Ind Eng 118:366–382. https://doi.org/10.1016/j.cie.2018.02.035
Liu F, Niu B, Xing M, Wu L, Feng Y (2021) Optimal cross-trained worker assignment for a hybrid seru production system to minimize makespan and workload imbalance. Comput Ind Eng 160:107552. https://doi.org/10.1016/j.cie.2021.107552
Liu C, Li Z, Tang J, Wang X, Yao MJ (2022) How seru production system improves manufacturing flexibility and firm performance: an empirical study in China. Ann Oper Res 316(1):529–554. https://doi.org/10.1007/s10479-020-03850-y
Miao Q, Bai Z, Liu X, Awais M (2022) Modelling and numerical analysis for seru system balancing with lot splitting. Int J Prod Res. https://doi.org/10.1080/00207543.2022.2149873
Ren Y, Tang J, Yu Y, Li X (2023) A two-stage stochastic programming model and parallel master-slave adaptive GA for flexible seru system formation. Int J Prod Res. https://doi.org/10.1080/00207543.2023.2177087
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484–489. https://doi.org/10.1038/nature16961
Sun W, Yu Y, Lou Q, Wang J, Guan Y (2020) Reducing the total tardiness by seru production: model, exact and cooperative coevolution solutions. Int J Prod Res 58(21):6441–6452. https://doi.org/10.1080/00207543.2019.1680898
Sutton RS, McAllester D, Singh S, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems, p 12
Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. Advances in neural information processing systems, p 28
Wang Y, Tang J (2022) Optimized skill configuration for the seru production system under an uncertain demand. Ann Oper Res 316:445–465. https://doi.org/10.1007/s10479-020-03805-3
Wu Y, Wang L, Chen J, Zheng J, Pan Z (2023a) A reinforcement learning driven two-stage evolutionary optimisation for hybrid seru system scheduling with worker transfer. Int J Prod Res. https://doi.org/10.1080/00207543.2023.2252523
Wu Y, Wang L, Zhuang X, Wang JJ, Chen JF, Zheng J (2023b) A cooperative coevolutionary algorithm with problem-specific knowledge for energy-efficient scheduling in seru system. Knowl-Based Syst 274:110663. https://doi.org/10.1016/j.knosys.2023.110663
Yılmaz ÖF (2020a) Attaining flexibility in seru production system by means of Shojinka: an optimization model and solution approaches. Comput Oper Res 119:104917. https://doi.org/10.1016/j.cor.2020.104917
Yılmaz ÖF (2020b) Operational strategies for seru production system: a bi-objective optimisation model and solution methods. Int J Prod Res 58(11):3195–3219. https://doi.org/10.1080/00207543.2019.1669841
Yılmaz BG, Yılmaz ÖF, Çevikcan E (2023) Lot streaming in workforce scheduling problem for seru production system under Shojinka philosophy. Comput Ind Eng 185:109680. https://doi.org/10.1016/j.cie.2023.109680
Yin Y, Stecke KE, Swink M, Kaku I (2017) Lessons from seru production on manufacturing competitively in a high cost environment. J Oper Manag 49:67–76. https://doi.org/10.1016/j.jom.2017.01.003
Yin Y, Stecke KE, Li D (2018) The evolution of production systems from Industry 2.0 through Industry 4.0. Int J Prod Res 56(1–2):848–861. https://doi.org/10.1080/00207543.2017.1403664
Ying KC, Tsai YJ (2017) Minimising total cost for training and assigning multiskilled workers in seru production systems. Int J Prod Res 55(10):2978–2989. https://doi.org/10.1080/00207543.2016.1277594
Yu Y, Sun W, Tang J, Wang J (2017a) Line-hybrid seru system conversion: models, complexities, properties, solutions and insights. Comput Ind Eng 103:282–299. https://doi.org/10.1016/j.cie.2016.11.035
Yu Y, Sun W, Tang J, Kaku I, Wang J (2017b) Line-seru conversion towards reducing worker (s) without increasing makespan: models, exact and meta-heuristic solutions. Int J Prod Res 55(10):2990–3007. https://doi.org/10.1080/00207543.2017.1284359
Yu Y, Wang J, Ma K, Sun W (2018) Seru system balancing: definition, formulation, and exact solution. Comput Ind Eng 122:318–325. https://doi.org/10.1016/j.cie.2018.05.048
Zhang X, Liu C, Li W, Evans S, Yin Y (2017) Effects of key enabling technologies for seru production on sustainable performance. Omega 66:290–307. https://doi.org/10.1016/j.omega.2016.01.013
Zhang Z, Song X, Huang H, Yin Y, Lev B (2022a) Scheduling problem in seru production system considering DeJong’s learning effect and job splitting. Ann Oper Res 312(2):1119–1141. https://doi.org/10.1007/s10479-021-04515-0
Zhang Z, Song X, Huang H, Zhou X, Yin Y (2022b) Logic-based Benders decomposition method for the seru scheduling problem with sequence-dependent setup time and DeJong’s learning effect. Eur J Oper Res 297(3):866–877. https://doi.org/10.1016/j.ejor.2021.06.017
Zhang Z, Gong X, Song X, Yin Y, Lev B, Chen J (2022c) A column generation-based exact solution method for seru scheduling problems. Omega 108:102581. https://doi.org/10.1016/j.omega.2021.102581
Zhang Z, Song X, Gong X, Yin Y, Lev B, Zhou X (2022d) An exact quadratic programming approach based on convex reformulation for seru scheduling problems. Nav Res Logist 69(8):1096–1107. https://doi.org/10.1002/nav.22078
Zhang Z, Wang L, Song X, Huang H, Yin Y (2022e) Improved genetic-simulated annealing algorithm for seru loading problem with downward substitution under stochastic environment. J Oper Res Soc 73(8):1800–1811. https://doi.org/10.1080/01605682.2021.1939172
Zhang Z, Song X, Gong X, Yin Y, Lev B, Zhou X (2023) An effective heuristic based on 3-opt strategy for seru scheduling problems with learning effect. Int J Prod Res 61(6):1938–1954. https://doi.org/10.1080/00207543.2022.2054744
Zhang Z, Song X, Gong X, Yin Y, Lev B, Zhou X (2024) Coordinated seru scheduling and distribution operation problems with DeJong’s learning effects. Eur J Oper Res 313(3):452–464. https://doi.org/10.1016/j.ejor.2023.08.022
Zhou X, Wei X, Lin J, Tian X, Lev B, Wang S (2021) Supply chain management under carbon taxes: a review and bibliometric analysis. Omega 98:102295. https://doi.org/10.1016/j.omega.2020.102295
Acknowledgements
We would like to give our great appreciation to all the reviewers and editors who contributed this research.
Funding
This research is sponsored by the Fundamental Research Funds for the Central Universities (No. 30922011406), and System Science and Enterprise Development Research Center (Grant No. Xq22B06).
Author information
Authors and Affiliations
Contributions
Yiran Xiang: conceptualization, methodology, model formulation, writing original draft. Zhe Zhang: supervision, methodology, coding, validation, formal analysis. Xue Gong: conceptualization, resources, writing review and editing. Xiaoling Song: investigation, writing review and editing. Yong Yin: methodology, language proof, writing review and editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Human participants and/or animals
This manuscript does not contain any studies with human participants or animals performed by any of the authors.
Ethical approval
All analyses are based on numerical analysis, thus no ethical approval is required.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xiang, Y., Zhang, Z., Gong, X. et al. A strip-packing constructive algorithm with deep reinforcement learning for dynamic resource-constrained seru scheduling problems. Soft Comput 28, 9785–9802 (2024). https://doi.org/10.1007/s00500-024-09815-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-024-09815-8