Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model

IDZIK Christian, GERLACH Jannik, BAILLY David, HIRT Gerhard

Abstract. Rolling is a well-established forming process for producing finished or semi-finished products in various industries. Although highly automated, most rolling processes are designed manually by experts based on their knowledge, highly specialized heuristics and analytical process models or numerical simulations. This manual design approach does not lead to an optimization accounting for multiple objectives. Previous work [1] has shown the potential of coupling reinforcement learning (RL) with fast analytical rolling models (FRM) to optimize hot rolling processes. However, the designed pass schedules do not robustly reach the desired final height within typical industrial tolerances. Therefore, in this paper the existing approach of coupling RL with an FRM is extended by dynamically ranges for height reductions. This extension guarantees that the target height is always reached exactly. In addition to the height reduction, the RL algorithm can determine the inter-pass time, initial slab temperature and rolling velocity. For the optimization, an objective function, called reward function, considering all relevant optimization objectives such as the final grain size and energy consumption, was developed. An exemplary training was performed for a defined starting (140 mm) and final height (25 mm). The resulting, automatically designed pass schedules reach the target height and fulfill all defined optimization objective including the required average austenite grain size.

Keywords
Hot Rolling, Reinforcement Learning, Multi-Objective Optimization

Published online 4/19/2023, 10 pages
Copyright © 2023 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA

Citation: IDZIK Christian, GERLACH Jannik, BAILLY David, HIRT Gerhard, Advanced automatic pass schedule design for hot rolling by coupling reinforcement learning with a fast rolling model, Materials Research Proceedings, Vol. 28, pp 601-610, 2023

DOI: https://doi.org/10.21741/9781644902479-65

The article was published as article 65 of the book Material Forming

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

References
[1] C. Scheiderer, T. Thun, C. Idzik, A.F. Posada-Moreno, A. Krämer, J. Lohmar, G. Hirt, T. Meisen, Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes, Procedia Manuf. 51 (2020) 897-903. https://doi.org/10.1016/j.promfg.2020.10.126
[2] J.M. Allwood, J.M. Cullen, M.A. Carruth, Sustainable materials. With both eyes open ; [future buildings, vehicles, products and equipment – made efficiently and made with less new material]. UIT Cambridge, Cambridge, 2012
[3] World Steel Association, 2022 World Steel in Figures. World crude steel production 1950 to 2021. https://worldsteel.org/steel-topics/statistics/world-steel-in-figures-2022/
[4] D.S. Svietlichnyj, M. Pietrzyk, On-Line Model for Control of Hot Plate Rolling. In: Beynon JH (Hrsg) 3rd International Conference on Modelling of Metal Rolling Processes. IOM Communications, London, S (1999) 62-71
[5] M. Schmidtchen, R. Kawalla, Fast Numerical Simulation of Symmetric Flat Rolling Processes for Inhomogeneous Materials Using a Layer Model − Part I. Basic Theory, Steel Res. Int. 87 (2016) 1065-1081. https://doi.org/10.1002/srin.201600047
[6] A. Özgür, Y. Uygun, M.-T. Hütt, A review of planning and scheduling methods for hot rolling mills in steel production, Comput. Ind. Eng. 151 (2021) 106606. https://doi.org/10.1016/j.cie.2020.106606
[7] V. Pandey, P.S. Rao, S. Singh, M. Pandey, A Calculation Procedure and Optimization for Pass Scheduling in Rolling Process. A Review, J. Mater. Sci. Mech. Eng. 5 (2018) 126-130
[8] S. Wu, X. Zhou, J. Ren, G. Cao, Z. Liu, N. Shi, Optimal design of hot rolling process for C-Mn steel by combining industrial data-driven model and multi-objective optimization algorithm. J. Iron Steel Res. Int. 25 (2018) 700–705. https://doi.org/10.1007/s42243-018-0101-8
[9] C.A. Hernández Carreón, J.E. Mancilla Tolama, G. Castilla Valdez, I. Hernández González, Multi-Objective Optimization of the Hot Rolling Scheduling of Steel Using a Genetic Algorithm. MRS Adv. 4 (2019) 3373-3380. https://doi.org/10.1557/adv.2019.436
[10] J.H. Beynon, C.M. Sellars, Modelling Microstructure and Its Effects during Multipass Hot Rolling. Iron Steel Inst. Jap. 32 (1992) 359–367. https://doi.org/10.2355/isijinternational.32.359
[11] J. Lohmar, S. Seuren, M. Bambach, G. Hirt, Design and Application of an Advanced Fast Rolling Model with Through Thickness Resolution for Heavy Plate Rolling. In: J. Guzzoni, M. Manning (Hrsg) 2nd International Conference on Ingot Casting Rolling Forging, ICRF 2014
[12] S. Shen, D. Guye, X. Ma, S. Yue, N. Armanfard, Multistep networks for roll force prediction in hot strip rolling mill, Machine Learning with Applications 7 (2022) 100245. https://doi.org/10.1016/j.mlwa.2021.100245
[13] R.S. Sutton, A. Barto, Reinforcement learning. An introduction. Adaptive computation and machine learning, The MIT Press, Cambridge, MA, London, 2018
[14] C. Li, P. Zheng, Y. Yin, B. Wang, L .Wang, Deep Reinforcement Learning in Smart Manufacturing: A Review and Prospects, CIRP J. Manuf. Sci. Technol. 40 (2022) 75-101. https://doi.org/10.1016/j.cirpj.2022.11.003
[15] A. Esteso, D. Peidro, J. Mula, M. Díaz-Madroñero, Reinforcement learning applied to production planning and control, Int. J. Prod. Res. (2022) 1-18. https://doi.org/10.1080/00207543.2022.2104180
[16] M. Panzer, B. Bender, Deep reinforcement learning in production systems: a systematic literature review, Int. J. Prod. Res. 60 (2022) 4316-4341. https://doi.org/10.1080/00207543.2021.1973138
[17] N. Reinisch, F. Rudolph, S. Günther, D. Bailly, G. Hirt, Successful Pass Schedule Design in Open-Die Forging Using Double Deep Q-Learning, Processes 9 (2021) 1084. https://doi.org/10.3390/pr9071084
[18] O. Gamal, M.I.P. Mohamed, C.G. Patel, H. Roth, Data-Driven Model-Free Intelligent Roll Gap Control of Bar and Wire Hot Rolling Process Using Reinforcement Learning, IJMERR 10 (2021) 349–356. https://doi.org/10.18178/ijmerr.10.7.349-356
[19] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, Deterministic Policy Gradient Algorithms, Proceedings of the 31 st International Conference on Machine Learning, 32. Aufl, Beijing, China, 2014, 387-395