한국건설관리학회논문집 (Korean Journal of Construction Engineering and Management)

  • 제24권5호
  • /
  • Pages.73-82
  • /
  • 2023
  • /
  • 2005-6095(pISSN)
  • /
  • 2465-9703(eISSN)
한국건설관리학회 (Korea Institute of Construction Engineering and Management)
권호 표지
DOI QR코드

DOI QR Code

Leveraging Visibility-Based Rewards in DRL-based Worker Travel Path Simulation for Improving the Learning Performance

  • Kim, Minguk (Division of Architecture & Urban Design, Incheon National University) ;
  • Kim, Tae Wan (Division of Architecture & Urban Design, Incheon National University)
  • 투고 : 2023.05.25
  • 심사 : 2023.08.24
  • 발행 : 2023.09.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Optimization of Construction Site Layout Planning (CSLP) heavily relies on workers' travel paths. However, traditional path generation approaches predominantly focus on the shortest path, often neglecting critical variables such as individual wayfinding tendencies, the spatial arrangement of site objects, and potential hazards. These oversights can lead to compromised path simulations, resulting in less reliable site layout plans. While Deep Reinforcement Learning (DRL) has been proposed as a potential alternative to address these issues, it has shown limitations. Despite presenting more realistic travel paths by considering these variables, DRL often struggles with efficiency in complex environments, leading to extended learning times and potential failures. To overcome these challenges, this study introduces a refined model that enhances spatial navigation capabilities and learning performance by integrating workers' visibility into the reward functions. The proposed model demonstrated a 12.47% increase in the pathfinding success rate and notable improvements in the other two performance measures compared to the existing DRL framework. The adoption of this model could greatly enhance the reliability of the results, ultimately improving site operational efficiency and safety management such as by reducing site congestion and accidents. Future research could expand this study by simulating travel paths in dynamic, multi-agent environments that represent different stages of construction.

키워드

과제정보

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2021R1A2C1013188).

참고문헌

  1. Abotaleb, Ibrahim, Khaled Nassar, and Ossama Hosny. (2016). "Layout Optimization of Construction Site Facilities with Dynamic Freeform Geometric Representations." Automation in Construction, Elsevier, 66, pp. 15-28. https://doi.org/10.1016/j.autcon.2016.02.007
  2. Algfoor, Zeyad Abd, Mohd Shahrizal Sunar, and Hoshang Kolivand. (2015). "A comprehensive study on pathfinding techniques for robotics and video games." International Journal of Computer Games Technology, Hindawi, p. 7.
  3. Andayesh, Mohsen, and Farnaz Sadeghpour. (2013). "Dynamic Site Layout Planning through Minimization of Total Potential Energy." Automation in Construction, Elsevier, 31, pp. 92-102. https://doi.org/10.1016/j.autcon.2012.11.039
  4. Andayesh, Mohsen, Sadeghpour, Farnaz. (2014). "A Comparative Study of Different Approaches for Finding the Shortest Path on Construction Sites." Procedia Engineering, Elsevier, 85, pp. 33-41. https://doi.org/10.1016/j.proeng.2014.10.526
  5. Bellman, Richard. (1957). "A Markovian Decision Process." Journal of mathematics and mechanics , INDIANA UNIVERSITY, pp. 679-684.
  6. Bengio, Yoshua, Jer̂ome Louradour, Ronan Collobert, and Jason Weston. (2009). "Curriculum Learning." Proceedings of the 26th Annual International Conference on Machine Learning , ACM, Montreal, Quebec, Canada.
  7. Benjaoran, Vacharapoom, and Vachara Peansupap. (2020). "Grid-Based Construction Site Layout Planning with Particle Swarm Optimisation and Travel Path Distance." Construction Management and Economics, Taylor & Francis, 38(8), pp. 673-688.
  8. Bergstra, James, and Yoshua Bengio. (2012). "Random Search for Hyper-Parameter Optimization." Journal of Machine Learning Research, 13, pp. 281-305.
  9. El-Rayes, Khaled, and Ahmed Khalafallah. (2005). "Tradeoff between Safety and Cost in Planning Construction Site Layouts." Journal of Construction Engineering and Management, ASCE, 131(11), pp. 1186-1195. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:11(1186)
  10. El-Rayes, Khaled, and Hisham Said. (2009). "Dynamic Site Layout Planning Using Approximate Dynamic Programming." Journal of Computing in Civil Engineering, ASCE, 23(2), pp. 119-127. https://doi.org/10.1061/(ASCE)0887-3801(2009)23:2(119)
  11. Juliani, A., Berges, V.P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., and Lange, D. (2018). "Unity: A General Platform for Intelligent Agents." ArXiv Preprint ArXiv:1809.02627.
  12. Kim, H.S., Lee, H.S., Park, M.S. Chung, B.Y. and Hwang, S.J. (2016). "Automated Hazardous Area Identification Using Laborers' Actual and Optimal Routes." Automation in Construction, Elsevier, 65, pp. 21-32. https://doi.org/10.1016/j.autcon.2016.01.006
  13. Kim, M.G., Ham, Y.J. Koo, C.W., and Kim, T.W. (2023). "Simulating Travel Paths of Construction Site Workers via Deep Reinforcement Learning Considering Their Spatial Cognition and Wayfinding Behavior." Automation in Construction, Elsevier, 147, 104715.
  14. Kim, M.G., Ryu, H.G., and Kim, T.W. (2021). "A Typology Model of Temporary Facility Constraints for Automated Construction Site Layout Planning." Applied Sciences (Switzerland), MDPI, 11(3), pp. 1-21. https://doi.org/10.3390/app11031027
  15. Kumar, Srinath S., and Jack C.P. Cheng. (2015). "A BIM-Based Automated Site Layout Planning Framework for Congested Construction Sites." Automation in Construction, Elsevier, 59, pp. 24-37. https://doi.org/10.1016/j.autcon.2015.07.008
  16. Mawdesley, Michael J., Saad H. Al-jibouri, and Hongbo Yang. (2002). "Genetic Algorithms for Construction Site Layout in Project Planning." Journal of Construction Engineering and Management, ASCE, 128(5), pp. 418-426. https://doi.org/10.1061/(ASCE)0733-9364(2002)128:5(418)
  17. Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M.E., and Stone, P. (2020). "Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey." Journal of Machine Learning Research, 21, pp. 7382-7431.
  18. Rezaee, M., Shakeri, E., Ardeshir, A., and Malekitabar, H. (2021). "Optimizing Travel Distance of Construction Workers Considering Their Behavioral Uncertainty Using Fuzzy Graph Theory." Automation in Construction, Elsevier, 124, 103574.
  19. Sanad, Haytham M., Mohammad A. Ammar, and Moheeb E. Ibrahim. (2008). "Optimal Construction Site Layout Considering Safety and Environmental Aspects." Journal of Construction Engineering and Management, ASCE, 134(7), pp. 536-544. https://doi.org/10.1061/(ASCE)0733-9364(2008)134:7(536)
  20. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). "Proximal Policy Optimization Algorithms." arXiv preprint arXiv:1707.06347.
  21. Sutton, R.S., and A.G. Barto. (2018). Reinforcement Learning: An Introduction, MIT press.
  22. Wilson, David Bruce. (1996). "Generating Random Spanning Trees More Quickly than the Cover Time." Proceedings of the Annual ACM Symposium on Theory of Computing, ACM, Philadelphia PA, USA.