International conference on construction engineering and project management (국제학술발표논문집)

  • 2024.07a
  • /
  • Pages.439-446
  • /
  • 2024
  • /
  • 2508-9048(eISSN)
Korea Institute of Construction Engineering and Management (한국건설관리학회)
권호 표지
DOI QR코드

DOI QR Code

Retrieval-Augmented Generation-based Question Answering Technology for Construction Safety

  • Minwoo Jeong (Department of Civil and Environmental Engineering, Yonsei University) ;
  • Taegeon Kim (Department of Civil and Environmental Engineering, Yonsei University) ;
  • Seokhwan Kim (Department of Civil and Environmental Engineering, Yonsei University) ;
  • Hongjo Kim (Department of Civil and Environmental Engineering, Yonsei University)
  • Published : 2024.07.29
Download PDF
⟨ Previous Next ⟩

Abstract

This study investigates the potential of Retrieval-Augmented Generation (RAG)-based Question Answering (QA) technology for accurate and relevant responses of Large Language Models (LLMs) to construction safety-related queries. Despite LLMs' advancements, their application, especially a Q&A Chatbot faces challenges due to hallucination and lack of domain-specific details. This study explores RAG's potentials to mitigate these issues by making LLM refer to external databases, such as the OSHA Field Safety and Health Manual, for generating precise and factual contents. A comparative analysis of different RAG technologies-Naïve-RAG, Rerank-RAG, and Iterative Retrieval-Generation-demonstrates their effectiveness over traditional LLM approaches. The findings highlight RAG's significance in producing structured, fact-based responses, underscoring its superiority in addressing the domain-specific informational needs regarding construction safety practices. This research marks a step forward in the application of generative AI technologies to enhance safety standards and practices within the construction industry.

Keywords

Acknowledgement

This research was conducted with the support of the "2023 Yonsei University Future-Leading Research Initiative (No. 2023-22-0114)" and the "National R&D Project for Smart Construction Technology (No. RS-2020-KA156488)" funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure and Transport, and managed by the Korea Expressway Corporation. This study referenced the codes from 'LlamaIndex [16]'. All the related codes and information can be accessed through 'LlamaIndex (https://github.com/run-llama/llama_index)'.

References

  1. P. Ghimire, K. Kim, M. Acharya, Generative AI in the Construction Industry: Opportunities & Challenges, (n.d.).
  2. M. Regona, T. Yigitcanlar, B. Xia, R.Y.M. Li, Opportunities and Adoption Challenges of AI in the Construction Industry: A PRISMA Review, Journal of Open Innovation: Technology, Market, and Complexity 8 (2022) 45. https://doi.org/10.3390/joitmc8010045.
  3. S. Paneru, I. Jeelani, Computer vision applications in construction: Current state, opportunities & challenges, Automation in Construction 132 (2021) 103940. https://doi.org/10.1016/j.autcon.2021.103940.
  4. A. Saka, R. Taiwo, N. Saka, B.A. Salami, S. Ajayi, K. Akande, H. Kazemi, GPT models in construction industry: Opportunities, limitations, and a use case validation, Developments in the Built Environment 17 (2024) 100300. https://doi.org/10.1016/j.dibe.2023.100300.
  5. V. Rawte, S. Chakraborty, A. Pathak, A. Sarkar, S.M.T.I. Tonmoy, A. Chadha, A. Sheth, A. Das, The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, 2023: pp. 2541-2573. https://doi.org/10.18653/v1/2023.emnlp-main.155.
  6. Y. Ding, M. Liu, X. Luo, Safety compliance checking of construction behaviors using visual question answering, Automation in Construction 144 (2022) 104580. https://doi.org/10.1016/j.autcon.2022.104580.
  7. A.B. Saka, L.O. Oyedele, L.A. Akanbi, S.A. Ganiyu, D.W.M. Chan, S.A. Bello, Conversational artificial intelligence in the AEC industry: A review of present status, challenges and opportunities, Advanced Engineering Informatics 55 (2023) 101869. https://doi.org/10.1016/j.aei.2022.101869.
  8. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Kuttler, M. Lewis, W. Yih, T. Rocktaschel, S. Riedel, D. Kiela, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, (2021). http://arxiv.org/abs/2005.11401 (accessed October 23, 2023).
  9. C. Brown, OSHA Field Safety and Health Manual, (n.d.).
  10. Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, Q. Guo, M. Wang, H. Wang, Retrieval-Augmented Generation for Large Language Models: A Survey, (2024). http://arxiv.org/abs/2312.10997 (accessed February 5, 2024).
  11. Brown et al., Language Models are Few-Shot Learners, (2020). http://arxiv.org/abs/2005.14165 (accessed September 19, 2023).
  12. OpenAI et al., "GPT-4 Technical Report." arXiv, Dec. 18, 2023. doi: 10.48550/arXiv.2303.08774.
  13. Llama Hub, (n.d.). https://llamahub.ai/ (accessed February 15, 2024).
  14. M. Glass, G. Rossiello, M.F.M. Chowdhury, A.R. Naik, P. Cai, A. Gliozzo, Re2G: Retrieve, Rerank, Generate, (2022). http://arxiv.org/abs/2207.06300 (accessed February 15, 2024).
  15. Z. Shao, Y. Gong, Y. Shen, M. Huang, N. Duan, W. Chen, Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy, (2023). https://doi.org/10.48550/arXiv.2305.15294.
  16. J. Liu, LlamaIndex, (2022). https://doi.org/10.5281/zenodo.1234.
  17. F. Amer, Y. Jung, M. Golparvar-Fard, Transformer machine learning language model for auto-alignment of long-term and short-term plans in construction, Automation in Construction 132 (2021) 103929. https://doi.org/10.1016/j.autcon.2021.103929.
  18. J. Zheng, M. Fischer, Dynamic prompt-based virtual assistant framework for BIM information search, Automation in Construction 155 (2023) 105067. https://doi.org/10.1016/j.autcon.2023.105067.
  19. Y. Kim, S. Bang, J. Sohn, H. Kim, Question answering method for infrastructure damage information retrieval from textual data using bidirectional encoder representations from transformers, Automation in Construction 134 (2022) 104061. https://doi.org/10.1016/j.autcon.2021.104061.
  20. J. Kim, S. Chung, S. Moon, S. Chi, Feasibility Study of a BERT-based Question Answering Chatbot for Information Retrieval from Construction Specifications, in: 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), 2022: pp. 0970-0974. https://doi.org/10.1109/IEEM55944.2022.9989625.
  21. Chowdhery et al., PaLM: Scaling Language Modeling with Pathways, (2022). https://doi.org/10.48550/arXiv.2204.02311.
  22. Touvron et al., LLaMA: Open and Efficient Foundation Language Models, (2023). https://doi.org/10.48550/arXiv.2302.13971.
  23. J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, (2019). http://arxiv.org/abs/1810.04805 (accessed July 24, 2023).
  24. F. Guzman, A. Abdelali, I. Temnikova, H. Sajjad, S. Vogel, How do Humans Evaluate Machine Translation, in: O. Bojar, R. Chatterjee, C. Federmann, B. Haddow, C. Hokamp, M. Huck, V. Logacheva, P. Pecina (Eds.), Proceedings of the Tenth Workshop on Statistical Machine Translation, Association for Computational Linguistics, Lisbon, Portugal, 2015: pp. 457-466. https://doi.org/10.18653/v1/W15-3059.
  25. C.-H. Chiang, H. Lee, Can Large Language Models Be an Alternative to Human Evaluations?, (2023). https://doi.org/10.48550/arXiv.2305.01937.
  26. A. Allam, M. Haggag, The Question Answering Systems: A Survey, International Journal of Research and Reviews in Information Sciences 2 (2012) 211-221