산업경영시스템학회지 (Journal of Korean Society of Industrial and Systems Engineering)

  • 제47권2호
  • /
  • Pages.21-29
  • /
  • 2024
  • /
  • 2005-0461(pISSN)
  • /
  • 2287-7975(eISSN)
한국산업경영시스템학회 (Society of Korea Industrial and System Engineering)
권호 표지
DOI QR코드

DOI QR Code

NIST AI 위험 관리 프레임워크 적용: NTIS 데이터베이스 분석의 MAP, MEASURE, MANAGE 접근 사례 연구

Applying NIST AI Risk Management Framework: Case Study on NTIS Database Analysis Using MAP, MEASURE, MANAGE Approaches

  • 임정선 (한국과학기술정보연구원) ;
  • 배성훈 (한국국토정보공사 공간정보연구원) ;
  • 권태훈 (한국과학기술정보연구원)
  • 투고 : 2024.02.16
  • 심사 : 2024.03.06
  • 발행 : 2024.06.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Fueled by international efforts towards AI standardization, including those by the European Commission, the United States, and international organizations, this study introduces a AI-driven framework for analyzing advancements in drone technology. Utilizing project data retrieved from the NTIS DB via the "drone" keyword, the framework employs a diverse toolkit of supervised learning methods (Keras MLP, XGboost, LightGBM, and CatBoost) enhanced by BERTopic (natural language analysis tool). This multifaceted approach ensures both comprehensive data quality evaluation and in-depth structural analysis of documents. Furthermore, a 6T-based classification method refines non-applicable data for year-on-year AI analysis, demonstrably improving accuracy as measured by accuracy metric. Utilizing AI's power, including GPT-4, this research unveils year-on-year trends in emerging keywords and employs them to generate detailed summaries, enabling efficient processing of large text datasets and offering an AI analysis system applicable to policy domains. Notably, this study not only advances methodologies aligned with AI Act standards but also lays the groundwork for responsible AI implementation through analysis of government research and development investments.

키워드

과제정보

This research received partial funding from the program (K24L3M2C4) provided by the Korea Institute of Science and Technology Information (KISTI) in South Korea.

참고문헌

  1. Ahn, J.M., Kim, J.W., and Kim K.H., Ensemble Machine Learning of Gradient Boosting (XGBoost, LightGBM, CatBoost) and Attention-Based CNN-LSTM for Harmful Algal Blooms Forecasting, Toxins, 2023, Vol. 15, No. 10, https://doi.org/10.3390/toxins15100608.
  2. Asmussen., C.B. and Moller, C., Smart Literature Reivew: A Practical Topic Modelling Approach to Exploratory Literature Review, Journal of Big Data, 2019, pp. 6-93.
  3. BERTopic, c-TF-IDF, 2024, Available: https://maartengr.github.io/BERTopic/getting_started/ctfidf/ctfidf.html.
  4. BERTopic, Hierarchical Topic Modeling, 2024, Available: https://maartengr.github.io/BERTopic/getting_started/hierarchicaltopics/hierarchicaltopics.html.
  5. Candice, B., Anna, C., and and Gonzalo, M.M., A Comparative Analysis of Gradient Boosting Algorithms, Artificial Intelligence, 2020, Vol. 54, pp. 1937-1967. https://doi.org/10.1007/s10462-020-09896-5
  6. Carnegie Mellon University, Carnegie Mellon University Human-Computer Interaction Institute, 2024, [Online]. Available: https://hcii.cmu.edu/
  7. ChatGPT, ChatGPT4, 2024, [Online]. Available: https://chat.openai.com/.
  8. EC(European Commission), Feasibility study for and EU monitoring mechanism on key enabling technologies, Belgium: European Commission, 2012.
  9. EC, KETs obsevatory phase ii methodology report, Belgium: European Commission, 2017.
  10. EC, Advanced Technologies for Industry - Methodological Report, Belgium: European Commission, 2021.
  11. EC, Monitoring European Industryal Ecosystems- Conceptual, Monitoring and Indicator Framework, Belgium: European Commission, 2023.
  12. EC, EU AI Act: first regulation on artificial intelligence, 2024, [Online] https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence.
  13. Hovart, M., The New Framework for EU Research and Innovation, Science, 2011, Vol. 334, pp. 1066-1068. https://doi.org/10.1126/science.1214295
  14. IBM, What is human-centered AI?, 2024, [online] https://research.ibm.com/blog/what-is-human-centeredai.
  15. ISO(International Organization for Standardization), ISO 3100: 2018 Risk-Management Guidelines, 2024, [Online]. Available: https://www.iso.org/standard/65694.html.
  16. Jeong, E.H., The Analysis of Changes in East Coast Tourism using Topic Modeling, JKIIECT, 2020, Vol. 13, No. 6, pp. 489-495. https://doi.org/10.17661/JKIIECT.2020.13.6.489
  17. Lane, J., Let's make science metrics more scientific, Nature, 2010, Vol. 464, pp. 488-489. https://doi.org/10.1038/464488a
  18. Lane, J. and Bertuzzi, S., The STAR METRICS Project: Current and Future Uses for S&E Workforce Data, 2010, [Online]. Available: https://www.nsf.gov/sbe/sosp/workforce/lane.pdf.
  19. Lim, J.S. and Kim, S.G., Quantifying Innovation Characteristics of National R&D Investment through Cross-Cutting Data Analysis: Case Study on NTIS Nanotechnology, Journal of The Korean Operations Research and Management Science Society, 2020, Vol. 45, No. 2, pp. 13-23. https://doi.org/10.7737/JKORMS.2020.45.2.013
  20. Lim, J.S., Kim, S.G., Bae, S.H., Kim, K.H., and Won, D.K., Research on Regional Spatial Information Analysis Platform about NTIS Raw Data, Journal of Cadastre & Land InformatiX, 2020, Vol. 50, No. 2, pp. 21-35.
  21. Lim, J.S., Park, J.H., and Kim, S.G., Study on Measurement of National R&D Investment Spillover: Linking Multilateral Big Data, Korea CEO Academy, 2021, Vol. 24, No. 1, pp. 311-330. https://doi.org/10.37674/CEOMS.24.1.16
  22. Lim, J.S. and Bae, S.H., Text Based Explainable AI for Monitoring National Innovations, Journal of Korean Society of Industrial and Systems Engineering, 2022, Vol. 45, No. 4, pp. 1-7. https://doi.org/10.11627/jksie.2022.45.4.001
  23. Lim, J.S., Bae, S.H., Ryu, K.H., and Kim, S.K., A Study on Human-AI Collaboration Process to Support Evidence-Based National Innovation Monitoring : Case Study on Ministry of Oceans and Fisheries, Korean Society of Industrial and Systems Engineering, 2023, Vol. 46, No. 2, pp. 22-31. https://doi.org/10.11627/jksie.2023.46.2.022
  24. National Assembly of Korea, Legislation on fostering the artificial intelligence industry and creating a foundation for trust, 2024, [Online]. Available: https://likms.assembly.go.kr/bill/billDetail.do?billId=PRC_Y2B1M0R6G2I2P1B0V2X9H4Z0X3M3J2.
  25. NIST(National Institute of Standards and Technology), AI Risk Management Frame 1.0, Washington DC: US Department of Commerce, 2023.
  26. NTIS(National Science & Technology Information Service), 2024, [Online]. Available: https://www.ntis.go.kr.
  27. OECD, Advancing accountability in AI; Governing and managing risks throughout the lifecycle for trustworthy AI.
  28. OECD, R&D Platform for Investment and Evaluation ("R&D PIE"), 2024, [Online], Available: https://oecd-op si.org/innovations/rd-platform-for-investment-and-evalu ation-rd-pie.
  29. Park, D.J., Oh, H.S., Kim, H.G., and Yoon, M., Topic Modeling Analysis Comparison for Research Topic in Korean Society of Industrial and Systems Engineering: Concentrated on Research Papers from 1978- 1999, Journal of Korean Society of Industrial and Systems Engineering, Vol. 44, No. 4, pp. 113-127.
  30. Park, J.H. and Lim, J.S., National R&D investment performance analysis based on NTIS raw data: Gyeongsangnam-do Smart Farm Industrialization Case, Regional Industry Review, 2022, Vol. 45, No. 4, pp. 243-263. https://doi.org/10.33932/rir.45.4.12
  31. Stanford University, Stanford University Human-Centered AI, 2024, [Online]. Available: https://hai.stanford.edu/.
  32. So, B.H., Enhanced Gradient Boosting for Zero-Inflated Insurance Claims and Comparative Analysis of CatBoost, XGBoost, and LightGBM, arXiv, 2023, arXiv:2307.07771.
  33. University of California Berkeley, UC Berkeley Center for Human-Compatible AI, 2024, [Online]. Available: https://humancompatible.ai/.
  34. Yoon, S.U. and Kim, M.C., Topic Modeling on Fine Dust Issues Using LDA Analysis, Journal of Energy Engineering, 2020, Vol. 29, pp. 23-29. https://doi.org/10.5855/ENERGY.2020.29.2.023