Journal of the Korean earth science society (한국지구과학회지)

The Korean Earth Science Society (한국지구과학회)
권호 표지
DOI QR코드

DOI QR Code

Comparison of Machine Learning Model Performance based on Observation Methods using Naked-eye and Visibility-meter

머신러닝을 이용한 안개 예측 시 목측과 시정계 계측 방법에 따른 모델 성능 차이 비교

  • Changhyoun Park (Institute of Environmental Studies, Pusan National University) ;
  • Soon-hwan Lee (Department of Earth Science Education, Pusan National University)
  • 박창현 (부산대학교 환경연구원) ;
  • 이순환 (부산대학교 지구과학교육과)
  • Received : 2023.02.06
  • Accepted : 2023.03.08
  • Published : 2023.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we predicted the presence of fog with a one-hour delay using the XGBoost DART machine learning algorithm for Andong, which had the highest occurrence of fog among inland stations from 2016 to 2020. We used six datasets: meteorological data, agricultural observation data, additional derived data, and their expanded data. The weather phenomenon numbers obtained through naked-eye observations and the visibility distances measured by visibility meters were classified as fog [1] or no-fog [0]. We set up twelve machine learning modeling experiments and used data from 2021 for model validation. We mainly evaluated model performance using recall and AUC-ROC, considering the harmful effects of fog on society and local communities. The combination of oversampled meteorological data features and the target induced by weather phenomenon numbers showed the best performance. This result highlights the importance of naked-eye observations in predicting fog using machine learning algorithms.

본 연구에서는 2016년부터 2020년까지 내륙 관측소 중 안개 최다발 지역인 안동을 대상으로 XGBoost-DART 머신러닝 알고리즘을 이용하여 1 시간 후 안개 유무를 예측하였다. 기상자료, 농업관측자료, 추가 파생자료와 각 자료를 오버 샘플링한 확장자료, 총 6개의 데이터 세트를 사용하였다. 목측으로 획득한 기상현상번호와 시정계 관측으로 측정된 시정거리 자료를 각각 안개 유[1]무[0]로 이진 범주화하였다. 총 12개의 머신러닝 모델링 실험을 설계하였고, 안개가 사회와 지역사회에 미치는 유해성을 고려하여 모델의 성능은 재현율과 AUC-ROC를 중심으로 평가하였다. 전체적으로, 오버샘플링한 기상자료와 기상현상번호 기반의 예측 목표를 조합한 실험이 최고 성능을 보였다. 이 연구 결과는 머신러닝 알고리즘을 활용한 안개 예측에 있어서, 목측으로 획득한 기상현상번호의 중요성을 암시한다.

Keywords

Acknowledgement

본 연구는 2020년도 한국연구재단 이공분야기초연구사업(2020R1I1A1A01055060)과 중견연구자 지원사업(2022R1A2C1093229)의 지원을 받아 수행된 연구임. 연구에 도움을 주신 Berkeley Coding Academy의 Corey Wade에게 감사드린다.

References

  1. Akimoto, Y., Kusaka, H., 2015, A Climatological study of fog in Japan based on event data, Atmos. Res., 11, 200-211 https://doi.org/10.1016/j.atmosres.2014.04.003
  2. Anber, U., Gentine, P., Wang, S., Sobel, A.H., 2015, Fog and rain in the Amazon, Proc. Natl. Acad. Sci., 112(37), 11473-11477 https://doi.org/10.1073/pnas.1505077112
  3. Baldocchi, D., Waller, E., 2014, Winter fog is decreasing in the fruit growing region of the central valley of California Geophys. Res. Lett., 41(9), 3251-3256 https://doi.org/10.1002/2014GL060018
  4. Bartok, A. Bott, M. Gera, 2012, Fog prediction for road traffic safety in a coastal desert region Bound.-Layer Meteorol., 145(3), 485-506 https://doi.org/10.1007/s10546-012-9750-5
  5. Bang, C. H., Lee, J. W., Hong, S. Y,, 2008, Predictability experiments of fog and visibility in local airports over Korea using the WRF model, J. Korean Soc. Atmos. Environ., 24, 92-101 (in Korean with English abstract)
  6. Benjamin, S. G., Devenyi, D., Weygandt, S. S., Brundage, K. J., Brown, J. M., Grell, G. A., Kim, D. S., Schwartz, B. E. Smirnova, T. G, Smith, T. L., Manikin, G. S., 2004, An Hourly assimilation forecast cycle: The RUC. Mon. Weather Rev., 132, 495-518. https://doi.org/10.1175/1520-0493(2004)132<0495:AHACTR>2.0.CO;2
  7. Bergot, T., Terradellas, E., Cuxart, J., Mira, A., Liechti, O.,Mueller, M., Nielsen, N. W., 2007, Intercomparison of single-column numerical models for the prediction of radiation fog, J. Appl. Meteorol. Climatol., 46, 504-521. https://doi.org/10.1175/JAM2475.1
  8. Cho, Y. K., Kim, M. O., Kim, B. C., 2000, Sea fog around the Korean peninsula, J. Appl. Meteorol., 39, 2473-2479 https://doi.org/10.1175/1520-0450(2000)039<2473:SFATKP>2.0.CO;2
  9. Castillo-Boton, C., Casillas-Perez, D., Casanova-Mateo, C., Ghimire, S., Cerro-Prada, E., Gutierrez, P.A., Deo, R.C., Salcedo-Sanz, S., 2022, Machine learning regression and classification methods for fog events prediction, Atmos. Res., 272, 106157
  10. Chen, T., Guestrin, C. (2016). XGBoost: A scalable tree boosting sytem, KDD '16: Proceedings of the 222nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794
  11. Fabbian, D., De-Dear, R., Lellyett, S., 2007, Application of artificial neural network forecasts to predict fog at Canberra international airport Weather Forecast., 22(2), 372-381 https://doi.org/10.1175/WAF980.1
  12. Guerreiro, P.M.P, Soares, P.M.M, Cardoso, R.M., Ramos, A.M., 2020, An analysis of fog in the mainland portuguese international airports Atmos., 11(11), 1239
  13. Harris, D., Harris, S., 2012, Digital design and computer architecture (2nd ed.). San Francisco, Calif.: Morgan Kaufmann. p. 129. ISBN 978-0-12-394424-5.
  14. Heo, K. Y., Ha, K. J., Mahrt, L., Shim, J. S., 2010, Comparison of advection and steam fogs: from direct observation over the sea, Atmos. Res., 98, 426-437 https://doi.org/10.1016/j.atmosres.2010.08.004
  15. Huang, D.-S., 2004, A constructive approach for finding arbitrary roots of polynomials by neural networks, IEEE Trans. Neural Netw., 15(2), 477-491 https://doi.org/10.1109/TNN.2004.824424
  16. Huang, D.-S., Ip, H.H., Chi, Z., 2004; A neural root finder of polynomials based on root moments, Neural Comput., 16(8), 1721-1762 https://doi.org/10.1162/089976604774201668
  17. Kim, E.-J., Lee, K.-Y., Park, C., Lee, S.-H., 2021, A study on fog formation and visibility estimation in waterfront area using meteorological model, J. environ. Sci. Int., 30(4), 305-320 (in Korean with English abstract) https://doi.org/10.5322/JESI.2021.30.4.305
  18. Kim, W. H., Yum, S. S., 2015, Development and validation of the coupled system of Unified Model (UM) and PArameterized FOG (PAFOG), Atmos., 25, 149-154 (in Korean with English abstract) https://doi.org/10.14191/Atmos.2015.25.1.149
  19. Kim, Y. K., Kim, H., Seo, J. W., An, H. Y., Choi, Y. H., 2017, Meteorological analysis of the sea fog in winter season on Gyeonggi bay, Yellow sea: a case study for the 106-vehicle pileup on February 11, 2015. J. Coast. Res., 79, 124-128 (in Korean with English abstract) https://doi.org/10.2112/SI79-026.1
  20. Kim, M.W., R. K.-Y. Choi, Y. S. Lee, J. Choi, D. Y. Kim, K. H. Kim, 2021, Development of deep-learning based present weather. Journal of the Korean Data And Information Science Society, 32(5), 1007-1021 (in Korean with English abstract) https://doi.org/10.7465/jkdi.2021.32.5.1007
  21. Kuncheva, L. I., Rodgriguez, J. J., 2012, A weighted voting frameowrk fro classifiers ensembles, Knowledge and Information Systems, 38, 259-275. https://doi.org/10.1007/s10115-012-0586-6
  22. Klemm, O., Schemenauer, R.S., Lummerich, A., Cereceda, P., Marzol, V. Corell, D., van Heerden, J., Reinhard, D., Gherezghiher, T., Olivier, J., Osses, P., Sarsour, J., Frost, E., Estrela, M.J., Valiente, J.A., Fessehaye, G.M., 2012, Fessehaye Fog as a fresh-water resource: overview and perspectives AMBIO, 41, 221-234 https://doi.org/10.1007/s13280-012-0247-8
  23. Knapp, D. I., 1996, Development of a surface visibility algorithm for worldwide use with mesoscale model output. In conference of weather analysis and forecasting, Amer. Meteor. Soc., 15, 83-86.
  24. KoROAD, 2020, Statistical analysis of traffic accidents, Report, No. 2020-0202-006, Korea Road traffic authority, Wonju, Korea
  25. Lee, Y.-S., Choi, R. K.-Y., Kim, K.-H., Pa?, S. -H., Nam, H.-J. and Kim, S.-B., 2019, Improvement of automatic present weather observation with in situ visibility and humidity measurements, Atmos. Korean. Meteorol. Soc.m 29(4), 439-450 (in Korean with English abstract)
  26. Li, B., Du, J., Zhang, X.-P., 2016a, Feature extraction using maximum nonparametric margin projection, Neurocomputing, 188, 225-232 https://doi.org/10.1016/j.neucom.2014.11.105
  27. Li, B., Lei, L., Zhang, X.-P., 2016b, Constrained discriminant neighborhood embedding for high dimensional data feature extraction, Neurocomputing, 173, 137-144 https://doi.org/10.1016/j.neucom.2015.01.099
  28. Li, B. et al., 2019, Robust dimensionality reduction via feature space to feature space distance metric learning, IEEE Trans. Neural Netw., 112, 1-14 https://doi.org/10.1016/j.neunet.2019.01.001
  29. Mun, S. H., Lee, S. H., 2013, A Study on the change of fog frequency and duration hours in South Korea, J. Clim. Res. inst., 8(2), 93-104. https://doi.org/10.14383/cri.2013.8.2.93
  30. Miao, K.-C., Han, T.-T., Yao, Y.-Q., Lu, H., Chen, P., Wang, B., Zhang, J., 2020, Application of LSTM for short term fog forecasting based on meteorological elements, Neurocomputing, 408, 285-291 https://doi.org/10.1016/j.neucom.2019.12.129
  31. Montecinos, S., Carvajal, D., Cereceda, P., Concha, M.,2018, Collection efficiency of fog events Atmos. Res., 209,163-169 https://doi.org/10.1016/j.atmosres.2018.04.004
  32. Park, J. S., Lim, Y. K., Kim, K. R., Cho, C. B., Jang, J. Y., Kang, M. S., Kim, B. J., 2015, Atmospheric characteristics of fog incidents at the Nakdong river: Case study in Gangjeong-Goryung weir, J. Environ. Sci. Int., 24, 657-670 (in Korean with English abstract) https://doi.org/10.5322/JESI.2015.24.5.657
  33. Peng, Y., Abdel-Aty, M., Lee, J., Zou, Y., 2018, Analysis of the impact of fog-related reduced visibility on traffic parameters J. Transp. Eng. Part A, 144(2), Article 04017077
  34. Rashmi K. V., Gilad-Bachrach, R., 2015, DART: Dropouts Meet Multiple Additive Regression Trees in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS), San Diego, CA, USA. JMLR: W&CP vol.38
  35. Shim, H. N., Lee, Y. H., 2017, Influence of local wind on occurrence of fog at inland area, Atmos., 27, 213-224 (in Korean with English abstract)
  36. Shrestha, S., Moore, G.A., Peel, M.C., 2018, Trends in winter fog events in the Terai region of Nepal Agric. Forest Meteorol., 259, 118-130 https://doi.org/10.1016/j.agrformet.2018.04.018
  37. Sohn, 2010, Binary forecast model for fog occurrence over South Korea in the sprinttime, J. Korean Data Soc., 12(2), 949-960
  38. Tardif, R., 2007, The impact of vertical resolution in the explicit numerical forecasting of radiation fog: A case study, Pure Appl. Geophys., 164, 1221-1240 https://doi.org/10.1007/s00024-007-0216-5
  39. Tardif, R., Rasmussen, M., 2007, Event-based climatology and typology of fog in the New York city region, J. Appl. Meteorol. Climatol., 46, 1141-1168. https://doi.org/10.1175/JAM2516.1
  40. Wu, Y., Abdel-Aty, M., Lee, J., 2018, Crash risk analysis during fog conditions using real-time traffic data Accid. Anal. Prev., 114, 4-11 https://doi.org/10.1016/j.aap.2017.05.004
  41. Wantuch, F., 2001, Visibility and fog forecasting based on decision tree method, Idojaras, 105, 29-38.
  42. Zhu, B., 2019, Short_Term Photovoltaic Power Output Precition Based on k-Fold Cross-Validation and an Ensemble Model, " MIDPI Ergies