The Journal of Bigdata (한국빅데이터학회지)

Korea Bigdata Society (한국빅데이터학회)
권호 표지

Study on Anomaly Detection Method of Improper Foods using Import Food Big data

수입식품 빅데이터를 이용한 부적합식품 탐지 시스템에 관한 연구

  • Received : 2018.11.14
  • Accepted : 2018.12.31
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Owing to the increase of FTA, food trade, and versatile preferences of consumers, food import has increased at tremendous rate every year. While the inspection check of imported food accounts for about 20% of the total food import, the budget and manpower necessary for the government's import inspection control is reaching its limit. The sudden import food accidents can cause enormous social and economic losses. Therefore, predictive system to forecast the compliance of food import with its preemptive measures will greatly improve the efficiency and effectiveness of import safety control management. There has already been a huge data accumulated from the past. The processed foods account for 75% of the total food import in the import food sector. The analysis of big data and the application of analytical techniques are also used to extract meaningful information from a large amount of data. Unfortunately, not many studies have been done regarding analyzing the import food and its implication with understanding the big data of food import. In this context, this study applied a variety of classification algorithms in the field of machine learning and suggested a data preprocessing method through the generation of new derivative variables to improve the accuracy of the model. In addition, the present study compared the performance of the predictive classification algorithms with the general base classifier. The Gaussian Naïve Bayes prediction model among various base classifiers showed the best performance to detect and predict the nonconformity of imported food. In the future, it is expected that the application of the abnormality detection model using the Gaussian Naïve Bayes. The predictive model will reduce the burdens of the inspection of import food and increase the non-conformity rate, which will have a great effect on the efficiency of the food import safety control and the speed of import customs clearance.

FTA체결의 증가, 식품교역 증가 및 소비자의 다양한 식품 선호도 등으로 농축수산물 및 가공식품의 수입량은 매년 증가하고 있는 추세이다. 수입식품의 안전성을 확인하는 정밀검사는 전체 수입식품건수 대비 20%정도를 차지하고 계속 증가하고 있는 반면에 정부의 수입안전관리에 필요한 예산과 인력은 그 한계점에 다다르고 있다. 수입식품 안전사고가 발생하게 되면 막대한 사회적, 경제적 손실을 야기할 수 있으므로 수입식품의 수입허용여부를 정확하게 예측하여 선제 대응하는 것은 수입안전관리의 효율성과 경제성을 획기적으로 높일 수 있게 된다. 식품분야에서는 이미 엄청난 양의 정형 데이터가 과거로부터 쌓여 왔으며 이에 대한 충분한 분석을 통한 활용은 아직은 부족한 것이 현실이다. 전체 수입건수와 중량 중에서 차지하는 가공식품의 비중은 평균 75%에 달하고 있어 식품분야에서도 빅데이터의 분석, 분석기법의 적용 등으로 다량의 데이터로부터 의미 있는 정보를 추출하는 과학적이고 자동화된 부적합탐지시스템의 연구가 절실한 상황이다. 이러한 배경에서 본 연구는 기계학습분야의 다양한 부적합 예측 모형을 적용하였으며 예측 모형의 정확도를 개선시키기 위한 방편으로 새로운 파생변수의 생성을 통한 데이터 전처리 방안을 제시하였다. 또한 본 연구에서는 기계학습분야의 일반적인 기저 분류기를 적용하여 예측 모형의 성능을 비교하였으며 여러 기저분류기 중 Gaussian Naïve Bayes예측 모형이 수입식품의 부적합을 탐지하여 예측하는 가장 좋은 성과를 보여주었다. 향후 Gaussian Naïve Bayes 예측 모형을 이용한 부적합 탐지 모형을 적용하여 수입식품의 정밀검사 비중을 낮추고 부적합률을 제고시킴으로써 수입안전관리 국가사무의 효율성과 수입통관의 신속성에 지대한 효과를 거둘 수 있으리라 기대한다.

Keywords

References

  1. Allen, Bradley P., "Case-Based Reasoning: Business Applications", Communications of the ACM, Vol.37, No.3(1994), 40-42. https://doi.org/10.1145/175247.175250
  2. Barandela, R., V. Garcia, E. Rangel, and J. S. Sanchez, "Strategies for Learning in Class Imbalance Problems", Pattern Recognition, Vol.36, No.3(2003), 849-865. https://doi.org/10.1016/S0031-3203(02)00257-1
  3. Beaver, William H., "Financial Ratios as Predictors of Failure", Journal of Accounting Research, Vol.4, No.3(1996), 71-111.
  4. Cao, Fuyuan, Jiye Liang, Deyu Li, and Xingwang Zhao, "A Weighting K-Modes Algorithm for Subspace Clustering of Categorical Data", Neurocomputing. Vol.108(2013) 23-30. https://doi.org/10.1016/j.neucom.2012.11.009
  5. Chang, D.S. and S.H. Lee, "A Study on the Us's Safety Control System for the Imported Food: Focused on the Processed Food", The Journal of International Commerce, Vol.31, No.4(2016), 325-50.
  6. Derrac, Joaquín, Chris Cornelis, Salvador García, and Francisco Herrera, "Enhancing Evolutionary Instance Selection Algorithms by Means of Fuzzy Rough Set Based Feature Selection", Information Sciences. Vol.186, No.1(2012), 73-92. https://doi.org/10.1016/j.ins.2011.09.027
  7. Fawcett, Tom, "An Introduction to Roc Analysis", PATTERN RECOGNITION LETTERS, Vol.27, No.8(2006), 861-74. https://doi.org/10.1016/j.patrec.2005.10.010
  8. Galar, M., A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera, "A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches", IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol.42, No.4(2012), 463-84. https://doi.org/10.1109/TSMCC.2011.2161285
  9. Ganganwar, Vaishali, "An Overview of Classification Algorithms for Imbalanced Datasets", International Journal of Emerging Technology and Advanced Engineering, Vol.2, No.4(2012), 42-47.
  10. Garcia, V., A. I. Marques, and J. S. Sanchez, "On the Use of Data Filtering Techniques for Credit Risk Prediction with Instance-Based Models", Expert Systems With Applications, Vol.39, No.18(2012), 267-76.
  11. Jin, Huang, and C. X. Ling, "Using Auc and Accuracy in Evaluating Learning Algorithms", IEEE Transactions on Knowledge and Data Engineering, Knowledge and Data Engineering, IEEE Transactions on, IEEE Trans. Knowl. Data Eng,Vol. 17, No.3(2005), 299-310.
  12. Jo, Taeho, and Nathalie Japkowicz, "Class Imbalances Versus Small Disjuncts", SIGKDD Explor. Newsl, Vol. 6, No.1(2004), 40-49. https://doi.org/10.1145/1007730.1007737
  13. Kang, P.S., H.J. Lee and S.Z. Cho, " Svm Ensemble Techniques for Class Imbalance Problem", KOREA INFORMATION SCIENCE SOCIETY, Vol.31, No.2(2004), 706-708.
  14. Kim, U.M. and T.H. Hong, "The Prediction of Customers based on Case Based Reasoning with Weighted Factors for imbalanced Data Sets", The Journal of Information Systems, Vo1.21, No.1(2014), 29-45.
  15. Koufakou, Anna, and Michael Georgiopoulos, "A Fast Outlier Detection Strategy for Distributed High-Dimensional Data Sets with Mixed Attributes", DATA MINING AND KNOWLEDGE DISCOVERY, Vol.20, No.2(2010) 259-89. https://doi.org/10.1007/s10618-009-0148-z
  16. Lee, J.S. and J.G. Kwon, "A Hybrid Svm Classifier for Imbalanced Data Sets", Journal of Intelligence and Information Systems. Vol.19, No.2(2013), 125-40. https://doi.org/10.13088/jiis.2013.19.2.125
  17. Mac Namee, B., P. Cunningham, S. Byrne, and O. I. Corrigan, "The Problem of Bias in Training Data in Regression Problems in Medical Decision Support", ARTIFICIAL INTELLIGENCE IN MEDICINE, Vol.24, No.1(2002), 51-70. https://doi.org/10.1016/S0933-3657(01)00092-6
  18. Min, S.H., "Bankruptcy prediction using the improved bagging ensemble algorithm", Journal of Intelligence and Information Systems. Vol.20, No.4(2014), 121-139. https://doi.org/10.13088/jiis.2014.20.4.121
  19. Ohlson, James A., "Financial Ratios and the Probabilistic Prediction of Bankruptcy", Journal of Accounting Research, Vol.18, No.1(1980), 109-31. https://doi.org/10.2307/2490395
  20. Otey, M. E., A. Ghoting, and S. Parthasarathy, "Fast Distributed Outlier Detection in Mixed-Attribute Data Sets", DATA MINING AND KNOWLEDGE DISCOVERY, Vol.12, No.2-3 (2006), 203-28. https://doi.org/10.1007/s10618-005-0014-6
  21. Pai, Hao-Ting, Fan Wu, and Pei-Yun S. Hsueh, "A Relative Patterns Discovery for Enhancing Outlier Detection in Categorical Data", DECISION SUPPORT SYSTEMS, Vol.67(2014), 90-99. https://doi.org/10.1016/j.dss.2014.08.006
  22. Park, M. S., H. N. Kim, and G. J. Bahk, "The Analysis of Food Safety Incidents in South Korea, 1998-2016", Food Control, Vol.81(2017), 196-199. https://doi.org/10.1016/j.foodcont.2017.06.013
  23. Said, A. M., D. D. Dominic, and B. B. Samir, "Frequent Pattern-Based Outlier Detection Measurements: A Survey", International Conference on Research and Innovation in Information Systems (ICRIIS), 2011
  24. Thompson, Christopher Glen, Rae Seon Kim, Ariel M. Aloe, and Betsy Jane Becker, "Extracting the Variance in Flation Factor and Other Multicollinearity Diagnostics from Typical Regression Results", Basic & Applied Social Psychology, Vol.39, No.2(2017), 81-90. https://doi.org/10.1080/01973533.2016.1277529
  25. Wu, X., V. Kumar, M. Steinbach, Q. J. Ross, J. Ghosh, Q. Yang, H. Motoda, G. J. McLachlan, A. Ng, B. Liu, P. S. Yu, Z. H. Zhou, D. J. Hand, and D. Steinberg, "Top 10 Algorithms in Data Mining", Knowledge and Information Systems, Vol.14, No.1(2008), 1-37. https://doi.org/10.1007/s10115-007-0114-2
  26. Zhao, Xingwang, Jiye Liang, and Fuyuan Cao, "A Simple and Effective Outlier Detection Algorithm for Categorical Data", INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, Vol.5, No.3(2014), 469-77. https://doi.org/10.1007/s13042-013-0202-4