Browse > Article
http://dx.doi.org/10.36498/kbigdt.2021.6.2.169

Prediction of Agricultural Purchases Using Structured and Unstructured Data: Focusing on Paprika  

Somakhamixay Oui (충북대학교 경영정보학과)
Kyung-Hee Lee ((주)빅데이터랩스)
HyungChul Rah (충북대학교 수의학연구소)
Eun-Seon Choi (충북대학교 빅데이터 협동과정)
Wan-Sup Cho (충북대학교 경영정보학과)
Publication Information
The Journal of Bigdata / v.6, no.2, 2021 , pp. 169-179 More about this Journal
Abstract
Consumers' food consumption behavior is likely to be affected not only by structured data such as consumer panel data but also by unstructured data such as mass media and social media. In this study, a deep learning-based consumption prediction model is generated and verified for the fusion data set linking structured data and unstructured data related to food consumption. The results of the study showed that model accuracy was improved when combining structured data and unstructured data. In addition, unstructured data were found to improve model predictability. As a result of using the SHAP technique to identify the importance of variables, it was found that variables related to blog and video data were on the top list and had a positive correlation with the amount of paprika purchased. In addition, according to the experimental results, it was confirmed that the machine learning model showed higher accuracy than the deep learning model and could be an efficient alternative to the existing time series analysis modeling.
Keywords
structured data; Unstructured data; LSTM; CNN; SVR; Random Forest; XGBoost; SHAP;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Prabhu, C. S. R., Sreevallabh Chivukula, A., Mogadala, A., Ghosh, R., & Livingston, L. M. J. "Predictive Modeling for Unstructured Data. Big Data Analytics: Systems, Algorithms, Applications", (2019). 167-194. 
2 Schoen, H., Gayo-Avello, D., Takis Metaxas, P., Mustafaraj, E., Strohmaier, M., & Gloor, P. "The power of prediction with social media. Internet Research",(2013). 23(5), 528-543.    DOI
3 Schoen, H.; Gayo-Avello, D.; Metaxas, P.T.; Mustafaraj, E.; Strohmaier, M.; Gloor, P. "The power of prediction with social media". Internet Research 2013. 
4 Bahceci, O.; Alsing, O. Stock Market Prediction using Social Media Analysis. 2015. 
5 Artola, C.; Pinto, F.; de Pedraza Garcia, P. Can internet searches forecast tourism inflows? International Journal of Manpower 2015, 36, 103-116.    DOI
6 Cho, W.-S.; Cho, A.; Kwon, K.; Yoo, K.-H. Implementation of smart chungbuk tourism based on SNS data analysis. Journal of the Korean Data and Information Science Society 2015, 26, 409-418.    DOI
7 Meza, X.V.; Park, H.W. Organic Products in Mexico and South Korea on Twitter. Journal of Business Ethics 2016, 135, 587-603.    DOI
8 Yoo, D.-i. Vegetable Price Prediction Using Atypical Web-Search Data. In Proceedings of 2016 Annual Meeting, July 31-August 2, 2016, Boston, Massachusetts. 
9 Lee, S.Y. Analysis on how media report regarding FMD(Foot and mouse disease) affects households' consumption of meat product. Sogang University, Seoul, 2014. 
10 Choi, K.D.; Kang, H.-G.; Joo, H.H. Does the Harmful Information Regarding Food Safety Affect the Consumption Pattern of Consumers? - Focusing on Fukushima Nuclear Accident. Journal of Korean Economics Studies 2016, 34, 41-83. 
11 Kim, J.; Cha, M.; Lee, J.G. A Model for Nowcasting Commodity Price based on Social Media Data. Journal of Korean Institute of Information Scientists and Engineers 2017, 44, 1258-1268. 
12 Cho, Y.; Oh, E.; Cho, W.-S.; Nasridinov, A.; Yoo, K.-H.; Rah, H. Relations Between Paprika Consumption and Unstructured Big Data, and Paprika Consumption Prediction. International Journal of Contents 2019, 15, 113-119.    DOI
13 Rah, H.; Oh, E.; Yoo, D.-i.; Cho, W.-S.; Nasridinov, A.; Park, S.; Cho, Y.; Yoo, K.-H. Prediction of Onion Purchase Using Structured and Unstructured Big Data. The Journal of the Korea Contents Association 2018, 18, 30-37. 
14 Som Akhamixay, O. Predictive Modeling of the Amount Purchased Paprika Using Deep Learning and Machine Learning. Chungbuk National University, Cheongju, 2021. 
15 Seungwon Oh, Namhui Im,Sang-Hyun Lee, Min Soo Kim. "Long-term Price Prediction and Trend Analysis of Garlic Using Prophet Model." Journal of the Korean Data Analysis Society 22.6 (2020): 2325-2336.    DOI
16 Shin, S., Lee, M., & Song, S. (2018). A Prediction Model for Agricultural Products Price with LSTM Network. The Journal of the Korea Contents Association, 18(11), 416-429.    DOI
17 Im, J., Kim, W.-Y., Byoun, W.-J., & Shin, S.-J. (2018). Fruit price prediction study using artificial intelligence. The Journal of the Convergence on Culture Technology, 4(2), 197-204.    DOI
18 Mi hye Kim, Sung min Hong,Yoon Sanghoo . (2018).The Comparison of Peach Price and Trading Volume Prediction Model Using Machine Learning Technique, .Journal of The Korean Data Analysis Society, 20(6), 2933-2940.    DOI
19 Yoona Noh, Seungwon Jung, Jaeuk Moon, Eenjun Hwang. "Explainable COVID-19 Forecasting Scheme Using Attention LSTM and SHAP." SIGDB 37.2 (2021): 37-51. 
20 Jeong-min Ju, Sun-mee Kang, Ji-wung Choi, Youngwoo Han. "A Study on the Prediction of Apartment Sale Price Using Machine Learning : Focused on the Collection of Internal and External Data and Price Prediction of Korean Apartments." Proceedings of the Korea Information Processing Society Conference 27.2 (2020): 956-959. 
21 Do Hyeon Lim, Yu-rin Lee, Jaejun Lee, Kee-Young Kwahk, Hyunchul Ahn. "LightGBM-based Dropout Prediction and Its Interpretation using SHAP." Proceedings of KIIT Conference. 2021.11 (2021): 91-93. 
22 Hyerin Jeong, Park Jung hoon, Yung-Seop Lee, Changwon Lim. (2020). Visualization of Explainable Artificial Intelligence Techniques Using Variable Importance with Its Applications to Health Information Data. Journal of Health Informatics and Statistics, 45(4), 317-334.   DOI