The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Document classification using a deep neural network in text mining

텍스트 마이닝에서 심층 신경망을 이용한 문서 분류

  • Lee, Bo-Hui (Department of Advertising and Public Relations, Silla University) ;
  • Lee, Su-Jin (Department of Statistics, Pusan National University) ;
  • Choi, Yong-Seok (Department of Statistics, Pusan National University)
  • Received : 2020.06.03
  • Accepted : 2020.08.21
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

The document-term frequency matrix is a term extracted from documents in which the group information exists in text mining. In this study, we generated the document-term frequency matrix for document classification according to research field. We applied the traditional term weighting function term frequency-inverse document frequency (TF-IDF) to the generated document-term frequency matrix. In addition, we applied term frequency-inverse gravity moment (TF-IGM). We also generated a document-keyword weighted matrix by extracting keywords to improve the document classification accuracy. Based on the keywords matrix extracted, we classify documents using a deep neural network. In order to find the optimal model in the deep neural network, the accuracy of document classification was verified by changing the number of hidden layers and hidden nodes. Consequently, the model with eight hidden layers showed the highest accuracy and all TF-IGM document classification accuracy (according to parameter changes) were higher than TF-IDF. In addition, the deep neural network was confirmed to have better accuracy than the support vector machine. Therefore, we propose a method to apply TF-IGM and a deep neural network in the document classification.

문서-용어 빈도행렬은 그룹정보가 존재하는 문서들의 용어를 추출한 것으로 일반적인 텍스트 마이닝에서의 자료이다. 본 연구에서는 연구 분야 성격에 따른 문서 분류를 위해 문서-용어 빈도행렬을 생성하고, 전통적인 용어 가중치 함수인 TF-IDF와 최근 잘 알려진 용어 가중치 함수인 TF-IGM을 적용하였다. 또 용어 가중치가 적용된 문서-용어 가중행렬에 문서분류 정확도 향상을 위해 핵심어를 추출하여 문서-핵심어 가중행렬을 생성하였다. 핵심어가 추출된 행렬을 바탕으로, 심층 신경망을 이용해 문서를 분류하였다. 심층 신경망에서 최적의 모델을 찾기 위해 매개변수인 은닉층과 은닉노드수를 변화해가며 문서 분류 정확도를 확인하였다. 그 결과 8개의 은닉층을 가진 심층 신경망 모델이 가장 높은 정확도를 보였으며 매개변수 변화에 따른 모든 TF-IGM 문서 분류 정확도가 TF-IDF 문서 분류 정확도보다 높은 것을 확인하였다. 또한 개별 범주에 대한 문서 분류 분석 결과를 서포트 벡터 머신과 비교했을 때 심층 신경망이 대부분의 결과에서 더 좋은 정확도를 보임을 확인하였다.

Keywords

References

  1. Bengio, Y., Courville, A., and Vincent, P. (2013). Representation learning: a review and new perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1798-1828. https://doi.org/10.1109/TPAMI.2013.50
  2. Chen, K., Zhang, Z., Long, J., and Zhang, H. (2016). Turning from TF-IDF to TF-IGM for term weighting in text classification, Expert System with Applications, 66, 245-260. https://doi.org/10.1016/j.eswa.2016.09.009
  3. Cho, H. Y., Kim, Y. H., and Im, H. H. (2018). Forecast of wind-shear alert using deep neural networks, Asia-Pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, 8, 749-757.
  4. Cho, S. G., Cho, J. H., and Kim, S. B. (2015). Discovering meaningful trends in the inaugural addresses of United States Presidents Via text mining, Journal of Korean Institute of Industrial Engineers, 41, 453-460. https://doi.org/10.7232/JKIIE.2015.41.5.453
  5. Choi, M. J. (2017). Forecasting the number of tourists in Jeju Island using deep learning algorithm (Master thesis), Hanyang University.
  6. Jeon, E. K. (2018). Implementation of arrhythmia classification system using deep neural network (Master thesis), Soonchunhyang University.
  7. Jeong, H. Y., Shin, S. M., and Choi, Y. S. (2019). Comparison of term weighting schemes for document classification, The Korean Journal of Applied Statistics, 32, 265-276. https://doi.org/10.5351/KJAS.2019.32.2.265
  8. Joo, W. K. (2018). Automatic classification method for atypical texts that include structure information using deep learning (Doctoral thesis), Chungnam National University.
  9. Jung, M. J. (2017). A study on clustering methods for proximity data in text mining (Master thesis), Pusan National University.
  10. Jung, M. J., Shin, S. M., and Choi, Y. S. (2019). Creation and clustering of proximity data for text data analysis, The Korean Journal of Applied Statistics, 32, 451-462. https://doi.org/10.5351/KJAS.2019.32.3.451
  11. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition, Neural Computation, 1, 541-551. https://doi.org/10.1162/neco.1989.1.4.541
  12. Lee, D. J., Yeon, J. H., Hwang, I. B., and Lee, S. G. (2010). KKMA: a tool for utilizing Sejong corpus based on relational database, Communications of the Korean Institute of Information Scientists and Engineers, 16, 1046-1050.
  13. Lee, G. G., Ha, H. S., Hong, H. G., and Kim, H. B. (2018). Exploratory research on automating the analysis of scientific argumentation using machine learning, Journal of the Korean Association for Science Education, 38, 219-234. https://doi.org/10.14697/JKASE.2018.38.2.219
  14. Lee, M. R. and Bae, H. K. (2002). Design of keyword extraction system using TFIDF, The Korean Society for Cognitive Science, 13, 1-11.
  15. Satopaa, V., Albrecht, J., Irwin, D., and Raghavan, B. (2011). Finding a "kneedle" in a haystack: detecting knee points in system behavior, Distributed Computing Systems Workshops (ICDCSW) 2011 31st International Conference on, IEEE, 166-171.
  16. Schmidhuber, J. (2015). Deep learning in neural networks: an overview, Neural Networks, 61, 85-117. https://doi.org/10.1016/j.neunet.2014.09.003