Browse > Article
http://dx.doi.org/10.13088/jiis.2022.28.3.185

Methodology for Classifying Hierarchical Data Using Autoencoder-based Deeply Supervised Network  

Kim, Younha (Graduate School of Business IT, Kookmin University)
Kim, Namgyu (Graduate School of Business IT, Kookmin University)
Publication Information
Journal of Intelligence and Information Systems / v.28, no.3, 2022 , pp. 185-207 More about this Journal
Abstract
Recently, with the development of deep learning technology, researches to apply a deep learning algorithm to analyze unstructured data such as text and images are being actively conducted. Text classification has been studied for a long time in academia and industry, and various attempts are being performed to utilize data characteristics to improve classification performance. In particular, a hierarchical relationship of labels has been utilized for hierarchical classification. However, the top-down approach mainly used for hierarchical classification has a limitation that misclassification at a higher level blocks the opportunity for correct classification at a lower level. Therefore, in this study, we propose a methodology for classifying hierarchical data using the autoencoder-based deeply supervised network that high-level classification does not block the low-level classification while considering the hierarchical relationship of labels. The proposed methodology adds a main classifier that predicts a low-level label to the autoencoder's latent variable and an auxiliary classifier that predicts a high-level label to the hidden layer of the autoencoder. As a result of experiments on 22,512 academic papers to evaluate the performance of the proposed methodology, it was confirmed that the proposed model showed superior classification accuracy and F1-score compared to the traditional supervised autoencoder and DNN model.
Keywords
Deep Learning; Hierarchical Classification; Deeply Supervised Network; Autoencoder; Auxiliary Classifier;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 Secker, A. D., Davies, M. N., Freitas, A. A., Timmis, J., Mendao, M., & Flower, D. R. (2007). An experimental comparison of classification algorithms for hierarchical prediction of protein function. Expert Update (Magazine of the British Computer Society's Specialist Group on AI), 9(3), 17-22.
2 한국과학기술정보연구원. (2021, 09.08). 국내 논문 전문 텍스트 데이터셋. 한국과학기술정보연구원. https://doi.org/10.23057/38.   DOI
3 Le, L., Patterson, A., & White, M. (2018). Supervised autoencoders: improving generalization performance with unsupervised regularizers. Advances in neural information processing systems, 31.
4 Mishra, D., Chaudhury, S., Sarkar, M., & Soin, A. S. (2018). Ultrasound image segmentation: a deeply supervised network with attention to boundaries. IEEE Transactions on Biomedical Engineering, 66(6), 1637-1648.
5 Phyu, T. N. (2009). Survey of classification techniques in data mining. Proceedings of the international multiconference of engineers and computer scientists, 1(5).
6 Zhu, X., & Bain, M. (2017). B-CNN: branch convolutional neural network for hierarchical classification. arXiv preprint arXiv:1709.09890.
7 김동규, 이동욱, 박장원, 오성우, 권성준, 이인용, & 최동원. (2022). KB-BERT: 금융 특화 한국어 사전학습 언어모델과 그 응용. 지능정보연구, 28(2), 191-206   DOI
8 Vlasenko, B., Prasad, R., & Magimai.-Doss, M. (2021). Fusion of acoustic and linguistic information using supervised autoencoder for improved emotion recognition. Proceedings of the 2nd on Multimodal Sentiment Analysis Challenge, 51-59.
9 Wang, J., & Perez, L. (2017). The effectiveness of data augmentation in image classification using deep learning. Convolutional Neural Networks Vis. Recognit, 11, 1-8.
10 Zhang, G. P. (2000). Neural networks for classification: a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 30(4), 451-462.   DOI
11 임소라, & 권용진. (2017). 특허문서 필드의 기능 적 특성을 활용한 IPC 다중 레이블 분류. 인터넷정보학회논문지, 18(1), 77-88.   DOI
12 Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 580-587.
13 김무성, & 김남규. (2021). 다중 레이블 분류의 정확도 향상을 위한 스킵 연결 오토인코더 기반 레이블 임베딩 방법론. 지능정보연구, 27(3), 175-197.   DOI
14 신병진, 이종훈, 한상진, & 박충식. (2021). ICT 인프라 이상탐지를 위한 조건부 멀티모달 오토인코더에 관한 연구. 지능정보연구, 27(3), 57-73.   DOI
15 유태우, 김윤욱, 정하민, 유현수, & 안용학. (2018). 효율적인 사물 이미지 분류를 위한 계층적 이미지 분류 체계의 설계 및 구현. 융합보안논문지, 18(3), 53-59.
16 이상아, & 신효필. (2020). 감정 분석을 위한 BERT 사전학습모델과 추가 자질 모델의 결합. 한국정보과학회 학술발표논문집, 275-277.
17 한국과학기술정보연구원. (2022, 04.04). 논문 연구분야 분류 데이터. 한국과학기술정보연구원. https://doi.org/10.23057/50.   DOI
18 Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2006). Greedy layer-wise training of deep networks. Advances in neural information processing systems, 19.
19 Chen, Y., Wang, Y., Gu, Y., He, X., Ghamisi, P., & Jia, X. (2019). Deep learning ensemble for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(6), 1882-1897.   DOI
20 Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 4171-4186
21 Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., & Heng, P. A. (2016). 3D deeply supervised network for automatic liver segmentation from CT volumes. International conference on medical image computing and computerassisted intervention, 149-157.
22 Lee, C. Y., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015). Deeply-supervised nets. Artificial intelligence and statistics, 562-570.
23 Li, R., Wang, X., Huang, G., Yang, W., Zhang, K., Gu, X., Tran, S. N., Garg, S., Alty, J., & Bai, Q. (2022). A comprehensive review on deep supervision: theories and applications. arXiv preprint arXiv:2207.02376.
24 Parida, S., Villatoro-Tello, E., Kumar, S., Motlicek, P., & Zhan, Q. (2020). Idiap submission to swiss-german language detection shared task. SwissText/KONVENS.
25 Pereira, G. T., Santos, B. Z., & Cerri, R. (2018). A genetic algorithm for transposable elements hierarchical classification rule induction. 2018 IEEE Congress on Evolutionary Computation (CEC), 1-8.
26 Romero, M., Finke, J., & Rocha, C. (2022). A top-down supervised learning approach to hierarchical multi-label classification in networks. Applied Network Science, 7(1), 1-17.   DOI
27 Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1986). Parallel distributed processing: explorations in the microstructure of cognition. Cambridge, MA, USA: MIT Press.
28 Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
29 Kaikhah, K. (2004). Automatic text summarization with neural networks. 2004 2nd International IEEE Conference on'Intelligent Systems'. Proceedings (IEEE Cat. No. 04EX791), 1, 40-44.
30 Silla, C. N., & Freitas, A. A. (2011). A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22(1), 31-72.   DOI
31 Shen, Z., Liu, Z., Li, J., Jiang, Y. G., Chen, Y., & Xue, X. (2017). Dsod: learning deeply supervised object detectors from scratch. Proceedings of the IEEE international conference on computer vision, 1919-1927.
32 Ullah, M. A., Marium, S. M., Begum, S. A., & Dipa, N. S. (2020). An algorithm and method for sentiment analysis using the text and emoticon. ICT Express, 6(4), 357-360.   DOI
33 Cai, L., & Hofmann, T. (2004). Hierarchical document categorization with support vector machines. In Proceedings of the thirteenth ACM international conference on Information and knowledge management, 78-87.
34 Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008). Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th international conference on Machine learning, 1096-1103.