한국문헌정보학회지 (Journal of the Korean Society for Library and Information Science)

한국문헌정보학회 (Korean Society For Library And Information Science)
권호 표지
DOI QR코드

DOI QR Code

BERT 모형을 이용한 주제명 자동 분류 연구

A Study on Automatic Classification of Subject Headings Using BERT Model

  • 이용구 (경북대학교 문헌정보학과)
  • 투고 : 2023.04.21
  • 심사 : 2023.05.19
  • 발행 : 2023.05.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

이 연구는 딥러닝 기법의 전이학습 모형인 BERT를 이용하여 주제명의 자동 분류를 실험하고 그 성능을 평가하였으며, 더 나아가 주제명이 부여된 KDC 분류체계와 주제명의 범주 유형에 따른 성능을 분석하였다. 실험 데이터는 국가서지를 이용하여 주제명의 부여 횟수에 따라 6개의 데이터셋을 구축하고 분류 자질로 서명을 이용하였다. 그 결과, 분류 성능으로 3,506개의 주제명이 포함된 데이터셋(레코드 1,539,076건)에서 마이크로 F1과 매크로 F1 척도가 각각 0.6059와 0.5626 값을 보였다. 또한 KDC 분류체계에 따른 분류 성능은 총류, 자연과학, 기술과학, 그리고 언어 분야에서 좋은 성능을 보이며 종교와 예술 분야는 낮은 성능을 보였다. 주제명의 범주 유형에 따른 성능은 '식물', '법률명', '상품명'이 높은 성능을 보인 반면, '국보/보물' 유형의 주제명에서 낮은 성능을 보였다. 다수의 주제명을 포함하는 데이터셋으로 갈수록 분류기가 주제명을 제대로 부여하지 못하는 비율이 늘어나 최종 성능의 하락을 가져오기 때문에, 저빈도 주제명에 대한 분류 성능을 높이기 위한 개선방안이 필요하다.

This study experimented with automatic classification of subject headings using BERT-based transfer learning model, and analyzed its performance. This study analyzed the classification performance according to the main class of KDC classification and the category type of subject headings. Six datasets were constructed from Korean national bibliographies based on the frequency of the assignments of subject headings, and titles were used as classification features. As a result, classification performance showed values of 0.6059 and 0.5626 on the micro F1 and macro F1 score, respectively, in the dataset (1,539,076 records) containing 3,506 subject headings. In addition, classification performance by the main class of KDC classification showed good performance in the class General works, Natural science, Technology and Language, and low performance in Religion and Arts. As for the performance by the category type of the subject headings, the categories of plant, legal name and product name showed high performance, whereas national treasure/treasure category showed low performance. In a large dataset, the ratio of subject headings that cannot be assigned increases, resulting in a decrease in final performance, and improvement is needed to increase classification performance for low-frequency subject headings.

키워드

참고문헌

  1. Baek, Ji-Won & Chung, Yeon Kyoung (2014). A study on improving access & retrieval system of the National Library of Korea subject headings. Journal of the Korean Society for Information Management, 31(1), 31-51. https://doi.org/10.3743/KOSIM.2014.31.1.031
  2. Choi, Yoon Kyung & Chung, Yeon Kyoung (2014). A study on improvements for high quality in National Library of Korea subject headings list. Journal of the Korean Society for Library and Information Science, 48(1), 75-95. https://doi.org/10.4275/KSLIS.2014.48.1.075
  3. Eom, Kihong & Kim, Dae-Sik (2021). Automated classification model for online public opinions in a political arena: KoBERT based sentiment analysis. Korean Party Studies Review, 20(3), 167-191. https://doi.org/10.30992/KPSR.2021.09.20.3.167
  4. Hwang, Sangheum & Kim, Dohyun (2020). BERT-based classification model for Korean documents. The Journal of Society for e-Business Studies, 25(1), 203-214. https://doi.org/10.7838/jsebs.2020.25.1.203
  5. Kim, In hu & Kim, Seonghee (2022). Automatic classification of academic articles using BERT model based on deep learning. Journal of the Korean Society for Information Management, 39(3), 293-310. https://doi.org/10.3743/KOSIM.2022.39.3.293
  6. National Library of Korea (2021. 4. 10.). Try it: Search and summary served by artificial intelligence. Available: https://www.nl.go.kr/NL/contents/N50603000000.do?schM=view&id=38537&schBcid=normal0302
  7. Oh, Wonseok (2021). A Study on the Development of Work Support Tools for Librarians Using Artificial Intelligence Technology(979-11-6513-187-6). National Library of Korea.
  8. Park, Jinwoo, Sim, Woochul, Lee, Sanghun, Ko, Bongsoo, & Noh, Hansung (2022). A study on automatic CPC classification based on Korean patent sentence: a deep learning approach using artificial intelligence language model KorPatBERT. The Journal of Intellectual Property, 17(3), 209-256. https://doi.org/10.34122/jip.2022.17.3.209
  9. Cho, K., Van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. https://doi.org/10.48550/arXiv.1406.1078
  10. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805. https://doi.org/10.48550/arXiv.1810.04805
  11. Hochreiter, S. & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  12. Jin, Q., Dhingra, B., Cohen, W., & Lu, X. (2018). AttentionMeSH: simple, effective and interpretable automatic MeSH indexer. Proceedings of the 6th BioASQ Workshop A challenge on large-scale biomedical semantic indexing and question answering, 47-56. https://doi.org/10.18653/v1/W18-5306 
  13. Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1746-1751. https://doi.org/10.3115/v1/D14-1181
  14. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324. https://doi.org/10.1109/5.726791
  15. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., & Khudanpur, S. (2011, May). Extensions of recurrent neural network language model. In 2011 IEEE international conference on acoustics, speech and signal processing, 5528-5531. IEEE. https://doi.org/10.1109/ICASSP.2011.5947611
  16. Mork, J. G., Jimeno-Yepes, A., & Aronson, A. R. (2013). The NLM Medical Text Indexer System for Indexing Biomedical Literature. BioASQ@ CLEF, 1.
  17. Peng, S., You, R., Wang, H., Zhai, C., Mamitsuka, H., & Zhu, S. (2016). Deepmesh: deep semantic representation for improving large-scale MeSH indexing. Bioinformatics, 32(12), i70-i79. https://doi.org/10.1093/bioinformatics/btw294
  18. Reich, P. & Biever, E. J. (1991). Indexing Consistency: the Input/Output Function of Thesauri. College & Research Libraries, 52(4), 336-342. https://doi.org/10.5860/crl_52_04_336
  19. Saarti, J. (2002). Consistency of subject indexing of novels by public library professionals and patrons. Journal of Documentation, 58(1), 49-65. https://doi.org/10.1108/00220410210425403
  20. Tonta, Y. (1991). A study of indexing consistency between library of congress and british library catalogers. Library Resources & Technical Services, 35(2), 177-185.
  21. Vaswani, A., Shazeer, N. M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. Advances in neural information processing systems, 30, 6000-6010. https://doi.org/10.48550/arXiv.1706.03762
  22. You, R., Liu, Y., Mamitsuka, H., & Zhu, S. (2021). BERTMeSH: deep contextual representation learning for large-scale high-performance MeSH indexing with full text. Bioinformatics, 37(5), 684-692. https://doi.org/10.1093/bioinformatics/btaa837