Journal of the Korean Society for Library and Information Science (한국문헌정보학회지)

Korean Society For Library And Information Science (한국문헌정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Comparative Study on Topic Modeling of LDA, Top2Vec, and BERTopic Models Using LIS Journals in WoS

LDA, Top2Vec, BERTopic 모형의 토픽모델링 비교 연구 - 국외 문헌정보학 분야를 중심으로 -

  • 이용구 (경북대학교 사회과학대학 문헌정보학과) ;
  • 김선욱 (대구가톨릭대학교 사회과학대학 문헌정보학과)
  • Received : 2023.12.23
  • Accepted : 2024.01.29
  • Published : 2024.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study is to extract topics from experimental data using the topic modeling methods(LDA, Top2Vec, and BERTopic) and compare the characteristics and differences between these models. The experimental data consist of 55,442 papers published in 85 academic journals in the field of library and information science, which are indexed in the Web of Science(WoS). The experimental process was as follows: The first topic modeling results were obtained using the default parameters for each model, and the second topic modeling results were obtained by setting the same optimal number of topics for each model. In the first stage of topic modeling, LDA, Top2Vec, and BERTopic models generated significantly different numbers of topics(100, 350, and 550, respectively). Top2Vec and BERTopic models seemed to divide the topics approximately three to five times more finely than the LDA model. There were substantial differences among the models in terms of the average and standard deviation of documents per topic. The LDA model assigned many documents to a relatively small number of topics, while the BERTopic model showed the opposite trend. In the second stage of topic modeling, generating the same 25 topics for all models, the Top2Vec model tended to assign more documents on average per topic and showed small deviations between topics, resulting in even distribution of the 25 topics. When comparing the creation of similar topics between models, LDA and Top2Vec models generated 18 similar topics(72%) out of 25. This high percentage suggests that the Top2Vec model is more similar to the LDA model. For a more comprehensive comparison analysis, expert evaluation is necessary to determine whether the documents assigned to each topic in the topic modeling results are thematically accurate.

이 연구는 토픽모델링 모형인 LDA, Top2Vec, BERTopic을 대상으로 실험데이터에서 토픽을 추출하고, 그 결과를 비교 분석함으로써 각각의 모형 간의 특성과 차이를 파악하는데 목적이 있다. 실험데이터는 Web of Science(WoS)에 등재된 문헌정보학 분야 학술지 85종에 게재된 논문 55,442편을 대상으로 하였다. 실험 과정으로 우선 각 모형의 파라미터를 기본값 그대로 이용하여 1차 토픽모델링 결과를 얻었고, 최적의 토픽 수를 설정하여 각 모형의 2차 토픽모델링 결과를 얻었으며, 이들을 각 모형과 단계별로 비교분석하였다. 1차 토픽모델링 단계에서는 LDA, Top2Vec, BERTopic 모형이 각각 100개, 350개, 550개의 토픽을 생성하여 세 모형은 각각 매우 다른 크기의 토픽 개수를 가져왔으며, LDA 모형에 비해 Top2Vec이나 BERTopic 모형이 토픽을 3배, 5배 더 세분화하였다. 또한 세 모형은 토픽 당 문서 수의 평균이나 표준편차에서도 많은 차이가 났다. 구체적으로 LDA 모형은 비교적 적은 수의 토픽에 많은 문서를 부여하는 반면, BERTopic 모형은 반대의 경향을 보였다. 25개의 토픽 수를 생성하는 2차 토픽모델링 단계에서는 다른 모형에 비해 Top2Vec 모형이 평균적으로 토픽 당 많은 문서를 부여하고 토픽간에 고르게 문서를 할당하여 상대적으로 편차가 작았다. 또한 모형간의 유사 토픽의 생성여부를 비교하면, LDA와 Top2Vec 모형이 전체 25개 중에 18개(72%)의 공통된 토픽을 생성하여 BERTopic 모형에 비해 두 모형이 더 유사한 결과를 보였다. 향후 토픽모델링 결과에서 각 토픽과 부여된 문서들이 주제적으로 올바르게 형성되었는지에 대한 전문가의 평가를 통해 보다 완전한 분석이 필요하다.

Keywords

References

  1. Kim, SeonWook & Yang, Kiduk (2022). Topic model augmentation and extension method using LDA and BERTopic. Journal of the Korean Society for Information Management, 39(3), 99-132. http://doi.org/10.3743/KOSIM.2022.39.3.099
  2. Kim, SeonWook, Yang, Kiduk, & Lee, HyeKyung (2022). Analysis of research topic trend in library and information science using dynamic topic modeling. Journal of Korean Library and Information Science Society, 53(2), 265-284. http://doi.org/10.16981/kliss.53.2.202206.265
  3. Kim, Tae Kyung & Kim, Changsik (2018). Research trends analysis of information security using text mining. Journal of the Korea Society of Digital Industry and Information Management, 14(2), 19-25. http://dx.doi.org/10.17662/ksdim.2018.14.2.019
  4. Lee, Ji-Yong, Choi, You Lee, Kim, Dae Geon, & Lee, Seungbak (2022). Types of violence appearing in the sports field: case law analysis using text mining. The Korean Journal of Physical Education, 61(5), 43-54. http://dx.doi.org/10.23949/kjpe.2022.09.61.5.4
  5. Lim, Jeonghoon (2022). Analysis of research trends in information literacy education using keyword network analysis and topic modeling. Journal of the Korean Society for Information Management, 39(4), 23-48. http://dx.doi.org/10.3743/KOSIM.2022.39.4.023
  6. Park, Jahyun & Song, Min (2013). A study on the research trends in Library & Information Science in Korea using topic modeling. Journal of the Korean Society for Information Management, 30(1), 7-32. https://doi.org/10.3743/KOSIM.2013.30.1.007
  7. Park, JunHyeong & Oh, Hyo-Jung (2017). Comparison of topic modeling methods for analyzing research trends of archives management in Korea: Focused on LDA and HDP. Journal of Korean Library and Information Science Society, 48(4), 235-258. https://doi.org/10.16981/kliss.48.4.201712.235
  8. Ali, I. & Naeem, M. A. (2022). Identifying and profiling user interest over time using social data. In 2022 24th International Multitopic Conference (INMIC), 1-6. https://doi.org/10.1109/INMIC56986.2022.9972955
  9. Angelov, D. (2020). Top2Vec: Distributed representations of topics. arXiv preprint arXiv:2008.09470. https://doi.org/10.48550/arXiv.2008.09470
  10. Blei, D. & Lafferty, J. (2005). Correlated topic models. Advances in Neural Information Processing Systems, 18, 147-154.
  11. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993-1022.
  12. Chen, A. T., Sheble, L., & Eichler, G. (2013). Topic modeling and network visualization to explore patient experiences. In Visual Analytics in Healthcare Workshop 2013.
  13. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391-407. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
  14. Dur, B. I. U. (2014). Data visualization and infographics in visual communication design education at the age of information. Journal of Arts and Humanities, 3(5), 39-50. https://doi.org/10.18533/journal.v3i5.460
  15. Egger, R. & Yu, J. (2022). A topic modeling comparison between LDA, NMF, Top2Vec, and BERTopic to demystify twitter posts. Frontiers in Sociology, 7, 886498. https://doi.org/10.3389/fsoc.2022.886498
  16. Gao, Q., Huang, X., Dong, K., Liang, Z., & Wu, J. (2022). Semantic-enhanced topic evolution analysis: a combination of the dynamic topic model and word2vec. Scientometrics, 127, 1543-1563. https://doi.org/10.1007/s11192-022-04275-z
  17. Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794. https://doi.org/10.48550/arXiv.2203.05794
  18. Hofmann, T. (1999). Probabilistic latent semantic indexing. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 50-57. https://doi.org/10.1145/3130348.3130370
  19. Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., & Zhao, L. (2019). Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey. Multimedia Tools and Applications, 78(11), 15169-15211. https://doi.org/10.1007/s11042-018-6894-4
  20. Jing, X. Y., Zhang, D., & Tang, Y. Y. (2004). An improved LDA approach. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 34(5), 1942-1951. https://doi.org/10.1109/tsmcb.2004.831770
  21. Li, C., Lu, Y., Wu, J., Zhang, Y., Xia, Z., Wang, T., Yu, D., Chen, X., Liu, P., & Guo, J. (2018). LDA meets Word2Vec: A novel model for academic abstract clustering. In Proceedings of the 2018 Web Conference Companion (WWW '18 Companion), 1699-1706. https://doi.org/10.1145/3184558.3191629
  22. Li, W. & McCallum, A. (2006). Pachinko allocation: DAG-structured mixture models of topic correlations. In Proceedings of the 23rd International Conference on Machine Learning, 577-584. https://doi.org/10.1145/1143844.1143917
  23. Mehrotra, R., Sanner, S., Buntine, W., & Xie, L. (2013). Improving LDA topic models for microblogs via tweet pooling and automatic labeling. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, 889-892. https://doi.org/10.1145/2484028.2484166
  24. Moody, C. E. (2016). Mixing Dirichlet topic models and word embeddings to make lda2vec. arXiv preprint arXiv:1605.02019. https://doi.org/10.48550/arXiv.1605.02019
  25. Reimers, N. & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3982-3992. https://doi.org/10.18653/v1/D19-1410
  26. Sia, S., Dalmia, A., & Mielke, S. J. (2020). Tired of topic models? Clusters of pretrained word embeddings make for fast and good topics too!. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP), 1728-1736. https://doi.org/10.18653/v1/2020.emnlp-main.135
  27. Vayansky, I. & Kumar, S. A. (2020). A review of topic modeling methods. Information Systems, 94, 1-15. https://doi.org/10.1016/j.is.2020.101582
  28. Yuan, C. & Yang, H. (2019). Research on K-value selection method of K-means clustering algorithm. J, 2(2), 226-235. https://doi.org/10.3390/j2020016