The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Comparison and Error Analysis of Korean Bio-medical Named Entity Recognition

한국어 생의학 개체명 인식 성능 비교와 오류 분석

  • Jae-Hong Lee (Dept. of Health & Medical Science, Jeonnam State University)
  • 이재홍 (전남도립대학교 보건의료과)
  • Received : 2024.06.17
  • Accepted : 2024.08.12
  • Published : 2024.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The advent of transformer architectures in deep learning has been a major breakthrough in natural language processing research. Object name recognition is a branch of natural language processing and is an important research area for tasks such as information retrieval. It is also important in the biomedical field, but the lack of Korean biomedical corpora for training has limited the development of Korean clinical research using AI. In this study, we built a new biomedical corpus for Korean biomedical entity name recognition and selected language models pre-trained on a large Korean corpus for transfer learning. We compared the name recognition performance of the selected language models by F1-score and the recognition rate by tag, and analyzed the errors. In terms of recognition performance, KlueRoBERTa showed relatively good performance. The error analysis of the tagging process shows that the recognition performance of Disease is excellent, but Body and Treatment are relatively low. This is due to over-segmentation and under-segmentation that fails to properly categorize entity names based on context, and it will be necessary to build a more precise morphological analyzer and a rich lexicon to compensate for the incorrect tagging.

딥러닝 분야에서 트랜스포머 아키텍쳐의 출현은 자연어 처리 연구가 획기적인 발전을 가져왔다. 개체명 인식은 자연어 처리의 한 분야로 정보 검색과 같은 태스크에 중요한 연구 분야이다. 생의학 분야에서도 그 중요성이 강조되나 학습용 한국어 생의학 말뭉치의 부족으로 AI를 활용한 한국어 임상 연구 발전에 제약이 되고 있다. 본 연구에서는 한국어 생의학 개체명 인식을 위해 새로운 생의학 말뭉치를 구축하고 대용량 한국어 말뭉치로 사전 학습된 언어 모델들을 선정하여 전이 학습시켰다. F1-score로 선정된 언어 모델의 개체명 인식 성능과 태그별 인식률을 비교하고 오류 분석을 하였다. 인식 성능에서는 KlueRoBERTa가 상대적인 좋은 성능을 보였다. 태깅 과정의 오류 분석 결과 Disease의 인식 성능은 우수하나 상대적으로 Body와 Treatment는 낮았다. 이는 문맥에 기반하여 제대로 개체명을 분류하지 못하는 과분할과 미분할로 인한 것으로, 잘못된 태깅들을 보완하기 위해서는 보다 정밀한 형태소 분석기와 풍부한 어휘사전 구축이 선행되어야 할 것이다.

Keywords

References

  1. J. Lee, O. Kwon, "Performance Assessment of Machine Learning and Deep Learning in Regional Name Identification and Classification in Scientific Documents," J. of The Korea Institute of Electronic Communication Sciences, vol. 19, no. 2, 2024, pp. 389-396. https://doi.org/10.13067/JKIECS.2024.19.2.389
  2. J. L, A. Sun, J. Han, and C. Li, "A Survey on Deep Learning for Named Entity Recognition," IEEE Trans. on Knowledge and Data Engineering, vol. 34, no. 1, 2020, pp. 50-70. https://doi.org/10.48550/arXiv.1812.09449
  3. J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. So, and J. Kang, "BioBERT: a pre-trained biomedical language representation model for biomedical text mining," Bioinformatics, vol. 36, issue 4, 2019, pp. 1234-1240. https://doi.org/10.48550/arXiv.1901.08746
  4. K. Huang, J. Altosaar, and R. Ranganath, "ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission", In Proc. of CHIL(Conference on Health, Inference, and Learning)'20 Workshop, Toronto, ON, Canada, April 2-4, 2020. https://doi.org/10.48550/arXiv.1904.05342
  5. Y. Li, R. Wehbe, F. Ahmed, H. Wang, and Y. Lao, "A Comparative study of pretrained language models for long clinical text", J. of the American Medical Informatics Association, vol. 30, no. 2, 2023, pp. 340-347. https://doi.org/10.48550/arXiv.2301.11847
  6. Y. Kim and T. Lee, "Korean clinical entity recognition from diagnosis text using BERT," BMC Medical Informatics and Decision Making, vol. 20, supplement 7, 2020. https://doi.org/10.1186/s12911-020-01241-8
  7. S. Byun, J. Hong, S. Park, D. Jang, J. Seo, M. Kim, C. Oh, and H. Shin, "Korean Bio-Medical Corpus (KBMC) for Medical Named Entity Recognition," In Pro. of the 2024 Joint Int. Conf. on Computational Linguistics, Language Resources and Evaluation, Torino, Italy, 2024, pp. 9941-9947. https://doi.org/10.48550/arXiv.2403.16158
  8. Y. Kim, J. Kim, J. Lee, M. Jang, Y. Yum, S. Kim, U. Shin, Y. Kim, H. Joo, and S. Song, "A pre-trained BERT for Korean medical natural language processing," Scientific Reports, vol. 12, no. 1, 2022. https://doi.org/10.1038/s41598-022-17806-8
  9. J. Lee, "Comparison of Sentiment Classification Performance of for RNN and Transformer- Based Models on Korean Reviews, " J. of The Korea Institute of Electronic Communication Sciences, vol. 18, no. 4, 2023, pp. 693-700. https://doi.org/10.13067/JKIECS.2023.18.4.693
  10. J. Lee, "Performance Comparison and Error Analysis of Transformer-based Korean Named Entity Recognition," J. of Jeonnam State University, vol. 25, 2023, pp. 293-302.