Journal of Advanced Information Technology and Convergence (한국정보기술학회 영문논문지)

Korean Institute of Information Technology (한국정보기술학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Performance Analysis of Entity Name Recognition Techniques Using Korean Patent Literature

  • Gim, Jangwon (Department of Software Convergence Engineering, Kunsan National University)
  • Received : 2020.12.10
  • Accepted : 2020.12.30
  • Published : 2020.12.31
⟨ Previous Next ⟩

Abstract

Entity name recognition is a part of information extraction that extracts entity names from documents and classifies the types of extracted entity names. Entity name recognition technologies are widely used in natural language processing, such as information retrieval, machine translation, and query response systems. Various deep learning-based models exist to improve entity name recognition performance, but studies that compared and analyzed these models on Korean data are insufficient. In this paper, we compare and analyze the performance of CRF, LSTM-CRF, BiLSTM-CRF, and BERT, which are actively used to identify entity names using Korean data. Also, we compare and evaluate whether embedding models, which are variously used in recent natural language processing tasks, can affect the entity name recognition model's performance improvement. As a result of experiments on patent data and Korean corpus, it was confirmed that the BiLSTM-CRF using FastText method showed the highest performance.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.NRF-2018R1C1B6008624).

References

  1. A. Goyal, V. Gupta, & M. Kumar, "Recent named entity recognition and classification techniques: a systematic review," Computer Science Review, Vol. 29, pp. 21-43, 2018. https://doi.org/10.1016/j.cosrev.2018.06.001
  2. A. Mehdi, S. Pouriyeh, M. Assefi, S. Safaei, E. D. Trippe, J. B. Gutierrez, and K. Kochut. (2017). A brief survey of text mining: Classification, clustering and extraction techniques. [Online]. Available: arXiv:1707.02919 [accessed: Dec. 22. 2020]
  3. M. Monica, J. Urbano, S. Sanchez-Cuadrado, J. Morato, and J. M. Gomez-Berbis, "Named entity recognition: fallacies, challenges and opportunities," Computer Standards & Interfaces Vol. 35, no. 5, pp. 482-489, 2013. https://doi.org/10.1016/j.csi.2012.09.004
  4. G. Bekoulis, J. Deleu, T. Demeester, and C. Develder, "Joint entity recognition and relation extraction as a multi-head selection problem," Expert Systems with Applications, Vol. 114, pp. 34-45, 2018. https://doi.org/10.1016/j.eswa.2018.07.032
  5. E., Tome, B. K. Seljak, and P. Korosec, "A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations," PloS one Vol. 12, no. 6, 2017.
  6. K. Lee, J. Lee, M. Choi, G. Kim, "Study on Named Entity Recognition in Korean Text," in Proc. of the KIISE Conference, The 12th Annual Conference on Human & Cognitive Language Technology, 2000, pp.292-299.
  7. Y. Hwang, H. Lee, E. Chung, B. Yun, S. Park, "Korean Named Entity Recognition Based on Supervised Learning Using Named Entity Construction Principles," In Proc. of the KIISE Conference, The 14th Annual Conference on Human & Cognitive Language Technology, 2002, pp.111-117.
  8. K. Shrutika, and S. Govilkar, "Survey of named entity recognition techniques for various indian regional languages," International Journal of Computer Applications, Vol. 164, no. 4, pp. 37-43, 2017. https://doi.org/10.5120/ijca2017913621
  9. M. Sudha, N. Jahan, and D. Chopra, "Named entity recognition using hidden Markov model (HMM)," International Journal on Natural Language Computing (IJNLC), Vol. 1, no. 4, pp. 15-23, 2012. https://doi.org/10.5121/ijnlc.2012.1402
  10. Y. Hwang, B. Yun, "HMM-based Korean Named Entity Recognition," In Proc. of the KIPS Transaction Vol.10(B), No.2, 2003, pp.229-236.
  11. B., Andrew, and R. Grishman, "A maximum entropy approach to named entity recognition," Ph.D. dissertation, New York University, Graduate School of Arts and Science, 1999.
  12. S. Kim, D. Ra, "Korean Named Entity Recognition Using Two-level Maximum Entropy Model," In Proc. of the KIISE Symposium, Vol. 2, No.1, 2008, pp.81-86.
  13. H. Mo, T. Khin, and M. Khin, "CRF-Based Named Entity Recognition for Myanmar Language," In Proc. of ICGEC, Springer, Cham, 2016, pp. 204-211.
  14. C. Lee, Y. Hwang, H. Oh, S. Lim, J. Heo, C. Lee, H. Kim, J. Wang, M. Jang, "Fine-Grained Named Entity Recognition using Conditional Random Fields for Question Answering," In Proc. of the KIISE Conference, The 18th Annual Conference on Human & Cognitive Language Technology, 2006, pp. 268-272.
  15. K. Lee, Y. Hwang, S. Kim, and H. Rim, "Biomedical named entity recognition using two-phase model based on SVMs," Journal of Biomedical Informatics, Vol. 37, no. 6, pp. 436-447, 2004. https://doi.org/10.1016/j.jbi.2004.08.012
  16. C. Lee, M. Jang, "Named Entity Recognition with Structural SVMs and Pegasos algorithm," In Proc. of KSCS Cognitive Science, Vol.21, No.4, 2010, pp. 655-667.
  17. T. Lee, S. Shin, and S. Kang, "Named Entity Recognition for Patent Documents Based on Conditional Random Fields," KIPS Transactions on Software and Data Engineering, Vol. 5, no. 9, pp. 419-424, 2016. https://doi.org/10.3745/KTSDE.2016.5.9.419
  18. J. Li, A. Sun, J. Han and C. Li. (2020). A Survey on Deep Learning for Named Entity Recognition. IEEE Transactions on Knowledge and Data Engineering [Online]. Available: doi: 10.1109/TKDE.2020.2981314 [accessed: Dec. 22. 2020]
  19. D. Chuanhai, J. Zhang, C. Zong, M. Hattori, and H. Di, "Character-based LSTM-CRF with radical-level features for Chinese named entity recognition," In Proc. of Natural Language Understanding and Intelligent Applications, Springer, Cham, 2016, pp. 239-250.
  20. Z. Huang, W. Xu, and K. Yu. (2015). Bidirectional LSTM-CRF models for sequence tagging. [Online]. Available: arXiv:1508.01991 [accessed: Dec. 22. 2020]
  21. Y. Peng, S. Yan, and Z. Lu. (2019). Transfer learning in biomedical natural language processing: An evaluation of BERT and ELMo on ten benchmarking datasets. [Online]. Available: arXiv:1906.05474 [accessed: Dec. 22. 2020]
  22. T. Pires, E. Schlinger, and D. Garrette. (2019). How multilingual is Multilingual BERT?. [Online]. Available: arXiv:1906.01502 [accessed: Dec. 22. 2020]
  23. A. Gillioz, J. Casas, E. Mugellini, and O. Khaled, "Overview of the Transformer-based Models for NLP Tasks," In Proc. of 2020 15th Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2020, pp. 179-183.
  24. C. Sun, X. Qiu, Y. Xu, and X. Huang, "How to fine-tune bert for text classification?," In Proc. of China National Conference on Chinese Computational Linguistics, Springer, Cham, 2019, pp. 194-206.
  25. SKTBrain. (2020). Korean BERT pre-trained cased (KoBERT). [Online]. Available: https://github.com/SKTBrain/KoBERT [accessed: Dec. 22. 2020]
  26. K. Kaliyaperumal, Z. Wang, S. Mayhew, and D. Roth, "Cross-lingual ability of multilingual bert: An empirical study," In Proc. of International Conference on Learning Representations, 2019.
  27. J. Li, A. Sun, J. Han, and C. Li. (2020). A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering [Online]. Available: arXiv:1812.09449 [accessed: Dec. 22. 2020]
  28. S. Na, and J. Min, "Character-based LSTM CRFs for named entity recognition," In Proc. of KISS Korea Computer Congress, 2016, pp. 729-731.
  29. J. Min, H. Oh, S. Na. "Character-Based LSTM CRFs for Food Domain Named Entity Recognition," In Proc. of KISS Korea Computer Congress, 2016, pp. 500-502.
  30. Z. Huang, W. Xu, and K. Yu. (2015). Bidirectional LSTM-CRF models for sequence tagging. [Online]. Available: arXiv:1508.01991 [accessed: Dec. 22. 2020]
  31. X. Ma and E. Hovy, ''End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF,'' in Proc. of 54th Annu. Meeting Assoc. Comput. Linguistics (ACL), Berlin, Germany, Vol. 1, Aug. 2016, pp. 1064-1074.
  32. J. P. C. Chiu and E. Nichols, ''Named entity recognition with bidirectional LSTM-CNNs,'' Trans. Assoc. Comput. Linguistics, Vol. 4, pp. 357-370, Dec. 2016. https://doi.org/10.1162/tacl_a_00104
  33. H. Yu, and Y. Ko, "Expansion of word representation for named entity recognition based on bidirectional lstm crfs," Journal of KIISE Vol. 44, no. 3. pp. 306-313, 2017. https://doi.org/10.5626/JOK.2017.44.3.306
  34. J. Devlin, M. Chang, K. Lee, and K. Toutanova. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. [Online]. Available: arXiv:1810.04805 [accessed: Dec. 22. 2020]
  35. Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon. (2020). Domain-specific language model pretraining for biomedical natural language processing. [Online]. Available: arXiv:2007.15779 [accessed: Dec. 22. 2020]
  36. L. Beltagy, K. Lo, and A. Cohan. (2019). SciBERT: A pretrained language model for scientific text. [Online]. Available: arXiv:1903.10676 [accessed: Dec. 22. 2020]
  37. K. Huang, J. Altosaar, and R. Ranganath. (2019). Clinicalbert: Modeling clinical notes and predicting hospital readmission. [Online]. Available: arXiv:1904.05342 [accessed: Dec. 22. 2020]
  38. Y. Lim, D. Seo, Y. Jung, "Fine-tuning BERT Models for Keyphrase Extraction in Scientific Articles," JOURNAL OF ADVANCED INFORMATION TECHNOLOGY AND CONVERGENCE, Vol. 10, no. 1, pp. 45-56, 2020. https://doi.org/10.14801/jaitc.2020.10.1.45
  39. F. Almeida, and G. Xexeo. (2019). Word embeddings: A survey. [Online]. Available: arXiv:1901.09069 [accessed: Dec. 22. 2020]
  40. Y. Goldberg, and O. Levy. (2014). word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method. [Online]. Available: arXiv:1402.3722 [accessed: Dec. 22. 2020]
  41. J. Pennington, R. Socher, and C. Manning, "Glove: Global vectors for word representation," In Proc. of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532-1543.
  42. A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jegou, and T. Mikolov. (2016). Fasttext. zip: Compressing text classification models. [Online]. Available: arXiv:1612.03651 [accessed: Dec. 22. 2020]
  43. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, ..., and S. Agarwal, (2020). Language models are few-shot learners. [Online]. Available: arXiv:2005.14165 [accessed: Dec. 22. 2020]