Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Improved Character-Based Neural Network for POS Tagging on Morphologically Rich Languages

  • Samat Ali (School of Computer Science Technology, Xinjiang Normal University) ;
  • Alim Murat (School of Computer Science Technology, Xinjiang Normal University)
  • Received : 2022.09.01
  • Accepted : 2022.10.29
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Since the widespread adoption of deep-learning and related distributed representation, there have been substantial advancements in part-of-speech (POS) tagging for many languages. When training word representations, morphology and shape are typically ignored, as these representations rely primarily on collecting syntactic and semantic aspects of words. However, for tasks like POS tagging, notably in morphologically rich and resource-limited language environments, the intra-word information is essential. In this study, we introduce a deep neural network (DNN) for POS tagging that learns character-level word representations and combines them with general word representations. Using the proposed approach and omitting hand-crafted features, we achieve 90.47%, 80.16%, and 79.32% accuracy on our own dataset for three morphologically rich languages: Uyghur, Uzbek, and Kyrgyz. The experimental results reveal that the presented character-based strategy greatly improves POS tagging performance for several morphologically rich languages (MRL) where character information is significant. Furthermore, when compared to the previously reported state-of-the-art POS tagging results for Turkish on the METU Turkish Treebank dataset, the proposed approach improved on the prior work slightly. As a result, the experimental results indicate that character-based representations outperform word-level representations for MRL performance. Our technique is also robust towards the-out-of-vocabulary issues and performs better on manually edited text.

Keywords

Acknowledgement

This work is supported by the Doctoral Fund of Xinjiang Normal University (No. XJNUBS2007).

References

  1. X. Yu and N. T. Vu, "Character composition model with convolutional neural networks for dependency parsing on morphologically rich languages," in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, Canada, 2017, pp. 672-678.
  2. T. V. Ngo, T. L. Ha, P. T. Nguyen, and L. M. Nguyen, "Overcoming the rare word problem for low-resource language pairs in neural machine translation," 2019 [Online]. Available: https://arxiv.org/abs/1910.03467.
  3. M. Grohe, "word2vec, node2vec, graph2vec, x2vec: towards a theory of vector embeddings of structured data," in Proceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, Portland, OR, 2020, pp. 1-16. 
  4. C. I. Eke, A. Norman, L. Shuib, F. B. Fatokun, and I. Omame, "The significance of global vectors representation in sarcasm analysis," in Proceedings of 2020 International Conference in Mathematics, Computer Engineering and Computer Science (ICMCECS), Ayobo, Nigeria, 2020, pp. 1-7. 
  5. P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching word vectors with subword information," Transactions of the Association for Computational Linguistics, vol. 5, pp. 135-146, 2017.  https://doi.org/10.1162/tacl_a_00051
  6. A. Chiche and B. Yitagesu, "Part of speech tagging: a systematic review of deep learning and machine learning approaches," Journal of Big Data, vol. 9, article no. 10, 2022. https://doi.org/10.1186/s40537-022-00561-y 
  7. R. D. Deshmukh and A. Kiwelekar, "Deep learning techniques for part of speech tagging by natural language processing," in Proceedings of 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bangalore, India, 2020, pp. 76-81. 
  8. K. K. Akhil, R. Rajimol, and V. S. Anoop, "Parts-of-Speech tagging for Malayalam using deep learning techniques," International Journal of Information Technology, vol. 12, pp. 741-748, 2020.  https://doi.org/10.1007/s41870-020-00491-z
  9. W. Khan, A. Daud, K. Khan, J. A. Nasir, M. Basheri, N. Aljohani, and F. S. Alotaibi, "Part of speech tagging in Urdu: comparison of machine and deep learning approaches," IEEE Access, vol. 7, pp. 38918-38936, 2019.  https://doi.org/10.1109/ACCESS.2019.2897327
  10. C. A. Bahcevan, E. Kutlu, and T. Yildiz, "Deep neural network architecture for part-of-speech tagging for Turkish language," in Proceedings of 2018 3rd International Conference on Computer Science and Engineering (UBMK), Sarajevo, Bosnia and Herzegovina, 2018, pp. 235-238. 
  11. K. K. Todi, P. Mishra, and D. M. Sharma, "Building a Kannada POS tagger using machine learning and neural network models," 2018 [Online]. Available: https://arxiv.org/abs/1808.03175. 
  12. S. Kumar, M. A. Kumar, and K. P. Soman, "Deep learning based part-of-speech tagging for Malayalam Twitter data (Special issue: deep learning techniques for natural language processing)," Journal of Intelligent Systems, vol. 28, no. 3, pp. 423-435, 2019.  https://doi.org/10.1515/jisys-2017-0520
  13. M. Maimaiti, A. Wumaier, K. Abiderexiti, and T. Yibulayin, "Bidirectional long short-term memory network with a conditional random field layer for Uyghur part-of-speech tagging," Information, vol. 8, no. 4, article no. 157, 2017. https://doi.org/10.3390/info8040157
  14. S. Fouladi, M. J. Ebadi, A. A. Safaei, M. Y. Bajuri, and A. Ahmadian, "Efficient deep neural networks for classification of COVID-19 based on CT images: virtualization via software defined radio," Computer Communications, vol. 176, pp. 234-248, 2021.  https://doi.org/10.1016/j.comcom.2021.06.011
  15. A. Javaheri, N. Moghadamnejad, H. Keshavarz, E. Javaheri, C. Dobbins, E. Momeni-Ortner, and R. Rawassizadeh, "Public vs media opinion on robots and their evolution over recent years," CCF Transactions on Pervasive Computing and Interaction, 2, 189-205, 2020.  https://doi.org/10.1007/s42486-020-00035-1
  16. S. Fouladi, A. A. Safaei, N. Mammone, F. Ghaderi, and M. J. Ebadi, "Efficient deep neural networks for classification of Alzheimer's disease and mild cognitive impairment from scalp EEG recordings," Cognitive Computation, vol. 14, pp. 1247-1268, 2022.  https://doi.org/10.1007/s12559-022-10033-3
  17. R. Collobert, "Deep learning for efficient discriminative parsing," in Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, 2011, pp. 224-232. 
  18. Y. Esref and B. Can, "Using morpheme-level attention mechanism for Turkish sequence labelling," in Proceedings of 2019 27th Signal Processing and Communications Applications Conference (SIU), Sivas, Turkey, 2019, pp. 1-4.