Browse > Article
http://dx.doi.org/10.9708/jksci.2017.22.04.041

A Study on Word Sense Disambiguation Using Bidirectional Recurrent Neural Network for Korean Language  

Min, Jihong (Dept. of Information and Communication Engineering, Inha University)
Jeon, Joon-Woo (Dept. of Information and Communication Engineering, Inha University)
Song, Kwang-Ho (Dept. of Information and Communication Engineering, Inha University)
Kim, Yoo-Sung (Dept. of Information and Communication Engineering, Inha University)
Abstract
Word sense disambiguation(WSD) that determines the exact meaning of homonym which can be used in different meanings even in one form is very important to understand the semantical meaning of text document. Many recent researches on WSD have widely used NNLM(Neural Network Language Model) in which neural network is used to represent a document into vectors and to analyze its semantics. Among the previous WSD researches using NNLM, RNN(Recurrent Neural Network) model has better performance than other models because RNN model can reflect the occurrence order of words in addition to the word appearance information in a document. However, since RNN model uses only the forward order of word occurrences in a document, it is not able to reflect natural language's characteristics that later words can affect the meanings of the preceding words. In this paper, we propose a WSD scheme using Bidirectional RNN that can reflect not only the forward order but also the backward order of word occurrences in a document. From the experiments, the accuracy of the proposed model is higher than that of previous method using RNN. Hence, it is confirmed that bidirectional order information of word occurrences is useful for WSD in Korean language.
Keywords
Word Sense Disambiguation; Neural Network Language Model; Context Vector; Bidirectional Recurrent Neural Network;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 M. Sundermeyer, and R. Schluter, "LSTM Neural Networks for Language Modeling" Interspeech, pp. 194-197, Sep. 2012.
2 mykang, bgkim and jslee, "Word Sense Disambiguation using Word2Vec", Proceeding of 27th Conference on Human & Cognitive Language Technology, pp. 81-84, Oct. 2015
3 jcshin, and, cyock, "Homograph Word Sense Disambiguation using Korean Lexical Semantic Map(UWordMap) and Word-Embedding", Proceeding of Korean Computer Congress 2016, pp. 702-704, Jun. 2016
4 D. Yuan, J. Richardson, R. Doherty, C. Evans and E. Altendorf, "Word Sense Disambiguation with Neural Language Models", arXivpreprint arXiv:1603.07012, 2016.
5 jhmin, jwjeon, khsong, and yskim, "Study on Word Sense Disambiguation Using Recurrent Neural Network for Korean", Proceeding of Winter Conference on Korean Association of Computer Education 2017, pp. 93-96, Jan. 2017.
6 jsbae, and cklee, "End-to-end Learning of Korean Semantic Role Labeling Using Bidirectional LSTM CRF", Proceeding of 42th Winter Conference on Korean Institute of Information Scientists and Engineers, pp. 566-568, Dec. 2015.
7 C. Irsoy, and C. Cardie, "Opinion Mining with Deep Recurrent Neural Networks", Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, pp. 720-728, 2014.
8 shjung, "Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithms", Journal of The Korea Society of Computer and Information, Vol. 17, No. 3, pp. 11-25, Mar. 2012   DOI
9 H. Sak, A. Senior, and F. Beaufays, " Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition", arXiv preprint arXiv:1402.1128, 2014.
10 Junyoung Chung, C. Gulcehre, KyungHyun Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling", arXiv preprint arXiv:1412.3555, 2014.
11 bmkang, "Text Context and Word Meaning: Latent Semantic Analysis", Journal of the Linguistic Society of Korea, Vol. 68, pp. 3-34, Apr. 2014.
12 hmkim, jmyoon, jhan, kmbae, and yjko, "Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp. 3-8, Oct. 2016
13 smhan, "Deep Learning Architectures and Applications", Journal of Intelligence Information System, Vol. 22, No. 2, pp. 127-142, Jun. 2016   DOI
14 Y. Bengio, P. Simard, and P. Frasconi, "Learning Long-Term Dependencies with Gradient Descent is Difficult", IEEE Transactions on Neural Networks, Vol. 5, No. 2, Mar. 1994
15 Gensim, https://radimrehurek.com/gensim/models/word2vec.html
16 T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space". arXivpreprint arXiv: 1301.3781. 2013.
17 mbchung, "Color matching application which can help color blind people based on smart phone", Journal of The Korea Society of Computer and Information, Vol.20, No. 5, pp. 65-72, May. 2015   DOI
18 National Institute of Korean Language, http://ithub.korean.go.kr,
19 shchoi, jsseol and sglee, "On Word Embedding Models and Parameters Optimized for Korean", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp. 252-256, Oct. 2016
20 hgkim, mbkang, and jhhong, "21st Century Sejong Modern Korean Corpora: Results and Expectations", Proceeding of 19th Conference on Human & Cognitive Language Technology, pp. 311-316, Oct. 2007
21 Keras, http://keras.io
22 Korean Wikipedia, https://ko.wikipedia.org/
23 Namuwiki, https://namu.wiki/
24 SENSEAVAL-2, http://www.hipposmond.com/senseval2/