Browse > Article
http://dx.doi.org/10.4275/KSLIS.2017.51.4.099

Research on the Utilization of Recurrent Neural Networks for Automatic Generation of Korean Definitional Sentences of Technical Terms  

Choi, Garam (경기대학교 일반대학원 문헌정보학과)
Kim, Han-Gook (한국과학기술정보연구원 산업정보분석실, 과학기술연합대학원대학교 과학기술정보과학과)
Kim, Kwang-Hoon (한국과학기술정보연구원 산업정보분석실)
Kim, You-eil (한국과학기술정보연구원 산업정보분석실)
Choi, Sung-Pil (경기대학교 문헌정보학과)
Publication Information
Journal of the Korean Society for Library and Information Science / v.51, no.4, 2017 , pp. 99-120 More about this Journal
Abstract
In order to develop a semiautomatic support system that allows researchers concerned to efficiently analyze the technical trends for the ever-growing industry and market. This paper introduces a couple of Korean sentence generation models that can automatically generate definitional statements as well as descriptions of technical terms and concepts. The proposed models are based on a deep learning model called LSTM (Long Sort-Term Memory) capable of effectively labeling textual sequences by taking into account the contextual relations of each item in the sequences. Our models take technical terms as inputs and can generate a broad range of heterogeneous textual descriptions that explain the concept of the terms. In the experiments using large-scale training collections, we confirmed that more accurate and reasonable sentences can be generated by CHAR-CNN-LSTM model that is a word-based LSTM exploiting character embeddings based on convolutional neural networks (CNN). The results of this study can be a force for developing an extension model that can generate a set of sentences covering the same subjects, and furthermore, we can implement an artificial intelligence model that automatically creates technical literature.
Keywords
Sentence Generation; Text Generation; Natural Language Generation(NLG); Automatic Report Generation; Deep Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Bahdanau, D., Cho, K., and Bengio, Y. 2014. "Neural Machine Translation by Jointly Learning to Align and Translate." In conference ICLR 2015.
2 Bauer, A., Hoedoro, N. and Schneider, A. 2015. "Rule-based Approach to Text Generation in Natural Language-Automated Text Markup Language (ATML3)." In Challenge+DC@ RuleML 2015.
3 Bian, J., Gao, B. and Liu, T. Y. 2014. "Knowledge-powered Deep Learning for Word Embedding." In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, September 15-19, 2014, Nancy: 132-148.
4 Bontcheva, K. and Wilks, Y. 2004. "Automatic Report Generation from Ontologies: the MIAKT Approach." In International Conference on Application of Natural Language to Information Systems, 324-335.
5 Boulanger-Lewandowski, N., Bengio, Y. and Vincent, P. 2012. "Modeling Temporal Dependencies in High-dimensional Sequences: Application to Polyphonic Music Generation and Transcription." In Proceedings of the Twenty-nine International Conference on Machine Learning ICML.
6 Bowman, S. et al. 2016. "Generating Sentences from a Continuous Space." In SIGNLL Conference on Computational Natural Language Learning (CONLL), 2016.
7 Deng, L. and Yu, D. 2014. "Deep Learning: Methods and Applications." Foundations and Trends(R) in Signal Processing, 7(3-4): 197-387.   DOI
8 Hochreiter, S. and Schmidhuber, J. 1997. "Long Short-Term Memory. Neural Computation." Neural Computation, 9(8): 1735-1780.   DOI
9 Graves, A., Jaitly, N. and Mohamed, A. R. 2013. "Hybrid Speech Recognition with Deep Bidirectional LSTM." In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, 273-278.
10 Hochreiter, S. 1991. Untersuchungen zu Dynamischen Neuronalen Netzen. Ph.D. diss., Institut fur Informatik, Technische Universitat, Munchen.
11 Kalchbrenner, N., Grefenstette, E. and Blunsom, P. 2014. "A Convolutional Neural Network for Modelling Sentences." In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 655-665.
12 Krizhevsky, A., Sutskever, I. and Hinton, G. E. 2012. "Imagenet Classification with Deep Convolutional Neural Networks." In Advances in Neural Information Processing Systems, 60(6): 1097-1105.
13 Langkilde-Geary, I. 2002. "An Empirical Verification of Coverage and Correctness for a General-purpose Sentence Generator." In Proceedings of the 12th International Natural Language Generation Workshop, 17-24.
14 Nallapati, R. et al. 2016. "Abstractive Text Summarization using Sequence-to-sequence Rnns and Beyond." In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), August 7-12, 2016, Berlin: 280-290.
15 Mairesse, F. 2005. Natural Language Generation: APT on Dialogue Models and Dialogue Systems. [online] [cited 2017. 6. 30.]
16 Srivastava, R. K., Greff, K. and Schmidhuber, J. 2015. "Tranining Very Deep Networks." In Advances in Neural Information Processing Systems, (2015a): 2377-2385.
17 Woodward, A., Sood, B. and Hare, J. 2016. Market Share Analysis: Business Intelligence and Analytics Software, 2015. [online] [cited 2017. 6. 2.]
18 Sundermeyer, M., Schlüter, R. and Ney, H. 2012. "LSTM Neural Networks for Language Modeling." In Thirteenth Annual Conference of the International Speech Communication Association.
19 Sutskever, I., Martens, J. and Hinton, G. E. 2011. "Generating Text with Recurrent Neural Networks." In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 1017-1024.
20 Wikipedia. 2017. kowiki-latest-abstract.xml. [online] [cited 2017. 6. 27.]
21 Zheng, X., Chen, H. and Xu, T. 2013. "Deep Learning for Chinese Word Segmentation and POS Tagging." In Conference on Empirical Methods in Natural Language Processing (EMNLP), 647-657.
22 Zilly, J. G. et al. 2016. "Recurrent Highway Networks." In Proceedings of the 34 th International Conference on Machine Learning.