Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Neural Theorem Prover with Word Embedding for Efficient Automatic Annotation

효율적인 자동 주석을 위한 단어 임베딩 인공 신경 정리 증명계 구축

  • 양원석 (한국과학기술원 전산학부) ;
  • 박한철 (한국과학기술원 전산학부) ;
  • 박종철 (한국과학기술원 전산학부)
  • Received : 2016.11.21
  • Accepted : 2017.02.11
  • Published : 2017.04.15
⟨ Previous Next ⟩

Abstract

We present a system that automatically annotates unverified Web sentences with information from credible sources. The system turns to neural theorem proving for an annotating task for cancer related Wikipedia data (1,486 propositions) with Korean National Cancer Center data (19,304 propositions). By switching the recursive module in a neural theorem prover to a word embedding module, we overcome the fundamental problem of tremendous learning time. Within the identical environment, the original neural theorem prover was estimated to spend 233.9 days of learning time. In contrast, the revised neural theorem prover took only 102.1 minutes of learning time. We demonstrated that a neural theorem prover, which encodes a proposition in a tensor, includes a classic theorem prover for exact match and enables end-to-end differentiable logic for analogous words.

본 연구는 전문기관에서 생산되는 검증된 문서의 정보를 웹상의 수많은 검증되지 않은 문서에 자동 주석하여 신뢰도를 향상하고 심화 정보를 추가하는 시스템을 제안한다. 제안하는 시스템은 국가암정보센터의 검증된 문서들에서 추출한 19,304개 명제를 위키피디아 암 관련 문서에서 추출한 1,486개 명제에 주석하는 과제를 수행하기 위해, 기존 인공 신경 정리 증명계의 순환 모듈을 단어 임베딩 모듈로 교체하였다. 이를 통해 기존의 근본적인 문제점이었던 학습 시간 문제를 해결하였고, 동일한 환경에서 기존 시스템의 학습 시간이 233.9일로 추정된 것에 비해 재구축한 시스템은 102.1분 내로 학습이 완료되었다. 제안하는 시스템의 장점은 명제를 텐서로 인코딩하여 미분 가능하게 전체적인 연산을 진행하는 인공 신경 정리 증명계가 단어의 정확한 일치를 파악하는 전통적인 정리 증명계를 포함하며 동시에 유사어 관계로부터의 논리 전개 역시 가능하게 한다는 점을 실제 문서 데이터에서 입증했다는 것이다.

Keywords

Acknowledgement

Grant : 상대방의 감정을 추론, 판단하여 그에 맞추어 대화하고 대응할 수 있는 감성지능 기술 연구 개발

Supported by : 정보통신기술진흥센터, 한국연구재단

References

  1. H. Robert, et al., "Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions," J Am Osteopath Assoc, 114.5, pp. 368-373, 2014. https://doi.org/10.7556/jaoa.2014.035
  2. W. Richard, et al., "Deductive Question Answering from Multiple Resources," New Directions in Question Answering, pp. 253-262, 2004.
  3. S. Kim, Y. Hur, S. Kim, Y. Kim, "Efficient Retrieval Technique of the Web Document based on Fuzzy Logic," Communications of the Korean Institute of Information Scientists and Engineers, 38(2B), pp. 287- 290, 2011.
  4. T. Rocktäschel and S. Riedel, "Learning Knowledge Base Inference with Neural Theorem Provers," NAACL Workshop on Automated Knowledge Base Construction, 2016.
  5. L. Jin, et al., "Fuzzy keyword search over encrypted data in cloud computing," IEEE INFOCOM, 2010.
  6. K. Janusz and S. Zadrozny, "Queries with Fuzzy Linguistic Quantifiers for Data of Variable Quality Using Some Extended OWA Operators," Flexible Query Answering Systems 2015, Springer International Publishing, pp. 295-305, 2016.
  7. S. Ilya, O. Vinyals, and Q. Le, "Sequence to sequence learning with neural networks," Advances in Neural Information Processing Systems, 2014.
  8. K. Cho, et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," Conference on Empirical Methods in Natural Language Processing, 2014.
  9. T. Mikolov and J. Dean, "Distributed representations of words and phrases and their compositionality," Advances in Neural Information Processing Systems, 2013.
  10. J. Pennington, R. Socher, and C. Manning, "Glove: Global Vectors for Word Representation," Conference on Empirical Methods in Natural Language Processing, 2014.
  11. B. Peng, et al., "Towards neural network-based reasoning," IEEE Conference on Robotics, Automation and Mechatronics, 2015.
  12. R. Scott and N. Freitas, "Neural programmer- interpreters," International Conference on Learning Representations, 2015.
  13. S. J. Russell and P. Norvig, Artificial Intelligence: Modern Approach, 3rd edition, Prentice Hall, 1995.
  14. T. H. Park, J. W. Cha, "Feature Selection for Korean Sematic Role Labeling Using CRFs," Communications of the Korean Institute of Information Scientists and Engineers, 34(8), 2016.8, 37-41, Korea Information Science Society.
  15. K. Diederik and J. Ba, "Adam: A method for stochastic optimization," International Conference on Learning Representations, 2015.
  16. M. Xuezhe and E. Hovy, "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF," Proc. of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.
  17. W. Felix, J. Bergmann, and B. Schuller, "Introducing CURRENNT-the Munich open-source CUDA RecurREnt neural network toolkit," Journal of Machine Learning Research 16.3, pp. 547-551, 2015.
  18. G. Alex, N. Beringer, and J. Schmidhuber, "Rapid retraining on speech data with LSTM recurrent networks," Technical Report IDSIA-05-05, 2005.