Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

I-QANet: Improved Machine Reading Comprehension using Graph Convolutional Networks

I-QANet: 그래프 컨볼루션 네트워크를 활용한 향상된 기계독해

  • Kim, Jeong-Hoon (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University) ;
  • Kim, Jun-Yeong (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University) ;
  • Park, Jun (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University) ;
  • Park, Sung-Wook (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University) ;
  • Jung, Se-Hoon (Dept. of Computer Engineering, Sunchon National University) ;
  • Sim, Chun-Bo (Dept. of Artificial Intelligence Engineering, Sunchon National University)
  • Received : 2022.08.18
  • Accepted : 2022.10.21
  • Published : 2022.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Most of the existing machine reading research has used Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) algorithms as networks. Among them, RNN was slow in training, and Question Answering Network (QANet) was announced to improve training speed. QANet is a model composed of CNN and self-attention. CNN extracts semantic and syntactic information well from the local corpus, but there is a limit to extracting the corresponding information from the global corpus. Graph Convolutional Networks (GCN) extracts semantic and syntactic information relatively well from the global corpus. In this paper, to take advantage of this strength of GCN, we propose I-QANet, which changed the CNN of QANet to GCN. The proposed model performed 1.2 times faster than the baseline in the Stanford Question Answering Dataset (SQuAD) dataset and showed 0.2% higher performance in Exact Match (EM) and 0.7% higher in F1. Furthermore, in the Korean Question Answering Dataset (KorQuAD) dataset consisting only of Korean, the learning time was 1.1 times faster than the baseline, and the EM and F1 performance were also 0.9% and 0.7% higher, respectively.

Keywords

Acknowledgement

This work was supported by Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry(IPET) through Smart Farm Innovation Technology Development Program, funded by Ministry of Agriculture, Food and Rural Affairs (MAFRA) and Rural Development Administration (RDA) and Ministry of Science and ICT(MSIT)(421028-3); and this research was supported by the MSIT(Ministry of Science and ICT), Korea, under the Grand Information Technology Research Center support program (IITP-2022-2020-0-01489) supervised by the IITP(Institute for Information & communications Technology Planning & Evaluation).

References

  1. S. Liu, X. Zhang, S. Zhang, H. Wang, and W. Zhang, "Neural Machine Reading Comprehension: Methods and Trends," Applied Sciences, Vol. 9, No. 18, pp. 3698, 2019. https://doi.org/10.3390/app9183698
  2. C. Zeng, S. Li, Q. Li, J. Hu, and J. Hu, "A Survey on Machine Reading Comprehension -Tasks, Evaluation Metrics and Benchmark Datasets," Applied Sciences, Vol. 10, No. 21, pp. 7640, 2020. https://doi.org/10.3390/app10217640
  3. Z. Zhang, H. Zhao, and R. Wang, "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond," arXiv Preprint, arXiv:2005.06249, 2020.
  4. A.W. Yu, D. Dohan, M. Luong, R. Zhao, K. Chen, M. Norouzi, et al., "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension," Proceedings of International Conference on Learning Representations 2018: the 6th International Conference on Learning Representations, 2018.
  5. W.L. Hamilton, R. Ying, and J. Leskovec, "Representation Learning on Graphs: Methods and Applications," Institute of Electrical and Electronics Engineers Data Engineering Bulletin, Vol. 40, No. 3, pp. 52-74, 2017.
  6. J. Zhoua, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, et al., "Graph Neural Networks: A Review of Methods and Applications," AI Open, Vol. 1, pp. 57-81, 2020. https://doi.org/10.1016/j.aiopen.2021.01.001
  7. S. Wang and J. Jiang, "Learning Natural Language Inference with LSTM," Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1442-1451, 2016.
  8. W. Shuohang and J. Jing, "Machine Comprehension Using Match-LSTM and Answer Pointer," arXiv Preprint, arXiv:1608.07905, 2016.
  9. Y. Cui, Z. Chen, S. Wei, S. Wang, T. Liu, and G. Hu, "Attention-over-Attention Neural Networks for Reading Comprehension," Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 593-602, 2017.
  10. W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou, "Gated Self-Matching Networks for Reading Comprehension and Question Answering," Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 189-198, 2017.
  11. C. Tan, F. Wei, N. Yang, W. Lv, and M. Zhou, "S-Net: From Answer Extraction to Answer Generation for Machine Reading ComprehenSion," arXiv Preprint, arXiv:1706.04815, 2017.
  12. M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi, "Bidirectional Attention Flow for Machine Comprehension," International Conference on Learning Representations, 2016.
  13. C. Xiong, V. Zhong, and R. Socher, "Dynamic Coattention Networks for Question Answering," arXiv Preprint, arXiv:1611.01604, 2016.
  14. A. Chaturvedi, O. Pandit, and U. Garain, "CNN for Text-Based Multiple Choice Question Answering," Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 4, pp. 272-277, 2018.
  15. Z. Chen, Y. Cui, W. Ma, S. Wang, and G. Hu, "Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions," Association for the Advancement of Artificial Intelligence, Vol. 33, No. 1, pp. 6276-6283, 2019.
  16. P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, "Graph Attention Networks," International Conference on Learning Representations, 2018.
  17. L. Yao, C. Mao, and Y. Luo, "Graph Convolutional Networks for Text Classification," Proceedings of the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence, Vol. 33, pp. 7370-7377, 2019.
  18. K. Xu, W. Hu, J. Leskovec, and S. Jegelka, "How Powerful are Graph Neural Networks," Proceedings of International Conference on Learning Representations, pp. 1-17, 2019.
  19. Q. Ran, Y. Lin, P. Li, J. Zhou, and Z. Liu, "NumNet: Machine Reading Comprehension with Numerical Reasoning," Conference on Empirical Methods in Natural Language Processing & International Joint Conference on Natural Language Processing, pp. 2474-2484, 2019.
  20. J. Zhang, H. Zhang, L. Sun, and C. Xia, "Graph-BERT: Only Attention is Needed for Learning Graph Representations," arXiv Preprint, arXiv:2001.05140, 2020.
  21. S.W. Park and D.Y. Kim, "Comparison of Image Classification Performance in Convolutional Neural Network according to Transfer Learning," Journal of Korea Multimedia Society, Vol. 21, No. 12, pp. 1387-1395, 2018. https://doi.org/10.9717/KMMS.2018.21.12.1387
  22. R.K. Srivastava, K. Greff, and J. Schmidhuber, "Highway Networks," Clinical Orthopaedics and Related Research, abs/1505.00387, 2015.
  23. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, et al., "Attention is All You Need," Advances in Neural Information Processing Systems, 2017.
  24. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, 2016.
  25. S. Lim, M. Kim, and J. Lee, "KorQuAD: Korean QA Dataset for Machine Comprehension," Proceedings of the Korean Information Science Society Conference, pp. 539-541, 2018.