Browse > Article
http://dx.doi.org/10.9717/kmms.2022.25.11.1643

I-QANet: Improved Machine Reading Comprehension using Graph Convolutional Networks  

Kim, Jeong-Hoon (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University)
Kim, Jun-Yeong (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University)
Park, Jun (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University)
Park, Sung-Wook (Interdisciplinary Program in IT-Bio Convergence System, Sunchon National University)
Jung, Se-Hoon (Dept. of Computer Engineering, Sunchon National University)
Sim, Chun-Bo (Dept. of Artificial Intelligence Engineering, Sunchon National University)
Publication Information
Journal of Korea Multimedia Society / v.25, no.11, 2022 , pp. 1643-1652 More about this Journal
Abstract
Most of the existing machine reading research has used Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) algorithms as networks. Among them, RNN was slow in training, and Question Answering Network (QANet) was announced to improve training speed. QANet is a model composed of CNN and self-attention. CNN extracts semantic and syntactic information well from the local corpus, but there is a limit to extracting the corresponding information from the global corpus. Graph Convolutional Networks (GCN) extracts semantic and syntactic information relatively well from the global corpus. In this paper, to take advantage of this strength of GCN, we propose I-QANet, which changed the CNN of QANet to GCN. The proposed model performed 1.2 times faster than the baseline in the Stanford Question Answering Dataset (SQuAD) dataset and showed 0.2% higher performance in Exact Match (EM) and 0.7% higher in F1. Furthermore, in the Korean Question Answering Dataset (KorQuAD) dataset consisting only of Korean, the learning time was 1.1 times faster than the baseline, and the EM and F1 performance were also 0.9% and 0.7% higher, respectively.
Keywords
Machine Reading Comprehension; QANet; Graph Convolutional Networks;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 W.L. Hamilton, R. Ying, and J. Leskovec, "Representation Learning on Graphs: Methods and Applications," Institute of Electrical and Electronics Engineers Data Engineering Bulletin, Vol. 40, No. 3, pp. 52-74, 2017.
2 W. Shuohang and J. Jing, "Machine Comprehension Using Match-LSTM and Answer Pointer," arXiv Preprint, arXiv:1608.07905, 2016.
3 C. Tan, F. Wei, N. Yang, W. Lv, and M. Zhou, "S-Net: From Answer Extraction to Answer Generation for Machine Reading ComprehenSion," arXiv Preprint, arXiv:1706.04815, 2017.
4 A. Chaturvedi, O. Pandit, and U. Garain, "CNN for Text-Based Multiple Choice Question Answering," Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 4, pp. 272-277, 2018.
5 J. Zhang, H. Zhang, L. Sun, and C. Xia, "Graph-BERT: Only Attention is Needed for Learning Graph Representations," arXiv Preprint, arXiv:2001.05140, 2020.
6 A.W. Yu, D. Dohan, M. Luong, R. Zhao, K. Chen, M. Norouzi, et al., "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension," Proceedings of International Conference on Learning Representations 2018: the 6th International Conference on Learning Representations, 2018.
7 L. Yao, C. Mao, and Y. Luo, "Graph Convolutional Networks for Text Classification," Proceedings of the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence, Vol. 33, pp. 7370-7377, 2019.
8 S. Liu, X. Zhang, S. Zhang, H. Wang, and W. Zhang, "Neural Machine Reading Comprehension: Methods and Trends," Applied Sciences, Vol. 9, No. 18, pp. 3698, 2019.   DOI
9 J. Zhoua, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, et al., "Graph Neural Networks: A Review of Methods and Applications," AI Open, Vol. 1, pp. 57-81, 2020.   DOI
10 M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi, "Bidirectional Attention Flow for Machine Comprehension," International Conference on Learning Representations, 2016.
11 Y. Cui, Z. Chen, S. Wei, S. Wang, T. Liu, and G. Hu, "Attention-over-Attention Neural Networks for Reading Comprehension," Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 593-602, 2017.
12 W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou, "Gated Self-Matching Networks for Reading Comprehension and Question Answering," Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 189-198, 2017.
13 C. Xiong, V. Zhong, and R. Socher, "Dynamic Coattention Networks for Question Answering," arXiv Preprint, arXiv:1611.01604, 2016.
14 P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, "Graph Attention Networks," International Conference on Learning Representations, 2018.
15 K. Xu, W. Hu, J. Leskovec, and S. Jegelka, "How Powerful are Graph Neural Networks," Proceedings of International Conference on Learning Representations, pp. 1-17, 2019.
16 Q. Ran, Y. Lin, P. Li, J. Zhou, and Z. Liu, "NumNet: Machine Reading Comprehension with Numerical Reasoning," Conference on Empirical Methods in Natural Language Processing & International Joint Conference on Natural Language Processing, pp. 2474-2484, 2019.
17 S.W. Park and D.Y. Kim, "Comparison of Image Classification Performance in Convolutional Neural Network according to Transfer Learning," Journal of Korea Multimedia Society, Vol. 21, No. 12, pp. 1387-1395, 2018.   DOI
18 Z. Chen, Y. Cui, W. Ma, S. Wang, and G. Hu, "Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions," Association for the Advancement of Artificial Intelligence, Vol. 33, No. 1, pp. 6276-6283, 2019.
19 C. Zeng, S. Li, Q. Li, J. Hu, and J. Hu, "A Survey on Machine Reading Comprehension -Tasks, Evaluation Metrics and Benchmark Datasets," Applied Sciences, Vol. 10, No. 21, pp. 7640, 2020.   DOI
20 Z. Zhang, H. Zhao, and R. Wang, "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond," arXiv Preprint, arXiv:2005.06249, 2020.
21 S. Wang and J. Jiang, "Learning Natural Language Inference with LSTM," Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1442-1451, 2016.
22 A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, et al., "Attention is All You Need," Advances in Neural Information Processing Systems, 2017.
23 P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, 2016.
24 S. Lim, M. Kim, and J. Lee, "KorQuAD: Korean QA Dataset for Machine Comprehension," Proceedings of the Korean Information Science Society Conference, pp. 539-541, 2018.
25 R.K. Srivastava, K. Greff, and J. Schmidhuber, "Highway Networks," Clinical Orthopaedics and Related Research, abs/1505.00387, 2015.