Browse > Article
http://dx.doi.org/10.30693/SMJ.2022.11.10.89

Open Domain Machine Reading Comprehension using InferSent  

Jeong-Hoon, Kim (순천대학교 IT-Bio융합시스템전공)
Jun-Yeong, Kim (순천대학교 IT-Bio융합시스템전공)
Jun, Park (순천대학교 IT-Bio융합시스템전공)
Sung-Wook, Park (순천대학교 IT-Bio융합시스템전공)
Se-Hoon, Jung (순천대학교 컴퓨터공학과)
Chun-Bo, Sim (순천대학교 인공지능공학부)
Publication Information
Smart Media Journal / v.11, no.10, 2022 , pp. 89-96 More about this Journal
Abstract
An open domain machine reading comprehension is a model that adds a function to search paragraphs as there are no paragraphs related to a given question. Document searches have an issue of lower performance with a lot of documents despite abundant research with word frequency based TF-IDF. Paragraph selections also have an issue of not extracting paragraph contexts, including sentence characteristics accurately despite a lot of research with word-based embedding. Document reading comprehension has an issue of slow learning due to the growing number of parameters despite a lot of research on BERT. Trying to solve these three issues, this study used BM25 which considered even sentence length and InferSent to get sentence contexts, and proposed an open domain machine reading comprehension with ALBERT to reduce the number of parameters. An experiment was conducted with SQuAD1.1 datasets. BM25 recorded a higher performance of document research than TF-IDF by 3.2%. InferSent showed a higher performance in paragraph selection than Transformer by 0.9%. Finally, as the number of paragraphs increased in document comprehension, ALBERT was 0.4% higher in EM and 0.2% higher in F1.
Keywords
Open Domain Machine Reading Comprehension; Document Search; Document Reader; ALBERT; InferSent;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 D. Chen, A. Fisch, J. Weston and A. Bordes, "Reading wikipedia to answer open-domain questions," arXiv preprint, arXiv:1704.00051, Mar. 2017.
2 M. Kim, S. Cho, D. Park, and H.C. Kwon, "Machine Reading Comprehension-based Question and Answering System for Search and Analysis of Safety Standards," Journal of Korea Multimedia Society, Vol. 23, No. 2, pp. 351-360, Feb. 2020.
3 S.E. Robertson, S. Walker, S. Jones, M.H. Beaulieu and M. Gatford, "Okapi at TREC-3," Third Text REtrieval Conference, pp. 109-126, London, UK, 1995.
4 M.J. Lim, W.H. Park and J.H. Shin, "Parting Lyrics Emotion Classification using Word2Vec and LSTM," Smart Media Journal, Vol. 9, No. 3, pp. 90-97, Sep. 2020.   DOI
5 J. Park, J.Y. Kim, S.W. Park, S.H. Jung and C.B. Sim, "Development of ResNet based Crop Growth Stage Estimation Model," Smart Media Journal, Vol. 11, No. 2, pp. 53-62, Mar. 2022.
6 J.H. Bang, J. Park, S.W. Park J.Y. Kim, S.H. Jung and C.B. Sim, "A System for Determining the Growth Stage of Fruit Tree Using a Deep Learning-Based Object Detection Model," Smart Media Journal, Vol. 11, No. 4, pp. 9-18, May, 2022.
7 S. Liu, X. Zhang, S. Zhang, H. Wang, and W. Zhang, "Neural Machine Reading Comprehension: Methods and Trends," Applied Sciences, Vol. 9, No. 18, pp. 3698, Sep. 2019.
8 C. Zeng, S. Li, Q. Li, J. Hu, and J. Hu, "A Survey on Machine Reading Comprehension-Tasks, Evaluation Metrics and Benchmark Datasets," Applied Sciences, Vol. 10, No. 21, pp. 7640, Oct. 2020.
9 A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser and I. Polosukhin, "Attention is All you Need," Part of Advances in Neural Information Processing Systems 30 (NIPS 2017), Vol. 30, pp. 5998-6008, 2017.
10 J. Devlin, M.W. Chang, K. Lee and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint, arXiv:1810.04805, Oct. 2018.
11 A. Conneau, D. Kiela, H. Schwenk, L. Barrault and A. Bordes, "Supervised learning of universal sentence representations from natural language inference data," arXiv preprin, arXiv:1705.02364, May, 2017.
12 Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma and R. Soricut, "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations," arXiv preprint, arXiv:1909.11942, Sep. 2019.
13 C. Xiong, V. Zhong, and R. Socher, "Dynamic coattention networks for question answering," arXiv preprint, arXiv:1611.01604, 2016.
14 P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," arXiv:1606.05250, pp. 2383-2392, Jun. 2016.
15 J. Chung, C. Gulcehre, K.H. Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling," arXiv preprint, arXiv:1412.3555, Dec. 2014.
16 Y.H. Lee and S.B. Lee, "A Research on Enhancement of Text Categorization Performance by using Okapi BM25 Word Weight Method," Journal of the Korea Academia-Industrial cooperation Society, Vol. 11, No. 12, pp. 5089-5096, Dec. 2010.   DOI
17 Z. Huang, W. Xu, and K. Yu, "Bidirectional LSTM-CRF models for sequence tagging," arXiv preprint, arXiv:1508.01991, 2015.
18 M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi, "Bidirectional attention flow for machine comprehension," In International Conference on Learning Representations, Toulon, France, Apr. 2017.
19 J. Pennington, R. Socher, and C.D. Manning, " Glove: Global vectors for word representation," In Proceedings of the 2014 conference on empirical methods in natural language processing, pp. 1532-1543. Doha,Qatar, Oct. 2014.
20 W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou, "Gated self-matching networks for reading comprehension and question answering," In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 189-198, Vancouver, Canada, Jul. 2017.
21 Wikimedia Articles Download, https://dumps.wikimedia.org/enwikisource (accessed Aug., Jun. 30, 2022).