Browse > Article
http://dx.doi.org/10.9708/jksci.2021.26.01.057

Text Augmentation Using Hierarchy-based Word Replacement  

Kim, Museong (Graduate School of Business IT, Kookmin University)
Kim, Namgyu (Graduate School of Business IT, Kookmin University)
Publication Information
Journal of the Korea Society of Computer and Information / v.26, no.1, 2021 , pp. 57-67 More about this Journal
Abstract
Recently, multi-modal deep learning techniques that combine heterogeneous data for deep learning analysis have been utilized a lot. In particular, studies on the synthesis of Text to Image that automatically generate images from text are being actively conducted. Deep learning for image synthesis requires a vast amount of data consisting of pairs of images and text describing the image. Therefore, various data augmentation techniques have been devised to generate a large amount of data from small data. A number of text augmentation techniques based on synonym replacement have been proposed so far. However, these techniques have a common limitation in that there is a possibility of generating a incorrect text from the content of an image when replacing the synonym for a noun word. In this study, we propose a text augmentation method to replace words using word hierarchy information for noun words. Additionally, we performed experiments using MSCOCO data in order to evaluate the performance of the proposed methodology.
Keywords
Deep Learning; Generative Adversarial Network; Text to Image Synthesis; Data Augmentation; WordNet;
Citations & Related Records
연도 인용수 순위
  • Reference
1 I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," Advances in Neural Information Processing Systems 27, 2014.
2 S. Surya, A. Setlur, A. Biswas, and S. Negi, "ReStGAN: A Step towards Visually Guided Shopper Experience via Text to Image Synthesis," 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Mar, 2020.
3 C. Shorten and T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," Journal of Big Data, No. 60, Feb, 2019.
4 X. Zhang, J. Zhao, and Y. LeCun, "Character-level Convolutional Networks for Text Classification," Advances in Neural Information Processing Systems 28, 2015.
5 W. Y. Wang and D. Yang, "That's So Annoying!!!: A Lexical and Frame Semantic Embedding-based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors Using #petpeeve Tweets," Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2557-2563, Sep, 2015.
6 T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, "Improved Techniques for Training GANs," Advances in Neural Information Processing Systems 29, 2016.
7 T. Mikolov, S. Kombrink, L. Burget, J. Cernocky, and S. Khudanpur, "Extensions of Recurrent Neural Network Language Model," Proceedings of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5528-5531, 2011.
8 S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative Adversarial Text to Image Synthesis," arXiv:1605.05396, May, 2016.
9 T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, "AttnGAN: Fine Grained Text to Image Generation with Attentional Generative Adversarial Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1316-1324, 2018.
10 H. Zhang, T. Xu, H. Li, S. Zhang, X. Wang, X. Huang, and D. N. Metaxas, "StackGAN: Text to Photo Realistic Image Synthesis with Stacked Generative Adversarial Networks," Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5907-5915, 2017.
11 T. DeVries and G. W. Taylor, "Dataset Augmentation in Feature Space," arXiv:1702.05538, Feb, 2017.
12 Y. Li, N. Wang, J. Liu, and X. Hou, "Demystifying Neural Style Transfer," arXiv:1701.01036, Jul, 2017.
13 C. Bowles, L. Chen, R. Guerrero, P. Bentley, R. Gunn, A. Hammers, D. A. Dickie, M. V. Hernandez, J. Wardlaw, and D. Rueckert, "GAN Augmentation: Augmenting Training Data Using Generative Adversarial Networks," arXiv:1810.10863, Oct, 2018.
14 T. Mikolov, C. Kai, G. Corrado, and J. Dean, "Efficient Estimation of Word Representations in Vector Space," arXiv:1301.3781, Jan, 2013.
15 Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based Learning Applied to Document Recognition," Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998.   DOI
16 P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching Word Vectors with Subword Information," arXiv:1607.04606, Jul, 2016.
17 Q. Xie, Z. Dai, E. Hovy, M. T. Luong, and Q. V. Le, "Unsupervised Data Augmentation for Consistency Training," arXiv:1904.12848, Jun, 2020.
18 J. Pennington, R. Socher, and C. D. Manning, "Glove: Global Vectors for Word Representation," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014.
19 J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," arXiv:1810.04805, May, 2019.
20 A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language Models are Unsupervised Multitask Learners," https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf, Feb, 2019.
21 A. Anaby-Tavor, B. Carmeli, E. Goldbraich, A. Kantor, G. Kour, S. Shlomov, N. Tepper, and N. Zwerdling, "Not Enough Data? Deep Learning to the Rescue!," arXiv:1911.03118, Nov, 2019.
22 V. Kumar, A. Choudhary, and E. Cho, "Data Augmentation Using Pre-trained Transformer Models," arXiv:2003.02245, Mar, 2020.
23 E. Loper and S. Bird, "NLTK: The Natural Language Toolkit," arXiv:cs/0205028, May, 2002.
24 Y. Tian and D. Lo, "A comparative study on the effectiveness of part-of-speech tagging techniques on bug reports," 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Mar, 2015.
25 T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar and C. L. Zitnick, "Microsoft COCO: Common Objects in Context," European Conference on Computer Vision, pp. 740-755, 2014.