Proceedings of the Korea Information Processing Society Conference (한국정보처리학회:학술대회논문집)

  • 2020.11a
  • /
  • Pages.1014-1017
  • /
  • 2020
  • /
  • 2005-0011(pISSN)
  • /
  • 2671-7298(eISSN)
Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

SG-Drop: Faster Skip-Gram by Dropping Context Words

  • Kim, DongJae (Dept. of Electric and Electronics Engineering, Korea University) ;
  • Synn, DoangJoo (Dept. of Electric and Electronics Engineering, Korea University) ;
  • Kim, Jong-Kook (Dept. of Electric and Electronics Engineering, Korea University)
  • Published : 2020.11.05
Download PDF
⟨ Previous Next ⟩

Abstract

Many natural language processing (NLP) models utilize pre-trained word embeddings to leverage latent information. One of the most successful word embedding model is the Skip-gram (SG). In this paper, we propose a Skipgram drop (SG-Drop) model, which is a variation of the SG model. The SG-Drop model is designed to reduce training time efficiently. Furthermore, the SG-Drop allows controlling training time with its hyperparameter. It could train word embedding faster than reducing training epochs while better preserving the quality.

Keywords

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(NRF-2016R1D1A1B04933156) This work was supported in part by the National Research Foundation of Korea (NRF) through the Basic Science Research Program funded by the Ministry of Education under Grant 2014R1A1A2059527, and in part by the Information Technology Research Center (ITRC), Ministry of Science and ICT (MSIT), South Korea, through a Support Program under Grant IITP-2020-2018-0-01433, supervised by the Institute for Information and Communications Technology Promotion (IITP)