• Title/Summary/Keyword: embedding based context-sensitive spelling error correction

Search Result 2, Processing Time 0.015 seconds

Context-Sensitive Spelling Error Correction Techniques in Korean Documents using Generative Adversarial Network (생성적 적대 신경망(GAN)을 이용한 한국어 문서에서의 문맥의존 철자오류 교정)

  • Lee, Jung-Hun;Kwon, Hyuk-Chul
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.10
    • /
    • pp.1391-1402
    • /
    • 2021
  • This paper focuses use context-sensitive spelling error correction using generative adversarial network. Generative adversarial network[1] are attracting attention as they solve data generation problems that have been a challenge in the field of deep learning. In this paper, sentences are generated using word embedding information and reflected in word distribution representation. We experiment with DCGAN[2] used for the stability of learning in the existing image processing and D2GAN[3] with double discriminator. In this paper, we experimented with how the composition of generative adversarial networks and the change of learning corpus influence the context-sensitive spelling error correction In the experiment, we correction the generated word embedding information and compare the performance with the actual word embedding information.

Performance Improvement of Context-Sensitive Spelling Error Correction Techniques using Knowledge Graph Embedding of Korean WordNet (alias. KorLex) (한국어 어휘 의미망(alias. KorLex)의 지식 그래프 임베딩을 이용한 문맥의존 철자오류 교정 기법의 성능 향상)

  • Lee, Jung-Hun;Cho, Sanghyun;Kwon, Hyuk-Chul
    • Journal of Korea Multimedia Society
    • /
    • v.25 no.3
    • /
    • pp.493-501
    • /
    • 2022
  • This paper is a study on context-sensitive spelling error correction and uses the Korean WordNet (KorLex)[1] that defines the relationship between words as a graph to improve the performance of the correction[2] based on the vector information of the word embedded in the correction technique. The Korean WordNet replaced WordNet[3] developed at Princeton University in the United States and was additionally constructed for Korean. In order to learn a semantic network in graph form or to use it for learned vector information, it is necessary to transform it into a vector form by embedding learning. For transformation, we list the nodes (limited number) in a line format like a sentence in a graph in the form of a network before the training input. One of the learning techniques that use this strategy is Deepwalk[4]. DeepWalk is used to learn graphs between words in the Korean WordNet. The graph embedding information is used in concatenation with the word vector information of the learned language model for correction, and the final correction word is determined by the cosine distance value between the vectors. In this paper, In order to test whether the information of graph embedding affects the improvement of the performance of context- sensitive spelling error correction, a confused word pair was constructed and tested from the perspective of Word Sense Disambiguation(WSD). In the experimental results, the average correction performance of all confused word pairs was improved by 2.24% compared to the baseline correction performance.