• Title/Summary/Keyword: language processing

Search Result 2,686, Processing Time 0.024 seconds

Interference Typo Correction Method by using Surrounding Word N-gram and Syllable N-gram (좌우 어절 N-gram 및 음절 N-gram을 이용한 간섭 오타 교정 방법)

  • Son, Sung-Hwan;Kang, Seung-Shik
    • Annual Conference on Human and Language Technology
    • /
    • 2019.10a
    • /
    • pp.496-499
    • /
    • 2019
  • 스마트폰의 쿼티 자판 소프트 키보드의 버튼과 버튼 사이 좁은 간격으로 인해 사용자가 의도치 않은 간섭 오타가 발생하는 것에 주목하였다. 그리고 오타 교정의 성능은 사용자의 관점에서 얼마나 잘 오타를 교정하느냐도 중요한 부분이지만, 또한 오타가 아닌 어절을 그대로 유지하는 것이 더 중요하게 판단될 수 있다. 왜냐하면 현실적으로 오타인 어절 보다 오타가 아닌 어절이 거의 대부분을 차지하기 때문이다. 따라서 해당 관점에서 교정 방법을 바라보고 연구할 필요가 있다. 이에 맞춰 본 논문에서는 대용량 한국어 말뭉치 데이터를 가지고 확률에 기반한 한국어 간섭 오타 수정 방법에 대해 제안한다. 제안하는 방법은 목표 어절의 좌우 어절 N-gram과 어절 내 좌우 음절 N-gram 정보를 바탕으로 발생할 수 있는 간섭 오타 교정 후보들 중 가운데서 가장 적합한 후보 어절을 선택하는 방법이다.

  • PDF

A Reconsideration of Asymmetries of Bracketing Paradoxes in English Derivation: a Corpus-based Approach

  • Kim, Jin-hyung
    • Journal of English Language & Literature
    • /
    • v.55 no.3
    • /
    • pp.475-495
    • /
    • 2009
  • In this paper, I discuss some asymmetries of bracketing paradoxes from a corpus-based perspective. Through a critical examination of previous analyses of bracketing paradoxes, it is demonstrated that the cases of apparent asymmetries of bracketing paradoxes are consistently accounted for when combined with the frequency-based parsability in morphological processing. Based on the relative frequency, this paper argues that bracketing paradoxes are well-atttested when their immediate bases are frequent and productive enough to be accessed as a unit and stored as such in memory. This is an extension of Hay 2002 which conducted a comprehensive survey of differential frequency effects in suffix pairs. A frequency-based approach to bracketing paradoxes adopted in this paper can be a challenge to the conventional formal theory by assuming a major role of language use and have the potential to significantly advance our understanding of the asymmetries observed in the real language world.

Q&A Chatbot in Arabic Language about Prophet's Biography

  • Somaya Yassin Taher;Mohammad Zubair Khan
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.3
    • /
    • pp.211-223
    • /
    • 2024
  • Chatbots have become very popular in our times and are used in several fields. The emergence of chatbots has created a new way of communicating between human and computer interaction. A Chatbot also called a "Chatter Robot," or conversational agent CA is a software application that mimics human conversations in its natural format, which contains textual material and oral communication with artificial intelligence AI techniques. Generally, there are two types of chatbots rule-based and smart machine-based. Over the years, several chatbots designed in many languages for serving various fields such as medicine, entertainment, and education. Unfortunately, in the Arabic chatbots area, little work has been done. In this paper, we developed a beneficial tool (chatBot) in the Arabic language which contributes to educating people about the Prophet's biography providing them with useful information by using Natural Language Processing.

KOREAN TOPIC MODELING USING MATRIX DECOMPOSITION

  • June-Ho Lee;Hyun-Min Kim
    • East Asian mathematical journal
    • /
    • v.40 no.3
    • /
    • pp.307-318
    • /
    • 2024
  • This paper explores the application of matrix factorization, specifically CUR decomposition, in the clustering of Korean language documents by topic. It addresses the unique challenges of Natural Language Processing (NLP) in dealing with the Korean language's distinctive features, such as agglutinative words and morphological ambiguity. The study compares the effectiveness of Latent Semantic Analysis (LSA) using CUR decomposition with the classical Singular Value Decomposition (SVD) method in the context of Korean text. Experiments are conducted using Korean Wikipedia documents and newspaper data, providing insight into the accuracy and efficiency of these techniques. The findings demonstrate the potential of CUR decomposition to improve the accuracy of document clustering in Korean, offering a valuable approach to text mining and information retrieval in agglutinative languages.

Generative AI as a Virtual Conversation Partner in Language Learning

  • Ji-Young Seo;Seon-Ah, Kim
    • International Journal of Advanced Culture Technology
    • /
    • v.12 no.2
    • /
    • pp.7-15
    • /
    • 2024
  • Despite a recent surge in multifaceted research on AI-integrated language learning, empirical studies in this area remain limited. This study adopts a Human-Generative AI parallel processing model to examine students' perceptions, asking 182 college students to independently construct knowledge and then compare their efforts with the results generated through in-classroom conversations with ChatGPT 3.5. In questionnaire responses, most students indicated that they found these activities useful and expressed a keen interest in learning various ways to utilize generative AI for language learning with instructor guidance. The findings confirm that ChatGPT's potential as a virtual conversation partner. Identifying specific reasons for the perceived usefulness of conversation activities and drawbacks of ChatGPT, this study emphasizes the importance of teachers staying informed about both the latest advances in technology and their limitations. We recommend that teachers endeavor to creatively design various classroom activities using AI technology.

A Novel Instruction Set for Packet Processing of Network ASIP (패킷 프로세싱을 위한 새로운 명령어 셋에 관한 연구)

  • Chung, Won-Young;Lee, Jung-Hee;Lee, Yong-Surk
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.9B
    • /
    • pp.939-946
    • /
    • 2009
  • In this paper, we propose a new network ASIP(Application Specific Instruction-set Processor) which was designed for simulation models by a machine descriptions language LISA(Language for Instruction Set Architecture). This network ASIP is aimed for an exclusive engine undertaking packet processing in a router. To achieve the purpose, we added a new necessary instruction set for processing a general ASIP based on MIPS(Microprocessor without Interlock Pipeline Stages) architecture in high speed. The new instructions can be divided into two groups: a classification instruction group and a modification instruction group, and each group is to be processed by its own functional unit in an execution stage. The functional unit was optimized for area and speed through Verilog HDL, and the result after synthesis was compared with the area and operation delay time. Moreownr, it was allocated to the Macro function ana low-level standardized programming language C using CKF(Compiler Known Function). Consequently, we verified performance improvement achieved by analysis and comparison of execution cycles of application programs.

A Study on the Construction of Financial-Specific Language Model Applicable to the Financial Institutions (금융권에 적용 가능한 금융특화언어모델 구축방안에 관한 연구)

  • Jae Kwon Bae
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.29 no.3
    • /
    • pp.79-87
    • /
    • 2024
  • Recently, the importance of pre-trained language models (PLM) has been emphasized for natural language processing (NLP) such as text classification, sentiment analysis, and question answering. Korean PLM shows high performance in NLP in general-purpose domains, but is weak in domains such as finance, medicine, and law. The main goal of this study is to propose a language model learning process and method to build a financial-specific language model that shows good performance not only in the financial domain but also in general-purpose domains. The five steps of the financial-specific language model are (1) financial data collection and preprocessing, (2) selection of model architecture such as PLM or foundation model, (3) domain data learning and instruction tuning, (4) model verification and evaluation, and (5) model deployment and utilization. Through this, a method for constructing pre-learning data that takes advantage of the characteristics of the financial domain and an efficient LLM training method, adaptive learning and instruction tuning techniques, were presented.

Concept-based Translation System in the Korean Spoken Language Translation System (한국어 대화체 음성언어 번역시스템에서의 개념기반 번역시스템)

  • Choi, Un-Cheon;Han, Nam-Yong;Kim, Jae-Hoon
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.8
    • /
    • pp.2025-2037
    • /
    • 1997
  • The concept-based translation system, which is a part of the Korean spoken language translation system, translates spoken utterances from Korean speech recognizer into one of English, Japanese and Korean in a travel planning task. Our system regulates semantic rather than the syntactic category in order to process the spontaneous speech which tends to be regarded as the one ungrammatical and subject to recognition errors. Utterances are parsed into concept structures, and the generation module produces the sentence of the specified target language. We have developed a token-separator using base-words and an automobile grammar corrector for Korean processing. We have also developed postprocessors for each target language in order to improve the readability of the generation results.

  • PDF

Rate-Controlled Data-Driven Real-Time Stream Processing for an Autonomous Machine (자율 기기를 위한 속도가 제어된 데이터 기반 실시간 스트림 프로세싱)

  • Noh, Soonhyun;Hong, Seongsoo;Kim, Myungsun
    • The Journal of Korea Robotics Society
    • /
    • v.14 no.4
    • /
    • pp.340-347
    • /
    • 2019
  • Due to advances in machine intelligence and increased demands for autonomous machines, the complexity of the underlying software platform is increasing at a rapid pace, overwhelming the developers with implementation details. We attempt to ease the burden that falls onto the developers by creating a graphical programming framework we named Splash. Splash is designed to provide an effective programming abstraction for autonomous machines that require stream processing. It also enables programmers to specify genuine, end-to-end timing constraints, which the Splash framework automatically monitors for violation. By utilizing the timing constraints, Splash provides three key language semantics: timing semantics, in-order delivery semantics, and rate-controlled data-driven stream processing semantics. These three semantics together collectively serve as a conceptual tool that can hide low-level details from programmers, allowing developers to focus on the main logic of their applications. In this paper, we introduce the three-language semantics in detail and explain their function in association with Splash's language constructs. Furthermore, we present the internal workings of the Splash programming framework and validate its effectiveness via a lane keeping assist system.

A study on Korean language processing using TF-IDF (TF-IDF를 활용한 한글 자연어 처리 연구)

  • Lee, Jong-Hwa;Lee, MoonBong;Kim, Jong-Weon
    • The Journal of Information Systems
    • /
    • v.28 no.3
    • /
    • pp.105-121
    • /
    • 2019
  • Purpose One of the reasons for the expansion of information systems in the enterprise is the increased efficiency of data analysis. In particular, the rapidly increasing data types which are complex and unstructured such as video, voice, images, and conversations in and out of social networks. The purpose of this study is the customer needs analysis from customer voices, ie, text data, in the web environment.. Design/methodology/approach As previous study results, the word frequency of the sentence is extracted as a word that interprets the sentence has better affects than frequency analysis. In this study, we applied the TF-IDF method, which extracts important keywords in real sentences, not the TF method, which is a word extraction technique that expresses sentences with simple frequency only, in Korean language research. We visualized the two techniques by cluster analysis and describe the difference. Findings TF technique and TF-IDF technique are applied for Korean natural language processing, the research showed the value from frequency analysis technique to semantic analysis and it is expected to change the technique by Korean language processing researcher.