• Title/Summary/Keyword: Natural Language Process

Search Result 239, Processing Time 0.028 seconds

Intelligent Information Retrieval Using Interactive Query Processing Agent (대화형 질의 처리 에이전트를 이용한 지능형 정보검색)

  • 이현영;이기오;한용기
    • Journal of the Korea Computer Industry Society
    • /
    • v.4 no.12
    • /
    • pp.901-910
    • /
    • 2003
  • Generally, most commercial retrieval engines adopt boolean query as user's query type. Although boolean query is useful to retrieval engines that need fast retrieval, it is not easy for user to express his demands with boolean operators. So, many researches have been studied for decades about information retrieval systems using natural language query that is convenient for user. To retrieve documents that are suitable for user's demands, they have to express their demands correctly, So, this thesis proposes interactive query process agent using natural language. This agent expresses demands concrete through gradual interaction with user, When users input a natural language Query, this agent analyzes the query and generates boolean query by selecting proper keyword and feedbacks the state of the keyword selected. If the keyword is a synonymy or a polysemy, the agent expands or limits the keyword through interaction with user. It makes user express demands more concrete and improve system performance. So, this agent can improve the precision of Information Retrieval.

  • PDF

A Study on Efficient Natural Language Processing Method based on Transformer (트랜스포머 기반 효율적인 자연어 처리 방안 연구)

  • Seung-Cheol Lim;Sung-Gu Youn
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.4
    • /
    • pp.115-119
    • /
    • 2023
  • The natural language processing models used in current artificial intelligence are huge, causing various difficulties in processing and analyzing data in real time. In order to solve these difficulties, we proposed a method to improve the efficiency of processing by using less memory and checked the performance of the proposed model. The technique applied in this paper to evaluate the performance of the proposed model is to divide the large corpus by adjusting the number of attention heads and embedding size of the BERT[1] model to be small, and the results are calculated by averaging the output values of each forward. In this process, a random offset was assigned to the sentences at every epoch to provide diversity in the input data. The model was then fine-tuned for classification. We found that the split processing model was about 12% less accurate than the unsplit model, but the number of parameters in the model was reduced by 56%.

Exploratory Research on Automating the Analysis of Scientific Argumentation Using Machine Learning (머신 러닝을 활용한 과학 논변 구성 요소 코딩 자동화 가능성 탐색 연구)

  • Lee, Gyeong-Geon;Ha, Heesoo;Hong, Hun-Gi;Kim, Heui-Baik
    • Journal of The Korean Association For Science Education
    • /
    • v.38 no.2
    • /
    • pp.219-234
    • /
    • 2018
  • In this study, we explored the possibility of automating the process of analyzing elements of scientific argument in the context of a Korean classroom. To gather training data, we collected 990 sentences from science education journals that illustrate the results of coding elements of argumentation according to Toulmin's argumentation structure framework. We extracted 483 sentences as a test data set from the transcription of students' discourse in scientific argumentation activities. The words and morphemes of each argument were analyzed using the Python 'KoNLPy' package and the 'Kkma' module for Korean Natural Language Processing. After constructing the 'argument-morpheme:class' matrix for 1,473 sentences, five machine learning techniques were applied to generate predictive models relating each sentences to the element of argument with which it corresponded. The accuracy of the predictive models was investigated by comparing them with the results of pre-coding by researchers and confirming the degree of agreement. The predictive model generated by the k-nearest neighbor algorithm (KNN) demonstrated the highest degree of agreement [54.04% (${\kappa}=0.22$)] when machine learning was performed with the consideration of morpheme of each sentence. The predictive model generated by the KNN exhibited higher agreement [55.07% (${\kappa}=0.24$)] when the coding results of the previous sentence were added to the prediction process. In addition, the results indicated importance of considering context of discourse by reflecting the codes of previous sentences to the analysis. The results have significance in that, it showed the possibility of automating the analysis of students' argumentation activities in Korean language by applying machine learning.

Application Development for Text Mining: KoALA (텍스트 마이닝 통합 애플리케이션 개발: KoALA)

  • Byeong-Jin Jeon;Yoon-Jin Choi;Hee-Woong Kim
    • Information Systems Review
    • /
    • v.21 no.2
    • /
    • pp.117-137
    • /
    • 2019
  • In the Big Data era, data science has become popular with the production of numerous data in various domains, and the power of data has become a competitive power. There is a growing interest in unstructured data, which accounts for more than 80% of the world's data. Along with the everyday use of social media, most of the unstructured data is in the form of text data and plays an important role in various areas such as marketing, finance, and distribution. However, text mining using social media is difficult to access and difficult to use compared to data mining using numerical data. Thus, this study aims to develop Korean Natural Language Application (KoALA) as an integrated application for easy and handy social media text mining without relying on programming language or high-level hardware or solution. KoALA is a specialized application for social media text mining. It is an integrated application that can analyze both Korean and English. KoALA handles the entire process from data collection to preprocessing, analysis and visualization. This paper describes the process of designing, implementing, and applying KoALA applications using the design science methodology. Lastly, we will discuss practical use of KoALA through a block-chain business case. Through this paper, we hope to popularize social media text mining and utilize it for practical and academic use in various domains.

A Study on the Development of Structural Analysis Program using MATLAB Language (MATLAB 언어를 이용한 구조해석 프로그램 개발에 관한 연구)

  • 배동명;강상중
    • Journal of the Korean Society of Fisheries and Ocean Technology
    • /
    • v.36 no.4
    • /
    • pp.347-353
    • /
    • 2000
  • The construction and ability of CAE program are presented. The merit and ability of MATLAB which is widely using in the field of recently engineering and natural science are also introduced. Also, analysis program of frame structure used the MATLAB language which is divide in 4th generation language is presented. In this paper, the proposed program using MATLB language to be based upon the composition of general CAE program is composed to preprocess, solver and post-process procedure. And it is able to carried out the static and eigenvalue analysis of truss structure and two dimensional frame structure. Also, for the sample pre-processing and post-processing, it is used the characteristic of input window and plot window to be made of the various GUI function. Each finite elements to be required for analysis is formulated by the Galerkin's method, as a kind of weighted residual method. For check of the results of calculation for program used in this paper, the results to be calculated using program to be developed by the author was compared with its of ANSYS code for general structural analysis about two dimensional truss and frame structure.

  • PDF

A Study on the Language Independent Dictionary Creation Using International Phoneticizing Engine Technology (국제 음소 기술에 의한 언어에 독립적인 발음사전 생성에 관한 연구)

  • Shin, Chwa-Cheul;Woo, In-Sung;Kang, Heung-Soon;Hwang, In-Soo;Kim, Suk-Dong
    • The Journal of the Acoustical Society of Korea
    • /
    • v.26 no.1E
    • /
    • pp.1-7
    • /
    • 2007
  • One result of the trend towards globalization is an increased number of projects that focus on natural language processing. Automatic speech recognition (ASR) technologies, for example, hold great promise in facilitating global communications and collaborations. Unfortunately, to date, most research projects focus on single widely spoken languages. Therefore, the cost to adapt a particular ASR tool for use with other languages is often prohibitive. This work takes a more general approach. We propose an International Phoneticizing Engine (IPE) that interprets input files supplied in our Phonetic Language Identity (PLI) format to build a dictionary. IPE is language independent and rule based. It operates by decomposing the dictionary creation process into a set of well-defined steps. These steps reduce rule conflicts, allow for rule creation by people without linguistics training, and optimize run-time efficiency. Dictionaries created by the IPE can be used with the Sphinx speech recognition system. IPE defines an easy-to-use systematic approach that can lead to internationalization of automatic speech recognition systems.

An Efficient Machine Learning-based Text Summarization in the Malayalam Language

  • P Haroon, Rosna;Gafur M, Abdul;Nisha U, Barakkath
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.6
    • /
    • pp.1778-1799
    • /
    • 2022
  • Automatic text summarization is a procedure that packs enormous content into a more limited book that incorporates significant data. Malayalam is one of the toughest languages utilized in certain areas of India, most normally in Kerala and in Lakshadweep. Natural language processing in the Malayalam language is relatively low due to the complexity of the language as well as the scarcity of available resources. In this paper, a way is proposed to deal with the text summarization process in Malayalam documents by training a model based on the Support Vector Machine classification algorithm. Different features of the text are taken into account for training the machine so that the system can output the most important data from the input text. The classifier can classify the most important, important, average, and least significant sentences into separate classes and based on this, the machine will be able to create a summary of the input document. The user can select a compression ratio so that the system will output that much fraction of the summary. The model performance is measured by using different genres of Malayalam documents as well as documents from the same domain. The model is evaluated by considering content evaluation measures precision, recall, F score, and relative utility. Obtained precision and recall value shows that the model is trustable and found to be more relevant compared to the other summarizers.

On the Automatic Generation of Illustrations for Events in Storybooks: Representation of Illustrative Events (동화책에서의 삽화 자동 생성 -삽화를 위한 사건 표현)

  • Baek, Seung-Cheol;Lee, Hee-Jin;Park, Jong-C.
    • 한국HCI학회:학술대회논문집
    • /
    • 2008.02a
    • /
    • pp.390-396
    • /
    • 2008
  • Storybooks, especially those for children, may contain illustrations. An automated system for generating illustrations would help the production process of storybook publishing. In this paper, we propose a method for automatically generating layouts of objects during generating illustrations. In generated layouts, it is preferred to avoid unnecessary overlap between objects, corresponding to the spatial information in storybooks. We first define a representation scheme for spatial information in natural language sentences using tree structures and predicate-argument structures. Unification of tree structures and Region Connection Calculus are then used to manipulate the information and generate corresponding illustrations.

  • PDF

Automatic Scoring System for Korean Short Answers by Student Answer Analysis and Answer Template Construction (학생 답안 분석과 정답 템플릿 생성에 의한 한국어 서답형 문항의 자동채점 시스템)

  • Kang, SeungShik;Jang, EunSeo
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.5
    • /
    • pp.218-224
    • /
    • 2016
  • This paper proposes a computer-based practical automatic scoring system for Korean short answers through student answer analysis and natural language processing techniques. The proposed system reduces the overall scoring time and budget, while improving the ease-of-use to write answer templates from student answers as well as the accuracy and reliability of automatic scoring system. To evaluate the application of the automatic scoring system and compare to the human scoring process, we performed an experiment using the student answers of social science subject in 2014 National Assessment of Educational Achievement.

Chatbot UX in a Mobile Environment (모바일 환경에서의 챗봇 UX)

  • Lee, Young-Ju
    • Journal of Digital Convergence
    • /
    • v.17 no.11
    • /
    • pp.517-522
    • /
    • 2019
  • In many businesses, chatbots enhance the user experience by providing the most immediate and direct feedback to user questions. The area of use of chatbots is growing. In this study, the three types of chatbot definition, command method, function, and platform are classified according to their distinct factors. In the process, the functional delimiter element is necessary for the Chatbot UX, which is a key technical element of the functional part of pattern recognition, natural language processing, semantic web, text mining, and context-aware computing. However, the limitations at this stage were also known. Based on this, we analyzed the chatbot's UX elements for Facebook, Skype, Telegram, and Google Assistant for a better user experience. Basic UI elements such as cards, quick response, command, and application of persistent menus are needed as user experience elements.