• Electronics and Telecommunications Trends
• /
• v.39 no.1
• /
• pp.95-105
• /
• 2024

In natural language processing, large language models such as GPT-4 have recently been in the spotlight. The performance of natural language processing has advanced dramatically driven by an increase in the number of model parameters related to the number of acceptable input tokens and model size. Research on multimodal models that can simultaneously process natural language and image data is being actively conducted. Moreover, natural-language and image-based reasoning capabilities of large language models is being explored in robot artificial intelligence technology. We discuss research and related patent trends in robot task planning and code generation for robot control using large language models.

• International Journal of Internet, Broadcasting and Communication
• /
• v.12 no.2
• /
• pp.45-50
• /
• 2020

A Language model with neural networks commonly trained with likelihood loss. Such that the model can learn the sequence of human text. State-of-the-art results achieved in various language generation tasks, e.g., text summarization, dialogue response generation, and text generation, by utilizing the language model's next token output probabilities. Monotonous and boring outputs are a well-known problem of this model, yet only a few solutions proposed to address this problem. Several decoding techniques proposed to suppress repetitive tokens. Unlikelihood training approached this problem by penalizing candidate tokens probabilities if the tokens already seen in previous steps. While the method successfully showed a less repetitive generated token, the method has a large memory consumption because of the training need a big vocabulary size. We effectively reduced memory footprint by encoding words as sequences of subword units. Finally, we report competitive results with token level unlikelihood training in several automatic evaluations compared to the previous work.

• Journal of Korea Multimedia Society
• /
• v.20 no.2
• /
• pp.371-381
• /
• 2017

The performance of the statistical context-sensitive spelling error correction depends on the quality and quantity of the data for statistical language model. In general, the size and quality of data in a statistical language model are proportional. However, as the amount of data increases, the processing speed becomes slower and storage space also takes up a lot. We suggest the improved statistical language model to solve this problem. And we propose an effective spelling error candidate generation method based on a new statistical language model. The proposed statistical model and the correction method based on it improve the performance of the spelling error correction and processing speed.

• Electronics and Telecommunications Trends
• /
• v.38 no.6
• /
• pp.1-11
• /
• 2023

Large language models seem promising for handling reasoning problems, but their underlying solving mechanisms remain unclear. Large language models will establish a new paradigm in artificial intelligence and the society as a whole. However, a major challenge of large language models is the massive resources required for training and operation. To address this issue, researchers are actively exploring compact large language models that retain the capabilities of large language models while notably reducing the model size. These research efforts are mainly focused on improving pretraining, instruction tuning, and alignment. On the other hand, chain-of-thought prompting is a technique aimed at enhancing the reasoning ability of large language models. It provides an answer through a series of intermediate reasoning steps when given a problem. By guiding the model through a multistep problem-solving process, chain-of-thought prompting may improve the model reasoning skills. Mathematical reasoning, which is a fundamental aspect of human intelligence, has played a crucial role in advancing large language models toward human-level performance. As a result, mathematical reasoning is being widely explored in the context of large language models. This type of research extends to various domains such as geometry problem solving, tabular mathematical reasoning, visual question answering, and other areas.

• Journal of the Korean Society for information Management
• /
• v.16 no.2
• /
• pp.49-80
• /
• 1999

This study analyses the researches on automatic query modification, expansion and combination based on search results of natural language queries and gives a conceptual framework for the factors affecting the effectiveness of the relevance feedback. The operating and experimental systems based on the vector space model, the binary independence model and the inference net model are reviewed, and it is found that the effectiveness of query expansion is affected by conceptual models, algorithms for weighting terms and documents and selecting query terms to be added, size of relevant and non-relevant documents to be used and size of terms to be added in relevance feedback, query length, type and size of DBs, etc.

• Language and Information
• /
• v.7 no.2
• /
• pp.103-115
• /
• 2003

In recent years, many organizations have been constructing their own large corpora to achieve corpus representativeness. However, there is no reliable guideline as to how large corpus resources should be compiled, especially for Korean corpora. In this study, we have contrived a new statistical model, ARIMA (Autoregressive Integrated Moving Average), for predicting the relationship between linguistic items (the number of types) and corpus size (the number of tokens), overcoming the major flaws of several previous researches on this issue. Finally, we shall illustrate that the ARIMA model presented is valid, accurate and very reliable. We are confident that this study can contribute to solving some inherent problems of corpus linguistics, such as corpus predictability, corpus representativeness and linguistic comprehensiveness.

• Journal of Korean language education
• /
• v.28 no.4
• /
• pp.87-119
• /
• 2017

The purpose of this study is to classify research methods used in Korean language education studies with a focus on identifying how and what quantitative research methods are utilized in these studies. Analyzing articles published in the Journal of Korean Language Education from 2005 to 2016, we found a trend that as a replacement for secondary research, primary research played a more prominent role after 2010, as the number of quantitative studies and studies using mixed methods increased. We also found that within quantitative studies of Korean language education, research themes and statistical analyses became diversified after 2010. In order for quantitative research to contribute continuously to Korean language education, the quality of research has to improve. In particular, quantitative researchers in this area should: (a) increase their general understanding of statistical methods, (b) conduct "power analysis" to determine the appropriate sample size for hypothesis testing, and (c) be aware of measurement issues such as measurement equivalence and DIF when measuring latent psychological constructs. It is also important to notice that these points above should be considered carefully in the planning and designing stage for researchers.

• Phonetics and Speech Sciences
• /
• v.6 no.2
• /
• pp.65-71
• /
• 2014

The purpose of this study was to investigate whether young children with cleft palate differ from those of noncleft typically developing children in terms of expressive vocabulary size, phonological characteristics and lexical selectivity. A total of 12 children with cleft palate and 12 noncleft children who were matched by age and gender participated in the study. The groups were compared by size of expressive vocabulary reported on Korean version of MacArthur-Bates Communicative Development Inventories and the number of different words, consonant inventory, the percentage of words beginning with obstruents and vowels, nasal, and glottal sounds, and the percentage of words which do not include obstruents in a language sample. Also, correlation analysis were performed to examine the relationship between measures on size of expressive vocabulary and phonological characteristics. The results showed that expressive vocabulary size and consonant inventory for children with cleft palate produced significantly smaller than those for noncleft children. Children with cleft palate produced significantly more words beginning with vowel or which do not include obstruents, and fewer words beginning with obstruents than noncleft children. The two groups showed different results on significant correlations between measures on size of expressive vocabulary and phonological characteristics indicating that children with cleft palate show different lexical selectivity from their noncleft peers. The results suggest that children with cleft palate aged 18-30 months demonstrate a slower rate of lexical and phonological development compared with their noncleft peers and they develop lexical selectivity reflecting cleft palate speech. The results will have a clinical implication on speech-language intervention for young children with cleft palates.

• Proceedings of the Korean Society of Propulsion Engineers Conference
• /
• 2012.05a
• /
• pp.403-407
• /
• 2012

The purpose of this study is to find the optimum skirt size for a composite pressure vessel using optimum analysis technique. The size optimization for skirt shape of a composite pressure vessel was conducted using sub-problem approximation method and batch processing codes programmed by APDL(ANSYS Parametric Design Language). The thickness and length of skirt part were selected as design variables for the optimum analysis. The objective function and constraints were chosen as weight and displacement of skirt part, respectively. The numerical results showed that the weight of skirt of a composite pressure vessel would be saved by maximum 4.38% through the size optimization analysis for the skirt shape.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.16 no.3
• /
• pp.1707-1714
• /
• 2015

The neural bases underlying within or between-language picture naming was investigated by using event-related fMRI. The present suudy explorered the following two goals: The first is to compare cortical activation areas relevant to naming process in native and foreign language, and to decide whether the activation pattern of the foreign word will be the same as native words or not. The next is to find the cerebral areas involved only in alternating language switching between native and foreign language condition. Differential activation patterns were observed for language switching against one-language. Both naming tasks all activated the left inferior frontal gyrus (LIFG) as expected. However the differences in naming between languages were reflected in the activation amount of the LIFG, namely more activation in naming the native language than the foreign language. Especially, naming of the foreign word from English showed the similar area and size in activation with native language suggesting that the process of borrowed noun resembles that of native common noun. And the language switching between languages newly activated the right middle frontal gyrus as well as the left inferior frontal areas. The right middle frontal gyrus engagement in switching conditions obviously identified that right hemisphere is recruited in code switching possibly with respect to meta-cognition controlling language index at a subconscious level.