• ETRI Journal
• /
• v.46 no.1
• /
• pp.106-117
• /
• 2024

Large language models (LLMs) have revolutionized various applications in natural language processing and exhibited proficiency in generating programming code. We propose a framework for evaluating the code generation ability of LLMs and introduce a new metric, pass-ratio@n, which captures the granularity of accuracy according to the pass rate of test cases. The framework is intended to be fully automatic to handle the repetitive work involved in generating prompts, conducting inferences, and executing the generated codes. A preliminary evaluation focusing on the prompt detail, problem publication date, and difficulty level demonstrates the successful integration of our framework with the LeetCode coding platform and highlights the applicability of the pass-ratio@n metric.

• The KIPS Transactions:PartD
• /
• v.8D no.3
• /
• pp.233-240
• /
• 2001

As software technology is in progress and software quantification is getting more important, many metrics have been proposed to quantify a variety of system entities. These metrics can be classified into two different forms : scalar metric and metric vector. Though some recent studies pointed out the composition problem of the scalar metric form, many scalar metrics are successfully used in software development organizations due to their practical applications. In this paper, it is concluded that hybrid metric form weighting external complexity is most suitable for scalar metric form. With this concept, a general framework for hybrid metrics construction independent of the development methodologies and target system type is proposed. This framework was successfully used in two projects that quantify the analysis phase of the structured methodology and the design phase of the object oriented real-time system, respectively. Any organization can quantify system entities in a short time using this framework.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.7 no.4
• /
• pp.819-833
• /
• 2013

Computing global features based on local features using a bio-inspired framework has shown promising performance. However, for some tough applications with large intra-class variances, a single local feature is inadequate to represent all the attributes of the images. To integrate the complementary abilities of multiple local features, in this paper we have extended the efficacy of the bio-inspired framework, HMAX, to adapt heterogeneous features for global feature extraction. Given multiple global features, we propose an approach, designated as parameterized metric learning, for high dimensional feature fusion. The fusion parameters are solved by maximizing the canonical correlation with respect to the parameters. Experimental results show that our method achieves significant improvements over the benchmark bio-inspired framework, HMAX, and other related methods on the Caltech dataset, under varying numbers of training samples and feature elements.

• Proceedings of the Korean Operations and Management Science Society Conference
• /
• 2005.05a
• /
• pp.579-584
• /
• 2005

A metric is a measure of one factor of a company's performance. The metrics are used to monitor the overall performance of the company for achieving business objectives. Insufficient metrics cannot reflect company's conditions. Therefore, it is important to be equipped with 'good' metrics. This study introduces the concept of metric quality and proposes its dimensions. The study also presents application scenarios that show the role and usefulness of the metric quality.

• ETRI Journal
• /
• v.32 no.5
• /
• pp.810-818
• /
• 2010

This paper presents a new framework for integrating untranscribed spoken content into the acoustic training of an automatic speech recognition system. Untranscribed spoken content plays a very important role for under-resourced languages because the production of manually transcribed speech databases still represents a very expensive and time-consuming task. We proposed two new methods as part of the training framework. The first method focuses on combining initial acoustic models using a data-driven metric. The second method proposes an improved acoustic training procedure based on unsupervised transcriptions, in which word endings were modified by broad phonetic classes. The training framework was applied to baseline acoustic models using untranscribed spoken content from parliamentary debates. We include three types of acoustic models in the evaluation: baseline, reference content, and framework content models. The best overall result of 18.02% word error rate was achieved with the third type. This result demonstrates statistically significant improvement over the baseline and reference acoustic models.

• Nonlinear Functional Analysis and Applications
• /
• v.28 no.3
• /
• pp.753-774
• /
• 2023

The fundamental aim of the proposed work is to introduce the concept of soft rectangular b-metric spaces, which involves generalizing the notions of rectangular metric spaces and b-metric spaces. Furthermore, an investigation into specific characteristics and topological aspects of the underlying generalization of metric spaces is conducted. Moreover, the research establishes fixed point theorems for mappings that satisfy essential criteria within soft rectangular b-metric spaces. These theorems offer a broader perspective on established results in fixed point theory. Additionally, several congruous examples are presented to enhance the understanding of the introduced spatial framework.

• Journal of KIISE:Software and Applications
• /
• v.34 no.6
• /
• pp.572-583
• /
• 2007

This paper investigates Centering-based metrics to evaluate ordering of utterances for text structuring. We point out a problem of MIN.NOCB metric which has been regarded as the simplest and best measure to evaluate coherence of ordering within Centering framework, and propose a new Centering-based metric, MAX.CPS as an alternative or supplementary one. This paper introduces a framework which pre-estimates the effectiveness of a metric on a given input ordering, and selects an applicable metric according to the pre-estimation result. Using this framework, we propose a new policy which can generate more optimal ordering within Centering framework. Moreover, we evaluate several kinds of Cf-ranking methods in terms of Centering-based metrics, and find that simply ranking entities by their linear order is generally the most suitable because of characteristics in Korean.

• The Journal of the Korea Contents Association
• /
• v.10 no.6
• /
• pp.134-143
• /
• 2010

This paper proposes a framework for classifying metric-based software quality prediction models, especially case of software criticality, into four types. Models are classified along two vectors: input metric forms and the necessity of past project data. Each type has its own characteristics and its strength and weakness are compared with those of other types using newly defined criteria. Through this qualitative evaluation each organization can choose a proper model to suit its environment. My earlier studies of criticality prediction model implemented specific models in each type and evaluated their prediction performances. In this paper I analyze the experimental results and show that the characteristics of a model type is the another key of successful model selection.

• Journal of Korea Multimedia Society
• /
• v.11 no.6
• /
• pp.737-745
• /
• 2008

In this paper, a novel neighborhood metric of histogram equalization (HE) algorithm for contrast enhancement is presented. We present a refinement of HE using neighborhood metrics with a general framework which orders pixels based on a sequence of sorting functions which uses both global and local information to remap the image greylevels. We tested a novel sorting key with the suggestion of using the original image greylevel as the primary key and a novel neighborhood distinction metric as the secondary key, and compared HE using proposed distinction metric and other HE methods such as global histogram equalization (GHE), HE using voting metric and HE using contrast difference metric. We found that our method can preserve advantages of other metrics, while reducing drawbacks of them and avoiding undesirable over-enhancement that can occur with local histogram equalization (LHE) and other methods.

• Journal of applied mathematics & informatics
• /
• v.31 no.5_6
• /
• pp.643-649
• /
• 2013

The purpose of this paper is to establish some coupled coincidence point theorems for a pair of mappings having a strict mixed g-monotone property satisfying a contractive condition of rational type in the framework of partially ordered metric spaces. Also, we present a result on the existence and uniqueness of coupled common fixed points. The results presented in the paper generalize and extend several well-known results in the literature.