• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• v.2 no.1
• /
• pp.41-60
• /
• 1998

This paper derives a consistent and bias corrected extension of Akaike's Information Criterion (AIC), $AIC_{bc}$, based on Kullback-Leibler information. This criterion has terms that penalize the overparametrization more strongly for small and large samples than that of AIC. The overfitting problem of the asymptotically efficient model selection criteria for small and large samples will be overcome. The $AIC_{bc}$ also provides a consistent model order selection. Thus, it is widely applicable to data with small and/or large sample sizes, and to cases where the number of free parameters is a relatively large fraction of the sample size. Relationships with other model selection criteria such as $AIC_c$ of Hurvich, CAICF of Bozdogan and etc. are discussed. Empirical performances of the $AIC_{bc}$ are studied and discussed in better model order choices of a linear regression model using a Monte Carlo experiment.

• The Pure and Applied Mathematics
• /
• v.4 no.1
• /
• pp.97-104
• /
• 1997

The large deviations theorem of Cramer is extended to conditional probabilities in the following sense. Consider a random sample of pairs of random vectors and the sample means of each of the pairs. The probability that the first falls outside a certain convex set given that the second is fixed is shown to decrease with the sample size at an exponential rate which depends on the Kullback-Leibler distance between two distributions in an associated exponential familiy of distributions. Examples are given which include a method of computing the Bahadur exact slope for tests of certain composite hypotheses in exponential families.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2008.11a
• /
• pp.40-43
• /
• 2008

본 논문에서는 고차원 영역의 데이터 집합을 저차원 영역으로 표현하는 방법에 대해서 제안한다. 특별히 고차원 영역을 2 차원 영역으로 실험하였다. 제안한 방법은 사람이 데이터 객체 사이의 거리나 관계를 직관적으로 인지할 수 있도록 하는 방법이다. 데이터 객체 사이의 거리나 관계를 계산하기 위하여 Kullback-Leibler divergence 를 사용하였다. 이 방법은 확률 분포를 갖는 벡터들 사이의 거리를 계산하여 사용한다. Kullback-Leibler divergence 를 사용하여 계산된 거리 값들은 저차원 영역에서 객체들의 좌표를 계산하기 위하여 사용된다. 좌표계산을 위해서 Simulated Annealing 란 최적화 기법을 사용하였다. 실험 결과를 통해 다차원 데이터를 2 차원 영역으로 표현한 것이 충분히 직관적임을 보였다.

• The Korean Journal of Applied Statistics
• /
• v.20 no.1
• /
• pp.79-89
• /
• 2007

A test for normality introduced by Arizono and Ohta(1989) is based on fullback-Leibler discrimination information. The test statistic is derived from the discrimination information estimated using sample entropy of Vasicek(1976) and the maximum likelihood estimator of the variance. However, these estimators are biased and so it is reasonable to make use of unbiased estimators to accurately estimate the discrimination information. In this paper, Arizono-Ohta test for normality is improved. The derived test statistic is based on the bias-corrected entropy estimator and the uniformly minimum variance unbiased estimator of the variance. The properties of the improved KL test are investigated and Monte Carlo simulation is performed for power comparison.

• The Korean Journal of Applied Statistics
• /
• v.34 no.2
• /
• pp.167-175
• /
• 2021

Semi-supervised learning uses both labeled data and unlabeled data. Recently consistency regularization is very popular in semi-supervised learning. Unsupervised data augmentation (UDA) that uses unlabeled data augmentation is also based on the consistency regularization. The Kullback-Leibler divergence is used for the loss of unlabeled data and cross-entropy for the loss of labeled data through UDA learning. UDA uses techniques such as training signal annealing (TSA) and confidence-based masking to promote performance. In this study, we propose to use Jensen-Shannon divergence instead of Kullback-Leibler divergence, reverse-TSA and not to use confidence-based masking for performance improvement. Through experiment, we show that the proposed technique yields better performance than those of UDA.

• Journal of the Korean Statistical Society
• /
• v.25 no.2
• /
• pp.195-203
• /
• 1996

A bootstrap method is used to correct the apparent downward bias of a naive plug-in bootstrap model selection criterion, which is shown to enjoy a high degree of accuracy. Comparison of bootstrap method with the asymptotic method is made through an illustrative example.

• Journal of the Korean Statistical Society
• /
• v.32 no.3
• /
• pp.261-270
• /
• 2003

This paper introduces a new density estimator which is Hellinger consistent under a simple condition. A number of issues are discussed, such as extension to Kullback-Leibler consistency, robustness, the Bayes version of the estimator and the maximum likelihood case. An illustration is presented.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.41 no.6
• /
• pp.89-97
• /
• 2004

An object to reflect or emit light is captured by imaging system as distorted image due to various distortion. It is called image restoration that estimates original object by removing distortion. There are two categories in image restoration method. One is a deterministic method and the other is a stochastic method. In this paper, image restoration using Minimum Fisher Information(MFI), derived from B. Roy Frieden is proposed. In MFI restoration, experimental results to be made according to noise control parameter were investigated. And cross entropy(Kullback-Leibler entropy) was used as a standard measure of restoration accuracy, It is confirmed that restoration results using MFI have various roughness according to noise control parameter.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.6 no.5
• /
• pp.1463-1478
• /
• 2012

Based on the theory of local-world network, the composition self-similarity (CSS) of network traffic is presented for the first time in this paper for the study of DoS detection. We propose the concept of composition distribution graph and design the relative operations. The $(R/S)^d$ algorithm is designed for calculating the Hurst parameter. Based on composition distribution graph and Kullback Leibler (KL) divergence, we propose the composition self-similarity anomaly detection (CSSD) method for the detection of DoS attacks. We evaluate the effectiveness of the proposed method. Compared to other entropy based anomaly detection methods, our method is more accurate and with higher sensitivity in the detection of DoS attacks.

• Structural Engineering and Mechanics
• /
• v.48 no.3
• /
• pp.291-308
• /
• 2013

Structural health monitoring is crucial to maintain the structural performance safely. Moreover, the Kullback-Leibler divergence (KLD) is applied usually to asset the similarity between different probability density functions in the pattern recognition. In this study, the KLD is employed to detect the damage. However the asymmetry of the KLD is a shortcoming for the damage detection, to overcoming this shortcoming, two other divergences and one statistic distribution are proposed. Then the damage identification by the KLD and its three descriptions from the symmetric point of view is investigated. In order to improve the reliability and accuracy of the four divergences, the gapped smoothing method (GSM) is adopted. On the basis of the damage index approach, the new damage index (DI) for detect damage more accurately based on the four divergences is developed. In the last, the grey relational coefficient and hypothesis test (GRCHT) is utilized to obtain the more precise damage identification results. Finally, a clear remarkable improvement can be observed. To demonstrate the feasibility and accuracy of the proposed method, examples of an isotropic beam with different damage scenarios are employed so as to check the present approaches numerically. The final results show that the developed approach successfully located the damaged region in all cases effect and accurately.