• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• v.2 no.1
• /
• pp.41-60
• /
• 1998

This paper derives a consistent and bias corrected extension of Akaike's Information Criterion (AIC), $AIC_{bc}$, based on Kullback-Leibler information. This criterion has terms that penalize the overparametrization more strongly for small and large samples than that of AIC. The overfitting problem of the asymptotically efficient model selection criteria for small and large samples will be overcome. The $AIC_{bc}$ also provides a consistent model order selection. Thus, it is widely applicable to data with small and/or large sample sizes, and to cases where the number of free parameters is a relatively large fraction of the sample size. Relationships with other model selection criteria such as $AIC_c$ of Hurvich, CAICF of Bozdogan and etc. are discussed. Empirical performances of the $AIC_{bc}$ are studied and discussed in better model order choices of a linear regression model using a Monte Carlo experiment.

• The Korean Journal of Applied Statistics
• /
• v.20 no.1
• /
• pp.79-89
• /
• 2007

A test for normality introduced by Arizono and Ohta(1989) is based on fullback-Leibler discrimination information. The test statistic is derived from the discrimination information estimated using sample entropy of Vasicek(1976) and the maximum likelihood estimator of the variance. However, these estimators are biased and so it is reasonable to make use of unbiased estimators to accurately estimate the discrimination information. In this paper, Arizono-Ohta test for normality is improved. The derived test statistic is based on the bias-corrected entropy estimator and the uniformly minimum variance unbiased estimator of the variance. The properties of the improved KL test are investigated and Monte Carlo simulation is performed for power comparison.

• Journal of KIISE:Databases
• /
• v.37 no.6
• /
• pp.285-291
• /
• 2010

In this paper, we propose a new paradigm of weighting methods for naive Bayesian learning. We propose more fine-grained weighting methods, called value weighting method, in the context of naive Bayesian learning. While the current weighting methods assign a weight to an attribute, we assign a weight to an attribute value. We develop new methods, using Kullback-Leibler function, for both value weighting and feature weighting in the context of naive Bayesian. The performance of the proposed methods has been compared with the attribute weighting method and general naive bayesian. The proposed method shows better performance in most of the cases.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.5 no.6
• /
• pp.1133-1146
• /
• 2011

Cognitive radio (CR) is considered one of the most promising next-generation communication systems; it has the ability to sense and make use of vacant channels that are unused by licensed users. Reliable detection of the licensed users' signals is an essential element for a CR network. Cooperative spectrum sensing (CSS) is able to offer better sensing performance as compared to individual sensing. The presence of malicious users who falsify sensing data can severely degrade the sensing performance of the CSS scheme. In this paper, we investigate a secure CSS scheme, based on the Kullback-Leibler Divergence (KL-divergence) theory, in order to identify malicious users and mitigate their harmful effect on the sensing performance of CSS in a CR network. The simulation results prove the effectiveness of the proposed scheme.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2008.11a
• /
• pp.40-43
• /
• 2008

본 논문에서는 고차원 영역의 데이터 집합을 저차원 영역으로 표현하는 방법에 대해서 제안한다. 특별히 고차원 영역을 2 차원 영역으로 실험하였다. 제안한 방법은 사람이 데이터 객체 사이의 거리나 관계를 직관적으로 인지할 수 있도록 하는 방법이다. 데이터 객체 사이의 거리나 관계를 계산하기 위하여 Kullback-Leibler divergence 를 사용하였다. 이 방법은 확률 분포를 갖는 벡터들 사이의 거리를 계산하여 사용한다. Kullback-Leibler divergence 를 사용하여 계산된 거리 값들은 저차원 영역에서 객체들의 좌표를 계산하기 위하여 사용된다. 좌표계산을 위해서 Simulated Annealing 란 최적화 기법을 사용하였다. 실험 결과를 통해 다차원 데이터를 2 차원 영역으로 표현한 것이 충분히 직관적임을 보였다.

• Journal of the Korean Statistical Society
• /
• v.29 no.1
• /
• pp.1-8
• /
• 2000

A spectrum minimizing the frequency-domain Kullback-Leibler information number has been proposed and used to modify a spectrum estimate. Some numerical examples have illustrated the KL spectrum estimate is superior to the initial estimate, i.e., the autocovariances obtained by the inverse Fourier transformation of the KL spectrum estimate are closer to the sample autocovariances of the given observations than those of the initial spectrum estimate. Also, it has been shown that a Gaussian autoregressive process associated with the KL spectrum is the closest in the timedomain Kullback-Leibler sense to a Gaussian white noise process subject to given autocovariance constraints. In this paper a corresponding conditional probability theorem is presented, which gives another rationale to the KL spectrum.

• The Pure and Applied Mathematics
• /
• v.4 no.1
• /
• pp.97-104
• /
• 1997

The large deviations theorem of Cramer is extended to conditional probabilities in the following sense. Consider a random sample of pairs of random vectors and the sample means of each of the pairs. The probability that the first falls outside a certain convex set given that the second is fixed is shown to decrease with the sample size at an exponential rate which depends on the Kullback-Leibler distance between two distributions in an associated exponential familiy of distributions. Examples are given which include a method of computing the Bahadur exact slope for tests of certain composite hypotheses in exponential families.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.41 no.6
• /
• pp.89-97
• /
• 2004

An object to reflect or emit light is captured by imaging system as distorted image due to various distortion. It is called image restoration that estimates original object by removing distortion. There are two categories in image restoration method. One is a deterministic method and the other is a stochastic method. In this paper, image restoration using Minimum Fisher Information(MFI), derived from B. Roy Frieden is proposed. In MFI restoration, experimental results to be made according to noise control parameter were investigated. And cross entropy(Kullback-Leibler entropy) was used as a standard measure of restoration accuracy, It is confirmed that restoration results using MFI have various roughness according to noise control parameter.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.6 no.5
• /
• pp.1463-1478
• /
• 2012

Based on the theory of local-world network, the composition self-similarity (CSS) of network traffic is presented for the first time in this paper for the study of DoS detection. We propose the concept of composition distribution graph and design the relative operations. The $(R/S)^d$ algorithm is designed for calculating the Hurst parameter. Based on composition distribution graph and Kullback Leibler (KL) divergence, we propose the composition self-similarity anomaly detection (CSSD) method for the detection of DoS attacks. We evaluate the effectiveness of the proposed method. Compared to other entropy based anomaly detection methods, our method is more accurate and with higher sensitivity in the detection of DoS attacks.

• Nuclear Engineering and Technology
• /
• v.55 no.3
• /
• pp.827-838
• /
• 2023

Centrifugal pump is a key part of nuclear power plant systems, and its health status is critical to the safety and reliability of nuclear power plants. Therefore, fault diagnosis is required for centrifugal pump. Traditional fault diagnosis methods have difficulty extracting fault features from nonlinear and non-stationary signals, resulting in low diagnostic accuracy. In this paper, a new fault diagnosis method is proposed based on the improved particle swarm optimization (IPSO) algorithm-based variational modal decomposition (VMD) and relevance vector machine (RVM). Firstly, a simulation test bench for rotor faults is built, in which vibration displacement signals of the rotor are also collected by eddy current sensors. Then, the improved particle swarm algorithm is used to optimize the VMD to achieve adaptive decomposition of vibration displacement signals. Meanwhile, a screening criterion based on the minimum Kullback-Leibler (K-L) divergence value is established to extract the primary intrinsic modal function (IMF) component. Eventually, the factors are obtained from the primary IMF component to form a fault feature vector, and fault patterns are recognized using the RVM model. The results show that the extraction of the fault information and fault diagnosis classification have been improved, and the average accuracy could reach 97.87%.