• Journal of the Korean Statistical Society
• /
• v.26 no.1
• /
• pp.131-146
• /
• 1997

In this paper the problem of measuring the stochastic dependence among sets fo random variates is considered, and attention is specifically directed to forming a single well-defined measure of the dependence among sets of normal variates. A new information theoretic measure of the dependence called dependence index (DI) is introduced and its several properties are studied. The development of DI is based on the generalization and normalization of the mutual information introduced by Kullback(1968). For data analysis, minimum cross entropy estimator of DI is suggested, and its asymptotic distribution is obtained for testing the existence of the dependence. Monte Carlo simulations demonstrate the performance of the estimator, and show that is is useful not only for evaluation of the dependence, but also for independent model testing.

• Journal of Korea Society of Industrial Information Systems
• /
• v.5 no.2
• /
• pp.39-46
• /
• 2000

Recent]y van Es (1992) and Correa (1995) proposed an estimator of entropy. In this paper, we proposed the goodness of fit test statistics for exponentiality based on Vasicek's estimator and Correa's estimator of Kullback-Leibier Information. And we compare the power of the proposed test statistics with Kolmogorov-Sminov, Kuiper, Cramer von Mises, Watson, Andersen-Darling and Finkelstein and Schefer statistics.

• Journal of Convergence Society for SMB
• /
• v.6 no.4
• /
• pp.71-77
• /
• 2016

In this paper, we proposed a verification of speaker change utilizing the KL distance based on GMM-UBM to improve the performance of conventional BIC based Speaker Change Detection(SCD). We have verified Conventional BIC-based SCD using KL-distance based SCD which is robust against difference of information volume than BIC-based SCD. And we have applied GMM-UBM to compensate asymmetric information volume. Conventional BIC-based SCD was composed of two steps. Step 1, to detect the Speaker Change Candidate Point(SCCP). SCCP is positive local maximum point of dissimilarity d. Step 2, to determine the Speaker Change Point(SCP). If ${\Delta}BIC$ of SCCP is positive, it decides to SCP. We examined verification of SCP using GMM-UBM based KL distance D. If the value of D on each SCP is higher than threshold, we accepted that point to the final SCP. In the experimental condition MDR(Missed Detection Rate) is 0, FAR(False Alarm Rate) when the threshold value of 0.028 has been improved to 60.7%.

• Journal of Convergence for Information Technology
• /
• v.7 no.6
• /
• pp.159-166
• /
• 2017

In this paper, we have experimented to improve the verification performance of speaker change detection on broadcast news. It is to enhance the input noisy speech and to apply the KL distance $D_s$ using the SNR-based weighting function $w_m$. The basic experimental system is the verification system of speaker change using GMM-UBM based KL distance D(Experiment 0). Experiment 1 applies the input noisy speech enhancement using MMSE Log-STSA. Experiment 2 applies the new KL distance $D_s$ to the system of Experiment 1. Experiments were conducted under the condition of 0% MDR in order to prevent missing information of speaker change. The FAR of Experiment 0 was 71.5%. The FAR of Experiment 1 was 67.3%, which was 4.2% higher than that of Experiment 0. The FAR of experiment 2 was 60.7%, which was 10.8% higher than that of experiment 0.

• Journal of applied mathematics & informatics
• /
• v.26 no.1_2
• /
• pp.177-189
• /
• 2008

In this paper, we have considered an investigation on goodness of fit tests based on divergence measures. In the case of categorical data, under certain regularity conditions, we obtained asymptotic distribution of these tests. Also, we have proposed a modified test that improves the rate of convergence. In continuous case, we used our modified entropy estimator [10], for Kullback-Leibler information estimation. A comparative study based on simulation results is discussed also.

• Communications for Statistical Applications and Methods
• /
• v.12 no.3
• /
• pp.743-758
• /
• 2005

This paper consider penalized likelihood regression with data from exponential family. The fast computation method applied to Gaussian data(Kim and Gu, 2004) is extended to non Gaussian data through asymptotically efficient low dimensional approximations and corresponding algorithm is proposed. Also smoothing parameter selection is explored for various exponential families, which extends the existing cross validation method of Xiang and Wahba evaluated only with Bernoulli data.

• Proceedings of the IEEK Conference
• /
• 2008.06a
• /
• pp.915-916
• /
• 2008

We propose a clustering based object feature matching for identification of same object in multi-camera system. The method is focused on ease to system initialization and extension. Clustering is used to estimate parameters of Gaussian mixture models of objects. A similarity measure between models are determined by Kullback-Leibler divergence. This method can be applied to occlusion problem in tracking.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.41 no.8
• /
• pp.851-858
• /
• 2016

This paper presents a new statistical voice activity detection (VAD) based on the probabilistic interpretation of nonnegative matrix factorization (NMF). The objective function of the NMF using Kullback-Leibler divergence coincides with the negative log likelihood function of the data if the distribution of the data given the basis and encoding matrices is modeled as Poisson distributions. Based on this probabilistic NMF, the VAD is constructed using the likelihood ratio test assuming that speech and noise follow Poisson distributions. Experimental results show that the proposed approach outperformed the conventional Gaussian model-based and NMF-based methods at 0-15 dB signal-to-noise ratio simulation conditions.

• Journal of Computing Science and Engineering
• /
• v.7 no.2
• /
• pp.99-111
• /
• 2013

Approximating a divergence between two probability distributions from their samples is a fundamental challenge in statistics, information theory, and machine learning. A divergence approximator can be used for various purposes, such as two-sample homogeneity testing, change-point detection, and class-balance estimation. Furthermore, an approximator of a divergence between the joint distribution and the product of marginals can be used for independence testing, which has a wide range of applications, including feature selection and extraction, clustering, object matching, independent component analysis, and causal direction estimation. In this paper, we review recent advances in divergence approximation. Our emphasis is that directly approximating the divergence without estimating probability distributions is more sensible than a naive two-step approach of first estimating probability distributions and then approximating the divergence. Furthermore, despite the overwhelming popularity of the Kullback-Leibler divergence as a divergence measure, we argue that alternatives such as the Pearson divergence, the relative Pearson divergence, and the $L^2$-distance are more useful in practice because of their computationally efficient approximability, high numerical stability, and superior robustness against outliers.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.20 no.4
• /
• pp.833-839
• /
• 2016

In this paper, we propose a selective feature extraction algorithm between Markov transition probability and co-occurrence probability for an effective image splicing detection. The Features used in our method are composed of the difference values between DCT coefficients in the adjacent blocks and the value of Kullback-Leibler divergence(KLD) is calculated to evaluate the differences between the distribution of original image features and spliced image features. KLD value is an efficient measure for selecting Markov feature or Co-occurrence feature because KLD shows non-similarity of the two distributions. After training the extracted feature vectors using the SVM classifier, we determine whether the presence of the image splicing forgery. To verify our algorithm we used grid search and 6-folds cross-validation. Based on the experimental results it shows that the proposed method has good detection performance with a limited number of features compared to conventional methods.