• Journal of Computing Science and Engineering
• /
• 제7권2호
• /
• pp.99-111
• /
• 2013

Approximating a divergence between two probability distributions from their samples is a fundamental challenge in statistics, information theory, and machine learning. A divergence approximator can be used for various purposes, such as two-sample homogeneity testing, change-point detection, and class-balance estimation. Furthermore, an approximator of a divergence between the joint distribution and the product of marginals can be used for independence testing, which has a wide range of applications, including feature selection and extraction, clustering, object matching, independent component analysis, and causal direction estimation. In this paper, we review recent advances in divergence approximation. Our emphasis is that directly approximating the divergence without estimating probability distributions is more sensible than a naive two-step approach of first estimating probability distributions and then approximating the divergence. Furthermore, despite the overwhelming popularity of the Kullback-Leibler divergence as a divergence measure, we argue that alternatives such as the Pearson divergence, the relative Pearson divergence, and the $L^2$-distance are more useful in practice because of their computationally efficient approximability, high numerical stability, and superior robustness against outliers.

• 한국음향학회지
• /
• 제35권2호
• /
• pp.83-91
• /
• 2016

음악 유사도 계산은 음악 검색 및 분류 등의 정보 처리 시스템 구현에 있어서 가장 중요한 부분이다. 본 논문은 최근 제안된 무게중심 모델을 이용한 음악 검색 방법에 대해서 살펴보고, 무게중심 모델의 확률 분포 유사도를 이용하여 음악 검색을 수행하고 성능을 평가하였다. 확률 분포간의 거리는 주어진 두 개의 확률 분포가 특정 기준에서 얼마나 가까운 지를 계산하는 것으로 다이버전스라고 불리기도 한다. 본 논문에서는 무게중심 모델에서 확률 분포 간의 거리 비교 시에 알파 다이버전스를 활용하였다. 알파 다이버전스는 알파 값에 따라 다양한 형태를 가지며, 널리 사용되고 있는 KLD(Kullback-Leibler)와 BD(Bhattacharyya Distance)를 포함한다. 음악 장르와 가수 데이터셋에서 검색 실험을 수행했고, 확률 분포 거리 기반 유사도와 벡터 거리 기반 유사도의 음악 검색 성능을 비교하였다. 알파 다이버전스를 통해서 무게중심 모델 기반 음악 검색 성능을 개선시킬 수 있음을 보였다.

• Journal of applied mathematics & informatics
• /
• 제26권1_2호
• /
• pp.177-189
• /
• 2008

In this paper, we have considered an investigation on goodness of fit tests based on divergence measures. In the case of categorical data, under certain regularity conditions, we obtained asymptotic distribution of these tests. Also, we have proposed a modified test that improves the rate of convergence. In continuous case, we used our modified entropy estimator [10], for Kullback-Leibler information estimation. A comparative study based on simulation results is discussed also.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제6권5호
• /
• pp.1463-1478
• /
• 2012

Based on the theory of local-world network, the composition self-similarity (CSS) of network traffic is presented for the first time in this paper for the study of DoS detection. We propose the concept of composition distribution graph and design the relative operations. The $(R/S)^d$ algorithm is designed for calculating the Hurst parameter. Based on composition distribution graph and Kullback Leibler (KL) divergence, we propose the composition self-similarity anomaly detection (CSSD) method for the detection of DoS attacks. We evaluate the effectiveness of the proposed method. Compared to other entropy based anomaly detection methods, our method is more accurate and with higher sensitivity in the detection of DoS attacks.

• Structural Engineering and Mechanics
• /
• 제48권3호
• /
• pp.291-308
• /
• 2013

Structural health monitoring is crucial to maintain the structural performance safely. Moreover, the Kullback-Leibler divergence (KLD) is applied usually to asset the similarity between different probability density functions in the pattern recognition. In this study, the KLD is employed to detect the damage. However the asymmetry of the KLD is a shortcoming for the damage detection, to overcoming this shortcoming, two other divergences and one statistic distribution are proposed. Then the damage identification by the KLD and its three descriptions from the symmetric point of view is investigated. In order to improve the reliability and accuracy of the four divergences, the gapped smoothing method (GSM) is adopted. On the basis of the damage index approach, the new damage index (DI) for detect damage more accurately based on the four divergences is developed. In the last, the grey relational coefficient and hypothesis test (GRCHT) is utilized to obtain the more precise damage identification results. Finally, a clear remarkable improvement can be observed. To demonstrate the feasibility and accuracy of the proposed method, examples of an isotropic beam with different damage scenarios are employed so as to check the present approaches numerically. The final results show that the developed approach successfully located the damaged region in all cases effect and accurately.

• Nuclear Engineering and Technology
• /
• 제55권3호
• /
• pp.827-838
• /
• 2023

Centrifugal pump is a key part of nuclear power plant systems, and its health status is critical to the safety and reliability of nuclear power plants. Therefore, fault diagnosis is required for centrifugal pump. Traditional fault diagnosis methods have difficulty extracting fault features from nonlinear and non-stationary signals, resulting in low diagnostic accuracy. In this paper, a new fault diagnosis method is proposed based on the improved particle swarm optimization (IPSO) algorithm-based variational modal decomposition (VMD) and relevance vector machine (RVM). Firstly, a simulation test bench for rotor faults is built, in which vibration displacement signals of the rotor are also collected by eddy current sensors. Then, the improved particle swarm algorithm is used to optimize the VMD to achieve adaptive decomposition of vibration displacement signals. Meanwhile, a screening criterion based on the minimum Kullback-Leibler (K-L) divergence value is established to extract the primary intrinsic modal function (IMF) component. Eventually, the factors are obtained from the primary IMF component to form a fault feature vector, and fault patterns are recognized using the RVM model. The results show that the extraction of the fault information and fault diagnosis classification have been improved, and the average accuracy could reach 97.87%.

• 한국음향학회지
• /
• 제29권3호
• /
• pp.216-221
• /
• 2010

본 논문에서는 Gaussian Mixture Model (GMM)-supervector의 Mahalanobis 거리측정 방법 기반의 Support Vector Machine (SVM) 커널을 이용한 새로운 화자인증 방법을 제안한다. 제안된 GMM-supervector SVM 커널방식은 GMM 방식과 SVM 방식을 결합한 방식으로서, GMM 파라미터에 의해 형성된 화자 및 비 화자 GMM-supervectors의 화자인증 임계값을 Mahalanobis 거리측정 방법기반의 SVM 커널에 적용함으로써 화자인증 정확도를 높인다. 제안한 방식의 성능 측정을 위해 20명의 화자를 대상으로 문장독립형 화자인증 실험을 수행하여 기존에 사용되고 있는 GMM, SVM, Kullback-Leibler (KL) divergence 거리측정 방법 기반의 GMM-supervector SVM 커널, Bhattacharyya 거리측정 방법기반의 GMM-supervector SVM 커널 방식을 통한 화자인증 결과들과 비교하였다.

• Communications for Statistical Applications and Methods
• /
• 제21권5호
• /
• pp.435-444
• /
• 2014

The most widely used optimal bandwidth is known to minimize the mean integrated squared error(MISE) of a kernel density estimator from a true density. In this article proposes, we propose a bandwidth which asymptotically minimizes the mean integrated density power divergence(MIDPD) between a true density and a corresponding kernel density estimator. An approximated form of the mean integrated density power divergence is derived and a bandwidth is obtained as a product of minimization based on the approximated form. The resulting bandwidth resembles the optimal bandwidth by Parzen (1962), but it reflects the nature of a model density more than the existing optimal bandwidths. We have one more choice of an optimal bandwidth with a firm theoretical background; in addition, an empirical study we show that the bandwidth from the mean integrated density power divergence can produce a density estimator fitting a sample better than the bandwidth from the mean integrated squared error.

• 대한전자공학회:학술대회논문집
• /
• 대한전자공학회 2008년도 하계종합학술대회
• /
• pp.915-916
• /
• 2008

We propose a clustering based object feature matching for identification of same object in multi-camera system. The method is focused on ease to system initialization and extension. Clustering is used to estimate parameters of Gaussian mixture models of objects. A similarity measure between models are determined by Kullback-Leibler divergence. This method can be applied to occlusion problem in tracking.

• Journal of Information Processing Systems
• /
• 제8권2호
• /
• pp.359-374
• /
• 2012

In this paper, we propose a new machine vision algorithm for automatic defect detection on patterned textures with the help of texture-periodicity and the Jensen-Shannon Divergence, which is a symmetrized and smoothed version of the Kullback-Leibler Divergence. Input defective images are split into several blocks of the same size as the size of the periodic unit of the image. Based on histograms of the periodic blocks, Jensen-Shannon Divergence measures are calculated for each periodic block with respect to itself and all other periodic blocks and a dissimilarity matrix is obtained. This dissimilarity matrix is utilized to get a matrix of true-metrics, which is later subjected to Ward's hierarchical clustering to automatically identify defective and defect-free blocks. Results from experiments on real fabric images belonging to 3 major wallpaper groups, namely, pmm, p2, and p4m with defects, show that the proposed method is robust in finding fabric defects with a very high success rates without any human intervention.