Browse > Article

Online Probability Density Estimation of Nonstationary Random Signal using Dynamic Bayesian Networks  

Cho, Hyun-Cheol (Dept. of Electrical Engineering, Dong-A University)
Fadali, M. Sami (Dept. of Electrical Engineering, Univ. of Nevada-Reno)
Lee, Kwon-Soon (Dept. of Electrical Engineering, Dong-A University)
Publication Information
International Journal of Control, Automation, and Systems / v.6, no.1, 2008 , pp. 109-118 More about this Journal
Abstract
We present two estimators for discrete non-Gaussian and nonstationary probability density estimation based on a dynamic Bayesian network (DBN). The first estimator is for off line computation and consists of a DBN whose transition distribution is represented in terms of kernel functions. The estimator parameters are the weights and shifts of the kernel functions. The parameters are determined through a recursive learning algorithm using maximum likelihood (ML) estimation. The second estimator is a DBN whose parameters form the transition probabilities. We use an asymptotically convergent, recursive, on-line algorithm to update the parameters using observation data. The DBN calculates the state probabilities using the estimated parameters. We provide examples that demonstrate the usefulness and simplicity of the two proposed estimators.
Keywords
Dynamic bayesian networks; nonstationary signal; online estimation; probability density function;
Citations & Related Records

Times Cited By Web Of Science : 2  (Related Records In Web of Science)
Times Cited By SCOPUS : 2
연도 인용수 순위
1 E. Parzen, "On estimation of a probability density function and mode," Analysis of Mathematical Statistics, vol. 33, pp. 1065-1076, 1962   DOI
2 D. S. Modha and Y. Fainman, "A learning law for density estimation," IEEE Trans. on Neural Networks, vol. 5, no. 3, pp. 519-523, 1994   DOI   ScienceOn
3 T. Kostiainen and J. Lampinen, "On the generative probability density model in the self-organizing map," Neurocomputing, vol. 48, pp. 217-228, 2002   DOI   ScienceOn
4 G. Miller and D. Horn, "Maximum entropy approach to probability density estimation," Proc. of Int. Conf. on Knowledge-Based Intelligent Electronic Systems, vol. 1, pp. 225-230, 1998
5 C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995
6 J. M. Mendel, Lessons in Estimation Theory for Signal Processing, Communications, and Control, Prentice Hall, New Jersey, 1995
7 S. Fiori and P. Bucciarelli, "Probability density estimation using adaptive activation function neurons," Neural Processing Letters, vol. 12, pp. 31-42, 2001
8 A. Likas, "Probability density estimation using artificial neural networks," Computer Physics Communications, vol. 135, no. 2, pp. 167-175, 2001   DOI   ScienceOn
9 A. Rodriguez and E. T. Horst, "Dynamic density estimation with financial applications," http://ftp.stat.duke.edu/WorkingPapers/06-21.pdf, 2006
10 A. Papoulis and S. U. Pillai, Probability, Random Variables and Stochastic Processes, McGraw Hill, 2002
11 R. J. Serfling, Approximation Theorems of Mathematical Statistics, Wiley & Sons, New York, 1980
12 W. J. Rugh, Linear System Theory, Prentice Hall, 1996
13 M. Strikanth, H. K. Kesavan, and P. H. Roe, "Probability density function estimation using the MinMax measure," IEEE Trans. on Systems, Man, and Cybernetics-Part C: Applications and Reviews, vol. 30, no. 1, pp. 77-83, 2000   DOI   ScienceOn
14 M. E. Tipping and C. M. Bishop, "Probability principle component analysis," J. Roy. Statist. Soc., Ser. B, vol. 21, no. 3, pp. 611-622, 1999
15 H. Yin and N. M. Allinson, "Self-organizing mixture networks for probability density estimation," IEEE Trans. on Neural Networks, vol. 12, no. 2, pp. 405- 411, 2001   DOI   ScienceOn
16 R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Wiley-Interscience Publication, New York, 2001
17 B. W. Silverman, Density Estimation for Statistics and Data Analysis, Chapman & Hall/CRC, 1986
18 K. Kokkinakis, "Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling," Signal Processing, vol. 85, no. 9, pp. 1852-1858, 2005   DOI   ScienceOn
19 C. Wang and W. Wang, "Links between PPCA and subspace methods for complete Gaussian density estimation," IEEE Trans. on Neural Networks, vol. 17, no. 3, pp. 789-792, 2006   DOI   ScienceOn
20 K. Murphy, Dynamic Bayesian networks: Representation, Inference and Learning, Ph.D. Dissertation, UC Berkeley, 2002
21 T. Kohonen, "The self-organization map," Proc. of the IEEE, vol. 78, no. 9, pp. 1464-1480, 1990   DOI   ScienceOn
22 Y. Baram and Z. Roth, "Forecasting by density shaping using neural networks," IEEE/IAFE Proc. of Computational Intelligence for Financial Engineering, pp. 57-71, 1995
23 H. Wang, A. Wang, and Y. Wang, "Online estimation algorithm for the unknown probability density functions of random parameters in auto-regression and exogenous stochastic systems," IEEE Proc.-Control Theory Applications, vol. 153, no. 4, pp. 462-468, 2006   DOI   ScienceOn
24 J. N. Kapur, G. Baciu, and H. K. Kesavan, "The MinMax information measure," Int. J. of System Science, vol. 26, no. 1, pp. 1-12, 1995   DOI   ScienceOn
25 A. Sarajedini, R. Hecht-Nielsen, and P. M. Chau, "Conditional probability density function estimation with sigmoidal neural networks," IEEE Trans. on Neural Networks, vol. 10, no. 2, pp. 231-238, 1999   DOI   ScienceOn