Browse > Article

Parameter Learning of Dynamic Bayesian Networks using Constrained Least Square Estimation and Steepest Descent Algorithm  

Cho, Hyun-Cheol (울산과학대학 전가전자학부)
Lee, Kwon-Soon (동아대학교 전기공학과)
Koo, Kyung-Wan (호서대학교 국방과학기술학과)
Publication Information
The Transactions of the Korean Institute of Electrical Engineers P / v.58, no.2, 2009 , pp. 164-171 More about this Journal
Abstract
This paper presents new learning algorithm of dynamic Bayesian networks (DBN) by means of constrained least square (LS) estimation algorithm and gradient descent method. First, we propose constrained LS based parameter estimation for a Markov chain (MC) model given observation data sets. Next, a gradient descent optimization is utilized for online estimation of a hidden Markov model (HMM), which is bi-linearly constructed by adding an observation variable to a MC model. We achieve numerical simulations to prove its reliability and superiority in which a series of non stationary random signal is applied for the DBN models respectively.
Keywords
Dynamic Bayesian Networks; Parameter Learning; LS Estimation; Steepest Descent Algorithm; Markov Chain; HMM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 K Murphy, "Dynamic Bayesian networks:Representation, Inference and Leaming" Ph.D. Dissertation, UC Berkeley, 2002
2 J. M. Pena, J. Bjorkegren, and J. Tebner, "Learning dynamic Bayesian network models via cross-validation," Pattern Recognition Letters, vol. 26, no.14, pp. 2295-2308. 2005   DOI   ScienceOn
3 J. A. Bilmes, "Buride Markov models: a graphical-modeling approach to automatic speech recognition," Computer Speech and Language, vol. 17, pp. 213-231, 2003   DOI   ScienceOn
4 J.-T. Chien, "Online unsupervised learning of hidden Markov models for adaptive speech recognition," IEEE Proc. of Vision, Image and Signal Processing, vol. 148, no.5, pp. 315-324, 2001   DOI   ScienceOn
5 V. Krishnamurthy, J.B.Moore, and S. -H.Chung, "Hidden Markov model signal processing in presence of unknown deterministic inferences." IEEE Trans. on Automatic Control, vol. 38, no.1, pp. 146-152, 1993   DOI   ScienceOn
6 L. Cohen and A. Bronstein, "Adaptive online learning of Bayesian network parameters," http://www.hpl.hp.com/techreports/2001/HPL-2001-156.pdf, 2001
7 R. E. Schapire, Bartlett, P. Y. Freund, and W. S. Lee, "Boosting the margin: a new explanation for the effectiveness of voting methods," Annals of Statistics, vol. 26, no.5, pp.1651-1686, 1998   DOI   ScienceOn
8 A. Garg, V. Pavlovic, and J. M.Rehg, "Boosted learning in dynamics Bayesian networks for multimodal speaker detection," Proc. of the IEEE, vol. 91, no.9, pp.1355-1369, 2003   DOI   ScienceOn
9 A Papoulis, S. U. Pillai, Probability, random variables and stochastic processes, McGraw Hill, 2002
10 X. Li, M. Parizeau, and R. Plamondon, "Training hidden Markov models with multiple observations - A combinatorial method," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.22, no.4, pp. 371-377, 2000   DOI   ScienceOn
11 M. Stone, "Cross-validatory choice and assenssment of statistical predictions", J. of the Royal Statistical Society. vol. B36, no.2, pp. 111-133, 1974
12 F. Tian, H. Zhang, and Y. Lu, "Research on modeling with dynamics Bayesian networks," Proc. of the IEEE/WIC Int. Conf. on Web Intelligene, pp. 606-609, 2003