Browse > Article
http://dx.doi.org/10.5302/J.ICROS.2011.17.8.777

Learning of Differential Neural Networks Based on Kalman-Bucy Filter Theory  

Cho, Hyun-Cheol (Ulsan College)
Kim, Gwan-Hyung (Tongmyong University)
Publication Information
Journal of Institute of Control, Robotics and Systems / v.17, no.8, 2011 , pp. 777-782 More about this Journal
Abstract
Neural network technique is widely employed in the fields of signal processing, control systems, pattern recognition, etc. Learning of neural networks is an important procedure to accomplish dynamic system modeling. This paper presents a novel learning approach for differential neural network models based on the Kalman-Bucy filter theory. We construct an augmented state vector including original neural state and parameter vectors and derive a state estimation rule avoiding gradient function terms which involve to the conventional neural learning methods such as a back-propagation approach. We carry out numerical simulation to evaluate the proposed learning approach in nonlinear system modeling. By comparing to the well-known back-propagation approach and Kalman-Bucy filtering, its superiority is additionally proved under stochastic system environments.
Keywords
differential neural networks; Kalman-Bucy filter; learning; augmented state estimation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
Times Cited By SCOPUS : 0
연도 인용수 순위
1 M. Choi, R. Sakthivel, and W. K. Chung, "Neural network-aided extended Kalman filter for SLAM problem," IEEE International Conference on Robotics & Automation, Roma, Italy, pp. 1686-1690, 2007.   DOI
2 K. A. Kramer and S. C. Stubberud, "Tracking of multiple target types with a single neural extended Kalman filter," International Journal of Intelligent Systems, vol. 25, pp. 440-459, 2010.
3 S. Li, "Comparative analysis of backpropagation and extended Kalman filter in pattern and batch forms for training neural networks," International Joint Conference on Neural Networks, Washington, DC. pp. 144-149, 2001.
4 C. M. Takenga, K. R. Anne, K. Kyamakya, and J. C. Chedjou, "Comparison of gradient descent method, Kalman filtering and decoupled Kalman in training neural networks used for fingerprint-based positioning," Proc. of the IEEE Vehicular Technology Conference, pp. 4146-4150, 2004.   DOI
5 P. H. G. Coelho and L. B. Neto, "Complex RTRL neural networks fast Kalman training," International Conference in Intelligent Systems Design and Applications, pp. 573-577, 2007.
6 A. S. Poznyak, E. N. Sancbez, and W. Yu, Differential Neural Networks for Robust Nonlinear Control, New Jersey, World Science, 2001.
7 H. C. Cho, J. W. Lee, Y. J. Lee, and K. S. Lee, "Online dynamic modeling of ubiquitous sensor based embedded robot systems using Kalman filter algorithm," Journal of Institute of Control, Robotics and Systems (in Korean), vol. 14, no. 8, pp. 779-784, 2008.   과학기술학회마을   DOI
8 S. Haykin, Neural Networks and Learning Machines, New Jersey, Prentice Hall, 2009.
9 W. Yu and J. de J. Rubio, "Recurrent neural networks training with stable risk-sensitive Kalman filter algorithm," Proc. of International Joint Conference on Neural Networks, Montreal, Canada, pp. 700-705, 2005.
10 J. Choi, A. C. Lima, and S. Haykin, "Kalman filter-trained recurrent neural equalizers for time-varying channels," IEEE Transaction on Communications, vol. 53, no. 3, pp. 472-480, 2005.   DOI
11 D. T. Mirikitani and N. Nikolaev, "Dynamic modeling with ensemble Kalman filter trained recurrent neural networks," International Conference on Machine Learning and Applications, pp. 843-848, 2008.
12 X. Wang and Y. Huang, "Convergence study in extended Kalman filter-based training of recurrent neural networks," IEEE Transaction on Neural Networks, vol. 22, no. 4, pp. 588-600, 2011.   DOI
13 B. Todorovic, M. Stankovic, and C. Moraga, "On-line learning in recurrent neural networks using nonlinear Kalman filters," Proc. of the IEEE Int. Symposium on Signal Processing and Information Technology, pp. 802-805, 2003.   DOI
14 J. M. Mendel, Lessons in Estimation Theory for Signal Processing, Communications and Control, New Jersey, Prentice Hall, 1995.
15 K.-W. Wong, C.-S. Leung, and S.-J. Chang, "Use pf periodic and monotonic activation functions in multilayer feedforward neural networks trained by extended Kalman filter algorithm," IEEE Image Signal Process, vol. 149, no. 4, pp. 217-224, 2002.   DOI
16 S. J. Julier and J. K. Uhlmann, "Unscented filtering and nonlinear estimation," Proc. of the IEEE, vol. 92, no. 3, pp. 401-422, 2004.   DOI
17 R. Zhan and J. Wan, "Neural network-aided adaptive unscented Kalman filter for nonlinear state estimation," IEEE Signal Processing Letters, vol. 13, no. 7, pp. 445-448, 2006.   DOI