Journal of Institute of Control, Robotics and Systems (제어로봇시스템학회논문지)

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Learning of Differential Neural Networks Based on Kalman-Bucy Filter Theory

칼만-버쉬 필터 이론 기반 미분 신경회로망 학습

  • 조현철 (울산과학대학 전기전자학부) ;
  • 김관형 (동명대학교 컴퓨터공학과)
  • Received : 2011.05.20
  • Accepted : 2011.06.20
  • Published : 2011.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

Neural network technique is widely employed in the fields of signal processing, control systems, pattern recognition, etc. Learning of neural networks is an important procedure to accomplish dynamic system modeling. This paper presents a novel learning approach for differential neural network models based on the Kalman-Bucy filter theory. We construct an augmented state vector including original neural state and parameter vectors and derive a state estimation rule avoiding gradient function terms which involve to the conventional neural learning methods such as a back-propagation approach. We carry out numerical simulation to evaluate the proposed learning approach in nonlinear system modeling. By comparing to the well-known back-propagation approach and Kalman-Bucy filtering, its superiority is additionally proved under stochastic system environments.

Keywords

References

  1. S. Li, "Comparative analysis of backpropagation and extended Kalman filter in pattern and batch forms for training neural networks," International Joint Conference on Neural Networks, Washington, DC. pp. 144-149, 2001.
  2. C. M. Takenga, K. R. Anne, K. Kyamakya, and J. C. Chedjou, "Comparison of gradient descent method, Kalman filtering and decoupled Kalman in training neural networks used for fingerprint-based positioning," Proc. of the IEEE Vehicular Technology Conference, pp. 4146-4150, 2004. https://doi.org/10.1109/VETECF.2004.1404859
  3. K.-W. Wong, C.-S. Leung, and S.-J. Chang, "Use pf periodic and monotonic activation functions in multilayer feedforward neural networks trained by extended Kalman filter algorithm," IEEE Image Signal Process, vol. 149, no. 4, pp. 217-224, 2002. https://doi.org/10.1049/ip-vis:20020515
  4. S. J. Julier and J. K. Uhlmann, "Unscented filtering and nonlinear estimation," Proc. of the IEEE, vol. 92, no. 3, pp. 401-422, 2004. https://doi.org/10.1109/JPROC.2003.823141
  5. B. Todorovic, M. Stankovic, and C. Moraga, "On-line learning in recurrent neural networks using nonlinear Kalman filters," Proc. of the IEEE Int. Symposium on Signal Processing and Information Technology, pp. 802-805, 2003. https://doi.org/10.1109/ISSPIT.2003.1341242
  6. R. Zhan and J. Wan, "Neural network-aided adaptive unscented Kalman filter for nonlinear state estimation," IEEE Signal Processing Letters, vol. 13, no. 7, pp. 445-448, 2006. https://doi.org/10.1109/LSP.2006.871854
  7. M. Choi, R. Sakthivel, and W. K. Chung, "Neural network-aided extended Kalman filter for SLAM problem," IEEE International Conference on Robotics & Automation, Roma, Italy, pp. 1686-1690, 2007. https://doi.org/10.1109/ROBOT.2007.363565
  8. K. A. Kramer and S. C. Stubberud, "Tracking of multiple target types with a single neural extended Kalman filter," International Journal of Intelligent Systems, vol. 25, pp. 440-459, 2010.
  9. W. Yu and J. de J. Rubio, "Recurrent neural networks training with stable risk-sensitive Kalman filter algorithm," Proc. of International Joint Conference on Neural Networks, Montreal, Canada, pp. 700-705, 2005.
  10. J. Choi, A. C. Lima, and S. Haykin, "Kalman filter-trained recurrent neural equalizers for time-varying channels," IEEE Transaction on Communications, vol. 53, no. 3, pp. 472-480, 2005. https://doi.org/10.1109/TCOMM.2005.843416
  11. P. H. G. Coelho and L. B. Neto, "Complex RTRL neural networks fast Kalman training," International Conference in Intelligent Systems Design and Applications, pp. 573-577, 2007.
  12. D. T. Mirikitani and N. Nikolaev, "Dynamic modeling with ensemble Kalman filter trained recurrent neural networks," International Conference on Machine Learning and Applications, pp. 843-848, 2008.
  13. X. Wang and Y. Huang, "Convergence study in extended Kalman filter-based training of recurrent neural networks," IEEE Transaction on Neural Networks, vol. 22, no. 4, pp. 588-600, 2011. https://doi.org/10.1109/TNN.2011.2109737
  14. J. M. Mendel, Lessons in Estimation Theory for Signal Processing, Communications and Control, New Jersey, Prentice Hall, 1995.
  15. A. S. Poznyak, E. N. Sancbez, and W. Yu, Differential Neural Networks for Robust Nonlinear Control, New Jersey, World Science, 2001.
  16. H. C. Cho, J. W. Lee, Y. J. Lee, and K. S. Lee, "Online dynamic modeling of ubiquitous sensor based embedded robot systems using Kalman filter algorithm," Journal of Institute of Control, Robotics and Systems (in Korean), vol. 14, no. 8, pp. 779-784, 2008. https://doi.org/10.5302/J.ICROS.2008.14.8.779
  17. S. Haykin, Neural Networks and Learning Machines, New Jersey, Prentice Hall, 2009.