Browse > Article
http://dx.doi.org/10.5391/JKIIS.2005.15.5.552

Parameter Estimation of Recurrent Neural Networks Using A Unscented Kalman Filter Training Algorithm and Its Applications to Nonlinear Channel Equalization  

Kwon Oh-Shin (School of Electronic and Information Engineering Kunsan National University)
Publication Information
Journal of the Korean Institute of Intelligent Systems / v.15, no.5, 2005 , pp. 552-559 More about this Journal
Abstract
Recurrent neural networks(RNNs) trained with gradient based such as real time recurrent learning(RTRL) has a drawback of slor convergence rate. This algorithm also needs the derivative calculation which is not trivialized in error back propagation process. In this paper a derivative free Kalman filter, so called the unscented Kalman filter(UKF), for training a fully connected RNN is presented in a state space formulation of the system. A derivative free Kalman filler learning algorithm makes the RNN have fast convergence speed and good tracking performance without the derivative computation. Through experiments of nonlinear channel equalization, performance of the RNNs with a derivative free Kalman filter teaming algorithm is evaluated.
Keywords
Recurrent Neural Network; A Derivative Free Kalman Filter; Unscented Kalman Filter; Channel Equalization; Time-varying Channel;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. J. julier and J. K. Uhlmann, ' A new extension of the Kalman filter to nonlinear systems,' in Proceeding of AeroSence: The 11th International Symposium on Aerospace /Defence Sensing, Simulation and Controls, 1997
2 S. Haykin, Adaptive Filter Theory, 4th Ed. Upper Saddle River, NJ Prentice Hall, 2002
3 S. Chen, G. J. Gibson, B. Mulgrew, and S. McLaughlin, 'Adaptive equalization of finite nonliner channels using multilayer perceptrons, ' Signal Processing, vol. 20, pp. 107-119, 1990
4 S. Julier, J. Uhlmann, and H. F. Durrant- Whyte, ' A new method for the nonlinear transformation of means and covaroances in filters and estimators,' IEEE Transaction on Auto matic Control, vol. 45, pp. 477-482, March 2000   DOI   ScienceOn
5 E. A. Wan and R van der Merwe, 'The unscented Kalman filter,' in kalman filtering and neural networks, Edited by S. Haykin. John Wiley and Sons, Inc., 2001
6 C.Cowan and Semnani, 'Time-variant equalization using a novel non-linear adaptive structure,' International Journal of Adaptive Control and Signal Processing, vol. 12, no.2, pp. 195-206, 1998
7 R J. Williams and D. Ziper, 'A learning algorithms for continually running fully recurrent neural networks,' Neural Computation, vol. 1, pp.270-280, 1989   DOI
8 M. Solazzi, A. Uncini, E. D. Di Clauio, and R.Parisi, 'Complex discriminative learning Bayesian neural equalizer,' Signal Processing, vol. 81, pp. 2493-2502, 2001
9 E. A. Wan and R van der Merwe, 'The unscented Kalman filter for nonlinear estimation,' in Proceeding of the IEEE 2000 daptiue Systems for Signal Processing, Communications and Control Symposium/As -SPCC), pp. 153-158, 2000
10 S. Haykin, Nerual Networks: a Comprehensive Foundation, 2nd Ed. Upper Saddle River, NJ: Prentice Hall, 1999
11 P. J. Werbos, 'Back-propagation through time: What it does and how to do it,' Proceedings of the IEEE, vol. 78, pp.1550-1560, October 1990
12 H. R. Jiang and K. S. Kwak, 'On modified complex recurrent neural network adaptive equalizer,' Journal of Circuits, Systems, and Computers, vol. 11, no. 1, pp. 93-101, 2002   DOI
13 B. Hammer and J. J. Steil, 'Tutorial: Perspective on learning with RNNs,' in Proc. of the European Symposium on Artifical Neural Networks(ESANN), pp.357-369, 2002
14 S. Chen. B. Mulgrew, and S. McLaughlin, 'Adaptive Bayesian equalizer with decision feedback,' IEEE Transactions on Signal Processing, vol.41, pp. 2918-2927, September 1993
15 S. Ong, C. You, S. Choi, and D. Hong, ' A decision feedback recurrent neural equalizer as an infinte impulse response filter,' IEEE Transcation on Signal Processing, vol. 45,pp. 2851 - 2858, November 1997
16 A. F. Atiya and A. G. Parlos, 'New results on recurrent network traning: Unifying the algorithms and accelerating convergence,' IEEE Transactions on Neural Networks, vol.11, pp.697-709, May 2000   DOI   ScienceOn
17 G. Kechriotis, E. Zervas, and E. S. Manolakos, 'Using recurrent neural networks for adaptive communication channel equalization,' IEEE Transaction on Neural Networks vol, 5, pp. 267-278, March 1994   DOI   ScienceOn
18 R. Parisi, E. D. Di Claudio, G. Orlandi, and B.D.Rao, 'Fast adaptive digital equalization by recurrent neural networks,' IEEE Transactions on Signal Processing, vol,45, pp. 2731- 2739, November 1997