Browse > Article
http://dx.doi.org/10.7472/jksii.2020.21.2.49

Step-size Normalization of Information Theoretic Learning Methods based on Random Symbols  

Kim, Namyong (Electronics, information and Communications Eng, Kangwon National University)
Publication Information
Journal of Internet Computing and Services / v.21, no.2, 2020 , pp. 49-55 More about this Journal
Abstract
Information theoretic learning (ITL) methods based on random symbols (RS) use a set of random symbols generated according to a target distribution and are designed nonparametrically to minimize the cost function of the Euclidian distance between the target distribution and the input distribution. One drawback of the learning method is that it can not utilize the input power statistics by employing a constant stepsize for updating the algorithm. In this paper, it is revealed that firstly, information potential input (IPI) plays a role of input in the cost function-derivative related with information potential output (IPO) and secondly, input itself does in the derivative related with information potential error (IPE). Based on these observations, it is proposed to normalize the step-size with the statistically varying power of the two different inputs, IPI and input itself. The proposed algorithm in an communication environment of impulsive noise and multipath fading shows that the performance of mean squared error (MSE) is lower by 4dB, and convergence speed is 2 times faster than the conventional methods without step-size normalization.
Keywords
Distribution distance; random symbol; step-size; information potential; Impulsive noise;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 J. Principe, D. Xu and J. Fisher, Information Theoretic Learning in: S. Haykin, Unsupervised Adaptive Filtering, Wiley, (New York, USA), pp. 265-319, 2000.
2 I. Santamaria, P. Pokharel, and J. Principe, "Generalized correlation function: Definition, properties, and application to blind equalization', IEEE Trans. Signal Process., vol. 54, pp. 2187-2197, Jun. 2006.   DOI
3 W. Liu, P. P. Pokharel, and J. C. Principe, "Correntropy: Properties and applications in non-gaussian signal processing", IEEE Trans. Signal Process., vol. 55, pp. 5286-5298, Nov. 2007.   DOI
4 N. Kim "Performance analysis of correntropy-based blind algorithms robust to impulsive noise", The Journal of Korean Institute of Communications and Information Sciences ' vol. 40, pp. 2324-2330, Dec. 2015.   DOI
5 N. Kim, "Information potential with shifted symbol points and related blind equalizer algorithms", Journal of The Institute of Electronics Engineers of Korea, vol. 50, pp. 293-300, Feb. 2013.
6 H. Ting, F. Jun, W. Qiang, Z. Ding-Xuan, "Learning theory approach to minimum error entropy criterion", Journal of Machine Learning Research, pp. 377-397, 2013.
7 E. Parzen, "On the estimation of a probability density function and the mode," Ann. Math. Stat., vol. 33, p. 1065, 1962.   DOI
8 N. Kim, and H. Byun, "Blind equalization based on Euclidian distance of information theoretic learning for impulsive noise environments", Proceedings of 2010 International conference on computer ccommunications and networks, pp. 53-56, July 2010.
9 B. David, L. Wayne, "Efficient hardware generation of random variates with arbitrary distributions", 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM'06) April, 2006.
10 S. Richard, Computer Generation of Statistical Distributions, Storming Media, 2000.
11 N. Kim, "Recursive estimation of Euclidean distance between probabilities based on a set of random symbols", Journal of Internet Computing and Services, vol. 15, pp. 119-124, Aug. 2014.   DOI
12 N. Kim, "Function analysis of the Euclidean distance between probability distributions", Entropy, vol. 20, no. 48; doi:10.3390/e20010048
13 L. Bharani, P. Radhika, "FPGA implementation of optimal step size NLMS algorithm and its performance analysis", IJRET, vol. 2, pp. 885-890, July 2013.   DOI
14 J. Proakis, Digital Communications, McGraw-Hill, 2th ed. USA, 1989.
15 R. Chinaboina, "Adaptive algorithms for acoustic echo cancellation in speech processing, IJRRAS, vol. 7, pp. 38-42, April 2011.
16 R. Treichler, R. B. Agee, "A new approach to multipath correction of constant modulus signals", IEEE Trans. ASSP., vol. ASSP-31, pp. 349-372, Nov. 1983.