The Transactions of The Korean Institute of Electrical Engineers (전기학회논문지)

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지

Blind Image Separation with Neural Learning Based on Information Theory and Higher-order Statistics

신경회로망 ICA를 이용한 혼합영상신호의 분리

  • Published : 2008.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

Blind source separation by independent component analysis (ICA) has applied in signal processing, telecommunication, and image processing to recover unknown original source signals from mutually independent observation signals. Neural networks are learned to estimate the original signals by unsupervised learning algorithm. Because the outputs of the neural networks which yield original source signals are mutually independent, then mutual information is zero. This is equivalent to minimizing the Kullback-Leibler convergence between probability density function and the corresponding factorial distribution of the output in neural networks. In this paper, we present a learning algorithm using information theory and higher order statistics to solve problem of blind source separation. For computer simulation two deterministic signals and a Gaussian noise are used as original source signals. We also test the proposed algorithm by applying it to several discrete images.

Keywords

References

  1. G. Huang and L. Yang, "A radar anti jamming technology based on blind source separation," Proc. of Int. Conf. on Signal Processing, vol. 3, pp. 2021-2024, 2004
  2. E. Visser and T. W. Lee, "Speech enhancement using blind source separation and two channel energy based speaker detection,", Proc. of IEEE Int. Conf. on Acoustics, Speech, & Signal Processing, vol. 1, pp. 884-887, 2003
  3. Q. Pan and T. Aboulnasr, "A postfilter perceptual convolutive blind source separation approach for speech signal," Proc. of Int. Conf. on Signal Processing, vol. 1, pp. 327-330, 2004
  4. D. Kim, H. Choi, and H. Bae, "Acoustic echo cancellation using blind source separation," IEEE Workshop on Signal Processing Systems, pp. 241-244, 2003
  5. C. W. Hesse and C. J. James, "On semi blind source separation using spatial constraints with applications in EEG analysis," IEEE Trans. on Biomedical Engineering, vol. 53, no. 12, pp. 2525-2534, 2006 https://doi.org/10.1109/TBME.2006.883796
  6. K. H. Ting, C. Chang, A. Leung, C. Chan, P. Fung, and F. Chan, "Second-order statistics based blind source separation for artifact correction of short ERP epochs," IEEE EMBS Asian‐Pacific Conf. on Biomedical Engineering, pp. 186-187, 2003
  7. S. Jayaraman, G. Sitaraman, and R. Seshadri, "Blind source separation of acoustic mixtures using time frequency domain independent component analysis," Proc. of Int. Conf. on Neural Information Processing, vol. 3, pp. 1383-1387, 2002
  8. A. T. Erdogan, "Globally convergent deflationary instantaneous blind source separation algorithm for digital communication signals," IEEE Trans. on Signal Processing, vol. 55, no. 5, pp. 2182-2192, 2007 https://doi.org/10.1109/TSP.2007.893214
  9. Q. H. Lin, F. L. Yin, T. M. Mei, and H. Liang, "A blind source separation based method for speech encryption," IEEE Trans. on Circuits and Systems I, vol. 53, no. 6, pp. 1320-1328, 2006 https://doi.org/10.1109/TCSI.2006.875164
  10. J. Karhunen, E. Oja, L. Wang, R. Vigario, and J. Joutsensalo, "A class of neural networks for independent component analysis," IEEE Trans. on Neural Networks, vol. 8, pp. 486-504, 1997 https://doi.org/10.1109/72.572090
  11. S. Amari and A. Cichocki, "Adaptive blind signal processing neural network approaches," Proc. of the IEEE, vol. 86, no. 10, pp. 2026-2048, 1998 https://doi.org/10.1109/5.720251
  12. E. Oja, "From neural network learning to independent components," Neurocomputing, vol. 22, pp. 187-199, 1998 https://doi.org/10.1016/S0925-2312(98)00057-5
  13. J. F. Cardoso, "Blind signal separation: statistical principles," Proc. of IEEE. vol. 86, no. 10, pp. 2009-2025, 1998 https://doi.org/10.1109/5.720250
  14. T. W. Lee, M. Girolami, A. J. Bell, and T. J. Sejnowski, "A unifying information-theoretic framework for independent component analysis," Computers & Mathematics with Applications, vol. 39, pp. 1-21, 2000
  15. K. Torkkola, "Blind separation of convolved sources based on information maximization," IEEE Workshop on Neural Networks for signal Processing, Kyoto, Japan, pp. 423-432, 1996
  16. S. Amari and A. Cichocki, and H. H. Yang, "A new learning algorithm for blind signals separation," Advances in Neural Information Processing 8, MIT Press, pp. 757-763, 1996
  17. J. F. Cardoso, "Infomax and maximum likelihood for blind separation," IEEE Letters on Signal Processing, vol. 4, pp. 112-114, 1997 https://doi.org/10.1109/97.566704
  18. J. M. Mendel, Lessons in estimation theory for signal processing, communications, and control, Prentice Hall, New Jersey, 1995
  19. T. Cover and J. Thomas, editors, Elements of information theory: volume 1, John Wiley and Sons, New York, 1991
  20. M. G. Kendall and A. Stuart, Kendall's advanced theory of statistics, volume 1: distribution theory, Oxford University Press, 1998
  21. P. Comon, "Independent component analysis, a new concept?," Signal Processing, vol. 36, pp. 287-314, 1994 https://doi.org/10.1016/0165-1684(94)90029-9
  22. S. Amari, "Natural gradient works efficiently in learning," Neural Computation, vol. 10, no. 2, pp. 251-276, 1998 https://doi.org/10.1162/089976698300017746
  23. E. Bingham and A. Hyvarinen, "A fast fixed-point algorithm for independent component analysis of complex-valued signals," Int. J. of Neural Systems, vol. 10, no. 1, pp. 1-8, 2000 https://doi.org/10.1142/S0129065700000028