Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Unsupervised Incremental Learning of Associative Cubes with Orthogonal Kernels

  • Kang, Hoon (School of Electrical and Electronics Engineering, Chung-Ang University) ;
  • Ha, Joonsoo (School of Electrical and Electronics Engineering, Chung-Ang University) ;
  • Shin, Jangbeom (School of Electrical and Electronics Engineering, Chung-Ang University) ;
  • Lee, Hong Gi (School of Electrical and Electronics Engineering, Chung-Ang University) ;
  • Wang, Yang (School of Electrical and Electronics Engineering, Chung-Ang University)
  • Received : 2014.06.27
  • Accepted : 2015.02.12
  • Published : 2015.02.25
Download PDF
⟨ Previous Next ⟩

Abstract

An 'associative cube', a class of auto-associative memories, is revisited here, in which training data and hidden orthogonal basis functions such as wavelet packets or Fourier kernels, are combined in the weight cube. This weight cube has hidden units in its depth, represented by a three dimensional cubic structure. We develop an unsupervised incremental learning mechanism based upon the adaptive least squares method. Training data are mapped into orthogonal basis vectors in a least-squares sense by updating the weights which minimize an energy function. Therefore, a prescribed orthogonal kernel is incrementally assigned to an incoming data. Next, we show how a decoding procedure finds the closest one with a competitive network in the hidden layer. As noisy test data are applied to an associative cube, the nearest one among the original training data are restored in an optimal sense. The simulation results confirm robustness of associative cubes even if test data are heavily distorted by various types of noise.

Keywords

References

  1. H. Kang, "Unsupervised learning with associative cubes for robust gray-scale image recognition," in Proc. ICNN&B 05, Beijing China, pp. 1864-1869, Oct. 2005.
  2. D. Hebb, Organization of Behavior. New York: Science Edition, Inc., 1961.
  3. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," in Proc. National Academic Sciences USA, Biophysics, vol. 79, pp. 2554-2558, Apr. 1982.
  4. B. Kosko, "Bidirectional associative memories," IEEE Trans. Syst., Man, Cybern., vol. 18, pp. 49-60, Jan./Feb. 1988. https://doi.org/10.1109/21.87054
  5. T. Kohonen, Self-Organization and Associative Memory. New York: Springer-Verlag, 1984.
  6. Y. F. Wang, J. B. Cruz, and J. H. Mulligan Jr., "Two coding strategies for bidirectional associative memory," IEEE Trans. Neural Netw., vol. 1, pp. 81-92, Mar. 1990. https://doi.org/10.1109/72.80207
  7. Y. J. Wang and D. L. Lee, "A modified bidirectional decoding strategy based on the bam structure," IEEE Trans. Neural Netw., vol. 4, pp. 710-717, July 1993. https://doi.org/10.1109/72.238325
  8. H. Kang, "Multilayered associative neural networks (m. a. n. n.): storage capacity vs. perfect recall," IEEE Trans. Neural Netw., vol. 5, pp. 812-822, Sept. 1994. https://doi.org/10.1109/72.317732
  9. H. Kang, "Multilayer associative neural networks (m. a. n. n.): storage capacity vs. noise-free recall," in Neural Network Theory, Technology, and Applications (P. K. Simpson, ed.), IEEE Technology Update Series, pp. 215-221, IEEE, 1996.
  10. G. E. Hinton, S. Osindero, and Y. W. Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006. https://doi.org/10.1162/neco.2006.18.7.1527
  11. Y. Bengio, "Learning deep architecture for ai," Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009. https://doi.org/10.1561/2200000006
  12. C. S. Burrus, R. A. Gopinath, and H. Guo, Introduction to Wavelets and Wavelet Transforms. New Jersey: Prentice-Hall International, Inc., 1998.
  13. H. Kang, "Associative cubes in unsupervised learning for robust gray-scale image recognition," in Proc. 3rd Int. Symposium on Neural Networks, Advances in Neural Networks - ISNN 2006, Springer-Verlag, Berlin Heidelberg (J. W. et al., ed.), vol. LNCS 3972, pp. 581-588, 2006.
  14. S. Haykin, Neural Networks: A Comprehensive Foundations. New Jersey: Prentice-Hall International, Inc., 1999.
  15. F. L. Lewis, Optimal Estimation. New York: John Wiley & Sons, Inc., 1986.
  16. M. Turk and A. Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71