Browse > Article

Development of Brain-Style Intelligent Information Processing Algorithm Through the Merge of Supervised and Unsupervised Learning: Generation of Exemplar Patterns for Training  

오상훈 (목원대학교 정보통신공학부)
Publication Information
Abstract
We propose a new algorithm to generate additional training patterns using the brain-style information processing algorithm, that is, supervised and unsupervised learning models. This will be useful in the case that we do not have enough number of training patterns because of limitation such as time consuming, economic problem, and so on. We adopt the independent component analysis as an unsupervised model for generating exempalr patterns and multilayer perceptions as supervised models for verifying usefulness of the generated patterns. After statistical analysis of the proposed pattern generation algorithm, we verify successful operations of our algorithm through simulation of handwritten digit recognition with various numbers of training patterns.
Keywords
교사학습;비교사학습;패턴생성;일반화 성능;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. H. Lee and S.-Y. Lee, 'On the efficient speech feature extraction based on independent component analysis,' Neural Processing Letters, vol. 15, no. 3, pp. 235-245, June 2002   DOI
2 M.Tatsuno, 'Computational Neuroscience-Methods in Neuronal modeling,' Educational Lecture, BSI Retreat, Oiso, Japan, Sept. 18-20, 2000
3 J. Albus, 'A theory of cerebellar function,' Mathematical Biosciences, vol. 10, pp. 25-61, 1971   DOI   ScienceOn
4 A. J. Bell and T. J. Sejnowski, 'The independent components of natural scenes are edge filters,' Vision Research, vol. 37, pp. 3327-3338, 1997   DOI   ScienceOn
5 T.-W. Lee, et al., 'A unifying information-theoretic framework for independent component analysis,' Computers & Mathematics with Applications, vol. 31, no. 11, pp. 1-21, March 2000   DOI   ScienceOn
6 R. Kamimura and S. Nakanishi, 'Hidden information maximization for feature detection and rule discovery,' Network: Computation in Neural Systems, vol. 6, pp. 577-602, 1995   DOI   ScienceOn
7 J. J. Hull, 'A database for handwritten text recognition research,' IEEE Trans. Pat. Ana. Mach. Int., vol. 15, no. 5, pp. 550-554, May 1994   DOI   ScienceOn
8 Z. Ghahramani and M. Jordan, 'Supervised learning from incomplete data via an EM approach,' Advances in Neural Information Processing Systems 6, pp. 120-127, Nov. 1994
9 B. Kamgar-Parsi, B. Kamgar-Parsi, J. E. Dayhoff, and A. K. Jain, 'Improving classification boundaries by exemplar generation for visual pattern discrimination,' Proc. IJCNN2001, vol. 4, pp. 2969-2974   DOI
10 B. A. Olshausen and D. J. Field, 'Emergence of simple-cell receptive field properties by learning a sparse code for natural images,' Nature, vol. 381, pp. 607-609, 13 June, 1996   DOI   ScienceOn
11 S.-H. Oh, A. Cichocki, S. Choi, S.-I. Amari, and S.-Y. Lee., 'Comparison of ICA/BSS algorithms in noisy environment,' Proc. ICONIP, vol. 2, pp. 1192-1197, Nov. 2000
12 M. Girolami, A. Cichocki, and S.-I. Amari, 'A common neural-network model for unsupervised exploratory data analysis and independent component analysis,' IEEE Trans. Neural Networks, vol. 9, no. 6, Nov. 1998   DOI   ScienceOn
13 L. Holmstrom and P. Koistinen, 'Using additive noise in back-propagation training,' IEEE Trans. Neural Networks, vol. 3, pp. 24-38, 1992   DOI   ScienceOn
14 J. Karhunen, et al., 'A class of neural networks for independent component analysis,' IEEE Trans. Neural Networks, vol. 8, pp. 486-504, 1997   DOI   ScienceOn
15 S.-H. Oh, 'Improving the error back-propagation algorithm with a modified error function,' IEEE Trans. Neural Networks, vol. 8, pp. 799-803, 1997   DOI   ScienceOn
16 J. C. Principe, et al., 'Learning from examples with information theoretic criteria,' The Journal of VLSI Signal Proc., special issue on neural networks, vol. 26, pp. 61-77, Aug. 2000   DOI   ScienceOn
17 K. Matsuoka, 'Noise injection into inputs in back-propagation learning,' IEEE Trans. Systems, Man, and Cybernetics, vol. 22, no. 3, pp. 436-440, 1992   DOI   ScienceOn
18 I. V. Tetko and A. E. P. Villa, 'Efficient partition of learning data sets for neural network training,' Neural Networks, vol. 10, pp. 1361-1374, 1997   DOI   ScienceOn
19 S.-Y. Yoon and S.-Y. Lee, 'Training algorithm with incomplete data for feed-forward neural networks,' Neural Processing Letters, vol. 10, pp. 171-179, 1999   DOI
20 U.-M. Bae and S.-Y. Lee, 'A complementary approach to blind signal separation for rea-world speech recognition,' Advances in Neural Information Processing Systems 13, pp, 765-771, Nov. 2000
21 A. Hyvarinen, P. O. Hoyer, and J. Hurri, 'Extensions of ICA as models of natural images and visual processing,' Proceedings of ICA, April 1-4, 2003, Nara, Japan
22 C. Wang and J. C. Principe, 'Training neural networks with additive noise in the desired signal,' IEEE Trans. Neural Networks, vol. 10, pp. 1511-1517, 1999   DOI   ScienceOn
23 A. Hyvarinen, P. O. Hoyer, and M. Inki, 'Topographic independent component analysis,' Neural Cmputation, vol.13, no.7, pp.1527-1558, 2001   DOI   ScienceOn