Browse > Article
http://dx.doi.org/10.5392/JKCA.2010.10.6.166

Hierarchical Architecture of Multilayer Perceptrons for Performance Improvement  

Oh, Sang-Hoon (목원대학교 정보통신공학과)
Publication Information
The Journal of the Korea Contents Association / v.10, no.6, 2010 , pp. 166-174 More about this Journal
Abstract
Based on the theoretical results that multi-layer feedforward neural networks with enough hidden nodes are universal approximators, we usually use three-layer MLP's(multi-layer perceptrons) consisted of input, hidden, and output layers for many application problems. However, this conventional three-layer architecture of MLP shows poor generalization performance in some applications, which are complex with various features in an input vector. For the performance improvement, this paper proposes a hierarchical architecture of MLP especially when each part of inputs has a special information. That is, one input vector is divided into sub-vectors and each sub-vector is presented to a separate MLP. These lower-level MLPs are connected to a higher-level MLP, which has a role to do a final decision. The proposed method is verified through the simulation of protein disorder prediction problem.
Keywords
Multilayer Perceptrons; Hierarchical Structure; Input Vector;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, Vol.313, pp.504-507, 2006.   DOI   ScienceOn
2 F. J. Owens, G. H. Zheng, and D. A. Irvine, "A multi-output-layer perceptron," Neural Computing & Applications, Vol.4, pp.10-20, 1996.   DOI
3 Y. Lee, S.-H. Oh, and M. W. Kim, "An analysis of premature saturation in back-propagation learning," Neural Networks, Vol.6, pp.719-728, 1993.   DOI   ScienceOn
4 Sang-Hoon Oh "On the Design of Multilayer perceptrons for Pattern Classifications," Proc. Int. Conf. on Convergence Content 2009, Hanoi, Vietnam, pp.59-62, Dec. 17-19 2009.
5 J. B. Hamshire II and A. H. Waibel, "A novel objective function for improved phoneme recognition using time-delay neural networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990.   DOI
6 A. S. Weigend and N. A. Gershenfeld, Time Series Prediction: Forecasting the future and understanding the past, Addison-Wesley Publishing Co., 1994.
7 Y.-M. Huang, C.-M. Hung, and H. C. Jiau, "Evaluation of neural networks and data mining methods on a credit assessment task for class imbalance problem," Nonlinear Analysis, Vol.7, pp.720-747, 2006.   DOI   ScienceOn
8 J. B. Hampshire II and A. H. Waibel, "A novel objective function for improved phoneme recognition using time-delay neural networks," IEEE Trans. Neural Networks, Vol.1, pp.216-228, 1990.   DOI
9 Z. R. Yang and R. Thomson, "Bio-basis function neural netwrok for prediction of protease cleavage sites in proteins," IEEE Trans. Neural Networks, Vol.16, pp.263-274, 2005.   DOI   ScienceOn
10 S.-H. Oh, "Improving the error back-propagation algorithm with a modified error function," IEEE Trans. Neural Networks, Vol.8, pp.799-803, 1997.   DOI   ScienceOn
11 A. van Ooyen and B. Nienhuis, "Improving the convergence of the back-propagation algorithm," Neural Networks, Vol.5, pp.465-471, 1992.   DOI   ScienceOn
12 오상훈, “다층퍼셉트론에 의한 불균형 데이터의 학습방법”, 한국콘텐츠학회 논문지, 제9권, 제7호, pp.141-148, 2009.
13 오상훈, “다층퍼셉트론의 출력노드 수 증가에 의한 성능향상”, 한국콘텐츠학회 논문지, 제9권, 제1호, pp.123-130, 2009.
14 D. Simard, P. Y. Steinkraus, and J. C. Platt, "Best practices for convolutional neural networks," Proc. Int. Conf. Document Analysis and Recognition(ICDAR), Washington DC, USA, pp.958-962, 2003.
15 Z.-H. Zhou and X.-Y. Liu, "Training cost-sensitive neural networks with methods addressing the class imbalance problem," IEEE Trans. Knowledgement and Data Eng., Vol.18, No.1, pp.63-77, 2006.   DOI   ScienceOn
16 Y. Xie and M. A. Jabri, "Analysis of the effects of quantization in multilayer neural networks using a statistical model," IEEE Trans. Neural Networks, Vol.3, pp.334-338, 1992.   DOI   ScienceOn
17 D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructures of Cognition, The MIT Press, 1986.
18 K. Hornik, M. Stincombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, Vol.2, pp.359-366, 1989.   DOI   ScienceOn
19 M. Stevenson, R. Winter, and B. Widrow, “Sensitiviety of feedforward neural networks to weight errors," IEEE Trans. Neural Networks, Vol.1, pp.71-90, 1990.   DOI
20 J. Y. Choi and C.-H. Choi, "Sensitivity analysis of multilayer perceptron with differentiable activation functions," IEEE Trans. Neural Networks, Vol.3, pp.101-107, 1992.   DOI   ScienceOn
21 Y. Lee and S.-H. Oh, "Input noise immunity of multilayer perceptrons," ETRI Journal, Vol.16, pp.35-43, 1994.   DOI
22 S.-H. Oh and Y. Lee, "Sensitivity analysis of single hidden-layer neural networks with threshold functions," IEEE Trans. Neural Networks, Vol.6, pp.1005-1007, 1995.   DOI   ScienceOn
23 R. P. Lippmann, "Pattern classification using neural networks," IEEE Communication Magazine, pp.47-64, 1989.