Browse > Article
http://dx.doi.org/10.5392/IJoC.2014.10.1.023

Comparison of Objective Functions for Feed-forward Neural Network Classifiers Using Receiver Operating Characteristics Graph  

Oh, Sang-Hoon (Department of Information Communication Engineering Mokwon University)
Wakuya, Hiroshi (Graduate School of Science and Engineering Saga University)
Publication Information
International Journal of Contents / v.10, no.1, 2014 , pp. 23-28 More about this Journal
Abstract
When developing a classifier using various objective functions, it is important to compare the performances of the classifiers. Although there are statistical analyses of objective functions for classifiers, simulation results can provide us with direct comparison results and in this case, a comparison criterion is considerably critical. A Receiver Operating Characteristics (ROC) graph is a simulation technique for comparing classifiers and selecting a better one based on a performance. In this paper, we adopt the ROC graph to compare classifiers trained by mean-squared error, cross-entropy error, classification figure of merit, and the n-th order extension of cross-entropy error functions. After the training of feed-forward neural networks using the CEDAR database, the ROC graphs are plotted to help us identify which objective function is better.
Keywords
Receiver Operating Characteristic Graph; Feed-Forward Neural Networks; Objective Function; Performance Comparison;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. Lee, S. H. Oh, and M. W. Kim, "An Analysis of Premature Saturation in Back-Propagation Learning," Neural Networks, vol. 6, 1993, pp. 719-728.   DOI   ScienceOn
2 J. J. Hull, "A Database for Handwritten Text recognition research," IEEE Trans. Pattern Anal. Machine Intell., vol. 16, 1994, pp. 550-554.   DOI   ScienceOn
3 K. Hornik, M. Stincombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, 1989, pp. 359-366.   DOI   ScienceOn
4 S. Suzuki, "Constructive Function Approximation by Three-layer Artificial Neural Networks," Neural Networks, vol. 11, 1998, pp. 1049-1058.   DOI   ScienceOn
5 Y. Liao, S. C. Fang, and H. L. W. Nuttle, "Relaxed Conditions for Radial-Basis Function Networks to be Universal Approximators," Neural Networks, vol. 16, 2003, pp. 1019-1028.   DOI   ScienceOn
6 D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing, Cambridge, MA, 1986.
7 A. van Ooyen and B. Nienhuis, "Improving the Convergence of the Backpropagation Algorithm," Neural Networks, vol. 5, 1992, pp. 465-471.   DOI   ScienceOn
8 S. H. Oh, "Improving the Error Back-Propagation Algorithm with a Modified Error Function," IEEE Trans. Neural Networks, vol. 8, 1997, pp. 799-803.   DOI   ScienceOn
9 J. B. Hampshare and A. H. Waibel, "A Novel Objective Function for Improved Phoneme Recognition Using Time Delay Neural Networks," IEEE Trans. Neural Networks, vol. 1, 1990, pp. 216-228.   DOI   ScienceOn
10 S. H. Oh, "Contour Plots of Objective Functions for Feed-Forward Neural Networks," Int. Journal of Contents, vol. 8, no. 4, 2012, pp. 30-35.
11 S. H. Oh, "A Statistical Perspective of Neural Networks for Imbalanced Data Problems," Int. Journal of Contents, vol. 7, 2011, pp. 1-5.
12 H. White, "Learning in Artificial Neural Networks: A Statistical Perspective," Neural Computation, vol. 1, no. 4, 1989, pp. 425-464.   DOI
13 T. Fawcett, "An Introduction to ROC Analysis," Pattern Recognition Letters, vol. 27, 2006, pp. 861-874.   DOI   ScienceOn
14 M. D. Richard and R. P. Lippmann, "Neural Network Classifier Estimate Bayesian a posteriori probabilities," Neural Computation, vol. 3, 1991, pp. 461-483.   DOI
15 S. H. Oh, "Statistical Analyses of Various Error functions for Pattern Classifiers," Proc. Convergence on Hybrid Information Technology, CCIS vol. 206, 2011, pp. 129-133.
16 I. Kononenko and I. Bratko, "Information-Based Evaluation criterion for Classifier's Performance," Machine Learning, vol. 6, 1991, pp. 67-80.
17 R. N. McDonough and A. D. Whalen, Detection of Signals in Noise, Academic Press, 1995.
18 Z. H. Michalopoulou, L. W. Nolte, and D. Alexandrou, "Performance Evaluation of Multilayer Perceptrons in Signal Detection and Classification," IEEE Trans. Neural Networks, vol. 6, 1995, pp. 381-386.   DOI   ScienceOn
19 R. Bi, Y. Zhou, F. Lu, and W. Wang, "Predicting gene ontology functions based on support vector machines and statistical significance estimation," Neurocomputing, vol. 70, 2007, pp. 718-725.   DOI   ScienceOn
20 L. Bruzzone and S. B. Serpico, "Classification of Remote-Sensing Data by Neural Networks," Pattern Recognition Letters, vol. 18, 1997, pp. 1323-1328.   DOI   ScienceOn
21 Y. Bengio, "Learning Deep Architecture for AI," Foundations and Trends in Machine Learning, vol.2, 2009, pp. 1-127.   DOI
22 S. H. Oh, "Error back-propagation algorithm for classification of imbalanced data", Neurocomputing, vol. 74, 2011, pp. 1058-1061.   DOI   ScienceOn
23 K. Hornik, "Approxiamtion Capabilitis of Multilayer Feedforward Networks," Neural Networks, vol. 4, 1991, pp. 251-257.   DOI   ScienceOn
24 Y. M. Huang, C. M. Hung, and H. C. Jiau, "Evaluation of Neural Networks and Data Mining Methods on a Credit Assessment Task for Class Imbalance Problem," Nonlinear Analysis, vol. 7, 2006, pp. 720-747.   DOI   ScienceOn