• The Journal of the Korea Contents Association
• /
• v.10 no.4
• /
• pp.10-18
• /
• 2010

This paper presents a optimization of sigmoid activation function parameter using genetic algorithms and pattern recognition analysis in input space of two spirals benchmark problem. To experiment, cascade correlation learning algorithm is used. In the first experiment, normal sigmoid activation function is used to analyze the pattern classification in input space of the two spirals problem. In the second experiment, sigmoid activation functions using different fixed values of the parameters are composed of 8 pools. In the third experiment, displacement of the sigmoid function to determine the value of the three parameters is obtained using genetic algorithms. The parameter values applied to the sigmoid activation functions for candidate neurons are used. To evaluate the performance of these algorithms, each step of the training input pattern classification shows the shape of the two spirals.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.15 no.3
• /
• pp.1724-1733
• /
• 2014

This paper presents a pattern recognition analysis of two spirals problem and optimization of Cascade Correlation learning algorithm using in combination with a non-monotone function as CosExp(cosine-modulated symmetric exponential function) and a monotone function as sigmoid function. In addition, the algorithm's optimization is attempted. By using genetic algorithms the optimization of the algorithm will attempt. In the first experiment, by using CosExp activation function for candidate neurons of the learning algorithm is analyzed the recognized pattern in input space of the two spirals problem. In the second experiment, CosExp function for output neurons is used. In the third experiment, the sigmoid activation functions with various parameters for candidate neurons in 8 pools and CosExp function for output neurons are used. In the fourth experiment, the parameters are composed of 8 pools and displacement of the sigmoid function to determine the value of the three parameters is obtained using genetic algorithms. The parameter values applied to the sigmoid activation functions for candidate neurons are used. To evaluate the performance of these algorithms, each step of the training input pattern classification shows the shape of the two spirals. In the optimizing process, the number of hidden neurons was reduced from 28 to15, and finally the learning algorithm with 12 hidden neurons was optimized.

• Journal of the Institute of Convergence Signal Processing
• /
• v.11 no.1
• /
• pp.80-87
• /
• 2010

The method of function synthesis is essential when we control the systems not known their characteristic, by predicting the function to satisfy a relation between input and output from the given pairs of input-output data. In general the most systems operate non-linearly, it is easy to come about problem is composed with combinations of parameter, constant, condition, and so on. Genetic programming is proposed by one of function synthesis methods. This is a search method of function tree to satisfy a relation between input and output, with appling genetic operation to function tree to convert function into tree structure. In this paper, we indicate problems of a function synthesis method by an existing genetic programming propose four type of new improved method. In other words, there are control of function tree growth, selection of local search method for early convergence, effective elimination of redundancy in function tree, and utilization of problem characteristic of object, for preventing function from complicating when the function tree is searched. In case of this improved method, we confirmed to obtain superior structure to function synthesis method by an existing genetic programming in a short period of time by means of computer simulation for the two-spirals problem.

• Journal of the Korea Society of Computer and Information
• /
• v.11 no.3
• /
• pp.107-115
• /
• 2006

This paper presents a new class of activation functions for Cascade Correlation learning algorithm, which herein will be called CosGauss function. This function is a cosine modulated gaussian function. In contrast to the sigmoidal, hyperbolic tangent and gaussian functions, more ridges can be obtained by the CosGauss function. Because of the ridges, it is quickly convergent and improves a pattern recognition speed. Consequently it will be able to improve a learning capability. This function was tested with a Cascade Correlation Network on the two spirals problem and results are compared with those obtained with other activation functions.

• Journal of the Korean Institute of Telematics and Electronics C
• /
• v.35C no.3
• /
• pp.79-89
• /
• 1998

The Error Back-Propagation(EBP) algorithm is widely applied to train a multi-layer perceptron, which is a neural network model frequently used to solve complex problems such as pattern recognition, adaptive control, and global optimization. However, the EBP is basically a gradient descent method, which may get stuck in a local minimum, leading to failure in finding the globally optimal solution. Moreover, a multi-layer perceptron suffers from locking a systematic determination of the network structure appropriate for a given problem. It is usually the case to determine the number of hidden nodes by trial and error. In this paper, we propose a new algorithm to efficiently train a multi-layer perceptron. OUr algorithm uses stochastic perturbation in the weight space to effectively escape from local minima in multi-layer perceptron learning. Stochastic perturbation probabilistically re-initializes weights associated with hidden nodes to escape a local minimum if the probabilistically re-initializes weights associated with hidden nodes to escape a local minimum if the EGP learning gets stuck to it. Addition of new hidden nodes also can be viewed asa special case of stochastic perturbation. Using stochastic perturbation we can solve the local minima problem and the network structure design in a unified way. The results of our experiments with several benchmark test problems including theparity problem, the two-spirals problem, andthe credit-screening data show that our algorithm is very efficient.