• International Journal of Control, Automation, and Systems
• /
• v.2 no.2
• /
• pp.189-200
• /
• 2004

In this paper, we introduce a new soft computing technique that dwells on the ideas of combining fuzzy rules in a fuzzy system with polynomial neural networks (PNN). The PNN is a flexible neural architecture whose structure is developed through the modeling process. Unfortunately, the PNN has a fatal drawback in that it cannot be constructed for nonlinear systems with only a small amount of input variables. To overcome this limitation in the conventional PNN, we employed one of three principal soft computing components such as a fuzzy system. As such, a space of input variables is partitioned into several subspaces by the fuzzy system and these subspaces are utilized as new input variables to the PNN architecture. The proposed soft computing technique is achieved by merging the fuzzy system and the PNN into one unified framework. As a result, we can find a workable synergistic environment and the main characteristics of the two modeling techniques are harmonized. Thus, the proposed method alleviates the problems of PNN while providing superb performance. Identification results of the three-input nonlinear static function and nonlinear system with two inputs will be demonstrated to demonstrate the performance of the proposed approach.

• Transactions on Control, Automation and Systems Engineering
• /
• v.3 no.1
• /
• pp.43-50
• /
• 2001

In this paper, we propose the design procedure of advance Polynomial Neural Networks(PNN) architecture for optimal model identification of complex and nonlinear system. The proposed PNN architecture is presented as the generic and advanced type. The essence of the design procedure dwells on the Group Method of Data Handling(GMDH). PNN is a flexible neural architecture whose structure is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but is generated in a dynamic way. In this sense, PNN is a self-organizing network. With the aid of three representative numerical examples, compari-sons show that the proposed advanced PNN algorithm can produce the model with higher accuracy than previous other works. And performance index related to approximation and generalization capabilities of model is evaluated and also discussed.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.66 no.12
• /
• pp.1772-1781
• /
• 2017

In this study, the design methodology for alleviating the overfitting problem of Polynomial Neural Networks(PNN) is realized with the aid of two kinds techniques such as L2 regularization and Sum of Squared Coefficients (SSC). The PNN is widely used as a kind of mathematical modeling methods such as the identification of linear system by input/output data and the regression analysis modeling method for prediction problem. PNN is an algorithm that obtains preferred network structure by generating consecutive layers as well as nodes by using a multivariate polynomial subexpression. It has much fewer nodes and more flexible adaptability than existing neural network algorithms. However, such algorithms lead to overfitting problems due to noise sensitivity as well as excessive trainning while generation of successive network layers. To alleviate such overfitting problem and also effectively design its ensuing deep network structure, two techniques are introduced. That is we use the two techniques of both SSC(Sum of Squared Coefficients) and $L_2$ regularization for consecutive generation of each layer's nodes as well as each layer in order to construct the deep PNN structure. The technique of $L_2$ regularization is used for the minimum coefficient estimation by adding penalty term to cost function. $L_2$ regularization is a kind of representative methods of reducing the influence of noise by flattening the solution space and also lessening coefficient size. The technique for the SSC is implemented for the minimization of Sum of Squared Coefficients of polynomial instead of using the square of errors. In the sequel, the overfitting problem of the deep PNN structure is stabilized by the proposed method. This study leads to the possibility of deep network structure design as well as big data processing and also the superiority of the network performance through experiments is shown.

• Proceedings of the KIEE Conference
• /
• 2001.07d
• /
• pp.2762-2764
• /
• 2001

In this study, we introduce and investigate a class of dynamic perceptron architectures, discuss a comprehensive design methodology and carry out a series of numeric experiments. The proposed dynamic perceptron architectures are called as Polynomial Neural Networks(PNN). PNN is a flexible neural architecture whose topology is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but is generated on the fly. In this sense, PNN is a self-organizing network. PNN has two kinds of networks, Polynomial Neuron(FPN)-based and Fuzzy Polynomial Neuron(FPN)-based networks, according to a polynomial structure. The essence of the design procedure of PN-based Self-organizing Polynomial Neural Networks(SOPNN) dwells on the Group Method of Data Handling (GMDH) [1]. Each node of the SOPNN exhibits a high level of flexibility and realizes a polynomial type of mapping (linear, quadratic, and cubic) between input and output variables. FPN-based SOPNN dwells on the ideas of fuzzy rule-based computing and neural networks. Simulations involve a series of synthetic as well as experimental data used across various neurofuzzy systems. A detailed comparative analysis is included as well.

• The Transactions of the Korean Institute of Electrical Engineers D
• /
• v.49 no.7
• /
• pp.378-388
• /
• 2000

In this paper, we propose Fuzzy Polynomial Neural Networks(FPNN) based on Polynomial Neural Networks(PNN) and Fuzzy Neural Networks(FNN) for model identification of complex and nonlinear systems. The proposed FPNN is generated from the mutually combined structure of both FNN and PNN. The one and the other are considered as the premise part and consequence part of FPNN structure respectively. As the consequence part of FPNN, PNN is based on Group Method of Data Handling(GMDH) method and its structure is similar to Neural Networks. But the structure of PNN is not fixed like in conventional Neural Networks and self-organizing networks that can be generated. FPNN is available effectively for multi-input variables and high-order polynomial according to the combination of FNN with PNN. Accordingly it is possible to consider the nonlinearity characteristics of process and to get better output performance with superb predictive ability. As the premise part of FPNN, FNN uses both the simplified fuzzy inference as fuzzy inference method and error back-propagation algorithm as learning rule. The parameters such as parameters of membership functions, learning rates and momentum coefficients are adjusted using genetic algorithms. And we use two kinds of FNN structure according to the division method of fuzzy space of input variables. One is basic FNN structure and uses fuzzy input space divided by each separated input variable, the other is modified FNN structure and uses fuzzy input space divided by mutually combined input variables. In order to evaluate the performance of proposed models, we use the nonlinear function and traffic route choice process. The results show that the proposed FPNN can produce the model with higher accuracy and more robustness than any other method presented previously. And also performance index related to the approximation and prediction capabilities of model is evaluated and discussed.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.3 no.3
• /
• pp.309-330
• /
• 2009

In this paper, we introduce the architecture of Genetic Algorithm(GA) based Feed-forward Polynomial Neural Networks(PNNs) and discuss a comprehensive design methodology. A conventional PNN consists of Polynomial Neurons, or nodes, located in several layers through a network growth process. In order to generate structurally optimized PNNs, a GA-based design procedure for each layer of the PNN leads to the selection of preferred nodes(PNs) with optimal parameters available within the PNN. To evaluate the performance of the GA-based PNN, experiments are done on a model by applying Medical Imaging System(MIS) data to a multi-variable software process. A comparative analysis shows that the proposed GA-based PNN is modeled with higher accuracy and more superb predictive capability than previously presented intelligent models.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2000.05a
• /
• pp.130-133
• /
• 2000

In this paper, a new design methodology named FNNN(Fuzzy Polynomial Neural Network) algorithm is proposed to identify the structure and parameters of fuzzy model using PNN(Polynomial Neural Network) structure and a fuzzy inference method. The PNN is the extended structure of the GMDH(Group Method of Data Handling), and uses several types of polynomials such as linear, quadratic and modified quadratic besides the biquadratic polynomial used in the GMDH. The premise of fuzzy inference rules defines by triangular and gaussian type membership function. The fuzzy inference method uses simplified and regression polynomial inference method which is based on the consequence of fuzzy rule expressed with a polynomial such as linear, quadratic and modified quadratic equation are used. Each node of the FPNN is defined as fuzzy rules and its structure is a kind of neuro-fuzzy architecture Several numerical example are used to evaluate the performance of out proposed model. Also we used the training data and testing data set to obtain a balance between the approximation and generalization of proposed model.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.63 no.4
• /
• pp.534-540
• /
• 2014

In this paper, we propose a fuzzy combined Polynomial Neural Network(PNN) for pattern classification. The fuzzy combined PNN comes from the generic TSK fuzzy model with several linear polynomial as the consequent part and is the expanded version of the fuzzy model. The proposed pattern classifier has the polynomial neural networks as the consequent part, instead of the general linear polynomial. PNNs are implemented by stacking the simple polynomials dynamically. To implement one layer of PNNs, the various types of simple polynomials are used so that PNNs have flexibility and versatility. Although the structural complexity of the implemented PNNs is high, the PNNs become a high order-multi input polynomial finally. To estimate the coefficients of a polynomial neuron, The weighted linear discriminant analysis. The output of fuzzy rule system with PNNs as the consequent part is the linear combination of the output of several PNNs. To evaluate the classification ability of the proposed pattern classifier, we make some experiments with several machine learning data sets.

• The Transactions of the Korean Institute of Electrical Engineers D
• /
• v.52 no.7
• /
• pp.424-433
• /
• 2003

Evolutionary design related to the optimal design of Polynomial Neural Networks (PNNs) structure for model identification of complex and nonlinear system is studied in this paper. The PNN structure is consisted of layers and nodes like conventional neural networks but is not fixed and can be changable according to the system environments. three types of polynomials such as linear, quadratic, and modified quadratic is used in each node that is connected with various kinds of multi-variable inputs. Inputs and order of polynomials in each node are very important element for the performance of model. In most cases these factors are decided by the background information and trial and error of designer. For the high reliability and good performance of the PNN, the factors must be decided according to a logical and systematic way. In the paper evolutionary algorithm is applied to choose the optimal input variables and order. Evolutionary (genetic) algorithm is a random search optimization technique. The evolved PNN with optimally chosen input variables and order is not fixed in advance but becomes fully optimized automatically during the identification process. Gas furnace and pH neutralization processes are used in conventional PNN version are modeled. It shows that the designed PNN architecture with evolutionary structure optimization can produce the model with higher accuracy than previous PNN and other works.

• The Transactions of the Korean Institute of Electrical Engineers C
• /
• v.52 no.7
• /
• pp.424-424
• /
• 2003

Evolutionary design related to the optimal design of Polynomial Neural Networks (PNNs) structure for model identification of complex and nonlinear system is studied in this paper. The PNN structure is consisted of layers and nodes like conventional neural networks but is not fixed and can be changable according to the system environments. three types of polynomials such as linear, quadratic, and modified quadratic is used in each node that is connected with various kinds of multi-variable inputs. Inputs and order of polynomials in each node are very important element for the performance of model. In most cases these factors are decided by the background information and trial and error of designer. For the high reliability and good performance of the PNN, the factors must be decided according to a logical and systematic way. In the paper evolutionary algorithm is applied to choose the optimal input variables and order. Evolutionary (genetic) algorithm is a random search optimization technique. The evolved PNN with optimally chosen input variables and order is not fixed in advance but becomes fully optimized automatically during the identification process. Gas furnace and pH neutralization processes are used in conventional PNN version are modeled. It shows that the designed PNN architecture with evolutionary structure optimization can produce the model with higher accuracy than previous PNN and other works.