• Journal of Advanced Information Technology and Convergence
• /
• v.8 no.2
• /
• pp.119-127
• /
• 2018

Training a very large deep neural network can be painfully slow and prone to overfitting. Many researches have done for overcoming the problem. In this paper, a combination of early stopping and ADAM based deep neural network was presented. This form of deep network is useful for handling the big data because it automatically stop the training before overfitting occurs. Also generalization ability is better than pure deep neural network model.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.66 no.12
• /
• pp.1772-1781
• /
• 2017

In this study, the design methodology for alleviating the overfitting problem of Polynomial Neural Networks(PNN) is realized with the aid of two kinds techniques such as L2 regularization and Sum of Squared Coefficients (SSC). The PNN is widely used as a kind of mathematical modeling methods such as the identification of linear system by input/output data and the regression analysis modeling method for prediction problem. PNN is an algorithm that obtains preferred network structure by generating consecutive layers as well as nodes by using a multivariate polynomial subexpression. It has much fewer nodes and more flexible adaptability than existing neural network algorithms. However, such algorithms lead to overfitting problems due to noise sensitivity as well as excessive trainning while generation of successive network layers. To alleviate such overfitting problem and also effectively design its ensuing deep network structure, two techniques are introduced. That is we use the two techniques of both SSC(Sum of Squared Coefficients) and $L_2$ regularization for consecutive generation of each layer's nodes as well as each layer in order to construct the deep PNN structure. The technique of $L_2$ regularization is used for the minimum coefficient estimation by adding penalty term to cost function. $L_2$ regularization is a kind of representative methods of reducing the influence of noise by flattening the solution space and also lessening coefficient size. The technique for the SSC is implemented for the minimization of Sum of Squared Coefficients of polynomial instead of using the square of errors. In the sequel, the overfitting problem of the deep PNN structure is stabilized by the proposed method. This study leads to the possibility of deep network structure design as well as big data processing and also the superiority of the network performance through experiments is shown.

• Korean Circulation Journal
• /
• v.53 no.11
• /
• pp.772-774
• /
• 2023

• Journal of the Korean Institute of Intelligent Systems
• /
• v.14 no.1
• /
• pp.82-87
• /
• 2004

ECG consists of various waveforms of electric signals of heat. Datamining can be used for analyzing and classifying the waveforms. Conventional studies classifying electrocardiogram have problems like extraction of distorted characteristics, overfitting, etc. This study classifies electrocardiograms by using BP algorithm and SVM to solve the problems. As results, this study finds that SVM provides an effective prohibition of overfitting in neural networks and guarantees a sole global solution, showing excellence in generalization performance.

• Journal of Korean Association for Spatial Structures
• /
• v.21 no.4
• /
• pp.39-48
• /
• 2021

In this paper, a GAN-based data augmentation method is proposed for topology optimization. In machine learning techniques, a total amount of dataset determines the accuracy and robustness of the trained neural network architectures, especially, supervised learning networks. Because the insufficient data tends to lead to overfitting or underfitting of the architectures, a data augmentation method is need to increase the amount of data for reducing overfitting when training a machine learning model. In this study, the Ganerative Adversarial Network (GAN) is used to augment the topology optimization dataset. The produced dataset has been compared with the original dataset.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.05a
• /
• pp.332-333
• /
• 2021

In this paper, we propose a new image classification method based on several trainings, which is mainly used to solve model overfitting and non-convergence in image classification tasks of small data sets and to improve classification accuracy.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.557-558
• /
• 2021

In this paper, we propose a new image classification method based on Convolutional Neural Network (CNN), which is mainly used to solve model overfitting and non-convergence and to improve classification accuracy in image classification tasks on small datasets.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.9 no.3
• /
• pp.25-30
• /
• 2009

Recent days intellectual systems using reinforcement learning are being researched at various fields of game and web searching applications. A good training models are called to be fitted with trainning data and also classified with new records accurately. A overfitted model with training data may possibly bring the unfavored fallacy of hasty generalization. But it would be unavoidable in actual world. The entropy and mutation model are suggested to reduce the overfitting problems on this paper. It explains variation of entropy and artificial development of entropy in datamining, which can tell development of mutation to survive in nature world. Periodical generation of maximum entropy are introduced in this paper to reduce overfitting. Maximum entropy model can be considered as a periodical generalization in intensified process of intellectual web searching.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.19 no.4
• /
• pp.161-166
• /
• 2018

In this paper, we propose a method for the random selection of pooling operations for the regularization and reduction of cross validation in convolutional neural networks. The pooling operation in convolutional neural networks is used to reduce the size of the feature map and for its shift invariant properties. In the existing pooling method, one pooling operation is applied in each pooling layer. Because this method fixes the convolution network, the network suffers from overfitting, which means that it excessively fits the models to the training samples. In addition, to find the best combination of pooling operations to maximize the performance, cross validation must be performed. To solve these problems, we introduce the probability concept into the pooling layers. The proposed method does not select one pooling operation in each pooling layer. Instead, we randomly select one pooling operation among multiple pooling operations in each pooling region during training, and for testing purposes, we use probabilistic weighting to produce the expected output. The proposed method can be seen as a technique in which many networks are approximately averaged using a different pooling operation in each pooling region. Therefore, this method avoids the overfitting problem, as well as reducing the amount of cross validation. The experimental results show that the proposed method can achieve better generalization performance and reduce the need for cross validation.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.16 no.4
• /
• pp.147-152
• /
• 2016

One of major issues in machine learning that extracts and acquires knowledge implicit in data is to find an appropriate way of representing it. Knowledge can be represented by a number of structures such as networks, trees, lists, and rules. The differences among these exist not only in their structures but also in effectiveness of the models for their problem solving capability. In this paper, we propose partition utility as a criterion function for clustering that can lead to simplification of the model and thus avoid overfitting problem. In addition, a heuristic is proposed as a way to construct balanced hierarchical models.