• International Journal of Advanced Culture Technology
• /
• 제5권2호
• /
• pp.82-89
• /
• 2017

The range of problems that can be handled by the activation of big data and the development of hardware has been rapidly expanded and machine learning such as deep learning has become a very versatile technology. In this paper, mnist data set is used as experimental data, and the Cross Entropy function is used as a loss model for evaluating the efficiency of machine learning, and the value of the loss function in the steepest descent method is We applied the GradientDescentOptimize algorithm to minimize and updated weight and bias via backpropagation. In this way we analyze optimal reliability value corresponding to the number of exercises and optimal reliability value without overfitting. And comparing the overfitting time according to the number of data changes based on the number of training times, when the training frequency was 1110 times, we obtained the result of 92%, which is the optimal reliability value without overfitting.

• 한국인공지능학회지
• /
• 제5권1호
• /
• pp.29-37
• /
• 2017

The range of problems that can be handled by the activation of big data and the development of hardware has been rapidly expanded and machine learning such as deep learning has become a very versatile technology. In this paper, mnist data set is used as experimental data, and the Cross Entropy function is used as a loss model for evaluating the efficiency of machine learning, and the value of the loss function in the steepest descent method is We applied the Gradient Descent Optimize algorithm to minimize and updated weight and bias via backpropagation. In this way we analyze optimal reliability value corresponding to the number of exercises and optimal reliability value without overfitting. And comparing the overfitting time according to the number of data changes based on the number of training times, when the training frequency was 1110 times, we obtained the result of 92%, which is the optimal reliability value without overfitting.

• 지능정보연구
• /
• 제5권1호
• /
• pp.95-101
• /
• 1999

In this paper, a comparison between different methods to combine predictions from neural networks will be given. These methods are bagging, bumping, and balancing. Those are based on the analysis of the ensemble generalization error into an ambiguity term and a term incorporating generalization performances of individual networks. Neural Networks and AI machine learning models are prone to overfitting. A strategy to prevent a neural network from overfitting, is to stop training in early stage of the learning process. The complete data set is spilt up into a training set and a validation set. Training is stopped when the error on the validation set starts increasing. The stability of the networks is highly dependent on the division in training and validation set, and also on the random initial weights and the chosen minimization procedure. This causes early stopped networks to be rather unstable: a small change in the data or different initial conditions can produce large changes in the prediction. Therefore, it is advisable to apply the same procedure several times starting from different initial weights. This technique is often referred to as training ensembles of neural networks. In this paper, we presented a comparison of three statistical methods to prevent overfitting of neural network.

• 방송공학회논문지
• /
• 제18권4호
• /
• pp.643-646
• /
• 2013

본 논문에서는 아다부스트의 과적합 문제를 해결하기 위해 샘플 군집화를 이용한 개선된 아다부스트 알고리즘을 제안한다. 아다부스트는 다양한 객체 검출 방법에서 좋은 성능을 보이는 방법으로 알려져 있지만 훈련 샘플에 노이즈가 존재하는 경우 과적합 현상이 발생하는 문제가 있다. 이를 해결하기 위해 제안하는 방법은 우선 훈련 샘플의 긍정 샘플을 k-평균 군집화 알고리즘을 이용하여 K개의 군집으로 나눈다. 이후 아다부스트의 약분류기 훈련 시 K개의 군집 중 훈련 오차를 최소화하는 하나의 군집만을 선택하여 사용한다. 이로써, 제안하는 방법은 매 회 반복되는 약분류기의 훈련 시 훈련 샘플들이 과분할 되는 것과 노이즈 샘플이 훈련에 사용되는 것을 방지함으로써 기존 아다부스트의 과적합 현상을 효과적으로 줄여준다. 실험 결과, 제안하는 방법은 다양한 실제 데이터셋에서 기존의 부스팅 기반 방법들에 비해 더 나은 분류 성능 및 일반화 성능을 보여주었다.

• 한국정보과학회논문지:소프트웨어및응용
• /
• 제34권1호
• /
• pp.11-17
• /
• 2007

패턴 분류에 많이 사용되는 기법 중의 하나인 메모리 기반 추론 알고리즘은 단순히 메모리에 저장된 학습패턴 또는 초월평면과 테스트 패턴간의 거리를 계산하여 가장 가까운 학습패턴의 클래스로 분류하기 때문에 테스트 패턴을 분류하는 기준을 설명할 수 없다는 문제점을 가지고 있다. 이 문제를 해결하기 위하여, 메모리 기반 학습 기법인 RPA를 기반으로 학습패턴들에 내재된 규칙성을 표현하는 IF-THEN 형태의 규칙을 생성하는 점진적 학습 알고리즘을 제안하였다. 하지만, RPA에 의해 생성된 규칙은 주어진 학습패턴 집합에만 충실히 학습되어 overfitting 현상을 보이게 되며, 또한 패턴 공간의 과도한 분할로 인하여 필요 이상으로 많은 개수의 규칙이 생성된다. 따라서, 본 논문에서는 생성된 규칙으로부터 불필요한 조건을 제거함으로써 ovefitting 현상을 해결함과 동시에 생성되는 규칙의 개수를 줄일 수 있는 점진적 규칙 추출 알고리즘을 제안하였으며, UCI Machine Learning Repository의 벤치마크 데이터를 이용하여 제안한 알고리즘의 성능을 입증하였다.

• 한국경영과학회:학술대회논문집
• /
• 대한산업공학회/한국경영과학회 2006년도 춘계공동학술대회 논문집
• /
• pp.1221-1226
• /
• 2006

The use of Artificial Neural Networks (ANN) has received increasing attention in the analysis and prediction of financial time series. Stationarity of the observed financial time series is the basic underlying assumption in the practical application of ANN on financial time series. In this paper, we will investigate whether it is feasible to relax the stationarity condition to non-stationary time series. Our result discusses the range of complexities caused by non-stationary behavior and finds that overfitting by ANN could be useful in the analysis of such non-stationary complex financial time series.

• International journal of advanced smart convergence
• /
• 제9권2호
• /
• pp.173-178
• /
• 2020

A common problem with neural network learning is that it is too suitable for the specificity of learning. In this paper, various methods were compared to avoid overfitting: regularization, drop-out, different numbers of data and different types of neural networks. Comparative studies of the above-mentioned methods have been provided to evaluate the test accuracy. I found that the more data using method is better than the regularization and dropout methods. Moreover, we know that deep convolutional neural networks outperform multi-layer neural networks and simple convolution neural networks.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• 제10권2호
• /
• pp.95-100
• /
• 2010

This paper proposes a new spatial regularization of Fisher linear discriminant analysis (LDA) to reduce the overfitting due to small size sample (SSS) problem in face recognition. Many regularized LDAs have been proposed to alleviate the overfitting by regularizing an estimate of the within-class scatter matrix. Spatial regularization methods have been suggested that make the discriminant vectors spatially smooth, leading to mitigation of the overfitting. As a generalized version of the spatially regularized LDA, the proposed regularized LDA utilizes the non-uniformity of spatial correlation structures in face images in adding a spatial smoothness constraint into an LDA framework. The region-dependent spatial regularization is advantageous for capturing the non-flat spatial correlation structure within face image as well as obtaining a spatially smooth projection of LDA. Experimental results on public face databases such as ORL and CMU PIE show that the proposed regularized LDA performs well especially when the number of training images per individual is quite small, compared with other regularized LDAs.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• 제6권3호
• /
• pp.179-183
• /
• 2006

ECG consists of various waveforms of electric signals of heat. Datamining can be used for analyzing and classifying the waveforms. Conventional studies classifying electrocardiogram have problems like extraction of distorted characteristics, overfitting, etc. This study classifies electrocardiograms by using BP algorithm and SVM to solve the problems. As results, this study finds that SVM provides an effective prohibition of overfitting in neural networks and guarantees a sole global solution, showing excellence in generalization performance.

• 한국데이터정보과학회:학술대회논문집
• /
• 한국데이터정보과학회 2006년도 PROCEEDINGS OF JOINT CONFERENCEOF KDISS AND KDAS
• /
• pp.171-179
• /
• 2006

For the problem of variable selection in linear models, we consider the errors are correlated with V covariance matrix. Hocking's theorems on the effects of the overfitting and the underfitting in linear model are extended to the less than full rank and correlated error model, and to the ANCOVA model.