• Proceedings of the Korea Institute of Convergence Signal Processing
• /
• 2000.08a
• /
• pp.157-160
• /
• 2000

본 논문은 HMM(Hidden Markov Model)을 이용하여 인식을 수행할 경우의 오류를 최소화 할 수 있는 후처리 과정으로 신경망을 결합시켜 HMM 단독으로 사용하였을 때 보다 높은 인식률을 얻을 수 있는 HMM과 신경망의 하이브리드시스템을 제안한다. HMM을 이용하여 학습한 후 학습에 참여하지 않은 데이터를 인식하였을 때 오인식 데이터를 정인식으로 인식하도록 HMM의 출력으로 얻은 각 출력확률을 후처리에 사용될 MLP(Multilayer Perceptrons)의 학습용으로 사용하여 MLP를 학습하여 HMM과 MLP을 결합한 하이브리드 모델을 만든다. 이와 같은 HMM과 신경망을 결합한 하이브리드 모델을 사용하여 단독 숫자음과 4연 숫자음 데이터에서 실험한 결과 HMM 단독으로 사용하였을 때 보다 각각 약 4.5%, 1.3%의 인식률 향상이 있었다. 기존의 하이브리드 시스템이 갖는 많은 학습시간이 소요되는 문제점과 실시간 음성인식시스템을 구현할 때의 학습데이터의 부족으로 인한 인식률 저하를 해결할 수 있는 방법임을 확인할 수 있었다.

• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.46.6-46
• /
• 2001

A servo gun welding system having a AC servo motor and a PC control system is presented for the improvement of quality control in the spot welding. The spot welding process is composed of the press stage, the weld stage, and the hold stage. The changes of gun press forces according to three stages in the spot welding process are controlled and measured through the load cell in order to know the influence on the welding quality. The relation between the measured force changes according to three stages and welding qualities is also implemented on the multilayer perceptrons, one of supervised learning method of neural network, which are powerful for realization of complex mapping characteristics. The estimated results and ...

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.9 no.7
• /
• pp.1607-1615
• /
• 2005

This Paper describes the implementation methods of MLP (mulilayer perceptrons) neural networks to recognize or reject handwritten numerals and non-nummerals. The MLP has known to be a very efficient classifier to recognize handwritten numerals in terms of recognition accuracy, speed, and memory requirements. In the previous researches, however, researchers have focused on the only numeral inputs and have not payed attention to the non-numeral inputs with respect to recognition accuracy, rejection rates, and other characteristics. In this paper, we present some implementation methods of the MLP in the environments that numeral and non-numerals are mixed. The MLPs have been developed by three methods, and investigated with three error types introduced. The experiments have been conducted on a total of 66,701 images of numerals and non-numerals. The promising method to recognize numerals and reject non-numerals has been described in terms of the three error types.

• Journal of the Korean Statistical Society
• /
• v.30 no.2
• /
• pp.179-192
• /
• 2001

When a statistical model has a hierarchical structure such as multilayer perceptrons in neural networks or Gaussian mixture density representation, the model includes distribution with unidentifiable parameters when the structure becomes redundant. Since the exact structure is unknown, we need to carry out statistical estimation or learning of parameters in such a model. From the geometrical point of view, distributions specified by unidentifiable parameters become a singular point in the parameter space. The problem has been remarked in many statistical models, and strange behaviors of the likelihood ratio statistics, when the null hypothesis is at a singular point, have been analyzed so far. The present paper studies asymptotic behaviors of the maximum likelihood estimator and the Bayesian predictive estimator, by using a simple cone model, and show that they are completely different from regular statistical models where the Cramer-Rao paradigm holds. At singularities, the Fisher information metric degenerates, implying that the cramer-Rao paradigm does no more hold, and that he classical model selection theory such as AIC and MDL cannot be applied. This paper is a first step to establish a new theory for analyzing the accuracy of estimation or learning at around singularities.

• The Transactions of the Korea Information Processing Society
• /
• v.7 no.6
• /
• pp.1834-1843
• /
• 2000

In this paper, we propose a modular neural network based classification method for handwritten numerals utilizing rotated and translated images of an input image. The whole numeral pattern space is divided into smaller spaces which overlap each other and form multiple clusters. On these multiple clusters, multiple multilayer perceptrons (MLP) neural networks, specialized in those clusters, are constructed. Thus, each MLP acts as an expert network on the corresponding cluster. An MLP is also used as a gating network functioning as a mediator among the multiple MLPs. In the learning phase, an input numeral image is dithered by tow geometric operations of translation and rotation so that new numeral images similar to original one are generated. In the recognition phase, we utilize not only input numeral image, but also nearly generated images through the rotation and the translation of the original image. Thus, multiple output values for those generated images were combined to make class decision by various combination methods. The experimental results confirm the validity of the proposed method.

• Journal of the Korean Institute of Telematics and Electronics C
• /
• v.35C no.6
• /
• pp.67-75
• /
• 1998

Although the nCE(n-th order cross-entropy) error function resolves the incorrect saturation problem of conventional EBP(error back-propagation) algorithm, the performance of MLP's (multilayer perceptrons) trained using the nCE function depends heavily on the order of the nCE function. In this paper, we propose an adaptive learning rate to make the performance of MLP's insensitive to the order of the nCE error. Additionally, we propose a limited error signal of output node to prevent unstable learning due to the adaptive learning rate. The effectiveness of the proposed method is demonstrated in simulations of handwritten digit recognition and thyroid diagnosis tasks.

• The Transactions of the Korean Institute of Electrical Engineers A
• /
• v.49 no.3
• /
• pp.107-117
• /
• 2000

This paper proposes a systematic method to develop short-term electrical load forecasting systems using neuro-fuzzy models. The primary goal of the proposed method is to improve the performance of the prediction model in terms of accuracy and reliability. For this, the proposed method explores the advantages of the structure learning of the neuro-fuzzy model. The proposed load forecasting system first builds an initial structure off-line for each hour of four day types and then stores the resultant initial structures in the initial structure bank. Whenever a prediction needs to be made, the proposed system initializes the neuro-fuzzy model with the appropriate initial structure stored and trains the initialized model. In order to demonstrate the viability of the proposed method, we develop an one hour ahead load forecasting system by using the real load data collected during 1993 and 1994 at KEPCO. Simulation results reveal that the prediction system developed in this paper can achieve a remarkable improvement on both accuracy and reliability compared with the prediction systems based on multilayer perceptrons, radial basis function networks, and neuro-fuzzy models without the structure learning.

• Journal of KIISE:Software and Applications
• /
• v.26 no.11
• /
• pp.1261-1269
• /
• 1999

다층 퍼셉트론은 다양한 응용 분야에 성공적으로 적용되고 있는 대표적인 신경회로망 모델이다. 그러나 다층 퍼셉트론의 학습에서 나타나는 플라토에 기인한 느린 학습 속도와 지역 극소는 실제 응용문제에 적용함에 있어서 가장 큰 문제로 지적되어왔다. 이 문제를 해결하기 위해 여러 가지 다양한 학습알고리즘들이 개발되어 왔으나, 계산의 비효율성으로 인해 실제 문제에는 적용하기 힘든 예가 많은 등, 현재까지 만족할 만한 해결책은 제시되지 못하고 있다. 본 논문에서는 다층퍼셉트론의 베이시스 함수로 사용되는 시그모이드 함수를 보다 일반화된 형태로 정의하여 사용함으로써 학습에 있어서의 플라토를 완화하고, 지역극소에 빠지는 것을 줄이는 접근방법을 소개한다. 본 방법은 기존의 변형된 가중치 수정식을 사용한 학습 속도 향상의 방법들과는 다른 접근 방법을 택함으로써 기존의 방법들과 함께 사용하는 것이 가능하다는 특징을 갖고 있다. 제안하는 방법의 성능을 확인하기 위하여 간단한 패턴 인식 문제들에의 적용 실험 및 기존의 학습 속도 향상 방법을 함께 사용하여 시계열 예측 문제에 적용한 실험을 수행하였고, 그 결과로부터 제안안 방법의 효율성을 확인할 수 있었다. Abstract A multilayer perceptron is the most well-known neural network model which has been successfully applied to various fields of application. Its slow learning caused by plateau and local minima of gradient descent learning, however, have been pointed as the biggest problems in its practical use. To solve such a problem, a number of researches on learning algorithms have been conducted, but it can be said that none of satisfying solutions have been presented so far because the problems such as computational inefficiency have still been existed in these algorithms. In this paper, we propose a new learning approach to minimize the effect of plateau and reduce the possibility of getting trapped in local minima by generalizing the sigmoidal function which is used as the basis function of a multilayer perceptron. Adapting a new approach that differs from the conventional methods with revised updating equation, the proposed method can be used together with the existing methods to improve the learning performance. We conducted some experiments to test the proposed method on simple problems of pattern recognition and a problem of time series prediction, compared our results with the results of the existing methods, and confirmed that the proposed method is efficient enough to apply to the real problems.

• International Journal of Control, Automation, and Systems
• /
• v.3 no.4
• /
• pp.571-579
• /
• 2005

In this paper, we proposed an algorithm for arrhythmia classification, which is associated with the reduction of feature dimensions by linear discriminant analysis (LDA) and a support vector machine (SVM) based classifier. Seventeen original input features were extracted from preprocessed signals by wavelet transform, and attempts were then made to reduce these to 4 features, the linear combination of original features, by LDA. The performance of the SVM classifier with reduced features by LDA showed higher than with that by principal component analysis (PCA) and even with original features. For a cross-validation procedure, this SVM classifier was compared with Multilayer Perceptrons (MLP) and Fuzzy Inference System (FIS) classifiers. When all classifiers used the same reduced features, the overall performance of the SVM classifier was comprehensively superior to all others. Especially, the accuracy of discrimination of normal sinus rhythm (NSR), arterial premature contraction (APC), supraventricular tachycardia (SVT), premature ventricular contraction (PVC), ventricular tachycardia (VT) and ventricular fibrillation (VF) were $99.307\%,\;99.274\%,\;99.854\%,\;98.344\%,\;99.441\%\;and\;99.883\%$, respectively. And, even with smaller learning data, the SVM classifier offered better performance than the MLP classifier.

• Journal of Computational Design and Engineering
• /
• v.3 no.4
• /
• pp.337-348
• /
• 2016

An experimental approach is presented for the measurement of wear that is common in the threading of cold-forged steel. In this work, the first objective is to measure wear on various types of roll taps manufactured to tapping holes in microalloyed HR45 steel. Different geometries and levels of wear are tested and measured. Taking their geometry as the critical factor, the types of forming tap with the least wear and the best performance are identified. Abrasive wear was observed on the forming lobes. A higher number of lobes in the chamber zone and around the nominal diameter meant a more uniform load distribution and a more gradual forming process. A second objective is to identify the most accurate data-mining technique for the prediction of form-tap wear. Different data-mining techniques are tested to select the most accurate one: from standard versions such as Multilayer Perceptrons, Support Vector Machines and Regression Trees to the most recent ones such as Rotation Forest ensembles and Iterated Bagging ensembles. The best results were obtained with ensembles of Rotation Forest with unpruned Regression Trees as base regressors that reduced the RMS error of the best-tested baseline technique for the lower length output by 33%, and Additive Regression with unpruned M5P as base regressors that reduced the RMS errors of the linear fit for the upper and total lengths by 25% and 39%, respectively. However, the lower length was statistically more difficult to model in Additive Regression than in Rotation Forest. Rotation Forest with unpruned Regression Trees as base regressors therefore appeared to be the most suitable regressor for the modeling of this industrial problem.