• Journal of Information Processing Systems
• /
• 제19권5호
• /
• pp.673-687
• /
• 2023

Convolutional neural network-based deep learning technology is the most commonly used in image identification, but it requires large-scale data for training. Therefore, application in specific fields in which data acquisition is limited, such as in the military, may be challenging. In particular, the identification of ground weapon systems is a very important mission, and high identification accuracy is required. Accordingly, various studies have been conducted to achieve high performance using small-scale data. Among them, the ensemble method, which achieves excellent performance through the prediction average of the pre-trained models, is the most representative method; however, it requires considerable time and effort to find the optimal combination of ensemble models. In addition, there is a performance limitation in the prediction results obtained by using an ensemble method. Furthermore, it is difficult to obtain the ensemble effect using models with imbalanced classification accuracies. In this paper, we propose a transfer learning-based feature fusion technique for heterogeneous models that extracts and fuses features of pre-trained heterogeneous models and finally, fine-tunes hyperparameters of the fully connected layer to improve the classification accuracy. The experimental results of this study indicate that it is possible to overcome the limitations of the existing ensemble methods by improving the classification accuracy through feature fusion between heterogeneous models based on transfer learning.

• 응용통계연구
• /
• 제32권2호
• /
• pp.319-330
• /
• 2019

과거의 전력소모량을 분석하여 미래의 전력소모량을 예측하는 것은 에너지 계획과 정책 결정에 있어 많은 이점을 가져다준다. 기계학습은 최근 전력소모량을 예측하는 분석 방법으로 많이 사용하고 있다. 그중 앙상블 학습은 모형의 과적합 현상을 방지하고 분산을 줄여 예측의 정확성을 높이는 방법으로 알려져 있다. 하지만 일별 데이터에 앙상블 학습을 적용했을 때 분석 방법의 특성으로 인해 피크를 잘 나타내지 못하고 중심값으로 예측하는 단점을 보였다. 본 연구에서는 앙상블 학습 전에 온도 변수와의 상관성을 고려하여 선형모형으로 적합함으로써 앙상블 학습의 단점을 보완한다. 그리고 9개의 모형을 비교한 결과 온도 변수를 선형모형으로 적합하고 랜덤포레스트를 사용한 모형이 결과가 가장 좋음을 보여준다.

• 지능정보연구
• /
• 제16권4호
• /
• pp.99-112
• /
• 2010

앙상블 학습은 분류 및 예측 알고리즘의 성과개선을 위하여 제안된 기계학습 기법이다. 그러나 앙상블 학습은 기저 분류자의 다양성이 부족한 경우 다중공선성 문제로 인하여 성과개선 효과가 미약하고 심지어는 성과가 악화될 수 있다는 문제점이 제기되었다. 본 연구에서는 기저 분류자의 다양성을 확보하고 앙상블 학습의 성과개선 효과를 제고하기 위하여 유전자 알고리즘 기반의 범위 최적화 기법을 제안하고자 한다. 본 연구에서 제안된 최적화 기법을 기업 부실예측 인공신경망 앙상블에 적용한 결과 기저 분류자의 다양성이 확보되고 인공신경망 앙상블의 성과가 유의적으로 개선되었음을 보여주었다.

• International Journal of Computer Science & Network Security
• /
• 제24권1호
• /
• pp.107-118
• /
• 2024

With the seamless growth of the technology, network usage requirements are expanding day by day. The majority of electronic devices are capable of communication, which strongly requires a secure and reliable network. Network-based intrusion detection systems (NIDS) is a new method for preventing and alerting computers and networks from attacks. Machine Learning is an emerging field that provides a variety of ways to implement effective network intrusion detection systems (NIDS). Bagging and Boosting are two ensemble ML techniques, renowned for better performance in the learning and classification process. In this paper, the study provides a detailed literature review of the past work done and proposed a novel ensemble approach to develop a NIDS system based on the voting method using bagging and boosting ensemble techniques. The test results demonstrate that the ensemble of bagging and boosting through voting exhibits the highest classification accuracy of 99.98% and a minimum false positive rate (FPR) on both datasets. Although the model building time is average which can be a tradeoff by processor speed.

• 정보처리학회논문지B
• /
• 제14B권4호
• /
• pp.303-310
• /
• 2007

기계학습에서 분류기틀의 집합으로 구성된 앙상블 분류기는 단일 분류기에 비해 정확도가 높다는 것이 입증되어왔다. 본 논문에서는 새로운 앙상블 학습으로서 데이터의 지역 기반 분류기들의 앙상블 학습을 제시하여 기존의 앙상블 학습과의 비교를 통해 성능을 검증하고자 한다. 지역 기반 분류기의 앙상블 학습은 데이터의 분포가 지역에 따라 다르다는 점에 착안하여 학습 데이터를 분할하여 해당하는 지역에 기반을 둔 분류기들을 만들어 나간다. 이렇게 만들어진 분류기들로부터 지역에 따라 가중치를 둔 투표를 적용하여 앙상블 방법을 이끌어낸다. 본 논문에서 제시한 앙상블 분류기의 성능평가를 위해 단일 분류기와 기존의 앙상블 분류기인 배깅과 부스팅 등을 UCI Machine Learning Repository에 있는 11개의 데이터 셋으로 정확도 비교를 하였다. 그 결과 새로운 앙상블 방법이 기본 분류기로 나이브 베이즈와 SVM을 사용했을 때 다른 방법보다 좋은 성능을 보이는 것을 알 수 있었다.

• Computers and Concrete
• /
• 제32권3호
• /
• pp.313-326
• /
• 2023

The seismic safety of the shear wall structure can be assessed through seismic fragility analysis, which requires high computational costs in estimating seismic demands. Accordingly, machine learning methods have been applied to such fragility analyses in recent years to reduce the numerical analysis cost, but it still remains a challenging task. Therefore, this study uses the ensemble machine learning method to present an improved framework for developing a more accurate seismic demand model than the existing ones. To this end, a rank-based selection method that enables determining an excellent model among several single machine learning models is presented. In addition, an index that can evaluate the degree of overfitting/underfitting of each model for the selection of an excellent single model is suggested. Furthermore, based on the selected single machine learning model, we propose a method to derive a more accurate ensemble model based on the bagging method. As a result, the seismic demand model for which the proposed framework is applied shows about 3-17% better prediction performance than the existing single machine learning models. Finally, the seismic fragility obtained from the proposed framework shows better accuracy than the existing fragility methods.

• 한국정보과학회논문지:컴퓨팅의 실제 및 레터
• /
• 제15권2호
• /
• pp.135-139
• /
• 2009

본 논문에서는 지역 전문가를 이용한 새로운 앙상블 방법을 제시하고자 한다. 이 앙상블 방법에서는 학습 데이타를 분할하여 속성 공간의 서로 다른 지역을 이용하여 전문가를 학습시킨다. 새로운 데이타를 분류할 때에는 그 데이타가 속한 지역을 담당하는 전문가들로 가중치 투표를 한다. UCI 기계 학습 데이타 저장소에 있는 10개의 데이타를 이용하여 단일 분류기, Bagging, Adaboost와 정확도를 비교하였다. 학습 알고리즘으로는 SVM, Naive Bayes, C4.5를 사용하였다. 그 결과 지역 전문가의 앙상블 학습 방법이 C4.5를 학습 알고리즘으로 사용한 Bagging, Adaboost와는 비슷한 성능을 보였으며 나머지 분류기보다는 좋은 성능을 보였다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제14권11호
• /
• pp.4557-4572
• /
• 2020

The Alaska competition provides an opportunity to study the practical problems of real-world steganalysis. Participants are required to solve steganalysis involving various embedding schemes, inconsistency JPEG Quality Factor and various processing pipelines. In this paper, we propose a method to ensemble multiple deep learning steganalyzers. We select SRNet and RESDET as our base models. Then we design a three-layers model ensemble network to fuse these base models and output the final prediction. By separating the three colors channels for base model training and feature replacement strategy instead of simply merging features, the performance of the model ensemble is greatly improved. The proposed method won second place in the Alaska 1 competition in the end.

• Smart Structures and Systems
• /
• 제32권1호
• /
• pp.61-81
• /
• 2023

This study presents an ensemble learning based Bayesian model updating approach for structural damage diagnosis. In the developed framework, the structure is initially decomposed into a set of substructures. The autoregressive moving average (ARMAX) model is established first for structural damage localization based structural motion equation. The wavelet packet decomposition is utilized to extract the damage-sensitive node energy in different frequency bands for constructing structural surrogate models. Four methods, including Kriging predictor (KRG), radial basis function neural network (RBFNN), support vector regression (SVR), and multivariate adaptive regression splines (MARS), are selected as candidate structural surrogate models. These models are then resampled by bootstrapping and combined to obtain an ensemble model by probabilistic ensemble. Meanwhile, the maximum entropy principal is adopted to search for new design points for sample space updating, yielding a more robust ensemble model. Through the iterations, a framework of surrogate ensemble learning based model updating with high model construction efficiency and accuracy is proposed. The specificities of the method are discussed and investigated in a case study.

• Journal of Information Processing Systems
• /
• 제10권3호
• /
• pp.443-458
• /
• 2014

An extreme learning machine (ELM) is a recently proposed learning algorithm for a single-layer feed forward neural network. In this paper we studied the ensemble of ELM by using a bagging algorithm for facial expression recognition (FER). Facial expression analysis is widely used in the behavior interpretation of emotions, for cognitive science, and social interactions. This paper presents a method for FER based on the histogram of orientation gradient (HOG) features using an ELM ensemble. First, the HOG features were extracted from the face image by dividing it into a number of small cells. A bagging algorithm was then used to construct many different bags of training data and each of them was trained by using separate ELMs. To recognize the expression of the input face image, HOG features were fed to each trained ELM and the results were combined by using a majority voting scheme. The ELM ensemble using bagging improves the generalized capability of the network significantly. The two available datasets (JAFFE and CK+) of facial expressions were used to evaluate the performance of the proposed classification system. Even the performance of individual ELM was smaller and the ELM ensemble using a bagging algorithm improved the recognition performance significantly.