• Computers and Concrete
• /
• 제32권3호
• /
• pp.233-246
• /
• 2023

Accurate prediction of concrete compressive strength can minimize the need for extensive, time-consuming, and costly mixture optimization testing and analysis. This study attempts to enhance the prediction accuracy of compressive strength using stacking ensemble machine learning (ML) with feature engineering techniques. Seven alternative ML models of increasing complexity were implemented and compared, including linear regression, SVM, decision tree, multiple layer perceptron, random forest, Xgboost and Adaboost. To further improve the prediction accuracy, a ML pipeline was proposed in which the feature engineering technique was implemented, and a two-layer stacked model was developed. The k-fold cross-validation approach was employed to optimize model parameters and train the stacked model. The stacked model showed superior performance in predicting concrete compressive strength with a correlation of determination (R²) of 0.985. Feature (i.e., variable) importance was determined to demonstrate how useful the synthetic features are in prediction and provide better interpretability of the data and the model. The methodology in this study promotes a more thorough assessment of alternative ML algorithms and rather than focusing on any single ML model type for concrete compressive strength prediction.

• 응용통계연구
• /
• 제32권2호
• /
• pp.319-330
• /
• 2019

과거의 전력소모량을 분석하여 미래의 전력소모량을 예측하는 것은 에너지 계획과 정책 결정에 있어 많은 이점을 가져다준다. 기계학습은 최근 전력소모량을 예측하는 분석 방법으로 많이 사용하고 있다. 그중 앙상블 학습은 모형의 과적합 현상을 방지하고 분산을 줄여 예측의 정확성을 높이는 방법으로 알려져 있다. 하지만 일별 데이터에 앙상블 학습을 적용했을 때 분석 방법의 특성으로 인해 피크를 잘 나타내지 못하고 중심값으로 예측하는 단점을 보였다. 본 연구에서는 앙상블 학습 전에 온도 변수와의 상관성을 고려하여 선형모형으로 적합함으로써 앙상블 학습의 단점을 보완한다. 그리고 9개의 모형을 비교한 결과 온도 변수를 선형모형으로 적합하고 랜덤포레스트를 사용한 모형이 결과가 가장 좋음을 보여준다.

• Journal of Information Processing Systems
• /
• 제19권5호
• /
• pp.673-687
• /
• 2023

Convolutional neural network-based deep learning technology is the most commonly used in image identification, but it requires large-scale data for training. Therefore, application in specific fields in which data acquisition is limited, such as in the military, may be challenging. In particular, the identification of ground weapon systems is a very important mission, and high identification accuracy is required. Accordingly, various studies have been conducted to achieve high performance using small-scale data. Among them, the ensemble method, which achieves excellent performance through the prediction average of the pre-trained models, is the most representative method; however, it requires considerable time and effort to find the optimal combination of ensemble models. In addition, there is a performance limitation in the prediction results obtained by using an ensemble method. Furthermore, it is difficult to obtain the ensemble effect using models with imbalanced classification accuracies. In this paper, we propose a transfer learning-based feature fusion technique for heterogeneous models that extracts and fuses features of pre-trained heterogeneous models and finally, fine-tunes hyperparameters of the fully connected layer to improve the classification accuracy. The experimental results of this study indicate that it is possible to overcome the limitations of the existing ensemble methods by improving the classification accuracy through feature fusion between heterogeneous models based on transfer learning.

• 한국통계학회:학술대회논문집
• /
• 한국통계학회 2005년도 추계 학술발표회 논문집
• /
• pp.193-197
• /
• 2005

Classification tree is one of the most suitable base learners for ensemble. For past decade, it was found that bagging gives the most accurate prediction when used with unpruned tree and boosting with stump. Researchers have tried to understand the relationship between the size of trees and the accuracy of ensemble. With experiment, it is found that large trees make boosting overfit the dataset and stumps help avoid it. It means that the accuracy of each classifier needs to be sacrificed for better weighting at each iteration. Hence, split effect in boosting can be explained with the trade-off between the accuracy of each classifier and better weighting on the misclassified points. In bagging, combining larger trees give more accurate prediction because bagging does not have such trade-off, thus it is advisable to make each classifier as accurate as possible.

• Journal of the Korean Data and Information Science Society
• /
• 제23권2호
• /
• pp.375-383
• /
• 2012

The purpose of the ensemble methods is to increase the accuracy of prediction through combining many classifiers. According to recent studies, it is proved that random forests and forward stagewise regression have good accuracies in classification problems. However they have great prediction error in separation boundary points because they used decision tree as a base learner. In this study, we use the kernel ridge regression instead of the decision trees in random forests and boosting. The usefulness of our proposed ensemble methods was shown by the simulation results of the prostate cancer and the Boston housing data.

• Communications for Statistical Applications and Methods
• /
• 제28권3호
• /
• pp.217-232
• /
• 2021

The Korea film industry has matured and the number of movie-watching per capita has reached the highest level in the world. Since then, movie industry growth rate is decreasing and even the total sales of movies per year slightly decreased in 2018. The number of moviegoers is the first factor of sales in movie industry and also an important factor influencing additional sales. Thus it is important to predict the number of movie audiences. In this study, we predict the cumulative number of audiences of films using stacking, an ensemble method. Stacking is a kind of ensemble method that combines all the algorithms used in the prediction. We use box office data from Korea Film Council and web comment data from Daum Movie (www.movie.daum.net). This paper describes the process of collecting and preprocessing of explanatory variables and explains regression models used in stacking. Final stacking model outperforms in the prediction of test set in terms of RMSE.

• 스마트미디어저널
• /
• 제13권4호
• /
• pp.9-15
• /
• 2024

Tree-based algorithms have been the dominant methods used build a prediction model for tabular data. This also includes personal credit data. However, they are limited to compatibility with categorical and numerical data only, and also do not capture information of the relationship between other features. In this work, we proposed an ensemble model using the Transformer architecture that includes text features and harness the self-attention mechanism to tackle the feature relationships limitation. We describe a text formatter module, that converts the original tabular data into sentence data that is fed into FinBERT along with other text features. Furthermore, we employed FT-Transformer that train with the original tabular data. We evaluate this multi-modal approach with two popular tree-based algorithms known as, Random Forest and Extreme Gradient Boosting, XGBoost and TabTransformer. Our proposed method shows superior Default Recall, F1 score and AUC results across two public data sets. Our results are significant for financial institutions to reduce the risk of financial loss regarding defaulters.

• 정보처리학회논문지:소프트웨어 및 데이터공학
• /
• 제9권5호
• /
• pp.161-168
• /
• 2020

스포츠 경기 결과예측은 전반적인 경기의 흐름과 승패에 영향을 미치는 변인들의 분석을 통해 팀의 전략 수립을 가능하게 해준다. 이와 같은 스포츠 경기결과 예측에 대한 연구는 주로 통계학적 기법과 기계학습 기법을 활용하여 진행되어 왔다. 승부예측 모델은 무엇보다 예측 성능이 가장 중요시된다. 그러나 최적의 성능을 보이는 예측 모델은 학습에 사용되는 데이터에 따라 다르게 나타나는 경향을 보였다. 본 논문에서는 이러한 문제를 해결하기 위해 데이터가 달라지더라도 해당 데이터에 대한 예측 시 가장 좋은 성능을 보이는 모델의 선택이 가능한 기존의 축구경기결과 예측에서 좋은 성능을 보여온 통계학적 모델과 기계학습 모델을 결합한 새로운 앙상블 모델을 제안한다. 본 논문에서 제안하는 앙상블 모델은 각 단일모델들의 경기 예측결과와 실제 경기결과를 병합한 데이터로부터 최종예측모델을 학습하여 경기 승부예측을 수행한다. 제안 모델에 대한 실험 결과, 기존 단일모델들에 비해 높은 성능을 보였다.

• 대한전자공학회논문지SP
• /
• 제49권1호
• /
• pp.41-48
• /
• 2012

적조란 유해조류의 일시적인 대 번식으로 바다를 적색으로 변화시키며 양식장의 어패류를 집단 폐사 시킬 뿐 아니라 연안환경 및 바다 생태계에 악영향을 미치는 자연 현상이다. 적조에 의한 양식어업의 피해는 매년 발생하고 있으며 매년 적조방제에 많은 비용을 소비하고 있다. 이 때문에 적조 발생을 미리 예측할 수 있으면 적조에 대한 피해 및 방재 비용을 최소화 시킬수 있다. 본 논문은 앙상블 학습은 이용한 적조발생 예측 방법을 제안한다. 제안방법은 앙상블 학습의 bagging과 boosting 방법을 이용하여서 적조를 예측의 성능을 향상시킨다. 실험결과 제안방법은 단일 분류기에 비하여서 더 좋은 적조 발생 예측 성능을 보였다.

• 산업경영시스템학회지
• /
• 제42권4호
• /
• pp.203-210
• /
• 2019

Recently, a number of researchers have produced research and reports in order to forecast more exactly air quality such as particulate matter and odor. However, such research mainly focuses on the atmospheric diffusion models that have been used for the air quality prediction in environmental engineering area. Even though it has various merits, it has some limitation in that it uses very limited spatial attributes such as geographical attributes. Thus, we propose the new approach to forecast an air quality using a deep learning based ensemble model combining temporal and spatial predictor. The temporal predictor employs the RNN LSTM and the spatial predictor is based on the geographically weighted regression model. The ensemble model also uses the RNN LSTM that combines two models with stacking structure. The ensemble model is capable of inferring the air quality of the areas without air quality monitoring station, and even forecasting future air quality. We installed the IoT sensors measuring PM2.5, PM10, H2S, NH3, VOC at the 8 stations in Jeonju in order to gather air quality data. The numerical results showed that our new model has very exact prediction capability with comparison to the real measured data. It implies that the spatial attributes should be considered to more exact air quality prediction.