• Title/Summary/Keyword: XGboost

Search Result 244, Processing Time 0.025 seconds

Objectivity in Korean News Reporting : Machine Learning-Based Verification of News Headline Accuracy (기계학습 기반 국내 뉴스 헤드라인의 정확성 검증 연구)

  • Baik, Jisoo;Lee, Seung Eon;Han, Jiyoung;Cha, Meeyoung
    • Annual Conference on Human and Language Technology
    • /
    • 2021.10a
    • /
    • pp.281-286
    • /
    • 2021
  • 뉴스 헤드라인에 제3자의 발언을 직접 인용해 전언하는 이른바 '따옴표 저널리즘'이 언론 보도의 객관주의 원칙을 해치는지는 언론학 및 뉴스 구독자에게 중요한 문제이다. 이 연구는 온라인 포털사이트를 통해 실시간 유통되는 한국어 기사의 정확성을 판별하기 위한 기계학습(Machine Learning) 모델을 제안한다. 이 연구에서 제안하는 모델은 Edit Distance와 FastText 기법을 활용해 기사 제목과 본문 내 인용구의 유사성을 측정하고, XGBoost 모델을 활용해 최종 분류한다. 아울러 이 모델을 통해 229만 건의 뉴스 헤드라인에 대해 직접 인용구가 포함된 기사가 취재원의 발언을 주관적인 윤색없이 독자들에게 전하고 있는지를 판별했다. 이뿐만 아니라 딥러닝 기반의 KoELECTRA 모델을 활용해 기사의 제목 내 인용구에 대한 감성 분석을 진행했다. 분석 결과, 윤색이 가미되지 않은 직접 인용형 기사의 비율이 지난 20년 동안 10% 이상 증가했으며, 기사 제목의 인용구에 나타나는 감정은 부정 감성이 긍정 감성의 2.8배 정도로 우세했다. 이러한 시도는 앞으로 계산사회과학 방법론과 빅데이터에 기반한 언론 보도의 평가 및 개선에 도움을 주리라 기대한다.

  • PDF

Study on Weather Data Interpolation of a Buoy Based on Machine Learning Techniques (기계 학습을 이용한 항로표지 기상 자료의 보간에 관한 연구)

  • Seong-Hun Jeong;Jun-Ik Ma;Seong-Hyun Jo;Gi-Ryun Lim;Jun-Woo Lee;Jun-Hee Han
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2022.06a
    • /
    • pp.72-74
    • /
    • 2022
  • Several types of data are collected from buoy due to the development of hardware technology.. However, the collected data are difficult to use due to errors including missing values and outliers depending on mechanical faults and meteorological environment. Therefore, in this study, linear interpolation is performed by adding the missing time data to enable machine learning to the insufficient meteorological data. After the linear interpolation, XGBoost and KNN-regressor, are used to forecast error data and suggested model is evaluated by using real-world data of a buoy.

  • PDF

Software Defect Prediction Based on SAINT (SAINT 기반의 소프트웨어 결함 예측)

  • Sriman Mohapatra;Eunjeong Ju;Jeonghwa Lee;Duksan Ryu
    • The Transactions of the Korea Information Processing Society
    • /
    • v.13 no.5
    • /
    • pp.236-242
    • /
    • 2024
  • Software Defect Prediction (SDP) enhances the efficiency of software development by proactively identifying modules likely to contain errors. A major challenge in SDP is improving prediction performance. Recent research has applied deep learning techniques to the field of SDP, with the SAINT model particularly gaining attention for its outstanding performance in analyzing structured data. This study compares the SAINT model with other leading models (XGBoost, Random Forest, CatBoost) and investigates the latest deep learning techniques applicable to SDP. SAINT consistently demonstrated superior performance, proving effective in improving defect prediction accuracy. These findings highlight the potential of the SAINT model to advance defect prediction methodologies in practical software development scenarios, and were achieved through a rigorous methodology including cross-validation, feature scaling, and comparative analysis.

A Study on Predicting Lung Cancer Using RNA-Sequencing Data with Ensemble Learning (앙상블 기법을 활용한 RNA-Sequencing 데이터의 폐암 예측 연구)

  • Geon AN;JooYong PARK
    • Journal of Korea Artificial Intelligence Association
    • /
    • v.2 no.1
    • /
    • pp.7-14
    • /
    • 2024
  • In this paper, we explore the application of RNA-sequencing data and ensemble machine learning to predict lung cancer and treatment strategies for lung cancer, a leading cause of cancer mortality worldwide. The research utilizes Random Forest, XGBoost, and LightGBM models to analyze gene expression profiles from extensive datasets, aiming to enhance predictive accuracy for lung cancer prognosis. The methodology focuses on preprocessing RNA-seq data to standardize expression levels across samples and applying ensemble algorithms to maximize prediction stability and reduce model overfitting. Key findings indicate that ensemble models, especially XGBoost, substantially outperform traditional predictive models. Significant genetic markers such as ADGRF5 is identified as crucial for predicting lung cancer outcomes. In conclusion, ensemble learning using RNA-seq data proves highly effective in predicting lung cancer, suggesting a potential shift towards more precise and personalized treatment approaches. The results advocate for further integration of molecular and clinical data to refine diagnostic models and improve clinical outcomes, underscoring the critical role of advanced molecular diagnostics in enhancing patient survival rates and quality of life. This study lays the groundwork for future research in the application of RNA-sequencing data and ensemble machine learning techniques in clinical settings.

Combination of fuzzy models via economic management for city multi-spectral remote sensing nano imagery road target

  • Weihua Luo;Ahmed H. Janabi;Joffin Jose Ponnore;Hanadi Hakami;Hakim AL Garalleh;Riadh Marzouki;Yuanhui Yu;Hamid Assilzadeh
    • Advances in nano research
    • /
    • v.16 no.6
    • /
    • pp.531-548
    • /
    • 2024
  • The study focuses on using remote sensing to gather data about the Earth's surface, particularly in urban environments, using satellites and aircraft-mounted sensors. It aims to develop a classification framework for road targets using multi-spectral imagery. By integrating Convolutional Neural Networks (CNNs) with XGBoost, the study seeks to enhance the accuracy and efficiency of road target identification, aiding urban infrastructure management and transportation planning. A novel aspect of the research is the incorporation of quantum sensors, which improve the resolution and sensitivity of the data. The model achieved high predictive accuracy with an MSE of 0.025, R-squared of 0.85, RMSE of 0.158, and MAE of 0.12. The CNN model showed excellent performance in road detection with 92% accuracy, 88% precision, 90% recall, and an f1-score of 89%. These results demonstrate the model's robustness and applicability in real-world urban planning scenarios, further enhanced by data augmentation and early stopping techniques.

Study on Fault Diagnosis and Data Processing Techniques for Substrate Transfer Robots Using Vibration Sensor Data

  • MD Saiful Islam;Mi-Jin Kim;Kyo-Mun Ku;Hyo-Young Kim;Kihyun Kim
    • Journal of the Microelectronics and Packaging Society
    • /
    • v.31 no.2
    • /
    • pp.45-53
    • /
    • 2024
  • The maintenance of semiconductor equipment is crucial for the continuous growth of the semiconductor market. System management is imperative given the anticipated increase in the capacity and complexity of industrial equipment. Ensuring optimal operation of manufacturing processes is essential to maintaining a steady supply of numerous parts. Particularly, monitoring the status of substrate transfer robots, which play a central role in these processes, is crucial. Diagnosing failures of their major components is vital for preventive maintenance. Fault diagnosis methods can be broadly categorized into physics-based and data-driven approaches. This study focuses on data-driven fault diagnosis methods due to the limitations of physics-based approaches. We propose a methodology for data acquisition and preprocessing for robot fault diagnosis. Data is gathered from vibration sensors, and the data preprocessing method is applied to the vibration signals. Subsequently, the dataset is trained using Gradient Tree-based XGBoost machine learning classification algorithms. The effectiveness of the proposed model is validated through performance evaluation metrics, including accuracy, F1 score, and confusion matrix. The XGBoost classifiers achieve an accuracy of approximately 92.76% and an equivalent F1 score. ROC curves indicate exceptional performance in class discrimination, with 100% discrimination for the normal class and 98% discrimination for abnormal classes.

A Study on the Retrieval of River Turbidity Based on KOMPSAT-3/3A Images (KOMPSAT-3/3A 영상 기반 하천의 탁도 산출 연구)

  • Kim, Dahui;Won, You Jun;Han, Sangmyung;Han, Hyangsun
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.6_1
    • /
    • pp.1285-1300
    • /
    • 2022
  • Turbidity, the measure of the cloudiness of water, is used as an important index for water quality management. The turbidity can vary greatly in small river systems, which affects water quality in national rivers. Therefore, the generation of high-resolution spatial information on turbidity is very important. In this study, a turbidity retrieval model using the Korea Multi-Purpose Satellite-3 and -3A (KOMPSAT-3/3A) images was developed for high-resolution turbidity mapping of Han River system based on eXtreme Gradient Boosting (XGBoost) algorithm. To this end, the top of atmosphere (TOA) spectral reflectance was calculated from a total of 24 KOMPSAT-3/3A images and 150 Landsat-8 images. The Landsat-8 TOA spectral reflectance was cross-calibrated to the KOMPSAT-3/3A bands. The turbidity measured by the National Water Quality Monitoring Network was used as a reference dataset, and as input variables, the TOA spectral reflectance at the locations of in situ turbidity measurement, the spectral indices (the normalized difference vegetation index, normalized difference water index, and normalized difference turbidity index), and the Moderate Resolution Imaging Spectroradiometer (MODIS)-derived atmospheric products(the atmospheric optical thickness, water vapor, and ozone) were used. Furthermore, by analyzing the KOMPSAT-3/3A TOA spectral reflectance of different turbidities, a new spectral index, new normalized difference turbidity index (nNDTI), was proposed, and it was added as an input variable to the turbidity retrieval model. The XGBoost model showed excellent performance for the retrieval of turbidity with a root mean square error (RMSE) of 2.70 NTU and a normalized RMSE (NRMSE) of 14.70% compared to in situ turbidity, in which the nNDTI proposed in this study was used as the most important variable. The developed turbidity retrieval model was applied to the KOMPSAT-3/3A images to map high-resolution river turbidity, and it was possible to analyze the spatiotemporal variations of turbidity. Through this study, we could confirm that the KOMPSAT-3/3A images are very useful for retrieving high-resolution and accurate spatial information on the river turbidity.

Estimation of Chlorophyll-a Concentration in Nakdong River Using Machine Learning-Based Satellite Data and Water Quality, Hydrological, and Meteorological Factors (머신러닝 기반 위성영상과 수질·수문·기상 인자를 활용한 낙동강의 Chlorophyll-a 농도 추정)

  • Soryeon Park;Sanghun Son;Jaegu Bae;Doi Lee;Dongju Seo;Jinsoo Kim
    • Korean Journal of Remote Sensing
    • /
    • v.39 no.5_1
    • /
    • pp.655-667
    • /
    • 2023
  • Algal bloom outbreaks are frequently reported around the world, and serious water pollution problems arise every year in Korea. It is necessary to protect the aquatic ecosystem through continuous management and rapid response. Many studies using satellite images are being conducted to estimate the concentration of chlorophyll-a (Chl-a), an indicator of algal bloom occurrence. However, machine learning models have recently been used because it is difficult to accurately calculate Chl-a due to the spectral characteristics and atmospheric correction errors that change depending on the water system. It is necessary to consider the factors affecting algal bloom as well as the satellite spectral index. Therefore, this study constructed a dataset by considering water quality, hydrological and meteorological factors, and sentinel-2 images in combination. Representative ensemble models random forest and extreme gradient boosting (XGBoost) were used to predict the concentration of Chl-a in eight weirs located on the Nakdong river over the past five years. R-squared score (R2), root mean square errors (RMSE), and mean absolute errors (MAE) were used as model evaluation indicators, and it was confirmed that R2 of XGBoost was 0.80, RMSE was 6.612, and MAE was 4.457. Shapley additive expansion analysis showed that water quality factors, suspended solids, biochemical oxygen demand, dissolved oxygen, and the band ratio using red edge bands were of high importance in both models. Various input data were confirmed to help improve model performance, and it seems that it can be applied to domestic and international algal bloom detection.

Analysis on the Determinants of Land Compensation Cost: The Use of the Construction CALS Data (토지 보상비 결정 요인 분석 - 건설CALS 데이터 중심으로)

  • Lee, Sang-Gyu;Seo, Myoung-Bae;Kim, Jin-Uk
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.21 no.10
    • /
    • pp.461-470
    • /
    • 2020
  • This study analyzed the determinants of land compensation costs using the CALS (Continuous Acquisition & Life-Cycle Support) system to generate data for the construction (planning, design, building, management) process. For analysis, variables used in the related research on land costs were used, which included eight variables (Land Area, Individual Public Land Price, Appraisal & Assessment, Land Category, Use District 1, Terrain Elevation, Terrain Shape, and Road). Also, the variables were analyzed using the machine learning-based Xgboost algorithm. Individual Public Land Price was identified as the most important variable in determining land cost. We used a linear multiple regression analysis to verify the determinants of land compensation. For this verification, the dependent variable included was the Individual Public Land Price, and the independent variables were the numeric variable (Land Area) and factor variables (Land Category, Use District 1, Terrain Elevation, Terrain Shape, Road). This study found that the significant variables were Land Category, Use District 1, and Road.

Using Mechanical Learning Analysis of Determinants of Housing Sales and Establishment of Forecasting Model (기계학습을 활용한 주택매도 결정요인 분석 및 예측모델 구축)

  • Kim, Eun-mi;Kim, Sang-Bong;Cho, Eun-seo
    • Journal of Cadastre & Land InformatiX
    • /
    • v.50 no.1
    • /
    • pp.181-200
    • /
    • 2020
  • This study used the OLS model to estimate the determinants affecting the tenure of a home and then compared the predictive power of each model with SVM, Decision Tree, Random Forest, Gradient Boosting, XGBooest and LightGBM. There is a difference from the preceding study in that the Stacking model, one of the ensemble models, can be used as a base model to establish a more predictable model to identify the volume of housing transactions in the housing market. OLS analysis showed that sales profits, housing prices, the number of household members, and the type of residential housing (detached housing, apartments) affected the period of housing ownership, and compared the predictability of the machine learning model with RMSE, the results showed that the machine learning model had higher predictability. Afterwards, the predictive power was compared by applying each machine learning after rebuilding the data with the influencing variables, and the analysis showed the best predictive power of Random Forest. In addition, the most predictable Random Forest, Decision Tree, Gradient Boosting, and XGBooost models were applied as individual models, and the Stacking model was constructed using Linear, Ridge, and Lasso models as meta models. As a result of the analysis, the RMSE value in the Ridge model was the lowest at 0.5181, thus building the highest predictive model.