• Title/Summary/Keyword: 자동화 머신러닝

Search Result 64, Processing Time 0.025 seconds

Dam Inflow Prediction and Evaluation Using Hybrid Auto-sklearn Ensemble Model (하이브리드 Auto-sklearn 앙상블 모델을 이용한 댐 유입량 예측 및 평가)

  • Lee, Seoro;Bae, Joo Hyun;Lee, Gwanjae;Yang, Dongseok;Hong, Jiyeong;Kim, Jonggun;Lim, Kyoung Jae
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.307-307
    • /
    • 2022
  • 최근 기후변화와 댐 상류 토지이용 변화 등과 같은 다양한 원인에 의해 댐 유입량의 변동성이 증가하면서 댐 관리 및 운영조작 의사 결정에 어려움이 발생하고 있다. 따라서 이러한 댐 유입량의 변동 특성을 반영하여 댐 유입량을 정확하고 효율적으로 예측할 수 있는 방안이 필요한 실정이다. 머신러닝 기술이 발전하면서 Auto-ML(Automated Machine Learning)이 다양한 분야에서 활용되고 있다. Auto-ML은 데이터 전처리, 최적 알고리즘 선택, 하이퍼파라미터 튜닝, 모델 학습 및 평가 등의 모든 과정을 자동화하는 기술이다. 그러나 아직까지 수문 분야에서 댐 유입량을 예측하기 위한 모델을 개발하는데 있어서 Auto-ML을 활용한 사례는 부족하고, 특히 댐 유입량의 예측 정확성을 확보하기 위해 High-inflow and low-inflow 의 변동 특성을 고려한 하이브리드 결합 방식을 통해 Auto-ML 기반 앙상블 모델을 개발하고 평가한 연구는 없다. 본 연구에서는 Auto-ML의 패키지 중 Auto-sklearn을 통해 홍수기, 비홍수기 유입량 변동 특성을 반영한 하이브리드 앙상블 댐 유입량 예측 모델을 개발하였다. 소양강댐을 대상으로 적용한 결과, 하이브리드 Auto-sklearn 앙상블 모델의 댐 유입량 예측 성능은 R2 0.868, RMSE 66.23 m3/s, MAE 16.45 m3/s로 단일 Auto-sklearn을 통해 구축 된 앙상블 모델보다 전반적으로 우수한 것으로 나타났다. 특히 FDC (Flow Duration Curve)의 저수기, 갈수기 구간에서 두 모델의 유입량 예측 경향은 큰 차이를 보였으며, 하이브리드 Auto-sklearn 모델의 예측 값이 관측 값과 더욱 유사한 것으로 나타났다. 이는 홍수기, 비홍수기 구간에 대한 앙상블 모델이 독립적으로 구축되는 과정에서 각 모델에 대한 하이퍼파라미터가 최적화되었기 때문이라 판단된다. 향후 본 연구의 방법론은 보다 정확한 댐 유입량 예측 자료를 생성하기 위한 방안 수립뿐만 아니라 다양한 분야의 불균형한 데이터셋을 이용한 앙상블 모델을 구축하는데도 유용하게 활용될 수 있을 것으로 사료된다.

  • PDF

Blockchain-based Important Information Management Techniques for IoT Environment (IoT 환경을 위한 블록체인 기반의 중요 정보 관리 기법)

  • Yoon-Su Jeong
    • Advanced Industrial SCIence
    • /
    • v.3 no.1
    • /
    • pp.30-36
    • /
    • 2024
  • Recently, the Internet of Things (IoT), which has been applied to various industrial fields, is constantly evolving in the process of automation and digitization. However, in the network where IoT devices are built, research on IoT critical information-related data sharing, personal information protection, and data integrity among intermediate nodes is still being actively studied. In this study, we propose a blockchain-based IoT critical information management technique that is easy to implement without burdening the intermediate node in the network environment where IoT is built. The proposed technique allocates a random value of a random size to the IoT critical information arriving at the intermediate node and manages it to become a decentralized P2P blockchain. In addition, the proposed technique makes it easier to manage IoT critical data by creating licenses such as time limit and device limitation according to the weight condition of IoT critical information. Performance evaluation and proposed techniques have improved delay time and processing time by 7.6% and 10.1% on average compared to existing techniques.

A Study on Atmospheric Data Anomaly Detection Algorithm based on Unsupervised Learning Using Adversarial Generative Neural Network (적대적 생성 신경망을 활용한 비지도 학습 기반의 대기 자료 이상 탐지 알고리즘 연구)

  • Yang, Ho-Jun;Lee, Seon-Woo;Lee, Mun-Hyung;Kim, Jong-Gu;Choi, Jung-Mu;Shin, Yu-mi;Lee, Seok-Chae;Kwon, Jang-Woo;Park, Ji-Hoon;Jung, Dong-Hee;Shin, Hye-Jung
    • Journal of Convergence for Information Technology
    • /
    • v.12 no.4
    • /
    • pp.260-269
    • /
    • 2022
  • In this paper, We propose an anomaly detection model using deep neural network to automate the identification of outliers of the national air pollution measurement network data that is previously performed by experts. We generated training data by analyzing missing values and outliers of weather data provided by the Institute of Environmental Research and based on the BeatGAN model of the unsupervised learning method, we propose a new model by changing the kernel structure, adding the convolutional filter layer and the transposed convolutional filter layer to improve anomaly detection performance. In addition, by utilizing the generative features of the proposed model to implement and apply a retraining algorithm that generates new data and uses it for training, it was confirmed that the proposed model had the highest performance compared to the original BeatGAN models and other unsupervised learning model like Iforest and One Class SVM. Through this study, it was possible to suggest a method to improve the anomaly detection performance of proposed model while avoiding overfitting without additional cost in situations where training data are insufficient due to various factors such as sensor abnormalities and inspections in actual industrial sites.

Sensor Fault Detection Scheme based on Deep Learning and Support Vector Machine (딥 러닝 및 서포트 벡터 머신기반 센서 고장 검출 기법)

  • Yang, Jae-Wan;Lee, Young-Doo;Koo, In-Soo
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.18 no.2
    • /
    • pp.185-195
    • /
    • 2018
  • As machines have been automated in the field of industries in recent years, it is a paramount importance to manage and maintain the automation machines. When a fault occurs in sensors attached to the machine, the machine may malfunction and further, a huge damage will be caused in the process line. To prevent the situation, the fault of sensors should be monitored, diagnosed and classified in a proper way. In the paper, we propose a sensor fault detection scheme based on SVM and CNN to detect and classify typical sensor errors such as erratic, drift, hard-over, spike, and stuck faults. Time-domain statistical features are utilized for the learning and testing in the proposed scheme, and the genetic algorithm is utilized to select the subset of optimal features. To classify multiple sensor faults, a multi-layer SVM is utilized, and ensemble technique is used for CNN. As a result, the SVM that utilizes a subset of features selected by the genetic algorithm provides better performance than the SVM that utilizes all the features. However, the performance of CNN is superior to that of the SVM.

An Image-based CAPTCHA System with Correction of Sub-images (서브 이미지의 교정을 통한 이미지 기반의 CAPTCHA 시스템)

  • Chung, Woo-Keun;Ji, Seung-Hyun;Cho, Hwan-Gue
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.16 no.8
    • /
    • pp.873-877
    • /
    • 2010
  • CAPTCHA is a security tool that prevents the automatic sign-up by a spam or a robot. This CAPTCHA usually depends on the smart readability of humans. However, the common and plain CAPTCHA with text-based system is not difficult to be solved by intelligent web-bot and machine learning tools. In this paper, we propose a new sub-image based CAPTCHA system totally different from the text based system. Our system offers a set of cropped sub-image from a whole digital picture and asks user to identify the correct orientation. Though there are some nice machine learning tools for this job, but they are useless for a cropped sub-images, which was clearly revealed by our experiment. Experiment showed that our sub-image based CAPTCHA is easy to human solver, but very hard to all kinds of machine learning or AI tools. Also our CAPTCHA is easy to be generated automatical without any human intervention.

Stochastic Real-time Demand Prediction for Building and Charging and Discharging Technique of ESS Based on Machine-Learning (머신러닝기반 확률론적 실시간 건물에너지 수요예측 및 BESS충방전 기법)

  • Yang, Seung Kwon;Song, Taek Ho
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.5 no.3
    • /
    • pp.157-163
    • /
    • 2019
  • K-BEMS System was introduced to reduce peak load and to save total energy of the 120 buildings that KEPCO headquarter and branch offices use. K-BEMS system is composed of PV, battery, and hybrid PCS. In this system, ESS, PV, lighting is used to save building energy based on demand prediction. Currently, neural network technique for short past data is applied to demand prediction, and fixed scheduling method by operator for ESS charging/discharging is used. To enhance this system, KEPCO research institute has carried out this K-BEMS research project for 3 years since January 2016. As the result of this project, we developed new real-time highly reliable building demand prediction technique with error free and optimized automatic ESS charging/discharging technique. Through several field test, we can certify the developed algorithm performance successfully. So we will describe the details in this paper.

Computer Vision-Based Measurement Method for Wire Harness Defect Classification

  • Yun Jung Hong;Geon Lee;Jiyoung Woo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.1
    • /
    • pp.77-84
    • /
    • 2024
  • In this paper, we propose a method for accurately and rapidly detecting defects in wire harnesses by utilizing computer vision to calculate six crucial measurement values: the length of crimped terminals, the dimensions (width) of terminal ends, and the width of crimped sections (wire and core portions). We employ Harris corner detection to locate object positions from two types of data. Additionally, we generate reference points for extracting measurement values by utilizing features specific to each measurement area and exploiting the contrast in shading between the background and objects, thus reflecting the slope of each sample. Subsequently, we introduce a method using the Euclidean distance and correction coefficients to predict values, allowing for the prediction of measurements regardless of changes in the wire's position. We achieve high accuracy for each measurement type, 99.1%, 98.7%, 92.6%, 92.5%, 99.9%, and 99.7%, achieving outstanding overall average accuracy of 97% across all measurements. This inspection method not only addresses the limitations of conventional visual inspections but also yields excellent results with a small amount of data. Moreover, relying solely on image processing, it is expected to be more cost-effective and applicable with less data compared to deep learning methods.

Prediction of Landslides and Determination of Its Variable Importance Using AutoML (AutoML을 이용한 산사태 예측 및 변수 중요도 산정)

  • Nam, KoungHoon;Kim, Man-Il;Kwon, Oil;Wang, Fawu;Jeong, Gyo-Cheol
    • The Journal of Engineering Geology
    • /
    • v.30 no.3
    • /
    • pp.315-325
    • /
    • 2020
  • This study was performed to develop a model to predict landslides and determine the variable importance of landslides susceptibility factors based on the probabilistic prediction of landslides occurring on slopes along the road. Field survey data of 30,615 slopes from 2007 to 2020 in Korea were analyzed to develop a landslide prediction model. Of the total 131 variable factors, 17 topographic factors and 114 geological factors (including 89 bedrocks) were used to predict landslides. Automated machine learning (AutoML) was used to classify landslides and non-landslides. The verification results revealed that the best model, an extremely randomized tree (XRT) with excellent predictive performance, yielded 83.977% of prediction rates on test data. As a result of the analysis to determine the variable importance of the landslide susceptibility factors, it was composed of 10 topographic factors and 9 geological factors, which was presented as a percentage for each factor. This model was evaluated probabilistically and quantitatively for the likelihood of landslide occurrence by deriving the ranking of variable importance using only on-site survey data. It is considered that this model can provide a reliable basis for slope safety assessment through field surveys to decision-makers in the future.

Study on High-speed Cyber Penetration Attack Analysis Technology based on Static Feature Base Applicable to Endpoints (Endpoint에 적용 가능한 정적 feature 기반 고속의 사이버 침투공격 분석기술 연구)

  • Hwang, Jun-ho;Hwang, Seon-bin;Kim, Su-jeong;Lee, Tae-jin
    • Journal of Internet Computing and Services
    • /
    • v.19 no.5
    • /
    • pp.21-31
    • /
    • 2018
  • Cyber penetration attacks can not only damage cyber space but can attack entire infrastructure such as electricity, gas, water, and nuclear power, which can cause enormous damage to the lives of the people. Also, cyber space has already been defined as the fifth battlefield, and strategic responses are very important. Most of recent cyber attacks are caused by malicious code, and since the number is more than 1.6 million per day, automated analysis technology to cope with a large amount of malicious code is very important. However, it is difficult to deal with malicious code encryption, obfuscation and packing, and the dynamic analysis technique is not limited to the performance requirements of dynamic analysis but also to the virtual There is a limit in coping with environment avoiding technology. In this paper, we propose a machine learning based malicious code analysis technique which improve the weakness of the detection performance of existing analysis technology while maintaining the light and high-speed analysis performance applicable to commercial endpoints. The results of this study show that 99.13% accuracy, 99.26% precision and 99.09% recall analysis performance of 71,000 normal file and malicious code in commercial environment and analysis time in PC environment can be analyzed more than 5 per second, and it can be operated independently in the endpoint environment and it is considered that it works in complementary form in operation in conjunction with existing antivirus technology and static and dynamic analysis technology. It is also expected to be used as a core element of EDR technology and malware variant analysis.

A Study on the Prediction of Strawberry Production in Machine Learning Infrastructure (머신러닝 기반 시설재배 딸기 생산량 예측 연구)

  • Oh, HanByeol;Lim, JongHyun;Yang, SeungWeon;Cho, YongYun;Shin, ChangSun
    • Smart Media Journal
    • /
    • v.11 no.5
    • /
    • pp.9-16
    • /
    • 2022
  • Recently, agricultural sites are automating into digital agricultural smart farms by applying technologies such as big data and Internet of Things (IoT). These smart farms aim to increase production and improve crop quality by measuring the environment of crops, investigating and processing data. Production prediction is an important study in smart farm digital agriculture, which is a high-tech agriculture, and it is necessary to analyze environmental data using big data and further standardized research to manage the quality of growth information data. In this paper, environmental and production data collected from smart farm strawberry farms were analyzed and studied. Based on regression analysis, crop production prediction models were analyzed using Ridge Regression, LightGBM, and XGBoost. Among the three models, the optimal model was XGBoost, and R2 showed 82.5 percent explanatory power. As a result of the study, the correlation between the amount of positive fluid absorption and environmental data was confirmed, and significant results were obtained for the production prediction study. In the future, it is expected to contribute to the prevention of environmental pollution and reduction of sheep through the management of sheep by studying the amount of sheep absorption, such as information on the growing environment of crops and the ingredients of sheep.