• Title/Summary/Keyword: 자동화 머신러닝

Search Result 68, Processing Time 0.028 seconds

Korean Text Classification Using Randomforest and XGBoost Focusing on Seoul Metropolitan Civil Complaint Data (RandomForest와 XGBoost를 활용한 한국어 텍스트 분류: 서울특별시 응답소 민원 데이터를 중심으로)

  • Ha, Ji-Eun;Shin, Hyun-Chul;Lee, Zoon-Ky
    • The Journal of Bigdata
    • /
    • v.2 no.2
    • /
    • pp.95-104
    • /
    • 2017
  • In 2014, Seoul Metropolitan Government launched a response service aimed at responding promptly to civil complaints. The complaints received are categorized based on their content and sent to the department in charge. If this part can be automated, the time and labor costs will be reduced. In this study, we collected 17,700 cases of complaints for 7 years from June 1, 2010 to May 31, 2017. We compared the XGBoost with RandomForest and confirmed the suitability of Korean text classification. As a result, the accuracy of XGBoost compared to RandomForest is generally high. The accuracy of RandomForest was unstable after upsampling and downsampling using the same sample, while XGBoost showed stable overall accuracy.

  • PDF

LSTM Model based on Session Management for Network Intrusion Detection (네트워크 침입탐지를 위한 세션관리 기반의 LSTM 모델)

  • Lee, Min-Wook
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.3
    • /
    • pp.1-7
    • /
    • 2020
  • With the increase in cyber attacks, automated IDS using machine learning is being studied. According to recent research, the IDS using the recursive learning model shows high detection performance. However, the simple application of the recursive model may be difficult to reflect the associated session characteristics, as the overlapping session environment may degrade the performance. In this paper, we designed the session management module and applied it to LSTM (Long Short-Term Memory) recursive model. For the experiment, the CSE-CIC-IDS 2018 dataset is used and increased the normal session ratio to reduce the association of mal-session. The results show that the proposed model is able to maintain high detection performance even in the environment where session relevance is difficult to find.

A Deep Learning Application for Automated Feature Extraction in Transaction-based Machine Learning (트랜잭션 기반 머신러닝에서 특성 추출 자동화를 위한 딥러닝 응용)

  • Woo, Deock-Chae;Moon, Hyun Sil;Kwon, Suhnbeom;Cho, Yoonho
    • Journal of Information Technology Services
    • /
    • v.18 no.2
    • /
    • pp.143-159
    • /
    • 2019
  • Machine learning (ML) is a method of fitting given data to a mathematical model to derive insights or to predict. In the age of big data, where the amount of available data increases exponentially due to the development of information technology and smart devices, ML shows high prediction performance due to pattern detection without bias. The feature engineering that generates the features that can explain the problem to be solved in the ML process has a great influence on the performance and its importance is continuously emphasized. Despite this importance, however, it is still considered a difficult task as it requires a thorough understanding of the domain characteristics as well as an understanding of source data and the iterative procedure. Therefore, we propose methods to apply deep learning for solving the complexity and difficulty of feature extraction and improving the performance of ML model. Unlike other techniques, the most common reason for the superior performance of deep learning techniques in complex unstructured data processing is that it is possible to extract features from the source data itself. In order to apply these advantages to the business problems, we propose deep learning based methods that can automatically extract features from transaction data or directly predict and classify target variables. In particular, we applied techniques that show high performance in existing text processing based on the structural similarity between transaction data and text data. And we also verified the suitability of each method according to the characteristics of transaction data. Through our study, it is possible not only to search for the possibility of automated feature extraction but also to obtain a benchmark model that shows a certain level of performance before performing the feature extraction task by a human. In addition, it is expected that it will be able to provide guidelines for choosing a suitable deep learning model based on the business problem and the data characteristics.

An Exploratory Study on Policy Decision Making with Artificial Intelligence: Applying Problem Structuring Typology on Success and Failure Cases (인공지능을 활용한 정책의사결정에 관한 탐색적 연구: 문제구조화 유형으로 살펴 본 성공과 실패 사례 분석)

  • Eun, Jong-Hwan;Hwang, Sung-Soo
    • Informatization Policy
    • /
    • v.27 no.4
    • /
    • pp.47-66
    • /
    • 2020
  • The rapid development of artificial intelligence technologies such as machine learning and deep learning is expanding its impact in the public administrative and public policy sphere. This paper is an exploratory study on policy decision-making in the age of artificial intelligence to design automated configuration and operation through data analysis and algorithm development. The theoretical framework was composed of the types of policy problems according to the degree of problem structuring, and the success and failure cases were classified and analyzed to derive implications. In other words, when the problem structuring is more difficult than others, the greater the possibility of failure or side effects of decision-making using artificial intelligence. Also, concerns about the neutrality of the algorithm were presented. As a policy suggestion, a subcommittee was proposed in which experts in technical and social aspects play a professional role in establishing the AI promotion system in Korea. Although the subcommittee works independently, it suggests that it is necessary to establish governance in which the results of activities can be synthesized and integrated.

A Predictive System for Equipment Fault Diagnosis based on Machine Learning in Smart Factory (스마트 팩토리에서 머신 러닝 기반 설비 장애진단 예측 시스템)

  • Chow, Jaehyung;Lee, Jaeoh
    • KNOM Review
    • /
    • v.24 no.1
    • /
    • pp.13-19
    • /
    • 2021
  • In recent, there is research to maximize production by preventing failures/accidents in advance through fault diagnosis/prediction and factory automation in the industrial field. Cloud technology for accumulating a large amount of data, big data technology for data processing, and Artificial Intelligence(AI) technology for easy data analysis are promising candidate technologies for accomplishing this. Also, recently, due to the development of fault diagnosis/prediction, the equipment maintenance method is also developing from Time Based Maintenance(TBM), being a method of regularly maintaining equipment, to the TBM of combining Condition Based Maintenance(CBM), being a method of maintenance according to the condition of the equipment. For CBM-based maintenance, it is necessary to define and analyze the condition of the facility. Therefore, we propose a machine learning-based system and data model for diagnosing the fault in this paper. And based on this, we will present a case of predicting the fault occurrence in advance.

Development of Medical Image Quality Assessment Tool Based on Chest X-ray (흉부 X-ray 기반 의료영상 품질평가 보조 도구 개발)

  • Gi-Hyeon Nam;Dong-Yeon Yoo;Yang-Gon Kim;Joo-Sung Sun;Jung-Won Lee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.6
    • /
    • pp.243-250
    • /
    • 2023
  • Chest X-ray is radiological examination for xeamining the lungs and haert, and is particularly widely used for diagnosing lung disease. Since the quality of these chest X-rays can affect the doctor's diagnosis, the process of evaluating the quality must necessarily go through. This process can involve the subjectivity of radiologists and is manual, so it takes a lot of time and csot. Therefore, in this paper, based on the chest X-ray quality assessment guidelines used in clinical settings, we propose a tool that automates the five quality assessments of artificial shadow, coverage, patient posture, inspiratory level, and permeability. The proposed tool reduces the time and cost required for quality judgment, and can be further utilized in the pre-processing process of selecting high-quality learning data for the development of a learning model for diagnosing chest lesions.

Development of Big Data and AutoML Platforms for Smart Plants (스마트 플랜트를 위한 빅데이터 및 AutoML 플랫폼 개발)

  • Jin-Young Kang;Byeong-Seok Jeong
    • The Journal of Bigdata
    • /
    • v.8 no.2
    • /
    • pp.83-95
    • /
    • 2023
  • Big data analytics and AI play a critical role in the development of smart plants. This study presents a big data platform for plant data and an 'AutoML platform' for AI-based plant O&M(Operation and Maintenance). The big data platform collects, processes and stores large volumes of data generated in plants using Hadoop, Spark, and Kafka. The AutoML platform is a machine learning automation system aimed at constructing predictive models for equipment prognostics and process optimization in plants. The developed platforms configures a data pipeline considering compatibility with existing plant OISs(Operation Information Systems) and employs a web-based GUI to enhance both accessibility and convenience for users. Also, it has functions to load user-customizable modules into data processing and learning algorithms, which increases process flexibility. This paper demonstrates the operation of the platforms for a specific process of an oil company in Korea and presents an example of an effective data utilization platform for smart plants.

Design of A New CAPTCHA System using Detecting Orientation of Polygonal Image (다각형 이미지의 방향 결정을 이용한 새로운 CAPTCHA 시스템의 설계)

  • Chung, WooKeun;Kim, JongWoo;Cho, HwanGue
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2010.04a
    • /
    • pp.766-769
    • /
    • 2010
  • CAPTCHA 시스템은 스팸이나 로봇에 의한 자동 가입, 계정 생성 방지도구로써 인간의 우수한 가독성을 통해 특정 언어 또는 그림을 해독할 수 있는 특성을 이용한 것으로 일반적으로 컴퓨터 프로그램이 해독하기 어려운 기호, 글자 등을 재입력하도록 하여 스팸을 위한 자동화 도구 등을 무력화 시키는 보안 기술이다. 하지만 기존에 존재하였던 텍스트 기반의 시스템은 웹봇이나 머신 러닝등을 통하여 쉽게 통과할 수 있는 단점을 나타냈다. 우리는 이러한 단점을 보완하고자 새로운 이미지 기반의 CAPTCHA 시스템을 제안하였다. 제안된 시스템은 일반적인 사진에서 부분 이미지를 출력, 무작위 회전을 가하여 사용자에게 올바른 교정을 요하는 시스템이었다. 본 논문에서는 일반적인 사진에서 출력되는 부분 이미지의 형태를 다각형으로 추출하여, 사용자에게 좀 더 인식률을 높일 수 있는 서브 이미지의 형태를 찾고, 좀 더 효과적이고 실용적일수 있는 CAPTCHA 시스템을 제안하고자 한다. 본 논문에서 제공하는 다각형의 형태는 정사각형, 정오각형, 정육각형, 정칠각형 그리고 정팔각형이다. 총 5가지 형태의 다각형 중에서 사용자에게 가장 효과적인 다각형을 실험을 통하여 찾을 것이다.

Machine Learning Based Automated Source, Sink Categorization for Hybrid Approach of Privacy Leak Detection (머신러닝 기반의 자동화된 소스 싱크 분류 및 하이브리드 분석을 통한 개인정보 유출 탐지 방법)

  • Shim, Hyunseok;Jung, Souhwan
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.4
    • /
    • pp.657-667
    • /
    • 2020
  • The Android framework allows apps to take full advantage of personal information through granting single permission, and does not determine whether the data being leaked is actual personal information. To solve these problems, we propose a tool with static/dynamic analysis. The tool analyzes the Source and Sink used by the target app, to provide users with information on what personal information it used. To achieve this, we extracted the Source and Sink through Control Flow Graph and make sure that it leaks the user's privacy when there is a Source-to-Sink flow. We also used the sensitive permission information provided by Google to obtain information from the sensitive API corresponding to Source and Sink. Finally, our dynamic analysis tool runs the app and hooks information from each sensitive API. In the hooked data, we got information about whether user's personal information is leaked through this app, and delivered to user. In this process, an automated Source/Sink classification model was applied to collect latest Source/Sink information, and the we categorized latest release version of Android(9.0) with 88.5% accuracy. We evaluated our tool on 2,802 APKs, and found 850 APKs that leak personal information.

Analyzing K-POP idol popularity factors using music charts and new media data using machine learning (머신러닝을 활용한 음원 차트와 뉴미디어 데이터를 활용한 K-POP 아이돌 인기 요인 분석)

  • Jiwon Choi;Dayeon Jung;Kangkyu Choi;Taein Lim;Daehoon Kim;Jongkyn Jung;Seunmin Rho
    • Journal of Platform Technology
    • /
    • v.12 no.1
    • /
    • pp.55-66
    • /
    • 2024
  • The K-POP market has become influential not only in culture but also in society as a whole, including diplomacy and environmental movements. As a result, various papers have been conducted based on machine learning to identify the success factors of idols by utilizing traditional data such as music and recordings. However, there is a limitation that previous studies have not reflected the influence of new media platforms such as Instagram releases, YouTube shorts, TikTok, Twitter, etc. on the popularity of idols. Therefore, it is difficult to clarify the causal relationship of recent idol success factors because the existing studies do not consider the daily changing media trends. To solve these problems, this paper proposes a data collection system and analysis methodology for idol-related data. By developing a container-based real-time data collection automation system that reflects the specificity of idol data, we secure the stability and scalability of idol data collection and compare and analyze the clusters of successful idols through a K-Means clustering-based outlier detection model. As a result, we were able to identify commonalities among successful idols such as gender, time of success after album release, and association with new media. Through this, it is expected that we can finally plan optimal comeback promotions for each idol, album type, and comeback period to improve the chances of idol success.

  • PDF