• Title/Summary/Keyword: 강화 학습

Search Result 1,608, Processing Time 0.032 seconds

A study on Deep Q-Networks based Auto-scaling in NFV Environment (NFV 환경에서의 Deep Q-Networks 기반 오토 스케일링 기술 연구)

  • Lee, Do-Young;Yoo, Jae-Hyoung;Hong, James Won-Ki
    • KNOM Review
    • /
    • v.23 no.2
    • /
    • pp.1-10
    • /
    • 2020
  • Network Function Virtualization (NFV) is a key technology of 5G networks that has the advantage of enabling building and operating networks flexibly. However, NFV can complicate network management because it creates numerous virtual resources that should be managed. In NFV environments, service function chaining (SFC) composed of virtual network functions (VNFs) is widely used to apply a series of network functions to traffic. Therefore, it is required to dynamically allocate the right amount of computing resources or instances to SFC for meeting service requirements. In this paper, we propose Deep Q-Networks (DQN)-based auto-scaling to operate the appropriate number of VNF instances in SFC. The proposed approach not only resizes the number of VNF instances in SFC composed of multi-tier architecture but also selects a tier to be scaled in response to dynamic traffic forwarding through SFC.

Worker Collision Safety Management System using Object Detection (객체 탐지를 활용한 근로자 충돌 안전관리 시스템)

  • Lee, Taejun;Kim, Seongjae;Hwang, Chul-Hyun;Jung, Hoekyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.9
    • /
    • pp.1259-1265
    • /
    • 2022
  • Recently, AI, big data, and IoT technologies are being used in various solutions such as fire detection and gas or dangerous substance detection for safety accident prevention. According to the status of occupational accidents published by the Ministry of Employment and Labor in 2021, the accident rate, the number of injured, and the number of deaths have increased compared to 2020. In this paper, referring to the dataset construction guidelines provided by the National Intelligence Service Agency(NIA), the dataset is directly collected from the field and learned with YOLOv4 to propose a collision risk object detection system through object detection. The accuracy of the dangerous situation rule violation was 88% indoors and 92% outdoors. Through this system, it is thought that it will be possible to analyze safety accidents that occur in industrial sites in advance and use them to intelligent platforms research.

Diagnosis of Sarcopenia in the Elderly and Development of Deep Learning Algorithm Exploiting Smart Devices (스마트 디바이스를 활용한 노약자 근감소증 진단과 딥러닝 알고리즘)

  • Yun, Younguk;Sohn, Jung-woo
    • Journal of the Society of Disaster Information
    • /
    • v.18 no.3
    • /
    • pp.433-443
    • /
    • 2022
  • Purpose: In this paper, we propose a study of deep learning algorithms that estimate and predict sarcopenia by exploiting the high penetration rate of smart devices. Method: To utilize deep learning techniques, experimental data were collected by using the inertial sensor embedded in the smart device. We implemented a smart device application for data collection. The data are collected by labeling normal and abnormal gait and five states of running, falling and squat posture. Result: The accuracy was analyzed by comparative analysis of LSTM, CNN, and RNN models, and binary classification accuracy of 99.87% and multiple classification accuracy of 92.30% were obtained using the CNN-LSTM fusion algorithm. Conclusion: A study was conducted using a smart sensoring device, focusing on the fact that gait abnormalities occur for people with sarcopenia. It is expected that this study can contribute to strengthening the safety issues caused by sarcopenia.

Resolving data imbalance through differentiated anomaly data processing based on verification data (검증데이터 기반의 차별화된 이상데이터 처리를 통한 데이터 불균형 해소 방법)

  • Hwang, Chulhyun
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.4
    • /
    • pp.179-190
    • /
    • 2022
  • Data imbalance refers to a phenomenon in which the number of data in one category is too large or too small compared to another category. Due to this, it has been raised as a major factor that deteriorates performance in machine learning that utilizes classification algorithms. In order to solve the data imbalance problem, various ovrsampling methods for amplifying prime number distribution data have been proposed. Among them, SMOTE is the most representative method. In order to maximize the amplification effect of minority distribution data, various methods have emerged that remove noise included in data (SMOTE-IPF) or enhance only border lines (Borderline SMOTE). This paper proposes a method to ultimately improve classification performance by improving the processing method for anomaly data in the traditional SMOTE method that amplifies minority classification data. The proposed method consistently presented relatively high classification performance compared to the existing methods through experiments.

Python Basic Programming Curriculum for Non-majors and Development Analysis of Evaluation Problems (비전공자를 위한 파이썬 기초 프로그래밍 커리큘럼과 평가문제 개발분석)

  • Hur, Kyeong
    • Journal of Practical Engineering Education
    • /
    • v.14 no.1
    • /
    • pp.75-83
    • /
    • 2022
  • Most of the courses that teach the Python programming language are liberal arts courses that all students in general universities must complete. Through this, non-major students who have learned the basic programming process based on computational thinking are strengthening their convergence capabilities to apply SW in various major fields. In the previous research results, various evaluation methods for understanding the concept of computational thinking and writing code were suggested. However, there are no examples of evaluation problems, so it is difficult to apply them in actual course operation. Accordingly, in this paper, a Python basic programming curriculum that can be applied as a liberal arts subject for non-majors is proposed according to the ADDIE model. In addition, the case of evaluation problems for each Python element according to the proposed detailed curriculum was divided into 1st and 2nd phases and suggested. Finally, the validity of the proposed evaluation problem was analyzed based on the evaluation scores of non-major students calculated in the course to which this evaluation problem case was applied. It was confirmed that the proposed evaluation problem case was applied as a real-time online non-face-to-face evaluation method to effectively evaluate the programming competency of non-major students.

An Analysis of Educational Capacity Prediction according to Pre-survey of Satisfaction using Random Forest (랜덤 포레스트를 활용한 만족도 사전조사에 따른 교육 역량 예측 분석)

  • Nam, Kihun
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.6
    • /
    • pp.487-492
    • /
    • 2022
  • Universities are looking for various methods to enhance educational competence level suitable for the rapidly changing social environment. This study suggests a method to promote academic and educational achievements by reducing drop-out rate from their majors through implementation of pre-survey of satisfaction that revised and complemented survey items. To supplement the CQI method implemented after a general satisfaction survey, a pre-survey of satisfaction was carried out. To consolidate students' competences, this study made prediction and analysis of data with more importance possible using the Random Forest of the machine learning technique that can be applied to AI Medici platform, whose design is underway. By pre-processing the pre-survey of satisfaction, the students information enrolled in classes were defined as an explanatory variable, and they were classified, and a model was created and learning was conducted. For the experimental environment, the algorithms and sklearn library related in Jupyter notebook 3.7.7, Python 3.7 were used together. This study carried out a comparative analysis of change in educational satisfaction survey, carried out after classes, and trends in the drop-out students by reflecting the results of the suggested method in the classes.

Impact Assessment of an Autonomous Demand Responsive Bus in a Microscopic Traffic Simulation (미시적 교통 시뮬레이션을 활용한 실시간 수요대응형 자율주행 버스 영향 평가)

  • Sang ung Park;Joo young Kim
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.21 no.6
    • /
    • pp.70-86
    • /
    • 2022
  • An autonomous demand-responsive bus with mobility-on-demand service is an innovative transport compensating for the disadvantages of an autonomous bus and a demand-responsive bus with mobility-on-demand service. However, less attention has been paid to the quantitative impact assessment of the autonomous demand-responsive bus due to the technological complexity of the autonomous demand-responsive bus. This study simulates autonomous demand-responsive bus trips by reinforcement learning on a microscopic traffic simulation to quantify the impact of the autonomous demand-responsive bus. The Chungju campus of the Korea National University of Transportation is selected as a testbed. Simulation results show that the introduction of the autonomous demand-responsive bus can reduce the wait time of passengers, average control delay, and increase the traffic speed compared to the results with fixed route bus service. This study contributes to the quantitative evaluation of the autonomous demand-responsive bus.

Reinforcement Learning-based Dynamic Weapon Assignment to Multi-Caliber Long-Range Artillery Attacks (다종 장사정포 공격에 대한 강화학습 기반의 동적 무기할당)

  • Hyeonho Kim;Jung Hun Kim;Joohoe Kong;Ji Hoon Kyung
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.45 no.4
    • /
    • pp.42-52
    • /
    • 2022
  • North Korea continues to upgrade and display its long-range rocket launchers to emphasize its military strength. Recently Republic of Korea kicked off the development of anti-artillery interception system similar to Israel's "Iron Dome", designed to protect against North Korea's arsenal of long-range rockets. The system may not work smoothly without the function assigning interceptors to incoming various-caliber artillery rockets. We view the assignment task as a dynamic weapon target assignment (DWTA) problem. DWTA is a multistage decision process in which decision in a stage affects decision processes and its results in the subsequent stages. We represent the DWTA problem as a Markov decision process (MDP). Distance from Seoul to North Korea's multiple rocket launchers positioned near the border, limits the processing time of the model solver within only a few second. It is impossible to compute the exact optimal solution within the allowed time interval due to the curse of dimensionality inherently in MDP model of practical DWTA problem. We apply two reinforcement-based algorithms to get the approximate solution of the MDP model within the time limit. To check the quality of the approximate solution, we adopt Shoot-Shoot-Look(SSL) policy as a baseline. Simulation results showed that both algorithms provide better solution than the solution from the baseline strategy.

Development of Optimal Design Technique of RC Beam using Multi-Agent Reinforcement Learning (다중 에이전트 강화학습을 이용한 RC보 최적설계 기술개발)

  • Kang, Joo-Won;Kim, Hyun-Su
    • Journal of Korean Association for Spatial Structures
    • /
    • v.23 no.2
    • /
    • pp.29-36
    • /
    • 2023
  • Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

Study on the Improvement of Special Staff Training for Multiple Majors of Secondary Informatics Teachers (중등 정보교사 복수전공 특별양성과정의 개선방안)

  • Shin, Soo-Bum
    • Journal of Creative Information Culture
    • /
    • v.5 no.3
    • /
    • pp.285-293
    • /
    • 2019
  • As the 2015 Revised Secondary Information Curriculum was reorganized into a software-focused curriculum, a number of information teachers were needed. For this purpose, a special training course for multiple majors for secondary information teachers was established, and produced about 100 teachers with multiple majors. To improve this training program, this study surveyed the trainees joined in this program. As the result of the survey, there were findings that the short training period made it difficult to understand the overall content of the curriculum, and that they had a loss of learning due to the burden on another major. Therefore, long-term training is necessary to train informatics teachers, and by realizing the reinforcement of specialty through pre-training and main training, and by organizing a gradual curriculum, substantiality should be sought.