• Title/Summary/Keyword: Q learning

Search Result 422, Processing Time 0.034 seconds

Predictions of dam inflow on Han-river basin using LSTM (LSTM을 이용한 한강유역 댐유입량 예측)

  • Kim, Jongho;Tran, Trung Duc
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2020.06a
    • /
    • pp.319-319
    • /
    • 2020
  • 최근 데이터 과학의 획기적인 발전 덕분에 딥러닝 (Deep Learning) 알고리즘이 개발되어 다양한 분야에 널리 적용되고 있다. 본 연구에서는 인공신경망 중 하나인 LSTM(Long-Short Term Memory) 네트워크를 사용하여 댐 유입량을 예측하였다. 구체적인 내용으로, (1) LSTM에 필요한 입력 데이터를 효율적으로 사전 처리하는 방법, (2) LSTM의 하이퍼 매개변수를 결정하는 방법 및 (3) 다양한 손실 함수(Loss function)를 선택하고 그 영향을 평가하는 방법 등을 다루었다. 제안된 LSTM 모델은 강우량(R), 댐유입량(Q) 기온(T), 기저유량(BF) 등을 포함한 다양한 입력 변수들의 함수로 가정하였으며, CCF(Cross Correlations), ACF(Autocorrelations) 및 PACF(Partial Autocorrelations) 등의 기법을 사용하여 입력 변수를 결정하였다. 다양한 sequence length를 갖는 (즉 t, t-1, … t-n의 시간 지연을 갖는) 입력 변수를 적용하여 데이터 학습에 최적의 시퀀스 길이를 결정하였다. LSTM 네트워크 모델을 적용하여 2014년부터 2020년까지 한강 유역 9개의 댐 유입량을 추정하였다. 본 연구로부터 댐 유입량을 예측하는 것은 홍수 및 가뭄 통제를 위한 필수 요건들 중 하나이며 수자원 계획 및 관리에 도움이 될 것이다.

  • PDF

Performance Comparison of Deep Reinforcement Learning based Computation Offloading in MEC (MEC 환경에서 심층 강화학습을 이용한 오프로딩 기법의 성능비교)

  • Moon, Sungwon;Lim, Yujin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2022.05a
    • /
    • pp.52-55
    • /
    • 2022
  • 5G 시대에 스마트 모바일 기기가 기하급수적으로 증가하면서 멀티 액세스 엣지 컴퓨팅(MEC)이 유망한 기술로 부상했다. 낮은 지연시간 안에 계산 집약적인 서비스를 제공하기 위해 MEC 서버로 오프로딩하는 특히, 태스크 도착률과 무선 채널의 상태가 확률적인 MEC 시스템 환경에서의 오프로딩 연구가 주목받고 있다. 본 논문에서는 차량의 전력과 지연시간을 최소화하기 위해 로컬 실행을 위한 연산 자원과 오프로딩을 위한 전송 전력을 할당하는 심층 강화학습 기반의 오프로딩 기법을 제안하였다. Deep Deterministic Policy Gradient (DDPG) 기반 기법과 Deep Q-network (DQN) 기반 기법을 차량의 전력 소비량과 큐잉 지연시간 측면에서 성능을 비교 분석하였다.

Development of An Autonomous Medicine Delivery Robot Using Facial Recognition for Unlocking Mechanisms (얼굴인식 알고리즘을 활용한 잠금해제 및 자율주행 약제배송로봇 개발)

  • Yu-Kyeong Kim;Ye-Rin Kim
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.874-875
    • /
    • 2023
  • 본 논문은 COVID-19와 같은 전염병 확산 방지를 위해 비대면 약제배송로봇을 제안한다. 제안한 로봇은 OpenCV와 Q-Learning기반의 모델을 사용하여 실시간 영상처리로 사람의 얼굴을 식별한다. 환자의 얼굴, 나이, 전달 약제 등을 환자 데이터베이스에 등록한다. 카메라로 인식된 환자의 얼굴과 데이터베이스 내 환자의 얼굴이 일치할 경우 잠금장치를 해제시켜 환자의 약제 수령을 허용한다. 또한 어플리케이션을 통해 약제가 올바르게 전달되었는지 2차적으로 확인한다. 따라서 본 논문에서 제안한 로봇은 비대면으로 환자에게 약을 전달함으로써 입원병동에서 발생할 수 있는 전염병 확상의 방지에 효과적으로 기여할 수 있을 것이다.

Novel Reward Function for Autonomous Drone Navigating in Indoor Environment

  • Khuong G. T. Diep;Viet-Tuan Le;Tae-Seok Kim;Anh H. Vo;Yong-Guk Kim
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.624-627
    • /
    • 2023
  • Unmanned aerial vehicles are gaining in popularity with the development of science and technology, and are being used for a wide range of purposes, including surveillance, rescue, delivery of goods, and data collection. In particular, the ability to avoid obstacles during navigation without human oversight is one of the essential capabilities that a drone must possess. Many works currently have solved this problem by implementing deep reinforcement learning (DRL) model. The essential core of a DRL model is reward function. Therefore, this paper proposes a new reward function with appropriate action space and employs dueling double deep Q-Networks to train a drone to navigate in indoor environment without collision.

Classification of Query E-Mail Using Neural Network (신경망을 이용한 사용자 질의 전자 메일 분류)

  • 변영철;홍영보
    • Journal of Korea Multimedia Society
    • /
    • v.7 no.3
    • /
    • pp.438-449
    • /
    • 2004
  • More and more users are using the query e-mail according to the increment of use of internet. The operator of internet site desires the users to check the FAQ and Q&A contents first before sending the query e-mail to the operator However the users try to get the solution for a problem easily by simply sending a query e-mail. Therefore the increment of query e-mail is inevitable, and the site operator is suffering from too heavy loads and spending too much time and cost to reply the query e-mail. In this paper, we are proposing an efficient method of classifying the query e-mail of users automatically by using a neural network. To verify the reasonability of our work, the query e-mails of KORNET are used as the test data, which is actually gathered in KT. A total of 210 learning data and 280 test data were used to test the performance of the proposed approach. From the experiments we got the encouraging result from the view point of application in real life. The proposed approach satisfied the request of users who wanted rapid response for their query e-mail.

  • PDF

Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithm (이진 삼차 재귀 신경망과 유전자 알고리즘을 이용한 문맥-자유 문법의 추론)

  • Jung, Soon-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.3
    • /
    • pp.11-25
    • /
    • 2012
  • We present the method to infer Context-Free Grammars by applying genetic algorithm to the Binary Third-order Recurrent Neural Networks(BTRNN). BTRNN is a multiple-layered architecture of recurrent neural networks, each of which is corresponding to an input symbol, and is combined with external stack. All parameters of BTRNN are represented as binary numbers and each state transition is performed with any stack operation simultaneously. We apply Genetic Algorithm to BTRNN chromosomes and obtain the optimal BTRNN inferring context-free grammar of positive and negative input patterns. This proposed method infers BTRNN, which includes the number of its states equal to or less than those of existing methods of Discrete Recurrent Neural Networks, with less examples and less learning trials. Also BTRNN is superior to the recent method of chromosomes representing grammars at recognition time complexity because of performing deterministic state transitions and stack operations at parsing process. If the number of non-terminals is p, the number of terminals q, the length of an input string k, and the max number of BTRNN states m, the parallel processing time is O(k) and the sequential processing time is O(km).

Efficiency Optimization Control of SynRM Drive using Multi-AFLC (다중 AFLC를 이용한 SynRM 드라이브의 효율 최적화 제어)

  • Choi, Jung-Sik;Ko, Jae-Sub;Jang, Mi-Geum;Chung, Dong-Hwa
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.24 no.5
    • /
    • pp.44-54
    • /
    • 2010
  • Optimal efficiency control of synchronous reluctance motor(SynRM) is very important in the sense of energy saving and conservation of natural environment because the efficiency of the SynRM is generally lower than that of other types of AC motors. This paper is proposed a novel efficiency optimization control of SynRM considering iron loss using multi adaptive fuzzy learning controller(AFLC). The optimal current ratio between torque current and exciting current is analytically derived to drive SynRM at maximum efficiency. This paper is proposed an efficiency optimization control for the SynRM which minimizes the copper and iron losses. There exists a variety of combinations of d and q-axis current which provide a specific motor torque. The objective of the efficiency optimization control is to seek a combination of d and q-axis current components, which provides minimum losses at a certain operating point in steady state. The control performance of the proposed controller is evaluated by analysis for various operating conditions. Analysis results are presented to show the validity of the proposed algorithm.

CAB: Classifying Arrhythmias based on Imbalanced Sensor Data

  • Wang, Yilin;Sun, Le;Subramani, Sudha
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.7
    • /
    • pp.2304-2320
    • /
    • 2021
  • Intelligently detecting anomalies in health sensor data streams (e.g., Electrocardiogram, ECG) can improve the development of E-health industry. The physiological signals of patients are collected through sensors. Timely diagnosis and treatment save medical resources, promote physical health, and reduce complications. However, it is difficult to automatically classify the ECG data, as the features of ECGs are difficult to extract. And the volume of labeled ECG data is limited, which affects the classification performance. In this paper, we propose a Generative Adversarial Network (GAN)-based deep learning framework (called CAB) for heart arrhythmia classification. CAB focuses on improving the detection accuracy based on a small number of labeled samples. It is trained based on the class-imbalance ECG data. Augmenting ECG data by a GAN model eliminates the impact of data scarcity. After data augmentation, CAB classifies the ECG data by using a Bidirectional Long Short Term Memory Recurrent Neural Network (Bi-LSTM). Experiment results show a better performance of CAB compared with state-of-the-art methods. The overall classification accuracy of CAB is 99.71%. The F1-scores of classifying Normal beats (N), Supraventricular ectopic beats (S), Ventricular ectopic beats (V), Fusion beats (F) and Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively. Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively.

Nuclear Magnetic Resonance (NMR)-Based Quantification on Flavor-Active and Bioactive Compounds and Application for Distinguishment of Chicken Breeds

  • Kim, Hyun Cheol;Yim, Dong-Gyun;Kim, Ji Won;Lee, Dongheon;Jo, Cheorun
    • Food Science of Animal Resources
    • /
    • v.41 no.2
    • /
    • pp.312-323
    • /
    • 2021
  • The purpose of this study was to use 1H nuclear magnetic resonance (1H NMR) to quantify taste-active and bioactive compounds in chicken breasts and thighs from Korean native chicken (KNC) [newly developed KNCs (KNC-A, -C, and -D) and commercial KNC-H] and white-semi broiler (WSB) used in Samgye. Further, each breed was differentiated using multivariate analyses, including a machine learning algorithm designed to use metabolic information from each type of chicken obtained using 1H-13C heteronuclear single quantum coherence (2D NMR). Breast meat from KNC-D chickens were superior to those of conventional KNC-H and WSB chickens in terms of both taste-active and bioactive compounds. In the multivariate analysis, meat portions (breast and thigh) and chicken breeds (KNCs and WSB) could be clearly distinguished based on the outcomes of the principal component analysis and partial least square-discriminant analysis (R2=0.945; Q2=0.901). Based on this, we determined the receiver operating characteristic (ROC) curve for each of these components. AUC analysis identified 10 features which could be consistently applied to distinguish between all KNCs and WSB chickens in both breast (0.988) and thigh (1.000) meat without error. Here, both 1H NMR and 2D NMR could successfully quantify various target metabolites which could be used to distinguish between different chicken breeds based on their metabolic profile.

An Artificial Neural Network-Based Drug Proarrhythmia Assessment Using Electrophysiological Characteristics of Cardiomyocytes (심근 세포의 전기생리학적 특징을 이용한 인공 신경망 기반 약물의 심장독성 평가)

  • Yoo, Yedam;Jeong, Da Un;Marcellinus, Aroli;Lim, Ki Moo
    • Journal of Biomedical Engineering Research
    • /
    • v.42 no.6
    • /
    • pp.287-294
    • /
    • 2021
  • Cardiotoxicity assessment of all drugs has been performed according to the ICH guidelines since 2005. Non-clinical evaluation S7B has focused on the hERG assay, which has a low specificity problem. The comprehensive in vitro proarrhythmia assay (CiPA) project was initiated to correct this problem, which presented a model for classifying the Torsade de pointes (TdP)-induced risk of drugs as biomarkers calculated through an in silico ventricular model. In this study, we propose a TdP-induced risk group classifier of artificial neural network (ANN)-based. The model was trained with 12 drugs and tested with 16 drugs. The ANN model was performed according to nine features, seven features, five features as an individual ANN model input, and the model with the highest performance was selected and compared with the classification performance of the qNet input logistic regression model. When the five features model was used, the results were AUC 0.93 in the high-risk group, AUC 0.73 in the intermediate-risk group, and 0.92 in the low-risk group. The model's performance using qNet was lower than the ANN model in the high-risk group by 17.6% and in the low-risk group by 29.5%. This study was able to express performance in the three risk groups, and it is a model that solved the problem of low specificity, which is the problem of hERG assay.