• The Journal of Korea Robotics Society
• /
• v.17 no.3
• /
• pp.264-272
• /
• 2022

Human-robot cooperative tasks are increasingly required in our daily life with the development of robotics and artificial intelligence technology. Interactive reinforcement learning strategies suggest that robots learn task by receiving feedback from an experienced human trainer during a training process. However, most of the previous studies on Interactive reinforcement learning have required an extra feedback input device such as a mouse or keyboard in addition to robot itself, and the scenario where a robot can interactively learn a task with human have been also limited to virtual environment. To solve these limitations, this paper studies training strategies of robot that learn table balancing tasks interactively using deep reinforcement learning with human's facial expression feedback. In the proposed system, the robot learns a cooperative table balancing task using Deep Q-Network (DQN), which is a deep reinforcement learning technique, with human facial emotion expression feedback. As a result of the experiment, the proposed system achieved a high optimal policy convergence rate of up to 83.3% in training and successful assumption rate of up to 91.6% in testing, showing improved performance compared to the model without human facial expression feedback.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• fall
• /
• pp.225-228
• /
• 2021

최근 모바일 비디오 스트리밍 서비스의 이용자 수가 증가하고 있다. 이에 따라 모바일 환경에 적합한 DASH 비디오 스트리밍 메커니즘이 연구되었고, 이것을 DQN 기법에 의해 개선한 알고리즘은 모바일 네트워크 환경에서 적절한 비디오 품질 선택을 통해 버퍼링을 크게 줄일 수 있었다. 그러나 이는 모바일 요금제로 비디오 스트리밍 서비스를 이용하는 사용자들에게 안정적인 서비스를 제공하기 어렵다. 이에 본 논문은 기존의 DQN 기법에 의한 알고리즘을 발전시켜 사용자의 모바일 요금제에 적합한 비디오 품질을 선택하는 알고리즘을 연구하고 성능 실험 결과를 분석한다. 또한 이 알고리즘을 전체 모바일 비디오 스트리밍 시스템과 통합하여 이용하도록 제안한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.144-146
• /
• 2022

차량 환경에서 발생하는 계산 집약적인 태스크가 증가하면서 모바일 엣지 컴퓨팅(MEC, Mobile Edge Computing)의 필요성이 높아지고 있다. 하지만 지상에 존재하는 MEC 서버는 출퇴근 시간과 같이 태스크가 일시적으로 급증하는 상황에 유동적으로 대처할 수 없으며, 이러한 상황을 대비하기 위해 지상 MEC 서버를 추가로 설치하는 것은 자원의 낭비를 불러온다. 최근 이 문제를 해결하기 위해 UAV(Unmanned Aerial Vehicle)기반 MEC 서버를 추가로 사용해 엣지 서비스를 제공하는 연구가 진행되고 있다. 그러나 UAV MEC 서버는 지상 MEC 서버와 달리 한정적인 배터리 용량으로 인해 서버 간 로드밸런싱을 통해 에너지 사용량을 최소화 하는 것이 필요하다. 본 논문에서는 UAV MEC 서버의 에너지 사용량을 고려한 마이그레이션 기법을 제안한다. 또한 GRU(Gated Recurrent Unit) 모델을 활용한 트래픽 예측을 바탕으로 한 마이그레이션을 통해 지연시간을 최소화할 수 있도록 한다. 제안 시스템의 성능을 평가하기 위해 MEC의 마이그레이션 시점을 결정하는 기준점와 차량의 밀도에 따라 실험을 진행하고, 서버의 로드 편차, UAV MEC 서버의 에너지 사용량 그리고 평균 지연 시간 측면에서 성능을 분석한다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.3
• /
• pp.551-569
• /
• 2024

With the rapid development of electric vehicles (EVs) industry, EV charging service becomes more and more important. Especially, in the case of suddenly drop of air temperature or open holidays that large-scale EVs seeking for charging devices (CDs) in a short time. In such scenario, inefficient EV charging scheduling algorithm might lead to a bad service quality, for example, long queueing times for EVs and unreasonable idling time for charging devices. To deal with this issue, this paper propose a Deep-Q-Network (DQN) based two-stage scheduling method for the large-scale EVs charging service. Fine-grained states with two delicate neural networks are proposed to optimize the sequencing of EVs and charging station (CS) arrangement. Two efficient algorithms are presented to obtain the optimal EVs charging scheduling scheme for large-scale EVs charging demand. Three case studies show the superiority of our proposal, in terms of a high service quality (minimized average queuing time of EVs and maximized charging performance at both EV and CS sides) and achieve greater scheduling efficiency. The code and data are available at THE CODE AND DATA.

• The Journal of the Convergence on Culture Technology
• /
• v.8 no.5
• /
• pp.697-703
• /
• 2022

Due to the recent economic downturn caused by Covid-19 and the unstable international situation, many investors are choosing the derivatives market as a means of investment. However, the derivatives market has a greater risk than the stock market, and research on the market of market participants is insufficient. Recently, with the development of artificial intelligence, machine learning has been widely used in the derivatives market. In this paper, reinforcement learning, one of the machine learning techniques, is applied to analyze the scalping technique that trades futures in minutes. The data set consists of 21 attributes using the closing price, moving average line, and Bollinger band indicators of 1 minute and 3 minute data for 6 months by selecting 4 products among futures products traded at trading firm. In the experiment, DNN artificial neural network model and three reinforcement learning algorithms, namely, DQN (Deep Q-Network), A2C (Advantage Actor Critic), and A3C (Asynchronous A2C) were used, and they were trained and verified through learning data set and test data set. For scalping, the agent chooses one of the actions of buying and selling, and the ratio of the portfolio value according to the action result is rewarded. Experiment results show that the energy sector products such as Heating Oil and Crude Oil yield relatively high cumulative returns compared to the index sector products such as Mini Russell 2000 and Hang Seng Index.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.05a
• /
• pp.606-608
• /
• 2023

최근 들어 극심한 가뭄이 지속됨에 따라 댐을 통한 물 수급에 어려움을 겪고 있다. 본 논문에서는 이러한 가뭄 상황에서 댐 자체 방류량 조절을 통해 낭비되고 있는 물을 절약하기 위한 기법을 제안한다. DQN 알고리즘을 사용해 방류량을 최적화하여 목표 저수량 이상의 상태를 60일간 유지하도록 설계하였으며, 해당 알고리즘 내 방류량의 가중치를 변경한 결과를 비교하여 그 성능을 분석하였다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.11a
• /
• pp.549-552
• /
• 2023

현재 전력망에서는 불규칙하거나 낭비되는 전력 문제를 해결하기 위한 한 방법으로 ESS(Energy Storage System)를 활용하는 방법이 많은 관심을 받고 있다. 본 연구에서는 업종별로 시간대에 따라 요금을 다르게 부과하는 배전망 시스템에서, 배터리를 보다 경제적으로 사용하는 동시에 여유 용량을 유지하도록 하는 DQN 기반 강화학습 기법을 제안하였다. 또한, 업종별로 다른 전력 소비 패턴을 에이전트의 동작성과 함께 그 성능을 분석하고 비교하였다.

• Journal of Energy Engineering
• /
• v.27 no.2
• /
• pp.26-31
• /
• 2018

This paper introduces a study to improve the thermal efficiency of the district heating user control facility based on reinforcement learning. As an example, it is proposed a general method of constructing a deep Q learning network(DQN) using deep Q learning, which is a reinforcement learning algorithm that does not specify a model. In addition, it is also introduced the big data platform system and the integrated heat management system which are specialized in energy field applied in processing huge amount of data processing from IoT sensor installed in many thermal energy control facilities.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.16 no.6
• /
• pp.1153-1160
• /
• 2021

This paper proposes a routing algorithm that determines the optimal path using deep reinforcement learning in software-defined networks. The deep reinforcement learning model for learning is based on DQN, the inputs are the current network state, source, and destination nodes, and the output returns a list of routes from source to destination. The routing task is defined as a discrete control problem, and the quality of service parameters for routing consider delay, bandwidth, and loss rate. The routing agent classifies the appropriate service class according to the user's quality of service profile, and converts the service class that can be provided for each link from the current network state collected from the SDN. Based on this converted information, it learns to select a route that satisfies the required service level from the source to the destination. The simulation results indicated that if the proposed algorithm proceeds with a certain episode, the correct path is selected and the learning is successfully performed.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.6
• /
• pp.61-67
• /
• 2023

The rise of 5G and the proliferation of smart devices have underscored the significance of multi-access edge computing (MEC). Amidst this trend, interest in effectively processing computation-intensive and latency-sensitive applications has increased. This study investigated a novel task offloading strategy considering the probabilistic MEC environment to address these challenges. Initially, we considered the frequency of dynamic task requests and the unstable conditions of wireless channels to propose a method for minimizing vehicle power consumption and latency. Subsequently, our research delved into a deep reinforcement learning (DRL) based offloading technique, offering a way to achieve equilibrium between local computation and offloading transmission power. We analyzed the power consumption and queuing latency of vehicles using the deep deterministic policy gradient (DDPG) and deep Q-network (DQN) techniques. Finally, we derived and validated the optimal performance enhancement strategy in a vehicle based MEC environment.