• 한국공간구조학회논문집
• /
• 제22권2호
• /
• pp.39-46
• /
• 2022

Recently, machine learning is widely used to solve optimization problems in various engineering fields. In this study, machine learning is applied to development of a control algorithm for a smart control device for reduction of seismic responses. For this purpose, Deep Q-network (DQN) out of reinforcement learning algorithms was employed to develop control algorithm. A single degree of freedom (SDOF) structure with a smart tuned mass damper (TMD) was used as an example structure. A smart TMD system was composed of MR (magnetorheological) damper instead of passive damper. Reward design of reinforcement learning mainly affects the control performance of the smart TMD. Various hyper-parameters were investigated to optimize the control performance of DQN-based control algorithm. Usually, decrease of the time step for numerical simulation is desirable to increase the accuracy of simulation results. However, the numerical simulation results presented that decrease of the time step for reward calculation might decrease the control performance of DQN-based control algorithm. Therefore, a proper time step for reward calculation should be selected in a DQN training process.

• 한국게임학회 논문지
• /
• 제20권3호
• /
• pp.15-24
• /
• 2020

2D 레고 퍼즐은 레고 브릭을 이용해서 다양한 영상을 완성하는 퍼즐로 많은 사람들의 사랑을 받고 있다. 본 연구에서는 입력된 픽셀 아트 영상으로부터 강화학습에 기반한 2D 레고 퍼즐을 구성하고 완성하는 방법을 제안한다. 먼저, 학습 단계에서는 바람직한 레고 퍼즐에 대한 휴리스틱을 안정성과 효율성으로 설정하고 이를 최대한 만족시키는 방향으로 DQN을 학습한다. 그리고, 레고화 단계에서는 이 DQN을 이용해서 실제 레고 브릭을 추가해가면서 퍼즐의 구성도를 완성하는 과정을 수행한다. 이 과정을 통해서 지금까지 기술로는 수행하기 힘들었던 매우 높은 해상도의 레고 퍼즐을 효율적으로 완성하는 기술을 제공한다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권1호
• /
• pp.334-349
• /
• 2022

The effectiveness of Wi-Fi networks is greatly influenced by the optimization of contention window (CW) parameters. Unfortunately, the conventional approach employed by IEEE 802.11 wireless networks is not scalable enough to sustain consistent performance for the increasing number of stations. Yet, it is still the default when accessing channels for single-users of 802.11 transmissions. Recently, there has been a spike in attempts to enhance network performance using a machine learning (ML) technique known as reinforcement learning (RL). Its advantage is interacting with the surrounding environment and making decisions based on its own experience. Deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments (such as continuous state spaces or actions spaces) and to get optimum rewards. As a result, we present a new approach of CW control mechanism, which is termed as contention window threshold (CW_Threshold). It uses the DRL principle to define the threshold value and learn optimal settings under various network scenarios. We demonstrate our proposed method, known as a smart exponential-threshold-linear backoff algorithm with a deep Q-learning network (SETL-DQN). The simulation results show that our proposed SETL-DQN algorithm can effectively improve the throughput and reduce the collision rates.

• 한국융합학회논문지
• /
• 제5권4호
• /
• pp.21-25
• /
• 2014

현재 대부분의 기업들이 정보기술을 기반으로 정보 시스템을 이용한 데이터 분석 기법을 활용하고 있다. 이러한 데이터 분석은 기업의 다양한 의사결정에 영향을 미치는 데이터의 품질 평가에 주목하고 있다. 이는 데이터 품질 평가가 기업의 효과적인 운영뿐만 아니라 여러 부분에서 중요한 역할을 하기 때문이다. 본 연구에서는 현재 다양하게 연구되고 있는 데이터 품질 평가 모델에 관하여 기술하고, 이를 기반으로 데이터 품질 분석에 활용되고 있는 융합적이며, 적응적 모델인 DQnA 모델에 관하여 서술하고, 이를 활용한 품질 분석 방법에 관하여 논의하고자 한다.

• 전기전자학회논문지
• /
• 제25권4호
• /
• pp.644-649
• /
• 2021

Self-Imitation Learning은 간단한 비활성 정책 actor-critic 알고리즘으로써 에이전트가 과거의 좋은 경험을 활용하여 최적의 정책을 찾을 수 있도록 해준다. 그리고 actor-critic 구조를 갖는 강화학습 알고리즘에 결합되어 다양한 환경들에서 알고리즘의 상당한 개선을 보여주었다. 하지만 Self-Imitation Learning이 강화학습에 큰 도움을 준다고 하더라도 그 적용 분야는 actor-critic architecture를 가지는 강화학습 알고리즘으로 제한되어 있다. 본 논문에서 Self-Imitation Learning의 알고리즘을 가치 기반 강화학습 알고리즘인 DQN에 적용하는 방법을 제안하고, Self-Imitation Learning이 적용된 DQN 알고리즘의 학습을 다양한 환경에서 진행한다. 아울러 그 결과를 기존의 결과와 비교함으로써 Self-Imitation Leaning이 DQN에도 적용될 수 있으며 DQN의 성능을 개선할 수 있음을 보인다.

• 한국공간구조학회논문집
• /
• 제23권2호
• /
• pp.29-36
• /
• 2023

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2023년도 추계학술발표대회
• /
• pp.906-907
• /
• 2023

본 연구는 변동성이 높은 주식시장에서 안정적인 수익창출에 기여할 수 있는 주가예측 강화학 모델을 제안한다. DQN 알고리즘과 LSTM 신경망을 이용하여 시장의 흐름에 따라 전략을 달리하는 모델을 개발하고, 이를 활용한 주식 트레이딩 시스템의 유용성을 확인하고 발전 방향을 제시한다.

• 한국멀티미디어학회논문지
• /
• 제21권2호
• /
• pp.188-198
• /
• 2018

With recent development of high-speed wide-area wireless networks and wide spread of highperformance wireless devices, the demand on seamless video streaming services in Long Term Evolution (LTE) network environments is ever increasing. To meet the demand and provide enhanced Quality of Experience (QoE) with mobile users, the Dynamic Adaptive Streaming over HTTP (DASH) has been actively studied to achieve QoE enhanced video streaming service in dynamic network environments. However, the existing DASH algorithm to select the quality of requesting video segments is based on a procedural algorithm so that it reveals a limitation to adapt its performance to dynamic network situations. To overcome this limitation this paper proposes a novel quality selection mechanism based on a Deep Q-Network (DQN) model, the DQN-based DASH ABR($DQN_{ABR}$) mechanism. The $DQN_{ABR}$ mechanism replaces the existing DASH ABR algorithm with an intelligent deep learning model which optimizes service quality to mobile users through reinforcement learning. Compared to the existing approaches, the experimental analysis shows that the proposed solution outperforms in terms of adapting to dynamic wireless network situations and improving QoE experience of end users.

• 한국멀티미디어학회논문지
• /
• 제25권2호
• /
• pp.145-156
• /
• 2022

Nowadays, the complexity of object tracking models among hardware applications has become a more in-demand duty to complete in various indeterminable environment tracking situations with multifunctional algorithm skills. In this paper, we propose a virtual city environment using AirSim (Aerial Informatics and Robotics Simulation - AirSim, CityEnvironment) and use the DQN (Deep Q-Learning) model of deep reinforcement learning model in the virtual environment. The proposed object tracking DQN network observes the environment using a deep reinforcement learning model that receives continuous images taken by a virtual environment simulation system as input to control the operation of a virtual drone. The deep reinforcement learning model is pre-trained using various existing continuous image sets. Since the existing various continuous image sets are image data of real environments and objects, it is implemented in 3D to track virtual environments and moving objects in them.

• 한국인터넷방송통신학회논문지
• /
• 제23권6호
• /
• pp.33-39
• /
• 2023

최근 비행 애드-훅 네트워크(Flying Ad-hoc Network) 환경에서 강화학습을 이용한 통신 성능 개선과 이동성 모델 설계에 관한 연구가 진행되고 있다. 무인항공기(UAV)에서의 이동성 모델은 움직임을 예측하고 제어하기 위한 핵심요소로 주목받고 있다. 본 논문에서는 무인항공기가 운용되는 3차원 가상 환경을 구현하고, 무인항공기의 경로 최적화를 위해 푸리에 기저 함수 근사를 적용한 Q-learning과 DQN 두 가지 강화학습 알고리즘을 적용하여 모델을 설계 및 성능을 분석하였다. 실험 결과를 통해 3차원 가상 환경에서 DQN 모델이 Q-learning 모델 대비 최적의 경로 탐색에 적합한 것을 확인하였다.