• 전자통신동향분석
• /
• 제34권4호
• /
• pp.1-14
• /
• 2019

Recent trends in deep reinforcement learning (DRL) have revealed the considerable improvements to DRL algorithms in terms of performance, learning stability, and computational efficiency. DRL also enables the scenarios that it covers (e.g., partial observability; cooperation, competition, coexistence, and communications among multiple agents; multi-task; decentralized intelligence) to be vastly expanded. These features have cultivated multi-agent reinforcement learning research. DRL is also expanding its applications from robotics to natural language processing and computer vision into a wide array of fields such as finance, healthcare, chemistry, and even art. In this report, we briefly summarize various DRL techniques and research directions.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권1호
• /
• pp.334-349
• /
• 2022

The effectiveness of Wi-Fi networks is greatly influenced by the optimization of contention window (CW) parameters. Unfortunately, the conventional approach employed by IEEE 802.11 wireless networks is not scalable enough to sustain consistent performance for the increasing number of stations. Yet, it is still the default when accessing channels for single-users of 802.11 transmissions. Recently, there has been a spike in attempts to enhance network performance using a machine learning (ML) technique known as reinforcement learning (RL). Its advantage is interacting with the surrounding environment and making decisions based on its own experience. Deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments (such as continuous state spaces or actions spaces) and to get optimum rewards. As a result, we present a new approach of CW control mechanism, which is termed as contention window threshold (CW_Threshold). It uses the DRL principle to define the threshold value and learn optimal settings under various network scenarios. We demonstrate our proposed method, known as a smart exponential-threshold-linear backoff algorithm with a deep Q-learning network (SETL-DQN). The simulation results show that our proposed SETL-DQN algorithm can effectively improve the throughput and reduce the collision rates.

• 한국정보통신학회:학술대회논문집
• /
• 한국정보통신학회 2022년도 춘계학술대회
• /
• pp.47-49
• /
• 2022

Robot navigation has seen a major improvement since the the rediscovery of the potential of Artificial Intelligence (AI) and the attention it has garnered in research circles. A notable achievement in the area was Deep Learning (DL) application in computer vision with outstanding daily life applications such as face-recognition, object detection, and more. However, robotics in general still depend on human inputs in certain areas such as localization, navigation, etc. In this paper, we propose a study case of robot navigation based on deep reinforcement technology. We look into the benefits of switching from traditional ROS-based navigation algorithms towards machine learning approaches and methods. We describe the state-of-the-art technology by introducing the concepts of Reinforcement Learning (RL), Deep Learning (DL) and DRL before before focusing on visual navigation based on DRL. The case study preludes further real life deployment in which mobile navigational agent learns to navigate unbeknownst areas.

• Journal of Advanced Research in Ocean Engineering
• /
• 제3권3호
• /
• pp.136-148
• /
• 2017

The paper presents dual arm ROV manipulation using deep reinforcement learning. The purpose of this underwater manipulator is to investigate and excavate natural resources in ocean, finding lost aircraft blackboxes and for performing other extremely dangerous tasks without endangering humans. This research work emphasizes on a self-learning approach using Deep Reinforcement Learning (DRL). DRL technique allows ROV to learn the policy of performing manipulation task directly, from raw image data. Our proposed architecture maps the visual inputs (images) to control actions (output) and get reward after each action, which allows an agent to learn manipulation skill through trial and error method. We have trained our network in simulation. The raw images and rewards are directly provided by our simple Lua simulator. Our simulator achieve accuracy by considering underwater dynamic environmental conditions. Major goal of this research is to provide a smart self-learning way to achieve manipulation in highly dynamic underwater environment. The results showed that a dual robotic arm trained for a 3DOF movement successfully achieved target reaching task in a 2D space by considering real environmental factor.

• 한국방사선학회논문지
• /
• 제14권7호
• /
• pp.991-1001
• /
• 2020

전산화단층영상 품질 개선을 위해 사용되는 지도학습 기반의 딥러닝 기술은 사전 학습을 위해 많은 양의 데이터를 필요로 하는 단점이 있다. 또한 지도학습 기반의 딥러닝 기술은 학습에 사용된 영상의 특징과 학습된 모델에 입력된 영상의 특징이 다른 경우 영상 내부 구조적 왜곡이 유발되는 한계점이 있다. 본 연구에서는 기존 지도학습 기반 딥러닝 기술의 단점을 보완하고 전산화단층영상의 잡음을 감소시킬 수 있는 심층강화학습 기반 영상화 모델을 개발하였다. 심층강화학습 기반 영상화 모델은 shared, value 및 policy 네트워크로 구성하였으며, 영상 잡음 특징 추출 및 모델의 성능 향상을 위해 합성곱, rectified linear unit(ReLU) 활성화 함수, dilation factor 및 게이트순환유닛을 사용하였다. 또한 기존 지도학습 기반 딥러닝 기술을 통해 획득한 영상의 영상품질 비교를 통해 본 연구에서 개발한 영상화 모델의 성능을 평가하였다. 연구결과 기존 기술에 비해 본 연구에서 개발한 영상화 모델 적용 시 전산화단층영상의 정량적 정확도는 큰 폭으로 향상, 잡음은 큰 폭으로 감소함을 확인하였다. 또한 영상화 모델 학습 시 사용한 영상과 구조적 특징이 다른 영상에 대해서도 잡음 감소 효과를 확인하였다. 따라서 본 연구에서 개발한 심층강화학습 기반 영상화 모델을 통해 전산화단층영상의 구조적 특징을 보전함과 동시에 잡음을 감소시킬 수 있다.

• Architectural research
• /
• 제26권1호
• /
• pp.1-12
• /
• 2024

This study introduces a deep reinforcement learning (DRL)-based methodology for optimizing apartment space arrangements, addressing the limitations of human capability in evaluating all potential spatial configurations. Leveraging computational power, the methodology facilitates the autonomous exploration and evaluation of innovative layout options, considering architectural principles, legal standards, and client re-quirements. Through comprehensive simulation tests across various apartment types, the research demonstrates the DRL approach's effec-tiveness in generating efficient spatial arrangements that align with current design trends and meet predefined performance objectives. The comparative analysis of AI-generated layouts with those designed by professionals validates the methodology's applicability and potential in enhancing architectural design practices by offering novel, optimized spatial configuration solutions.

• 한국건설관리학회논문집
• /
• 제24권5호
• /
• pp.73-82
• /
• 2023

Optimization of Construction Site Layout Planning (CSLP) heavily relies on workers' travel paths. However, traditional path generation approaches predominantly focus on the shortest path, often neglecting critical variables such as individual wayfinding tendencies, the spatial arrangement of site objects, and potential hazards. These oversights can lead to compromised path simulations, resulting in less reliable site layout plans. While Deep Reinforcement Learning (DRL) has been proposed as a potential alternative to address these issues, it has shown limitations. Despite presenting more realistic travel paths by considering these variables, DRL often struggles with efficiency in complex environments, leading to extended learning times and potential failures. To overcome these challenges, this study introduces a refined model that enhances spatial navigation capabilities and learning performance by integrating workers' visibility into the reward functions. The proposed model demonstrated a 12.47% increase in the pathfinding success rate and notable improvements in the other two performance measures compared to the existing DRL framework. The adoption of this model could greatly enhance the reliability of the results, ultimately improving site operational efficiency and safety management such as by reducing site congestion and accidents. Future research could expand this study by simulating travel paths in dynamic, multi-agent environments that represent different stages of construction.

• Nuclear Engineering and Technology
• /
• 제55권9호
• /
• pp.3277-3290
• /
• 2023

Nuclear power plant (NPP) operations with multiple objectives and devices are still performed manually by operators despite the potential for human error. These operations could be automated to reduce the burden on operators; however, classical approaches may not be suitable for these multi-objective tasks. An alternative approach is deep reinforcement learning (DRL), which has been successful in automating various complex tasks and has been applied in automation of certain operations in NPPs. But despite the recent progress, previous studies using DRL for NPP operations have limitations to handle complex multi-objective operations with multiple devices efficiently. This study proposes a novel DRL-based approach that addresses these limitations by employing a continuous action space and straightforward binary rewards supported by the adoption of a soft actor-critic and hindsight experience replay. The feasibility of the proposed approach was evaluated for controlling the pressure and volume of the reactor coolant while heating the coolant during NPP startup. The results show that the proposed approach can train the agent with a proper strategy for effectively achieving multiple objectives through the control of multiple devices. Moreover, hands-on testing results demonstrate that the trained agent is capable of handling untrained objectives, such as cooldown, with substantial success.

• 한국전자통신학회논문지
• /
• 제18권2호
• /
• pp.231-240
• /
• 2023

심층 강화 학습(Deep Reinforcement Learning)을 사용한 경로 계획에서 장애물을 자동으로 회피하기 위해 로봇을 학습시키는 일은 쉬운 일이 아니다. 많은 연구자가 DRL을 사용하여 주어진 환경에서 로봇 학습을 통해 장애물 회피하여 경로 계획을 수립하려는 가능성을 시도하였다. 그러나 다양한 환경에서 로봇과 장착된 센서의 오는 다양한 요인 때문에 주어진 시나리오에서 로봇이 모든 장애물을 완전히 회피하여 이동하는 것을 실현하는 일은 흔치 않다. 이러한 문제 해결의 가능성과 장애물을 회피 경로 계획 실험을 위해 테스트베드를 만들었고 로봇에 카메라를 장착하였다. 이 로봇의 목표는 가능한 한 빨리 벽과 장애물을 피해 시작점에서 끝점까지 도달하는 것이다. 본 논문에서는 벽과 장애물을 회피하기 위한 DRL의 가능성을 검증하기 위해 이중 심층 Q 네트워크(DDQN)를 제안하였다. 실험에 사용된 로봇은 Jetbot이며 자동화된 경로 계획에서 장애물 회피가 필요한 일부 로봇 작업 시나리오에 적용할 수 있을 것이다.

• Smart Structures and Systems
• /
• 제31권3호
• /
• pp.247-257
• /
• 2023

In structural health monitoring of large-scale structures, optimal sensor placement plays an important role because of the high cost of sensors and their supporting instruments, as well as the burden of data transmission and storage. In this study, a vibration sensor placement algorithm based on deep reinforcement learning (DRL) is proposed, which can effectively solve non-convex, high-dimensional, and discrete combinatorial sensor placement optimization problems. An objective function is constructed to estimate the quality of a specific vibration sensor placement scheme according to the modal assurance criterion (MAC). Using this objective function, a DRL-based algorithm is presented to determine the optimal vibration sensor placement scheme. Subsequently, we transform the sensor optimal placement process into a Markov decision process and employ a DRL-based optimization algorithm to maximize the objective function for optimal sensor placement. To illustrate the applicability of the proposed method, two examples are presented: a 10-story braced frame and a sea-crossing bridge model. A comparison study is also performed with a genetic algorithm and particle swarm algorithm. The proposed DRL-based algorithm can effectively solve the discrete combinatorial optimization problem for vibration sensor placements and can produce superior performance compared with the other two existing methods.