• Journal of Platform Technology
• /
• 제11권6호
• /
• pp.21-27
• /
• 2023

In this paper, we introduce a reinforcement learning-based algorithm for personalized tourist path recommendations. The algorithm employs a reinforcement learning agent to explore tourist regions and identify optimal paths that are expected to enhance tourism experiences. The concept of tourism experience is defined through points of interest (POI) located along tourist paths within the tourist area. These metrics are quantified through aggregated evaluation scores derived from reviews submitted by past visitors. In the experimental setup, the foundational learning model used to find tour paths is the Deep Q-Network (DQN). Despite the limited availability of historical tourist behavior data, the agent adeptly learns travel paths by incorporating preference scores of tourist POIs and spatial information of the travel area.

• 한국공간구조학회논문집
• /
• 제23권2호
• /
• pp.29-36
• /
• 2023

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

• 한국항해항만학회지
• /
• 제47권2호
• /
• pp.75-84
• /
• 2023

Ships need to ensure safety during their navigation, which makes route determination highly important. It must be accompanied by a route following controller that can accurately follow the route. This study proposes a method for automatically generating the ship route based on deep reinforcement learning algorithm and following it using a route following controller. To generate a ship route, under keel clearance was applied to secure the ship's safety and navigation chart information was used to apply ship navigation related regulations. For the experiment, a target ship with a draft of 8.23 m was designated. The target route in this study was to depart from Busan port and arrive at the pilot boarding place of the Ulsan port. As a route following controller, a velocity type fuzzy P ID controller that could compensate for the limitation of a linear controller was applied. As a result of using the deep Q network, a route with a total distance of 62.22 km and 81 waypoints was generated. To simplify the route, the Douglas-Peucker algorithm was introduced to reduce the total distance to 55.67 m and the number of way points to 3. After that, an experiment was conducted to follow the path generated by the target ship. Experiment results revealed that the velocity type fuzzy P ID controller had less overshoot and fast settling time. In addition, it had the advantage of reducing the energy loss of the ship because the change in rudder angle was smooth. This study can be used as a basic study of route automatic generation. It suggests a method of combining ship route generation with the route following control.

• Smart Structures and Systems
• /
• 제24권6호
• /
• pp.745-757
• /
• 2019

Road surface deteriorations such as potholes have caused motorists heavy monetary damages every year. However, effective road condition monitoring has been a continuing challenge to road owners. Depth cameras have a small field of view and can be easily affected by vehicle bouncing. Traditional image processing methods based on algorithms such as segmentation cannot adapt to varying environmental and camera scenarios. In recent years, novel object detection methods based on deep learning algorithms have produced good results in detecting typical objects, such as faces, vehicles, structures and more, even in scenarios with changing object distances, camera angles, lighting conditions, etc. Therefore, in this study, a Deep Learning Pothole Detector (DLPD) based on the deep region-based convolutional neural network is proposed for autonomous detection of potholes from images. About 900 images with potholes and road surface conditions are collected and divided into training and testing data. Parameters of the network in the DLPD are calibrated based on sensitivity tests. Then, the calibrated DLPD is trained by the training data and applied to the 215 testing images to evaluate its performance. It is demonstrated that potholes can be automatically detected with high average precision over 93%. Potholes can be differentiated from manholes by training and applying a manhole-pothole classifier which is constructed using the convolutional neural network layers in DLPD. Repeated detection of the same potholes can be prevented through feature matching of the newly detected pothole with previously detected potholes within a small region.

• 한국정보통신학회:학술대회논문집
• /
• 한국정보통신학회 2021년도 춘계학술대회
• /
• pp.538-540
• /
• 2021

본 논문에서는 드론의 표적탐지 임무 수행 시 상대운동 정보 제공을 위하여 지정된 표적을 탐지하고 인식하는 스마트 표적탐지 시스템을 제안하였다. 제안된 시스템은 적절한 정확도(i.e. mAP, IoU) 및 높은 실시간성을 동시에 확보할 수 있는 알고리즘을 개발하는데 중점을 두었다. 제안된 시스템은 Google Inception V2 딥러닝 모델의 100k 학습 후 test 결과가 1.0에 가까운 정확성을 보였고 실시간성도 Nvidia GTX 2070 Max-Q를 기반으로 한 고성능 노트북 활용 시에 추론 속도가 약 60-80[Hz]를 기록하였다. 제안된 스마트 표적탐지 시스템은 드론과 같이 운용되어 컴퓨터 영상처리를 활용하여 표적을 자동으로 인식하고 표적을 따라가면서 감시정찰 임무를 성공적으로 수행하는데 도움이 될 것이다.

• 한국인터넷방송통신학회논문지
• /
• 제21권6호
• /
• pp.163-171
• /
• 2021

본 논문은 분산형 전력 시스템에서 심층강화학습 기반의 전력 생산 환경 및 수요와 공급을 예측하며 자원 할당 알고리즘을 적용해 전력거래 시스템 연구의 최적화된 결과를 보여준다. 전력 거래시스템에 있어서 기존의 중앙집중식 전력 시스템에서 분산형 전력 시스템으로의 패러다임 변화에 맞추어 전력거래에 있어서 공동의 이익을 추구하며 장기적인 거래의 효율을 증가시키는 전력 거래시스템의 구축을 목표로 한다. 심층강화학습의 현실적인 에너지 모델과 환경을 만들고 학습을 시키기 위해 날씨와 매달의 패턴을 분석하여 데이터를 생성하며 시뮬레이션을 진행하는 데 있어서 가우시안 잡음을 추가해 에너지 시장 모델을 구축하였다. 모의실험 결과 제안된 전력 거래시스템은 서로 협조적이며 공동의 이익을 추구하며 장기적으로 이익을 증가시킨 것을 확인하였다.

• 인터넷정보학회논문지
• /
• 제23권3호
• /
• pp.13-20
• /
• 2022

차세대 와이파이 표준기술인 IEEE 802.11ay는 밀리미터파 대역에서 AP (Access Point)가 다수의 STA (Station)로 동시에 데이터를 전송하도록 MU-MIMO (Multiple User Multiple Input Multiple Output) 통신을 지원한다. 이를 위해, 주기적으로 MU-MIMO 빔포밍 훈련을 수행해야 하고, 효율적인 빔포밍 훈련을 위해서는 AP가 다수의 안테나로 다수의 빔을 동시에 전송할 때, 각 STA에서 측정되는 신호 세기를 정확히 예측하는 것이 중요하다. 본 논문에서는 딥러닝 기반 다중 빔 전송링크 성능 예측기법을 제안한다. 제안한 예측기법은 특정 실내 또는 실외 환경에서 미리 학습된 딥러닝 모델을 이용하여 다수의 빔이 동시에 전송될 때 STA에서 측정되는 신호 세기 예측의 정확성을 높인다. 이때, 딥러닝의 입력으로 개별 빔이 전송될 때 STA에서 측정되는 신호 세기 정보를 이용하고, 개별 빔의 신호 세기 정보를 얻는 과정은 이미 기존의 빔포밍 훈련에 포함되어 있으므로 정보 수집을 위해 추가적인 비용을 발생하지 않는다. 성능평가를 위해 NIST (National Institute of Standards and Technology)에 의해 개발된 Q-D 채널구현 (Quasi-Deterministic Channel Realization) 오픈소스 소프트웨어를 활용하였고 실측 데이터 기반으로 밀리미터파 채널을 구현하였다. 실험결과에서는 제안한 예측기법이 다른 비교기법보다 향상된 예측성능을 보였다.

• International Journal of Computer Science & Network Security
• /
• 제24권4호
• /
• pp.87-106
• /
• 2024

Vehicle-to-Home (V2H) and Home Centralized Photovoltaic (HCPV) systems can address various energy storage issues and enhance demand response programs. Renewable energy, such as solar energy and wind turbines, address the energy gap. However, no energy management system is currently available to regulate the uncertainty of renewable energy sources, electric vehicles, and appliance consumption within a smart microgrid. Therefore, this study investigated the impact of solar photovoltaic (PV) panels, electric vehicles, and Micro-Grid (MG) storage on maximum solar radiation hours. Several Deep Learning (DL) algorithms were applied to account for the uncertainty. Moreover, a Reinforcement Learning HCPV (RL-HCPV) algorithm was created for efficient real-time energy scheduling decisions. The proposed algorithm managed the energy demand between PV solar energy generation and vehicle energy storage. RL-HCPV was modeled according to several constraints to meet household electricity demands in sunny and cloudy weather. Simulations demonstrated how the proposed RL-HCPV system could efficiently handle the demand response and how V2H can help to smooth the appliance load profile and reduce power consumption costs with sustainable power generation. The results demonstrated the advantages of utilizing RL and V2H as potential storage technology for smart buildings.

• 스마트미디어저널
• /
• 제9권2호
• /
• pp.92-98
• /
• 2020

본 논문은 고성능의 서버 없이 안드로이드 스마트폰 단독으로 동작할 수 있도록 경량화 딥러닝 모델을 사용하여 구현한 자동차 번호판 인식 시스템을 제안한다. 자동차 번호판 인식시스템은 [번호판검출]-[문자영역 분할]-[문자인식]으로 3단계의 과정으로 구성되며, 번호판검출은 SSD-Mobilenet, 문자영역 분할은 ResNet에 localization을 추가하여 사용하였고 문자인식은 ResNet을 이용하여 구현하였다. 테스트한 기기는 삼성 갤럭시 S7, LG Q9이며 정확도는 약 85.3%, 실행속도는 약 1.1초가 소요된다.

• 디지털융복합연구
• /
• 제19권12호
• /
• pp.103-113
• /
• 2021

본 연구는 Q방법을 활용하여 대학생이 지각하는 교양교육에 대한 인식 유형을 알아보고 유형별 특성을 알아보기 위해 실시되었다. 문헌연구와 개방형 질문지, 심층 면접을 통해 수집한 Q 모집단에서 Q표본 33개를 추출하였다. Q분류는 B시에 소재한 A대학교 재학생 27명을 대상으로 실시하였으며, QUANL 프로그램을 사용하여 자료를 분석하였다. 연구 결과, 교양교육에 대한 인식 유형은 '다양한 경험 추구형', '실용학문 추구형', '사고확장 추구형', '사회변화 추구형' 등 4가지 유형으로 추출되었다. 본 연구 결과는 대학교육에서 교양교육의 의미를 재정립하고 다양한 교육 내용과 교수-학습 방법에 대한 고려가 필요함을 시사한다.