• 전기전자학회논문지
• /
• 제25권4호
• /
• pp.644-649
• /
• 2021

Self-Imitation Learning은 간단한 비활성 정책 actor-critic 알고리즘으로써 에이전트가 과거의 좋은 경험을 활용하여 최적의 정책을 찾을 수 있도록 해준다. 그리고 actor-critic 구조를 갖는 강화학습 알고리즘에 결합되어 다양한 환경들에서 알고리즘의 상당한 개선을 보여주었다. 하지만 Self-Imitation Learning이 강화학습에 큰 도움을 준다고 하더라도 그 적용 분야는 actor-critic architecture를 가지는 강화학습 알고리즘으로 제한되어 있다. 본 논문에서 Self-Imitation Learning의 알고리즘을 가치 기반 강화학습 알고리즘인 DQN에 적용하는 방법을 제안하고, Self-Imitation Learning이 적용된 DQN 알고리즘의 학습을 다양한 환경에서 진행한다. 아울러 그 결과를 기존의 결과와 비교함으로써 Self-Imitation Leaning이 DQN에도 적용될 수 있으며 DQN의 성능을 개선할 수 있음을 보인다.

• 한국지능시스템학회논문지
• /
• 제19권4호
• /
• pp.519-524
• /
• 2009

최근에 강화학습 기법은 기계학습 분야에서 많은 관심을 끌어왔다. 강화학습 관련 연구에서 가장 유력하게 사용되어 온 방법들로는 가치함수를 활용하는 기법, 제어규칙(policy) 탐색 기법 및 액터-크리틱 기법 등이 있는데, 본 논문에서는 이들 중 연속 상태 및 연속 입력을 갖는 문제를 위하여 액터-크리틱 기법의 틀에서 제안된 알고리즘들과 관련된 내용을 다룬다. 특히 본 논문은 퍼지 이론에 기반을 둔 액터-크리틱 계열 강화학습 기법인 ACFRL 알고리즘과, RLS 필터와 NAC(natural actor-critic) 기법에 기반을 둔 RLS-NAC 기법을 접목하는 방안을 집중적으로 고찰한다. 고찰된 방법론은 기는 로봇의 제어문제에 적용되고, 학습 성능의 비교로부터 얻어진 몇 가지 결과가 보고된다.

• Journal of information and communication convergence engineering
• /
• 제20권1호
• /
• pp.22-30
• /
• 2022

The reinforcement learning algorithm has proven its potential in solving sequential decision-making problems under uncertainties, such as finding paths to route data packets in wireless sensor networks. With reinforcement learning, the computation of the optimum path requires careful definition of the so-called reward function, which is defined as a linear function that aggregates multiple objective functions into a single objective to compute a numerical value (reward) to be maximized. In a typical defined linear reward function, the multiple objectives to be optimized are integrated in the form of a weighted sum with fixed weighting factors for all learning agents. This study proposes a reinforcement learning -based routing protocol for wireless sensor network, where different learning agents prioritize different objective goals by assigning weighting factors to the aggregated objectives of the reward function. We assign appropriate weighting factors to the objectives in the reward function of a sensor node according to its hop-count distance to the sink node. We expect this approach to enhance the effectiveness of multi-objective reinforcement learning for wireless sensor networks with a balanced trade-off among competing parameters. Furthermore, we propose SDN (Software Defined Networks) architecture with multiple controllers for constant network monitoring to allow learning agents to adapt according to the dynamics of the network conditions. Simulation results show that our proposed scheme enhances the performance of wireless sensor network under varied conditions, such as the node density and traffic intensity, with a good trade-off among competing performance metrics.

• 한국지능시스템학회:학술대회논문집
• /
• 한국퍼지및지능시스템학회 2007년도 춘계학술대회 학술발표 논문집 제17권 제1호
• /
• pp.33-36
• /
• 2007

최근에 인공지능 분야에서는, 국내외적으로 강화학습(reinforcement learning)에 관한 관심이 크게 증폭되고 있다. 강화학습의 최근 경향을 살펴보면, 크게 가치함수를 직접 활용하는 방법(value function-based methods), 제어 전략에 대한 탐색을 활용하는 방법(policy search methods), 그리고 액터-크리틱 방법(actor-critic methods)의 세가지 방향으로 발전하고 있음을 알 수 있다. 본 논문에서는 이중 세 번째 부류인 액터-크리틱 방법 중 NAC(natural actor-critic) 기법의 한 종류인 RLS-NAC(recursive least-squares based natural actor-critic) 알고리즘을 다양한 트레이스 감쇠계수를 사용하여 연속제어입력(real-valued control inputs)으로 제어되는 Kimura의 기는 로봇에 대해 적용해보고, 그 성능을 기존의 SGA(stochastic gradient ascent) 알고리즘을 이용하여 학습한 경우와 비교해보도록 한다.

• Computers and Concrete
• /
• 제32권2호
• /
• pp.119-138
• /
• 2023

Concrete carbonation is a prevalent phenomenon that leads to steel reinforcement corrosion in reinforced concrete (RC) structures, thereby decreasing their service life as well as durability. The process of carbonation results in a lower pH level of concrete, resulting in an acidic environment with a pH value below 12. This acidic environment initiates and accelerates the corrosion of steel reinforcement in concrete, rendering it more susceptible to damage and ultimately weakening the overall structural integrity of the RC system. Lower pH values might cause damage to the protective coating of steel, also known as the passive film, thus speeding up the process of corrosion. It is essential to estimate the carbonation factor to reduce the deterioration in concrete structures. A lot of work has gone into developing a carbonation model that is precise and efficient that takes both internal and external factors into account. This study presents an ML-based adaptive-neuro fuzzy inference system (ANFIS) approach to predict the carbonation depth of fly ash (FA)-based concrete structures. Cement content, FA, water-cement ratio, relative humidity, duration, and CO2 level have been used as input parameters to develop the ANFIS model. Six performance indices have been used for finding the accuracy of the developed model and two analytical models. The outcome of the ANFIS model has also been compared with the other models used in this study. The prediction results show that the ANFIS model outperforms analytical models with R-value, MAE, RMSE, and Nash-Sutcliffe efficiency index values of 0.9951, 0.7255 mm, 1.2346 mm, and 0.9957, respectively. Surface plots and sensitivity analysis have also been performed to identify the repercussion of individual features on the carbonation depth of FA-based concrete structures. The developed ANFIS-based model is simple, easy to use, and cost-effective with good accuracy as compared to existing models.

• Computers and Concrete
• /
• 제27권2호
• /
• pp.163-173
• /
• 2021

In this paper, the parameters of haunch height, reinforcement ratio and site condition were evaluated for the influence on the seismic performance of a composite precast fabricated utility tunnel by shaking table test and numerical simulation. The dynamic response laws of acceleration, interlayer displacement and steel strain under unidirectional horizontal seismic excitation were analyzed through four specimens with a similarity ratio of 1:6 in the test. And a numerical model was established and analyzed by the finite element software ABAQUS based on the structure of utility tunnel. The results indicated that composite precast fabricated utility tunnel with the good anti-seismic performance. In a certain range, increasing the height of haunch or the ratio of reinforcement could reduce the influence of seismic wave on the utility tunnel structure, which was beneficial to the structure earthquake resistance. The clay field containing the interlayer of liquefied sandy soil has a certain damping effect on the structure of the utility tunnel, and the displacement response could be reduced by 14.1%. Under the excitation of strong earthquake, the reinforcement strain at the side wall upper end and haunches of the utility tunnel was the biggest, which is the key part of the structure. The experimental results were in good agreement with the fitting results, and the results could provide a reference value for the anti-seismic design and application of composite precast fabricated utility tunnel.

• International Journal of Advanced Culture Technology
• /
• 제9권2호
• /
• pp.106-117
• /
• 2021

The aim is to find cyber measures in consideration of physical CPTED in order to prepare countermeasures for cybercrime prevention. For this, the six applied principles of CPTED were used as the standard. A new control item was created in connection with the control items of ISO27001. A survey was conducted on former and current investigators and security experts. As a result of the reliability analysis, the Kronbar alpha coefficient value was 0.947, indicating the reliability of the statistical value. As a result of factor analysis, it was reduced to six factors. The following are six factors and countermeasures. Nature monitoring blocks opportunities and strengthens business continuity. Access control is based on management system compliance, personnel security. Reinforcement of territoriality is reinforcement of each wife and ethics. Establishment of security policy to enhance readability, security system maintenance. Increasing usability is seeking ways to utilize, periodic incentives. For maintenance, security education is strength and security-related collective cooperation is conducted. The differentiation of this study was to find countermeasures against cybercrime in the psychological part of the past. However, they approached to find in cyber measures. The limitation of the study is to bring the concept of physical CPTED to the cyber concept.

• 융합정보논문지
• /
• 제10권6호
• /
• pp.1-7
• /
• 2020

수중 자원 탐색 및 해양 탐사, 환경 조사 등 수중 통신에 대한 수요가 급격하게 증가하고 있다. 하지만 수중 무선 통신을 사용하기 앞서 많은 문제점을 가지고 있다. 특히 수중 무선 네트워크에서 환경적 요인으로 인해 불가피하게 발생하는 불필요한 지연 시간과 노드 거리에 따른 공간적 불평등 문제가 존재한다. 본 논문은 이러한 문제를 해결하기 위해 ALOHA-Q를 기반으로 한 새로운 NAV 설정 방법을 제안한다. 제안 방법은 NAV 값을 랜덤하게 사용하고 통신 성공, 실패 유무에 따라 보상을 측정한다. 이후 보상 값에 따라 NAV 값을 설정 한다. 수중 무선 네트워크에서 에너지와 컴퓨팅 자원을 최대한 낮게 사용하면서 NAV 값을 강화 학습을 통하여 학습하고 한다. 시뮬레이션 결과 NAV 값이 해당 환경에 적응하고 최선의 값을 선택하여 불필요한 지연 시간문제와 공간적 불평등 문제를 해결할 수 있음을 보여준다. 시뮬레이션 결과 설정한 환경 내에서 기존 NAV 설정 시간 대비 약 17.5%의 시간을 감소하는 것을 보여준다.

• 콘크리트학회논문집
• /
• 제14권3호
• /
• pp.365-372
• /
• 2002

본 연구는 700kgf/$\textrm{cm}^2$ 고강도 콘크리트에서 횡보강근 형태, 체적비 그리고 횡보강근 항복강도에 따른 고강도 콘크리트기둥의 거동을 규명하기 위한 실험연구이다. 기둥은 중심축내력의 30%에 해당하는 일정축력과 수평방향의 반복 휨모멘트를 받는다. 본 연구에서 사용된 변수는 횡보강근 체적비(Ps=1.58, 2.25%), 횡보강근 형태(hoop-type, cross-type, diagonal-type) 그리고 횡보강근 항복강도(fy=5,600, 7,950 kgf/$\textrm{cm}^2$)이다. 실험결과로 모든 기둥의 휨강도는 현행규준의 등가응력블럭에 근거하여 산정된 휨강도보다 낮게 나타났다. 횡보강근을 ACI 규준 요구량보다 42%증가시킨 기둥 시험체는 연성적인 거동을 보였다. 그리고, 본 연구에서 적용한 축력비 0.3 P/PO하에서 고강도급 횡보강근을 사용한 시험체의 연성이 저강도급 횡보강근을 사용한 시험체의 경우보다 같거나 다소 큰 경향을 보이고 있었다.

• Structural Engineering and Mechanics
• /
• 제89권6호
• /
• pp.579-588
• /
• 2024

The present paper investigates the curvature ductility of confined reinforced concrete (RC) beams with normal (NSC) and high strength concrete (HSC). For the purpose of predicting the curvature ductility factor, an analytical model was developed based on the equilibrium of internal forces of confined concrete and reinforcement. In this context, the curvatures were calculated at first yielding of tension reinforcement and at ultimate when the confined concrete strain reaches the ultimate value. To best simulate the situation of confined RC beams in flexure, a modified version of an ancient confined concrete model was adopted for this study. In order to show the accuracy of the proposed model, an experimental database was collected from the literature. The statistical comparison between experimental and predicted results showed that the proposed model has a good performance. Then, the data generated from the validated theoretical model were used to train the artificial neural network (ANN) prediction model. The R² values for theoretical and experimental results are equal to 0.98 and 0.95, respectively which proves the high performance of the ANN model. Finally, a parametric study was implemented to analyze the effect of different parameters on the curvature ductility factor using theoretical and ANN models. The results are similar to those extracted from experiments, where the concrete strength, the compression reinforcement ratio, the yield strength, and the volumetric ratio of transverse reinforcement have a positive effect. In contrast, the ratio and the yield strength of tension reinforcement have a negative effect.