• 제목/요약/키워드: policy gradient

검색결과 73건 처리시간 0.031초

GPS를 이용한 차량 연료차단 관성주행의 감지에 관한 연구 (A Study for Detecting Fuel-cut Driving of Vehicle Using GPS)

  • 고광호
    • 디지털융복합연구
    • /
    • 제17권11호
    • /
    • pp.207-213
    • /
    • 2019
  • 대부분의 차량에 적용되어 있는 연료차단(fuel-cut) 관성주행은 변속기어 체결 상태에서 가속페달을 방치할 때 자동으로 작동하게 된다. 이 때 연료분사가 일시적으로 중단되므로 연비 향상 효과가 상당하다. 본 연구에서는 GPS를 이용하여 측정된 차속, 가속도, 도로구배 등의 신호를 바탕으로 하는 연료차단 관성주행 감지법을 제안하였다. 관성 주행시 작용하는 주행저항력에 의해 계산되는 가속도값과 GPS에서 실시간으로 측정되는 가속도값을 비교하는 방식이다. 실도로 주행 데이터를 측정하여 이 감지법을 평가한 결과 약 80% 수준의 정확도를 얻을 수 있었다. 도로구배가 다소 큰 12km 정도의 국도를 16분 동안 주행하면서 측정한 약 9,600개의 속도, 가속도, 도로구배 및 연료소모량 데이터에 감지법을 적용하여 얻은 결과이다. 인젝터 분사파형 분석을 위한 배선작업 등이 불필요하여 간단하게 연료차단여부를 판정할 수 있는 장점이 있다. 다만, 속도, 가속도 및 도로구배의 변화율이 연료소모량의 변화율에 비해 훨씬 크게 나타나기 때문에 감지법의 오차도 다소 증가하는 것을 알 수 있었다.

열대 홍수림 주변 해역 환경 전이대의 식물플랑크톤 및 박테리아의 분포 (Distribution of Phytoplankton and Bacteria in the Environmental Transitional Zone of Tropical Mangrove Area)

  • 최동한;노재훈;안성민;이미진;김동선;김경태;권문상;박흥식
    • Ocean and Polar Research
    • /
    • 제35권4호
    • /
    • pp.415-425
    • /
    • 2013
  • In order to understand phytoplankton and bacterial distribution in tropical coral reef ecosystems in relation to the mangrove community, their biomass and activities were measured in the sea waters of the Chuuk and the Kosrae lagoons located in Micronesia. Chlorophyll a and bacterial abundance showed maximal values in the seawater near the mangrove forests, and then steeply decreased as the distance increased from the mangrove forests, indicating that environmental conditions for these microorganisms changed greatly in lagoon waters. Together with chlorophyll a, abundance of Synechococcus and phototrophic picoeukaryotes and a variety of indicator pigments for dinoflagellates, diatoms, green algae and cryptophytes also showed similar spatial distribution patterns, suggesting that phytoplankton assemblages respond to the environmental gradient by changing community compositions. In addition, primary production and bacterial production were also highest in the bay surrounded by mangrove forest and lowest outside of the lagoon. These results suggest that mangrove waters play an important role in energy production and nutrient cycling in tropical coasts, undoubtedly receiving large inputs of organic matter from shore vegetation such as mangroves. However, the steep decrease of biomass and production of phytoplankton and heterotrophic bacteria within a short distance from the bay to the level of oligotrophic waters indicates that the effect of mangrove waters does not extend far away.

RLS 기반 Actor-Critic 학습을 이용한 로봇이동 (Robot Locomotion via RLS-based Actor-Critic Learning)

  • 김종호;강대성;박주영
    • 한국지능시스템학회논문지
    • /
    • 제15권7호
    • /
    • pp.893-898
    • /
    • 2005
  • 강화학습 방법론 중 하나의 부류인 액터-크리틱 알고리즘은 제어압력 선택 문제에 있어서 최소한의 계산만을 필요로 하고, 확률적 정책을 명시정으로 다룰 수 있는 장점 때문에 최근에 인공지능 분야에서 많은 관심을 끌고 있다. 액터-크리틱 네트워크는 제어압력 선택 전략을 위한 액터 네트워크와 가치 함수 근사를 위한 크리틱 네트워크로 구성되며, 우수한 제어입력의 서택과 정화한 가치 함수 관사를 최대한 신속하게 달성하기 위하여, 학습 과정 동안 액터와 크리틱은 자신들의 파라미터 백터를 적응적으로 변화시키는 전략을 구사한다. 본 논문은 크리틱의 학습을 위해 빠른 수렴성을 보장하는 RLS (Recursive Least Square)를 사용하고, 액터의 학습을 위해 정책의 기울기(Policy Gradient)를 이용하는 새로운 종류의 알고리즘을 고려한다. 고려된 알고리즘의 적용 가능성은 두개의 링크를 갖는 로봇에 대한 실험을 통하여 예시된다.

휴먼형 로봇 손의 사물 조작 수행을 이용한 인간 행동 복제 강화학습 정책 최적화 방법 성능 평가 (Evaluation of Human Demonstration Augmented Deep Reinforcement Learning Policy Optimization Methods Using Object Manipulation with an Anthropomorphic Robot Hand)

  • 박나현;오지헌;류가현;;;원다슬;정진균;장윤정;김태성
    • 한국정보처리학회:학술대회논문집
    • /
    • 한국정보처리학회 2020년도 추계학술발표대회
    • /
    • pp.858-861
    • /
    • 2020
  • 로봇이 사람과 같이 다양하고 복잡한 사물 조작을 하기 위해서 휴먼형 로봇손의 사물 파지 작업이 필수적이다. 자유도 (Degree of Freedom, DoF)가 높은 휴먼형(anthropomorphic) 로봇손을 학습시키기 위하여 사람 데모(human demonstration)가 결합된 강화학습 최적화 방법이 제안되었다. 본 연구에서는 강화학습 최적화 방법에 사람 데모가 결합된 Demonstration Augmented Natural Policy Gradient(DA-NPG)와 NPG 의 성능 비교를 통하여 행동 복제의 효율성을 확인하고, DA-NPG, DA-Trust Region Policy Optimization (DA-TRPO), DA-Proximal Policy Optimization (DA-PPO)의 최적화 방법의 성능 평가를 위하여 6 종의 물체에 대한 휴먼형 로봇손의 사물 조작 작업을 수행한다. 그 결과, DA-NPG 와 NPG를 비교한 결과를 통해 휴먼형 로봇손의 사물 조작 강화학습에 행동 복제가 효율적임을 증명하였다. 또한, DA-NPG 는 DA-TRPO 와 유사한 성능을 보이면서 모든 물체에 대한 사물 파지에 성공하여 가장 안정적이었다. 반면, DA-TRPO 와 DA-PPO 는 사물 조작에 실패한 물체가 존재하여 불안정한 성능을 보였다. 본 연구에서 제안하는 방법은 향후 실제 휴먼형 로봇에 적용하여 휴먼형 로봇 손의 사물조작 지능 개발에 유용할 것으로 전망된다.

Research on the Polarization Effects of the Shandong Processing Trade and Strategy to Coordinate Its Development

  • Xiao, Dan Dan
    • Asian Journal of Business Environment
    • /
    • 제3권2호
    • /
    • pp.17-22
    • /
    • 2013
  • Purpose - This dissertation is based on previous research, and analyzes processing trade, which constitutes a major section of foreign trade in Shandong Province. Research design, data, and methodology - The study uses the survey data on polarization, which is a vital index reflecting the unbalanced growth of regional economic development. The article introduces the processing trade polarization index, and the processing trade polarization fluctuation rate, to predict the geographical polarization posture and development trends in Shandong Province. Results -The development of processing trade in Shandong Province shows the level of gradient from east to west. The first-line growth pole has been formed and developed, and the initial formation of the diffusion mechanism has taken place. However, coordination problems in accompanying regional development have become increasingly prominent. Conclusions - This study focuses on the development of processing trade strategy and suggests overall coordination of development objectives, using non-balanced development goals. According to regional characteristics and development objectives of the processing trade in Shandong Province, the region around the city is divided into innovation diffusion region, enhanced growth areas, areas expected to undertake development, and areas to upgrade in four levels, given the different policy proposals.

제주도 표선유역의 물수지 평가를 위한 지하수 유동 모델링 (Groundwater Modeling for Estimating Water Balance over Pyosun Watershed in Jeju Island)

  • 송성호;이규상;안중기;전선금;이명재
    • 한국환경과학회지
    • /
    • 제24권4호
    • /
    • pp.495-504
    • /
    • 2015
  • To estimate water balance of Pyosun watershed in Jeju Island, a three-dimensional finite difference model MODFLOW was applied. Moreover, the accuracy of groundwater flow modeling was evaluated through the comparison of the recharge rate by flow modeling and the existing one from water balance model. The modeling result under the steady-state condition indicates that groundwater flow direction was from Mt. Halla to the South Sea and groundwater gradient was gradually lowered depending on the elevation. Annual recharge rate by the groundwater flow modeling in Pyosun watershed was calculated to 236 million $m^3/year$ and it was found to be very low as compared to the recharge rate 238 million $m^3/year$ by the existing water balance model. Therefore, groundwater flow modeling turned out to be useful to estimate the recharge rate in Pyosun watershed and it would be available to make groundwater management policy for watershed in the future.

4D 사이클링에 대한 기초 신체능력 프로토콜 (The Protocol of Basic Body Ability for 4D Cycling System)

  • 김기봉;이성한
    • 디지털융복합연구
    • /
    • 제11권11호
    • /
    • pp.313-320
    • /
    • 2013
  • 본 논문에서는 가상현실상의 내리막이나 오르막 상황, 경사도 또는 노면 상태를 그대로 인식할 수 있는 사이클 시뮬레이터를 개발하였다. 개발된 시뮬레이터는 전방에 설치되어 있는 LCD 모니터를 통해 디스플레이 되는 다양한 가상 운동경로의 상황에 맞추어 종전 운동용 사이클의 단조롭고 지루한 단점을 개선하였다. 또한 적절한 운동량과 재미요소를 추가하여 현실감과 게임의 재미를 느낄 수 있는 4D 사이클링 콘텐츠를 개발하였으며, 흥미 유발형 운동기기에 대한 사용자 편의성을 위한 기초 신체 능력에 대한 분석 및 평가 하였다.

보틀플리핑의 로봇 강화학습을 위한 효과적인 보상 함수의 설계 (Designing an Efficient Reward Function for Robot Reinforcement Learning of The Water Bottle Flipping Task)

  • 양영하;이상혁;이철수
    • 로봇학회논문지
    • /
    • 제14권2호
    • /
    • pp.81-86
    • /
    • 2019
  • Robots are used in various industrial sites, but traditional methods of operating a robot are limited at some kind of tasks. In order for a robot to accomplish a task, it is needed to find and solve accurate formula between a robot and environment and that is complicated work. Accordingly, reinforcement learning of robots is actively studied to overcome this difficulties. This study describes the process and results of learning and solving which applied reinforcement learning. The mission that the robot is going to learn is bottle flipping. Bottle flipping is an activity that involves throwing a plastic bottle in an attempt to land it upright on its bottom. Complexity of movement of liquid in the bottle when it thrown in the air, makes this task difficult to solve in traditional ways. Reinforcement learning process makes it easier. After 3-DOF robotic arm being instructed how to throwing the bottle, the robot find the better motion that make successful with the task. Two reward functions are designed and compared the result of learning. Finite difference method is used to obtain policy gradient. This paper focuses on the process of designing an efficient reward function to improve bottle flipping motion.

Predicting Reports of Theft in Businesses via Machine Learning

  • JungIn, Seo;JeongHyeon, Chang
    • International Journal of Advanced Culture Technology
    • /
    • 제10권4호
    • /
    • pp.499-510
    • /
    • 2022
  • This study examines the reporting factors of crime against business in Korea and proposes a corresponding predictive model using machine learning. While many previous studies focused on the individual factors of theft victims, there is a lack of evidence on the reporting factors of crime against a business that serves the public good as opposed to those that protect private property. Therefore, we proposed a crime prevention model for the willingness factor of theft reporting in businesses. This study used data collected through the 2015 Commercial Crime Damage Survey conducted by the Korea Institute for Criminal Policy. It analyzed data from 834 businesses that had experienced theft during a 2016 crime investigation. The data showed a problem with unbalanced classes. To solve this problem, we jointly applied the Synthetic Minority Over Sampling Technique and the Tomek link techniques to the training data. Two prediction models were implemented. One was a statistical model using logistic regression and elastic net. The other involved a support vector machine model, tree-based machine learning models (e.g., random forest, extreme gradient boosting), and a stacking model. As a result, the features of theft price, invasion, and remedy, which are known to have significant effects on reporting theft offences, can be predicted as determinants of such offences in companies. Finally, we verified and compared the proposed predictive models using several popular metrics. Based on our evaluation of the importance of the features used in each model, we suggest a more accurate criterion for predicting var.

Enhancing VANET Security: Efficient Communication and Wormhole Attack Detection using VDTN Protocol and TD3 Algorithm

  • Vamshi Krishna. K;Ganesh Reddy K
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권1호
    • /
    • pp.233-262
    • /
    • 2024
  • Due to the rapid evolution of vehicular ad hoc networks (VANETs), effective communication and security are now essential components in providing secure and reliable vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication. However, due to their dynamic nature and potential threats, VANETs need to have strong security mechanisms. This paper presents a novel approach to improve VANET security by combining the Vehicular Delay-Tolerant Network (VDTN) protocol with the Deep Reinforcement Learning (DRL) technique known as the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. A store-carry-forward method is used by the VDTN protocol to resolve the problems caused by inconsistent connectivity and disturbances in VANETs. The TD3 algorithm is employed for capturing and detecting Worm Hole Attack (WHA) behaviors in VANETs, thereby enhancing security measures. By combining these components, it is possible to create trustworthy and effective communication channels as well as successfully detect and stop rushing attacks inside the VANET. Extensive evaluations and simulations demonstrate the effectiveness of the proposed approach, enhancing both security and communication efficiency.