• Title/Summary/Keyword: reinforcement algorithms

검색결과 148건 처리시간 0.028초

애드혹 센서 네트워크 수명 연장을 위한 Q-러닝 기반 에너지 균등 소비 라우팅 프로토콜 기법 (Equal Energy Consumption Routing Protocol Algorithm Based on Q-Learning for Extending the Lifespan of Ad-Hoc Sensor Network)

  • 김기상;김승욱
    • 정보처리학회논문지:컴퓨터 및 통신 시스템
    • /
    • 제10권10호
    • /
    • pp.269-276
    • /
    • 2021
  • 최근 스마트 센서는 다양한 환경에서 사용되고 있으며, 애드혹 센서 네트워크 (ASN) 구현에 대한 연구가 활발하게 진행되고 있다. 그러나 기존 센서 네트워크 라우팅 알고리즘은 특정 제어 문제에 초점을 맞추며 ASN 작업에 직접 적용할 수 없는 문제점이 있다. 본 논문에서는 Q-learning 기술을 이용한 새로운 라우팅 프로토콜을 제안하는데, 제안된 접근 방식의 주요 과제는 균형 잡힌 시스템 성능을 확보하면서 효율적인 에너지 할당을 통해 ASN의 수명을 연장하는 것이다. 제안된 방법의 특징은 다양한 환경적 요인을 고려하여 Q-learning 효과를 높이며, 특히 각 노드는 인접 노드의 Q 값을 자체 Q 테이블에 저장하여 데이터 전송이 실행될 때마다 Q 값이 업데이트되고 누적되어 최적의 라우팅 경로를 선택하는 것이다. 시뮬레이션 결과 제안된 방법이 에너지 효율적인 라우팅 경로를 선택할 수 있으며 기존 ASN 라우팅 프로토콜에 비해 우수한 네트워크 성능을 얻을 수 있음을 확인하였다.

A vibration based acoustic wave propagation technique for assessment of crack and corrosion induced damage in concrete structures

  • Kundu, Rahul Dev;Sasmal, Saptarshi
    • Structural Engineering and Mechanics
    • /
    • 제78권5호
    • /
    • pp.599-610
    • /
    • 2021
  • Early detection of small concrete crack or reinforcement corrosion is necessary for Structural Health Monitoring (SHM). Global vibration based methods are advantageous over local methods because of simple equipment installation and cost efficiency. Among vibration based techniques, FRF based methods are preferred over modal based methods. In this study, a new coupled method using frequency response function (FRF) and proper orthogonal modes (POM) is proposed by using the dynamic characteristic of a damaged beam. For the numerical simulation, wave finite element (WFE), coupled with traditional finite element (FE) method is used for effectively incorporating the damage related information and faster computation. As reported in literature, hybrid combination of wave function based wave finite element method and shape function based finite element method can addresses the mid frequency modelling difficulty as it utilises the advantages of both the methods. It also reduces the dynamic matrix dimension. The algorithms are implemented on a three-dimensional reinforced concrete beam. Damage is modelled and studied for two scenarios, i.e., crack in concrete and rebar corrosion. Single and multiple damage locations with different damage length are also considered. The proposed methodology is found to be very sensitive to both single- and multiple- damage while being computationally efficient at the same time. It is observed that the detection of damage due to corrosion is more challenging than that of concrete crack. The similarity index obtained from the damage parameters shows that it can be a very effective indicator for appropriately indicating initiation of damage in concrete structure in the form of spread corrosion or invisible crack.

A Bio-inspired Hybrid Cross-Layer Routing Protocol for Energy Preservation in WSN-Assisted IoT

  • Tandon, Aditya;Kumar, Pramod;Rishiwal, Vinay;Yadav, Mano;Yadav, Preeti
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권4호
    • /
    • pp.1317-1341
    • /
    • 2021
  • Nowadays, the Internet of Things (IoT) is adopted to enable effective and smooth communication among different networks. In some specific application, the Wireless Sensor Networks (WSN) are used in IoT to gather peculiar data without the interaction of human. The WSNs are self-organizing in nature, so it mostly prefer multi-hop data forwarding. Thus to achieve better communication, a cross-layer routing strategy is preferred. In the cross-layer routing strategy, the routing processed through three layers such as transport, data link, and physical layer. Even though effective communication achieved via a cross-layer routing strategy, energy is another constraint in WSN assisted IoT. Cluster-based communication is one of the most used strategies for effectively preserving energy in WSN routing. This paper proposes a Bio-inspired cross-layer routing (BiHCLR) protocol to achieve effective and energy preserving routing in WSN assisted IoT. Initially, the deployed sensor nodes are arranged in the form of a grid as per the grid-based routing strategy. Then to enable energy preservation in BiHCLR, the fuzzy logic approach is executed to select the Cluster Head (CH) for every cell of the grid. Then a hybrid bio-inspired algorithm is used to select the routing path. The hybrid algorithm combines moth search and Salp Swarm optimization techniques. The performance of the proposed BiHCLR is evaluated based on the Quality of Service (QoS) analysis in terms of Packet loss, error bit rate, transmission delay, lifetime of network, buffer occupancy and throughput. Then these performances are validated based on comparison with conventional routing strategies like Fuzzy-rule-based Energy Efficient Clustering and Immune-Inspired Routing (FEEC-IIR), Neuro-Fuzzy- Emperor Penguin Optimization (NF-EPO), Fuzzy Reinforcement Learning-based Data Gathering (FRLDG) and Hierarchical Energy Efficient Data gathering (HEED). Ultimately the performance of the proposed BiHCLR outperforms all other conventional techniques.

해군분석모델용 AI-CGF를 위한 시나리오 생성 모델 설계(I): 진화학습 (Design of Scenario Creation Model for AI-CGF based on Naval Operations, Resources Analysis Model(I): Evolutionary Learning)

  • 김현근;강정석;박강문;김재우;김장현;박범준;지승도
    • 한국군사과학기술학회지
    • /
    • 제25권6호
    • /
    • pp.617-627
    • /
    • 2022
  • Military training is an essential item for the fundamental problem of war. However, there has always been a problem that many resources are consumed, causing spatial and environmental pollution. The concepts of defense modeling and simulation and CGF(Computer Generated Force) using computer technology began to appear to improve this problem. The Naval Operations, Resources Analysis Model(NORAM) developed by the Republic of Korea Navy is also a DEVS(Discrete Event Simulation)-based naval virtual force analysis model. The current NORAM is a battle experiment conducted by an operator, and parameter values such as maneuver and armament operation for individual objects for each situation are evaluated. In spite of our research conducted evolutionary, supervised, reinforcement learning, in this paper, we introduce our design of a scenario creation model based on evolutionary learning using genetic algorithms. For verification, the NORAM is loaded with our model to analyze wartime engagements. Human-level tactical scenario creation capability is secured by automatically generating enemy tactical scenarios for human-designed Blue Army tactical scenarios.

합 기반의 전건부를 가지는 뉴로-퍼지 시스템 설계 (Design of a Neuro-Fuzzy System Using Union-Based Rule Antecedent)

  • 한창욱;이돈규
    • 정보처리학회 논문지
    • /
    • 제13권2호
    • /
    • pp.13-17
    • /
    • 2024
  • 본 논문에서는 규칙의 수를 줄여 간결한 지식 기반을 보장할 수 있는 합 기반의 전건부를 가지는 뉴로-퍼지 제어기를 제안하였다. 제안된 뉴로-퍼지 제어기는 모든 입력 변수의 AND 조합을 전건부로 하는 구조의 퍼지 규칙보다 더 큰 입력 영역을 커버하기 위해 전건부에 입력 퍼지 집합의 합집합 연산을 허용하였다. 이러한 뉴로-퍼지 제어기를 구성하기 위해 본 논문에서는 OR 및 AND 퍼지 뉴런으로 구성된 multiple-term unified logic processor (MULP)를 고려하였다. 이러한 OR 및 AND 퍼지 뉴런은 조정 가능한 연결 강도 집합을 가지므로 학습을 통하여 최적의 연결 강도 집합을 찾을 수 있다. 초기 최적화 단계에서 유전 알고리즘은 제안된 뉴로 퍼지 제어기의 최적화된 이진 구조를 구성하고, 이후 확률에 기반한 강화 학습은 성능 지수를 더욱 향상시켜서 유전 알고리즘에 의해 최적화된 제어기의 이진 연결을 개선하였다. 역진자 시스템을 제어하기 위한 모의실험 및 실험을 통해 제안된 방법의 유효성을 검증하였다.

BIM기반 초기 설계 단계 철근 물량 산출 프로세스 구축 (Establishment of Rebar Quantity Estimation in BIM-based Initial Design Phase)

  • 송치호;김치경;이시은;최현철
    • 한국전산구조공학회논문집
    • /
    • 제29권5호
    • /
    • pp.447-454
    • /
    • 2016
  • 최근에 각광받고 있는 3D BIM기반 철근 물량 산출 현황을 살펴보면 상용 BIM 도구들이 철근 모델링 기능을 제공하지만 그 기능을 이용해서 철근을 모델링하려고 하면 방대한 양의 모델링 시간이 소요가 되어 현재 실용적으로 사용할 수 있는 BIM 소프트웨어는 없다. 본 연구에서는 BIM 기반 설계 현업에서 실용적인 철근 물량 산출 프로세스를 정리 및 제시하고, 초기설계단계의 단면 정보가 없는 상태에서 기존의 2D방식에 비하여 보다 정확하게 기동, 보, 슬래브, 벽체의 철근 단면 정보를 자동으로 배근해주는 프로그램인 Rebar Automatic Arrangement Program(이하 RAAP)을 개발하였다. 본 연구 결과는 건설프로젝트의 초기 설계단계에서 설계 진행상 구조 부재 단면 설계가 완료되지 않은 상태의 골조 물량 산출 시 적은 노력으로 기존의 2D기반 방식보다 신뢰성을 향상시킨 물량을 산출할 수 있는 기술로 발전시키는데 의미가 있으며, 합리적인 초기 설계단계의 물량 산출 프로그램을 구축할 수 있는 기초 연구로서 활용될 수 있을 것으로 기대된다.

360도 카메라 기반 건설현장 철근 배근 정보 원격 계측 기법 개발 (Development of Remote Measurement Method for Reinforcement Information in Construction Field Using 360 Degrees Camera)

  • 이명훈;우욱용;최하진;강수민;최경규
    • 한국구조물진단유지관리공학회 논문집
    • /
    • 제26권6호
    • /
    • pp.157-166
    • /
    • 2022
  • 철근 콘크리트 구조 건설현장에서 육안 검사 방식으로 수행되는 현재 단계의 구조감리는 그 필요성에 비하여 매우 노동 집약적이기에 현실적으로 현장의 모든 상황을 파악하기에 제한적이며, 검사자의 주관성도 배제될 수 없다. 따라서 본 연구는 철근을 대상으로 한 구조감리의 효율성 개선을 위해 360° 카메라를 통해 수집한 RGB 및 Depth 데이터 기반 3D model을 이용하여 배근 간격을 도출하고 실측값과의 비교를 통해 정확도를 검증하였다. 소규모 현장(약 265 m2)의 12개 지점에 대해 계측을 수행하였으며, 지점당 스캔시간은 약 20초, 이동 및 설치시간을 포함한 총 계측 시간은 약 15분이 소요되었다. 계측된 데이터는 SLAM 알고리즘을 통하여 RGB-based 3D model과 3D point cloud model을 생성하였으며, 각각의 모델에서의 계측값을 실측값과 비교하여 정확도 검증을 진행하였다. RGB-based 3D model과 3D point cloud model은 각각 10mm, 0.1mm의 최소분해능을 갖으며, 각 모델로부터 계측된 철근의 배근 간격 은 의 오차는 최대 28.4%, 최소 3.1% (RGB-based 3D model) 최대 10.8%, 최소 0.3% (3D point cloud model)로 확인되었다. 본 연구를 토대로 추후 자동화 기반의 원격구조 감리 기술개발을 통하여 현장적용 및 분석의 효율성을 증대시키고자 한다.

폭소노미 사이트를 위한 랭킹 프레임워크 설계: 시맨틱 그래프기반 접근 (A Folksonomy Ranking Framework: A Semantic Graph-based Approach)

  • 박현정;노상규
    • Asia pacific journal of information systems
    • /
    • 제21권2호
    • /
    • pp.89-116
    • /
    • 2011
  • In collaborative tagging systems such as Delicious.com and Flickr.com, users assign keywords or tags to their uploaded resources, such as bookmarks and pictures, for their future use or sharing purposes. The collection of resources and tags generated by a user is called a personomy, and the collection of all personomies constitutes the folksonomy. The most significant need of the folksonomy users Is to efficiently find useful resources or experts on specific topics. An excellent ranking algorithm would assign higher ranking to more useful resources or experts. What resources are considered useful In a folksonomic system? Does a standard superior to frequency or freshness exist? The resource recommended by more users with mere expertise should be worthy of attention. This ranking paradigm can be implemented through a graph-based ranking algorithm. Two well-known representatives of such a paradigm are Page Rank by Google and HITS(Hypertext Induced Topic Selection) by Kleinberg. Both Page Rank and HITS assign a higher evaluation score to pages linked to more higher-scored pages. HITS differs from PageRank in that it utilizes two kinds of scores: authority and hub scores. The ranking objects of these pages are limited to Web pages, whereas the ranking objects of a folksonomic system are somewhat heterogeneous(i.e., users, resources, and tags). Therefore, uniform application of the voting notion of PageRank and HITS based on the links to a folksonomy would be unreasonable, In a folksonomic system, each link corresponding to a property can have an opposite direction, depending on whether the property is an active or a passive voice. The current research stems from the Idea that a graph-based ranking algorithm could be applied to the folksonomic system using the concept of mutual Interactions between entitles, rather than the voting notion of PageRank or HITS. The concept of mutual interactions, proposed for ranking the Semantic Web resources, enables the calculation of importance scores of various resources unaffected by link directions. The weights of a property representing the mutual interaction between classes are assigned depending on the relative significance of the property to the resource importance of each class. This class-oriented approach is based on the fact that, in the Semantic Web, there are many heterogeneous classes; thus, applying a different appraisal standard for each class is more reasonable. This is similar to the evaluation method of humans, where different items are assigned specific weights, which are then summed up to determine the weighted average. We can check for missing properties more easily with this approach than with other predicate-oriented approaches. A user of a tagging system usually assigns more than one tags to the same resource, and there can be more than one tags with the same subjectivity and objectivity. In the case that many users assign similar tags to the same resource, grading the users differently depending on the assignment order becomes necessary. This idea comes from the studies in psychology wherein expertise involves the ability to select the most relevant information for achieving a goal. An expert should be someone who not only has a large collection of documents annotated with a particular tag, but also tends to add documents of high quality to his/her collections. Such documents are identified by the number, as well as the expertise, of users who have the same documents in their collections. In other words, there is a relationship of mutual reinforcement between the expertise of a user and the quality of a document. In addition, there is a need to rank entities related more closely to a certain entity. Considering the property of social media that ensures the popularity of a topic is temporary, recent data should have more weight than old data. We propose a comprehensive folksonomy ranking framework in which all these considerations are dealt with and that can be easily customized to each folksonomy site for ranking purposes. To examine the validity of our ranking algorithm and show the mechanism of adjusting property, time, and expertise weights, we first use a dataset designed for analyzing the effect of each ranking factor independently. We then show the ranking results of a real folksonomy site, with the ranking factors combined. Because the ground truth of a given dataset is not known when it comes to ranking, we inject simulated data whose ranking results can be predicted into the real dataset and compare the ranking results of our algorithm with that of a previous HITS-based algorithm. Our semantic ranking algorithm based on the concept of mutual interaction seems to be preferable to the HITS-based algorithm as a flexible folksonomy ranking framework. Some concrete points of difference are as follows. First, with the time concept applied to the property weights, our algorithm shows superior performance in lowering the scores of older data and raising the scores of newer data. Second, applying the time concept to the expertise weights, as well as to the property weights, our algorithm controls the conflicting influence of expertise weights and enhances overall consistency of time-valued ranking. The expertise weights of the previous study can act as an obstacle to the time-valued ranking because the number of followers increases as time goes on. Third, many new properties and classes can be included in our framework. The previous HITS-based algorithm, based on the voting notion, loses ground in the situation where the domain consists of more than two classes, or where other important properties, such as "sent through twitter" or "registered as a friend," are added to the domain. Forth, there is a big difference in the calculation time and memory use between the two kinds of algorithms. While the matrix multiplication of two matrices, has to be executed twice for the previous HITS-based algorithm, this is unnecessary with our algorithm. In our ranking framework, various folksonomy ranking policies can be expressed with the ranking factors combined and our approach can work, even if the folksonomy site is not implemented with Semantic Web languages. Above all, the time weight proposed in this paper will be applicable to various domains, including social media, where time value is considered important.