• Title/Summary/Keyword: data architecture

Search Result 6,976, Processing Time 0.036 seconds

Data anomaly detection for structural health monitoring using a combination network of GANomaly and CNN

  • Liu, Gaoyang;Niu, Yanbo;Zhao, Weijian;Duan, Yuanfeng;Shu, Jiangpeng
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.53-62
    • /
    • 2022
  • The deployment of advanced structural health monitoring (SHM) systems in large-scale civil structures collects large amounts of data. Note that these data may contain multiple types of anomalies (e.g., missing, minor, outlier, etc.) caused by harsh environment, sensor faults, transfer omission and other factors. These anomalies seriously affect the evaluation of structural performance. Therefore, the effective analysis and mining of SHM data is an extremely important task. Inspired by the deep learning paradigm, this study develops a novel generative adversarial network (GAN) and convolutional neural network (CNN)-based data anomaly detection approach for SHM. The framework of the proposed approach includes three modules : (a) A three-channel input is established based on fast Fourier transform (FFT) and Gramian angular field (GAF) method; (b) A GANomaly is introduced and trained to extract features from normal samples alone for class-imbalanced problems; (c) Based on the output of GANomaly, a CNN is employed to distinguish the types of anomalies. In addition, a dataset-oriented method (i.e., multistage sampling) is adopted to obtain the optimal sampling ratios between all different samples. The proposed approach is tested with acceleration data from an SHM system of a long-span bridge. The results show that the proposed approach has a higher accuracy in detecting the multi-pattern anomalies of SHM data.

A study on the improvement of MND-AMM for the expanded application to the architecture development of infra systems (기반체계 아키텍처 개발을 위한 MND-AMM 개선 연구)

  • Yoon, Tae Hun;Kim, Ui Hwan
    • Journal of the Korean Society of Systems Engineering
    • /
    • v.11 no.1
    • /
    • pp.25-31
    • /
    • 2015
  • Under the relevant regulations, it is required to develop a system architecture for the research and development of the information and communication infra system ; however, the national defense architecture development guides and MND-AF, which provide the instructions on the development and utilization of a system architecture, are still limited to the ITA level and merely providing the guidelines for developing the information system centric architecture. Thus, it is evident that we need a suitable architecture development methodology that corresponds to the growing needs for the communication infra system architecture, as well as the general weapon system architecture. Improving a meta-model is the core of improving a architecture framework. It determines a contents of a architecture and it influences a efficiency and a effectiveness of a architecture. The meta-model of the architecture framework must reflect concerns of various stakeholders and provide a traceability among them. Also, it should be easy to develop and use the architecture by securing the feasibility of the logical relationships and eliminating the duplication of the data inside the architecture. It is implemented through the development of the data-centric architecture and achieved through the "Fit-for-purpose" concept.

A Dynamic Data Grid Replication Strategy Based on Internet Architecture (인터넷 구조 기반의 동적 데이터 그리드 복제 정책)

  • Kim, Jun-Sang;Lee, Won-Joo;Jeon, Chang-Ho
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.45 no.3
    • /
    • pp.1-6
    • /
    • 2008
  • Data grid shares distributed large data via wide-band network. Such grid environment consumes much time for large data transmission. Because it is implemented on internet as physical network. Many replication strategies were proposed for solving this problem, but they are not optimal in real Data grid environments. Because they were proposed that based on logical topology without consideration of real internet architecture. Grid data access time is largely influenced by internet architecture as physical network of Data grid. In this paper, we propose a new data replication strategy RSIA(Replication Strategy based on Internet Architecture) based on internet architecture. The RSIA places replicas considering structural hierarchy in each element of internet, and avoid the performance bottlenecks to reduce system performance degradation when a data transfer. Through simulation, we show that the proposed RSIA data replication strategy improves the performance of Data Grid environment compared with previous strategies.

The Design and Implementation of Ontology for Simulation based Architecture Framework (ONT-AF) in Military Domain (SBA AF의 구축을 지원하는 온톨로지의 설계 및 구현(ONT-SAF))

  • Kwon, Youngmin;Sohn, Mye;Lee, Wookey
    • Journal of Information Technology and Architecture
    • /
    • v.9 no.3
    • /
    • pp.233-241
    • /
    • 2012
  • Architecture framework (AF) is a guideline to define components needed to develop and operate enterprise architecture (EA), and to define relationships among the components. There are many architecture frameworks to operate EA of governments and businesses such as Zachman framework, DoDAF, TOGAF, FEAF, and TEAF. DoDAF is the most representative AF to support the development of the EA in the military domain. DoDAF is composed of eight viewpoints and 40 views that are affiliated with the viewpoints. To develop an AF for a specific goal, system architects decide a set of views. Furthermore, they determine data that are needed for a view modeling. However, views and data in DoDAF are structurally inter-related explicitly and/or implicitly. So, developing an AF for a specific goal is going to be a project to be carried out over a long haul. To reduce the burden of its development, in this paper, we develop ONT-SAF (Ontology for DoDAF) that can infer inter-relationships like referential and transitive relationships and the sequences among the views. Furthermore, to promote reusability and consistency of the views and the data within an AF, we adopt the view-data separation strategy. ONT-DAT contains classes like 'viewpoint', 'view', 'data', 'expression method', and 'reference model', and 11 properties including 'hasView.' To prove the effectiveness of ONT-SAF, we perform a case study.

An Executive Information Systems Architecture for the Air Force Using Data Warehousing (데이터 웨어하우징을 이용한 공군 EIS 아키텍처)

  • Choi, Jun-Seob;Suh, Eui-Ho;Suh, Chang-Kyo
    • Asia pacific journal of information systems
    • /
    • v.8 no.3
    • /
    • pp.1-20
    • /
    • 1998
  • We propose an Executive Information Systems (EIS) Architecture for chief officers of the Air Force using data warehousing method. This architecture has two main proposes. The one is to provide the information for chief officers to control and command their organizations effectively by analyzing operation data at normal times. The other is to provide chief officers with the information about current situation so that they may make right and rapid decisions at emergency. The architecture introduced here is one that analyzes operational trends as well as current trends in a hierarchical organization environment. System analysis and design techniques for the Air Force EIS such as data flow diagram, system structure, entity-relational diagram, and third normal form of relational database were presented. After prototype screens are demonstrated, benefits of new EIS architecture were also discussed as a conclusion.

  • PDF

Lambda Architecture Used Apache Kudu and Impala (Apache Kudu와 Impala를 활용한 Lambda Architecture 설계)

  • Hwang, Yun-Young;Lee, Pil-Won;Shin, Yong-Tae
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.9
    • /
    • pp.207-212
    • /
    • 2020
  • The amount of data has increased significantly due to advances in technology, and various big data processing platforms are emerging, to handle it. Among them, the most widely used platform is Hadoop developed by the Apache Software Foundation, and Hadoop is also used in the IoT field. However, the existing Hadoop-based IoT sensor data collection and analysis environment has a problem of overloading the name node due to HDFS' Small File, which is Hadoop's core project, and it is impossible to update or delete the imported data. This paper uses Apache Kudu and Impala to design Lambda Architecture. The proposed Architecture classifies IoT sensor data into Cold-Data and Hot-Data, stores it in storage according to each personality, and uses Batch-View created through Batch and Real-time View generated through Apache Kudu and Impala to solve problems in the existing Hadoop-based IoT sensor data collection analysis environment and shorten the time users access to the analyzed data.

Bayesian forecasting approach for structure response prediction and load effect separation of a revolving auditorium

  • Ma, Zhi;Yun, Chung-Bang;Shen, Yan-Bin;Yu, Feng;Wan, Hua-Ping;Luo, Yao-Zhi
    • Smart Structures and Systems
    • /
    • v.24 no.4
    • /
    • pp.507-524
    • /
    • 2019
  • A Bayesian dynamic linear model (BDLM) is presented for a data-driven analysis for response prediction and load effect separation of a revolving auditorium structure, where the main loads are self-weight and dead loads, temperature load, and audience load. Analyses are carried out based on the long-term monitoring data for static strains on several key members of the structure. Three improvements are introduced to the ordinary regression BDLM, which are a classificatory regression term to address the temporary audience load effect, improved inference for the variance of observation noise to be updated continuously, and component discount factors for effective load effect separation. The effects of those improvements are evaluated regarding the root mean square errors, standard deviations, and 95% confidence intervals of the predictions. Bayes factors are used for evaluating the probability distributions of the predictions, which are essential to structural condition assessments, such as outlier identification and reliability analysis. The performance of the present BDLM has been successfully verified based on the simulated data and the real data obtained from the structural health monitoring system installed on the revolving structure.

An Architecture for Implementing Executive Information System Using Data Warehouse (데이터 웨어하우스를 이용한 임원정보시스템 구축용 아키텍쳐)

  • Lee, Hui-Seok;Hong, Eui-Gi;Kim, Tae-Hun
    • Asia pacific journal of information systems
    • /
    • v.7 no.1
    • /
    • pp.7-24
    • /
    • 1997
  • Executive information system (EIS) is a computer-based information system that supports decision makings ana management activities for senior executives. Data warehouse is a database that receives data copies from legacy systems and external data sources. Data warehouse is typically optimized for decision supports and can be an attractive solution for EIS implementation. This paper proposes an architecture for implementing EIS by the use of data warehouse. The architecture consists of ten implementation layers. Interrelationships among these layers are investigated for an effective EIS implementation. An EIS prototype for a real-fife enterprise is implemented to demonstrate the usefulness of the proposed architecture.

  • PDF

An architecture for implementing executive information system using data warehouse (데이터 웨어하우스를 이용한 임원정보시스템 구축용 아키텍쳐)

  • 이희석;홍의기;김태훈
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1996.10a
    • /
    • pp.254-257
    • /
    • 1996
  • Executive information system (EIS) is a computer-based information system that supports decision makings and management activities for senior executives. Data warehouse is a database that receives data copies from legacy systems and external data sources. Data warehouse is typically optimized for decision supports and can be an attractive solution for EIS implementation. This paper proposes an architecture for implementing EIS by the use of data warehouse. The architecture consists of ten implementation layers. Interrelationships among these layers are investigated for an effective EIS implementation. An EIS prototype for a real-life enterprise is implemented to demonstrate the usefulness of the proposed architecture.

  • PDF

Database Reverse Engineering Using Master Data in Microservice Architecture (마스터 데이터를 활용한 마이크로 서비스 아키텍처에서의 데이터베이스 리버스 엔지니어링)

  • Shin, Kwang-chul;Lee, Choon Y.
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.5
    • /
    • pp.523-532
    • /
    • 2019
  • Microservice architecture focuses on dividing it into small and lightweight services to build for the purpose of performing very close business functions. So it tends to concentrate only on agility, productivity, reliability, and ease of deployment of software development. Microservice architecture considers database as just a file or storage for storing and extracting data, emphasizing that data quality can be sacrificed for convenience and scalability of software development. Database reverse engineering for understanding database structure and data semantics is needed for data utilization for business decision making. However, it is difficult that reverse database engineering is applied in microservice architecture that neglects data quality. This study proposes database reverse engineering method that utilizes master data to restore the conceptual data model as a solution. The proposed method is applied to the return service database implemented by microservice architecture and verified its applicability.