• Title/Summary/Keyword: Very large real-time data

Search Result 132, Processing Time 0.024 seconds

Outlier Detection in Time Series Monitoring Datasets using Rule Based and Correlation Analysis Method (규칙기반 및 상관분석 방법을 이용한 시계열 계측 데이터의 이상치 판정)

  • Jeon, Jesung;Koo, Jakap;Park, Changmok
    • Journal of the Korean GEO-environmental Society
    • /
    • v.16 no.5
    • /
    • pp.43-53
    • /
    • 2015
  • In this study, detection methods of outlier in various monitoring data that fit into big data category were developed and outlier detections were conducted for both artificial data and real field monitoring data. Rule-based methods applied rate of change and probability of error for monitoring data are effective to detect a large-scale short faults and constant faults having no change within a certain period. There are however, problems with misjudgement that consider the normal data with a large scale variation as outlier caused by using independent single dataset. Rule-based methods for noise faults detection have a limit to application of real monitoring data due to the problem with a choice of proper window size of data and finding of threshold for outlier judgment. A correlation analysis among different two datasets were very effective to detect localized outlier and abnormal variation for short and long-term monitoring dataset if reasonable range of training data could be selected.

A Study on the Low Frequency Oscillation Using PMU Measurement Data (PMU 데이터를 이용한 저주파 진동분석 연구)

  • Kim, Yonghak;Nam, Suchul;Ko, Baekkyeong;Kang, Sungbum;Shim, Kwansik
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.6 no.3
    • /
    • pp.245-252
    • /
    • 2020
  • It is very important to evaluate on/off-line stability to operate the power system stably and economically. Until now, we have continuously secured the operation reliability of the power system through the evaluation of transient, voltage and small signal stability. This paper proposes that it is possible to operate in KWAMS by applying the multi-section analysis and subspace methods and verifying the reliability of the algorithms to directly estimate the dominant oscillation mode of the power system from the signal waveform acquired from the phasor measurement units. In addition, this paper shows that the dominant oscillation mode can be detected from real-time measurement data in power systems. Therefore, if we can monitor the state of the power system in real time, it is possible to avoid a large-scale power outage by knowing the possibility of the power system accident in advance.

Non-Causal Filter의 PC-NC에의 응용

  • 장현상;최종률
    • Proceedings of the Korean Society of Precision Engineering Conference
    • /
    • 1995.10a
    • /
    • pp.1039-1042
    • /
    • 1995
  • In real time application such as motion control, it is hard to find the application of non-causal filtering due to its need for future position data, even though it shows wide usage in off-line digital signal processing. Recently, some of motion control areas such as learning and repetitive control use non-causal filtering technique in their application. these kinds of zero-lag non-causal filter application are very usful not only to reduce the machine vibration, but also to increase control accuracy with comparatively less work. In this paper, genuine method to implement zero-lag non-causal filter in a CNC is introduced. Also the variation of this implementation for the learning operation is suggested to give the NC better control performance for a specific job. By adopting the new NC architecture call Soft-NC, all these implementions are made possible here, and especially large memory requirement which hinders their usage for many years is no longer barrier in their real world application.

  • PDF

PPFP(Push and Pop Frequent Pattern Mining): A Novel Frequent Pattern Mining Method for Bigdata Frequent Pattern Mining (PPFP(Push and Pop Frequent Pattern Mining): 빅데이터 패턴 분석을 위한 새로운 빈발 패턴 마이닝 방법)

  • Lee, Jung-Hun;Min, Youn-A
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.12
    • /
    • pp.623-634
    • /
    • 2016
  • Most of existing frequent pattern mining methods address time efficiency and greatly rely on the primary memory. However, in the era of big data, the size of real-world databases to mined is exponentially increasing, and hence the primary memory is not sufficient enough to mine for frequent patterns from large real-world data sets. To solve this problem, there are some researches for frequent pattern mining method based on disk, but the processing time compared to the memory based methods took very time consuming. There are some researches to improve scalability of frequent pattern mining, but their processes are very time consuming compare to the memory based methods. In this paper, we present PPFP as a novel disk-based approach for mining frequent itemset from big data; and hence we reduced the main memory size bottleneck. PPFP algorithm is based on FP-growth method which is one of the most popular and efficient frequent pattern mining approaches. The mining with PPFP consists of two setps. (1) Constructing an IFP-tree: After construct FP-tree, we assign index number for each node in FP-tree with novel index numbering method, and then insert the indexed FP-tree (IFP-tree) into disk as IFP-table. (2) Mining frequent patterns with PPFP: Mine frequent patterns by expending patterns using stack based PUSH-POP method (PPFP method). Through this new approach, by using a very small amount of memory for recursive and time consuming operation in mining process, we improved the scalability and time efficiency of the frequent pattern mining. And the reported test results demonstrate them.

Controller Design for Object Tracking with an Active Camera (능동 카메라 기반의 물체 추적 제어기 설계)

  • Youn, Su-Jin;Choi, Goon-Ho
    • Journal of the Semiconductor & Display Technology
    • /
    • v.10 no.1
    • /
    • pp.83-89
    • /
    • 2011
  • In the case of the tracking system with an active camera, it is very difficult to guarantee real-time processing due to the attribute of vision system which handles large amounts of data at once and has time delay to process. The reliability of the processed result is also badly influenced by the slow sampling time and uncertainty caused by the image processing. In this paper, we figure out dynamic characteristics of pixels reflected on the image plane and derive the mathematical model of the vision tracking system which includes the actuating part and the image processing part. Based on this model, we find a controller that stabilizes the system and enhances the tracking performance to track a target rapidly. The centroid is used as the position index of moving object and the DC motor in the actuating part is controlled to keep the identified centroid at the center point of the image plane.

A Study on Document Storage and Information Retrieval for Educational Informatization (교육정보화를 위한 문서저장 및 정보검색에 관한 연구)

  • Kang, Mu-Yeong;Lee, Sang-Gu
    • Journal of The Korean Association of Information Education
    • /
    • v.3 no.1
    • /
    • pp.1-12
    • /
    • 1999
  • By recent advances in the information management tools, Education Softwares and Study Wares.are rapidly developed and distributed over the Internet very fast. As Education Administrative Information systems are developed, information generated from the education field is growing large and fast. In this paper, we suggest a method of efficiently storing and retrieving such information in the large data basis, in real time. We also demonstrate an information system by the proposed method.

  • PDF

USING GIS TO IMPROVE ROADWAY CONSTRUCTION PLANNING

  • Sanghyeok Kang;Jongwon Seo
    • International conference on construction engineering and project management
    • /
    • 2005.10a
    • /
    • pp.643-648
    • /
    • 2005
  • The planning process for roadway construction involves a large amount of information on design, construction methods, quantities, unit costs, production rates, and site conditions. Therefore, it is very important to acquire, manage, and process the necessary information efficiently to produce a rigorous construction plan. GIS (Geographic Information System) is a very effective tool for integrating and managing various types of information including spatial and non-spatial data required for roadway construction planning. This paper proposes a GIS-based system for improving roadway construction planning with its 'Interactive Space Scheduling' and 'Operation Level Planning' functions. The proposed system can assist construction planners in a unique way by integrating design and construction information and creating modularized design elements for space scheduling in real time. It is expected that the proposed system could improve the efficiency of roadway construction planning.

  • PDF

A study on hypothetical switching software through of the analysis of failure data (고장 데이터 분석을 통한 교환 소프트웨어 특성 연구)

  • 이재기;신상권;이영목
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.23 no.8
    • /
    • pp.1915-1925
    • /
    • 1998
  • The switching system software is large scale, real-time multi-task system which requires high reliability. The reliability assessment of large-scale software is very important for the success of software development project. For this raeson, the software quality measurement is much more important. In this paper, we have learned about the software reliability, metho of the analysis of failure data and estimation of software quality. To estimate the software reliability, using the failure data found during of the system test. We apply the two software reliability growth models, named Goel-Okumoto(G-O) and S-shaped model, to estimate the software reliability. Also, we compared with the results and we reviewed fully not only development cycle but validation and verification of the test data, for each software versions. This paper presents a software reliability model that suitale the software development project and the activeity of quality control for the switching system.

  • PDF

Detecting Jaywalking Using the YOLOv5 Model

  • Kim, Hyun-Tae;Lee, Sang-Hyun
    • International Journal of Advanced Culture Technology
    • /
    • v.10 no.2
    • /
    • pp.300-306
    • /
    • 2022
  • Currently, Korea is building traffic infrastructure using Intelligent Transport Systems (ITS), but the pedestrian traffic accident rate is very high. The purpose of this paper is to prevent the risk of traffic accidents by jaywalking pedestrians. The development of this study aims to detect pedestrians who trespass using the public data set provided by the Artificial Intelligence Hub (AIHub). The data set uses training data: 673,150 pieces and validation data: 131,385 pieces, and the types include snow, rain, fog, etc., and there is a total of 7 types including passenger cars, small buses, large buses, trucks, large trailers, motorcycles, and pedestrians. has a class format of Learning is carried out using YOLOv5 as an implementation model, and as an object detection and edge detection method of an input image, a canny edge model is applied to classify and visualize human objects within the detected road boundary range. In this study, it was designed and implemented to detect pedestrians using the deep learning-based YOLOv5 model. As the final result, the mAP 0.5 showed a real-time detection rate of 61% and 114.9 fps at 338 epochs using the YOLOv5 model.

A Real-Time JPEG2000 Codec Implementation on ARM9 Processor (ARM9 프로세서용 실시간 JPEG2000 코덱의 구현)

  • Kim, Young-Tae;Cho, Shi-Won;Lee, Dong-Wook
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.8 no.3
    • /
    • pp.149-155
    • /
    • 2007
  • In this paper, we propose an real-time implementation of JPEG2000 codec on the ARM9 processor. The implemented codec is designed to separate control codes from data management codes in order to use effectively the system resources such as processor and memory. Especially, in embedded situations like cellular phones it is very important to provide good services using limited processor and internal memory. Since ARM9 series processors do not provide floating-point, large amount of computational time is required to perform the operation which needs highly repetitive floating-point computations like DWT(discrete wavelet transform). The proposed codec was programed using fixed-point to overcome this weakness. Also code optimization considering cache memory was applied to further improve the computational speed.

  • PDF