• Title/Summary/Keyword: data algorithm system

Search Result 6,152, Processing Time 0.035 seconds

An Abnormal Worker Movement Detection System Based on Data Stream Processing and Hierarchical Clustering

  • Duong, Dat Van Anh;Lan, Doi Thi;Yoon, Seokhoon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.4
    • /
    • pp.88-95
    • /
    • 2022
  • Detecting anomalies in human movement is an important task in industrial applications, such as monitoring industrial disasters or accidents and recognizing unauthorized factory intruders. In this paper, we propose an abnormal worker movement detection system based on data stream processing and hierarchical clustering. In the proposed system, Apache Spark is used for streaming the location data of people. A hierarchical clustering-based anomalous trajectory detection algorithm is designed for detecting anomalies in human movement. The algorithm is integrated into Apache Spark for detecting anomalies from location data. Specifically, the location information is streamed to Apache Spark using the message queuing telemetry transport protocol. Then, Apache Spark processes and stores location data in a data frame. When there is a request from a client, the processed data in the data frame is taken and put into the proposed algorithm for detecting anomalies. A real mobility trace of people is used to evaluate the proposed system. The obtained results show that the system has high performance and can be used for a wide range of industrial applications.

A Study on Efficient Cluster Analysis of Bio-Data Using MapReduce Framework

  • Yoo, Sowol;Lee, Kwangok;Bae, Sanghyun
    • Journal of Integrative Natural Science
    • /
    • v.7 no.1
    • /
    • pp.57-61
    • /
    • 2014
  • This study measured the stream data from the several sensors, and stores the database in MapReduce framework environment, and it aims to design system with the small performance and cluster analysis error rate through the KMSVM algorithm. Through the KM-SVM algorithm, the cluster analysis effective data was used for U-health system. In the results of experiment by using 2003 data sets obtained from 52 test subjects, the k-NN algorithm showed 79.29% cluster analysis accuracy, K-means algorithm showed 87.15 cluster analysis accuracy, and SVM algorithm showed 83.72%, KM-SVM showed 90.72%. As a result, the process speed and cluster analysis effective ratio of KM-SVM algorithm was better.

A Study of Boiler Control Loop Simulation in Thermal Power Plant (화력발전소 보일러 제어루프의 시뮬레이션에 관한 연구)

  • Lee, J.H.;Lee, C.J.
    • Proceedings of the KIEE Conference
    • /
    • 1999.07b
    • /
    • pp.868-870
    • /
    • 1999
  • In this paper we obtain a discrete mathmatical model of a Boiler control system from expermental data, we find appropriate input signal and parameter estimation algorithm for identification of the Boiler control system in power plant. Under these conditions experimental data are collected from real system and parameters are estimated by the Recursive Least Square algorithm. The computer simulation results show the parameter estimation algorithm for identification and the effectiveness of controller design of the Boiler control system.

  • PDF

A Implementation of Optimal Multiple Classification System using Data Mining for Genome Analysis

  • Jeong, Yu-Jeong;Choi, Gwang-Mi
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.12
    • /
    • pp.43-48
    • /
    • 2018
  • In this paper, more efficient classification result could be obtained by applying the combination of the Hidden Markov Model and SVM Model to HMSV algorithm gene expression data which simulated the stochastic flow of gene data and clustering it. In this paper, we verified the HMSV algorithm that combines independently learned algorithms. To prove that this paper is superior to other papers, we tested the sensitivity and specificity of the most commonly used classification criteria. As a result, the K-means is 71% and the SOM is 68%. The proposed HMSV algorithm is 85%. These results are stable and high. It can be seen that this is better classified than using a general classification algorithm. The algorithm proposed in this paper is a stochastic modeling of the generation process of the characteristics included in the signal, and a good recognition rate can be obtained with a small amount of calculation, so it will be useful to study the relationship with diseases by showing fast and effective performance improvement with an algorithm that clusters nodes by simulating the stochastic flow of Gene Data through data mining of BigData.

Design of a Consistency Algorithm for VOD Streaming Data (VOD 스트리밍 데이터를 위한 Consistency 알고리즘 설계)

  • Jang Seung-Ju
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.10 no.8
    • /
    • pp.1414-1421
    • /
    • 2006
  • This paper proposes a consistency algorithm that is able to serve streaming data efficiently in VOD system. The media data is stripping into several pieces of data by the Round Robin method in order to media data service. The barrier mechanism is changed into the minimum data factor(SH. GOP) in this paper. The shared memory is allocated at one host with one fragment size. Data is combined with RTP packet transmission data format using barrier mechanism. I experiment and program the suggested algorithm on the VOD system.

Real-time Reflection Light Detection Algorithm using Pixel Clustering Data (Pixel 군집화 Data를 이용한 실시간 반사광 검출 알고리즘)

  • Hwang, Dokyung;An, Jongwoo;Kang, Hosun;Lee, Jangmyung
    • The Journal of Korea Robotics Society
    • /
    • v.14 no.4
    • /
    • pp.301-310
    • /
    • 2019
  • A new algorithm has been propose to detect the reflected light region as disturbances in a real-time vision system. There have been several attempts to detect existing reflected light region. The conventional mathematical approach requires a lot of complex processes so that it is not suitable for a real-time vision system. On the other hand, when a simple detection process has been applied, the reflected light region can not be detected accurately. Therefore, in order to detect reflected light region for a real-time vision system, the detection process requires a new algorithm that is as simple and accurate as possible. In order to extract the reflected light, the proposed algorithm has been adopted several filter equations and clustering processes in the HSI (Hue Saturation Intensity) color space. Also the proposed algorithm used the pre-defined reflected light data generated through the clustering processes to make the algorithm simple. To demonstrate the effectiveness of the proposed algorithm, several images with the reflected region have been used and the reflected regions are detected successfully.

Genetic Algorithm Application to Machine Learning

  • Han, Myung-mook;Lee, Yill-byung
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.11 no.7
    • /
    • pp.633-640
    • /
    • 2001
  • In this paper we examine the machine learning issues raised by the domain of the Intrusion Detection Systems(IDS), which have difficulty successfully classifying intruders. There systems also require a significant amount of computational overhead making it difficult to create robust real-time IDS. Machine learning techniques can reduce the human effort required to build these systems and can improve their performance. Genetic algorithms are used to improve the performance of search problems, while data mining has been used for data analysis. Data Mining is the exploration and analysis of large quantities of data to discover meaningful patterns and rules. Among the tasks for data mining, we concentrate the classification task. Since classification is the basic element of human way of thinking, it is a well-studied problem in a wide variety of application. In this paper, we propose a classifier system based on genetic algorithm, and the proposed system is evaluated by applying it to IDS problem related to classification task in data mining. We report our experiments in using these method on KDD audit data.

  • PDF

TPC Algorithm for Fault Diagnosis of CAN-Based Multiple Sensor Network System (CAN 기반 다중센서 네트워크 시스템의 고장진단을 위한 TPC알고리즘)

  • Ha, Hwimyeong;Hwang, Yuseop;Jung, Kyungsuk;Kim, Hyunjun;Lee, Bongjin;Lee, Jangmyung
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.22 no.2
    • /
    • pp.147-152
    • /
    • 2016
  • This paper proposes a new TPC (Transmission Priority Change) algorithm which is used to diagnose failures of a CAN (Controller Area Network) based network system for the oil tank monitoring. The TPC algorithm is aimed to increase the total amount of data transmission and to minimize the latency for an urgent message by changing transmission priority. The urgency of the data transmission has been determined by the conditions of sensors. There are multiple sensors inside of the oil tank, such as temperature, valve, pressure and level sensors. When the sensors operate normally, the sensory data can be collected through the CAN network by the monitoring system. However when there is a dangerous situation or failure situation happened at a sensor, the data need to be handled quickly by the monitoring system, which is implemented by using the TPC algorithm. The effectiveness of the TPC algorithm has been verified by the real experiments. In addition, this paper introduces a method that people can figure out the condition of oil tanks and also can perform the fault diagnosis in real-time by using transmitted packet data. By applying this TPC algorithm to various industries, the convenience and reliability of multiple sensors network system can be improved.

Design and Implementation of the Intrusion Detection Pattern Algorithm Based on Data Mining (데이터 마이닝 기반 침입탐지 패턴 알고리즘의 설계 및 구현)

  • Lee, Sang-Hoon;Soh, Jin
    • The KIPS Transactions:PartC
    • /
    • v.10C no.6
    • /
    • pp.717-726
    • /
    • 2003
  • In this paper, we analyze the associated rule based deductive algorithm which creates the rules automatically for intrusion detection from the vast packet data. Based on the result, we also suggest the deductive algorithm which creates the rules of intrusion pattern fast in order to apply the intrusion detection systems. The deductive algorithm proposed is designed suitable to the concept of clustering which classifies and deletes the large data. This algorithm has direct relation with the method of pattern generation and analyzing module of the intrusion detection system. This can also extend the appication range and increase the detection speed of exiting intrusion detection system as the rule database is constructed for the pattern management of the intrusion detection system. The proposed pattern generation technique of the deductive algorithm is used to the algorithm is used to the algorithm which can be changed by the supporting rate of the data created from the intrusion detection system. Fanally, we analyze the possibility of the speed improvement of the rule generation with the algorithm simulation.

Implementation of Real Time 3 channel Transmission System Using ECG Data Compression Algorithm by Max-Min Slope Update (최대 및 최소 기울기 갱신에 의한 ECG 압축 알고리듬을 이용한 실시간 3채널 전송시스템 구현)

  • 조진호;김명남
    • Journal of Biomedical Engineering Research
    • /
    • v.16 no.3
    • /
    • pp.271-278
    • /
    • 1995
  • An ECG data compression algorithM using max-min slope update is proposed and a real time 3 channel ECG transmission system is implemented using the proposed algorithm. In order to effectively compress ECG data, we compare a threshold value with the max-min slope difference (MMSD) which is updated at each sample values. If this MMSD value is smaller than the threshold value, then the data is compressed. Conversely, when the MMSD value is larger than threshold value, the data is transmitted after storing the value and the length between the data which is beyond previous threshold level. As a result, it can accurately compress both the region of QRS, P, and T wave that has fast-changing and the region of the base line that slope is changing slow. Therefore, it Is possible to enhance the compression rate and the percent roms difference. In addition, because of the simplicity, this algorithm is more suitable for real-time implementation.

  • PDF