• Title/Summary/Keyword: Real-time data analysis

Search Result 2,781, Processing Time 0.03 seconds

Real-Time Stock Price Prediction using Apache Spark (Apache Spark를 활용한 실시간 주가 예측)

  • Dong-Jin Shin;Seung-Yeon Hwang;Jeong-Joon Kim
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.4
    • /
    • pp.79-84
    • /
    • 2023
  • Apache Spark, which provides the fastest processing speed among recent distributed and parallel processing technologies, provides real-time functions and machine learning functions. Although official documentation guides for these functions are provided, a method for fusion of functions to predict a specific value in real time is not provided. Therefore, in this paper, we conducted a study to predict the value of data in real time by fusion of these functions. The overall configuration is collected by downloading stock price data provided by the Python programming language. And it creates a model of regression analysis through the machine learning function, and predicts the adjusted closing price among the stock price data in real time by fusing the real-time streaming function with the machine learning function.

Conceptual Framework for Pattern-Based Real-Time Trading System using Genetic Algorithm (유전알고리즘 활용한 실시간 패턴 트레이딩 시스템 프레임워크)

  • Lee, Suk-Jun;Jeong, Suk-Jae
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.36 no.4
    • /
    • pp.123-129
    • /
    • 2013
  • The aim of this study is to design an intelligent pattern-based real-time trading system (PRTS) using rough set analysis of technical indicators, dynamic time warping (DTW), and genetic algorithm in stock futures market. Rough set is well known as a data-mining tool for extracting trading rules from huge data sets such as real-time data sets, and a technical indicator is used for the construction of the data sets. To measure similarity of patterns, DTW is used over a given period. Through an empirical study, we identify the ideal performances that were profitable in various market conditions.

CIM-Based System For Real-Time Voltage Stability Analysis (CIM 기반의 실시간 전압안정도 해석 시스템 구축)

  • Lee, Sung-Woo;Jang, Moon-Jong;Seo, Dong-Wan;Namkoong, Won
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.27 no.10
    • /
    • pp.41-49
    • /
    • 2013
  • There is a wide variety of system and applications in the power system. However, they have compatibility issues because they use different data standard and communication method. With the introduction of the smart grid, power system has been grow and diversified. Therefore power system need to be compatible with each other and the interoperability between applications is increasingly important. Thus, the IEC established IEC61970 and CIM Standard data exchange model for interoperability and system integration. Server-Client system was constructed which using CIM HSDA(Part4), a standard communication model, presented in IEC 619710. Also, self-developed real-time voltage stability analysis application and contingency analysis application was used. CIM HSDA was used for data input and real-time analysis. Tolerance of result which is in the range of allowable derived by Perform real-time voltage stability and contingency analysis of Jeju power system, and then compare it's result with PSS/E result.

RSP-DS: Real Time Sequential Patterns Analysis in Data Streams (RSP-DS: 데이터 스트림에서의 실시간 순차 패턴 분석)

  • Shin Jae-Jyn;Kim Ho-Seok;Kim Kyoung-Bae;Bae Hae-Young
    • Journal of Korea Multimedia Society
    • /
    • v.9 no.9
    • /
    • pp.1118-1130
    • /
    • 2006
  • Existed pattern analysis algorithms in data streams environment have researched performance improvement and effective memory usage. But when new data streams come, existed pattern analysis algorithms have to analyze patterns again and have to generate pattern tree again. This approach needs many calculations in real situation that needs real time pattern analysis. This paper proposes a method that continuously analyzes patterns of incoming data streams in real time. This method analyzes patterns fast, and thereafter obtains real time patterns by updating previously analyzed patterns. The incoming data streams are divided into several sequences based on time based window. Informations of the sequences are inputted into a hash table. When the number of the sequences are over predefined bound, patterns are analyzed from the hash table. The patterns form a pattern tree, and later created new patterns update the pattern tree. In this way, real time patterns are always maintained in the pattern tree. During pattern analysis, suffixes of both new pattern and existed pattern in the tree can be same. Then a pointer is created from the new pattern to the existed pattern. This method reduce calculation time during duplicated pattern analysis. And old patterns in the tree are deleted easily by FIFO method. The advantage of our algorithm is proved by performance comparison with existed method, MILE, in a condition that pattern is changed continuously. And we look around performance variation by changing several variable in the algorithm.

  • PDF

Nano Technology Trend Analysis Using Google Trend and Data Mining Method for Nano-Informatics (나노 인포매틱스 기반 구축을 위한 구글 트렌드와 데이터 마이닝 기법을 활용한 나노 기술 트렌드 분석)

  • Shin, Minsoo;Park, Min-Gyu;Bae, Seong-Hun
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.40 no.4
    • /
    • pp.237-245
    • /
    • 2017
  • Our research is aimed at predicting recent trend and leading technology for the future and providing optimal Nano technology trend information by analyzing Nano technology trend. Under recent global market situation, Users' needs and the technology to meet these needs are changing in real time. At this point, Nano technology also needs measures to reduce cost and enhance efficiency in order not to fall behind the times. Therefore, research like trend analysis which uses search data to satisfy both aspects is required. This research consists of four steps. We collect data and select keywords in step 1, detect trends based on frequency and create visualization in step 2, and perform analysis using data mining in step 3. This research can be used to look for changes of trend from three perspectives. This research conducted analysis on changes of trend in terms of major classification, Nano technology of 30's, and key words which consist of relevant Nano technology. Second, it is possible to provide real-time information. Trend analysis using search data can provide information depending on the continuously changing market situation due to the real-time information which search data includes. Third, through comparative analysis it is possible to establish a useful corporate policy and strategy by apprehending the trend of the United States which has relatively advanced Nano technology. Therefore, trend analysis using search data like this research can suggest proper direction of policy which respond to market change in a real time, can be used as reference material, and can help reduce cost.

CIM-based System for Real-time Voltage Stability Analysis (공통정보모델(CIM) 기반의 실시간 전압안정도 해석)

  • Lee, Sung-Woo;Jang, Moon-Jong;Seo, Dong-Wan;Namkoong, Won;Heo, Soung-Ouk
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.64 no.1
    • /
    • pp.48-56
    • /
    • 2015
  • There is a wide variety of system and applications in the power system. However, they have compatibility issues because they use different data standard and communication method. With the introduction of the smart grid, power system has been grow and diversified. Therefore power system need to be compatible with each other and the interoperability between applications is increasingly important. Thus, the IEC established IEC61970 and CIM Standard data exchange model for interoperability and system integration. Server-Client system was constructed which using CIM HSDA(Part4), a standard communication model, presented in IEC 619710. Also, self-developed real-time voltage stability analysis application and contingency analysis application was used. CIM HSDA was used for data input and real-time analysis. Tolerance of result which is in the range of allowable derived by Perform real-time voltage stability and contingency analysis of Jeju power system, and then compare it's result with PSS/E result.

Real-Time Analysis of Occupant Motion for Vehicle Simulator (차량 시뮬레이터 접목을 위한 실시간 인체거동 해석기법)

  • Oh, Kwangseok;Son, Kwon;Choi, Kyunghyun
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.26 no.5
    • /
    • pp.969-975
    • /
    • 2002
  • Visual effects are important cues for providing occupants with virtual reality in a vehicle simulator which imitates real driving. The viewpoint of an occupant is sensitively dependent upon the occupant's posture, therefore, the total human body motion must be considered in a graphic simulator. A real-time simulation is required for the dynamic analysis of complex human body motion. This study attempts to apply a neural network to the motion analysis in various driving situations. A full car of medium-sized vehicles was selected and modeled, and then analyzed using ADAMS in such driving conditions as bump-pass and lane-change for acquiring the accelerations of chassis of the vehicle model. A hybrid III 50%ile adult male dummy model was selected and modeled in an ellipsoid model. Multibody system analysis software, MADYMO, was used in the motion analysis of an occupant model in the seated position under the acceleration field of the vehicle model. Acceleration data of the head were collected as inputs to the viewpoint movement. Based on these data, a back-propagation neural network was composed to perform the real-time analysis of occupant motions under specified driving conditions and validated output of the composed neural network with MADYMO result in arbitrary driving scenario.

Development and Evaluation of a Real Time Runoff Modelling System using Weather Radar and Distributed Model (기상레이더와 분포형 모형을 이용한 실시간 유출해석 시스템 개발 및 평가)

  • Choi, Yun Seok;Kim, Kyung Tak;Kim, Joo Hun
    • Journal of Wetlands Research
    • /
    • v.14 no.3
    • /
    • pp.385-397
    • /
    • 2012
  • A grid based physically distributed model analyzes rainfall-runoff using physical parameters and grid-typed spatial and hydrological data. This study have developed a real time runoff modelling system using GRM RT(Grid based Rainfall-runoff Model Real Time) which is a real time flow analysis module in GRM, a grid based physically distributed rainfall-runoff model. Weather radar data received in real time are calibrated by using real time AWS from Korea Meteorological Administration(KMA), and they are applied to real time runoff modeling. And the runoff model is calibrated by using observed discharges from a water level gauge in real time. This study have designed and implemented the databases necessary to construct the real time runoff modelling system, and established the process of a real time runoff modelling. And the performances of the developed system have been evaluated. The system have been applied to Nerinheon watershed located in the upstream of Soyanggang Dam and the application results are evaluated.

Application Of Open Data Framework For Real-Time Data Processing (실시간 데이터 처리를 위한 개방형 데이터 프레임워크 적용 방안)

  • Park, Sun-ho;Kim, Young-kil
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.10
    • /
    • pp.1179-1187
    • /
    • 2019
  • In today's technology environment, most big data-based applications and solutions are based on real-time processing of streaming data. Real-time processing and analysis of big data streams plays an important role in the development of big data-based applications and solutions. In particular, in the maritime data processing environment, the necessity of developing a technology capable of rapidly processing and analyzing a large amount of real-time data due to the explosion of data is accelerating. Therefore, this paper analyzes the characteristics of NiFi, Kafka, and Druid as suitable open source among various open data technologies for processing big data, and provides the latest information on external linkage necessary for maritime service analysis in Korean e-Navigation service. To this end, we will lay the foundation for applying open data framework technology for real-time data processing.

Performance analysis of the data link layer of IEC/ISA fieldbus system by simulation model (시뮬레이션 모델을 이용한 IEC/ISA 필드버스 시스템의 데이터 링크 계층 성능 분석)

  • Lee, Seong-Geun;Hong, Seung-Ho
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.2 no.3
    • /
    • pp.209-219
    • /
    • 1996
  • Fieldbus provides a real-time data communication among field devices in the process control and manufacturing automation systems. In this paper, a Petri Net model of the 1993 draft of IEC/ISA fieldbus which is proposed as an international standard of fieldbus network is developed. Based on the Petri Net model, discrete-event simulation model of IEC/ISA fieldbus network is developed. This paper evaluates the network induced delay in the data link layer of IEC/ISA fieldbus using the simulation model. In addition, an integrated discrete-event/continuous-time simulation model of fieldbus system and distributed control system is developed. This paper investigates the real-time data processing capability of IEC/ISA fieldbus and the effect of network-induced delay to the performance of control system.

  • PDF