• The Journal of Society for e-Business Studies
• /
• v.15 no.4
• /
• pp.1-19
• /
• 2010

Recently, it is necessary to process real time sensor data, which is generated from ubiquitous environments. Data, which are written by XML, are small, but, large volumes of data. Therefore, weneed to use an efficient method for processing a large amount of it. An XQuery has two types for sensor data: one is to get sensor identification and value from sensor data; the other is restructuring for user's convenience. Existing XQuery engines don't have efficient method for batch processing of sensor data. This paper proposed the twig query processing over reverse path summary, and we developed and applied restructuring batch processing method for real time processing of a large amount of sensor data. Finally, we do performance evaluation using XMark and RFID EPC data, and comparison analysis with MonetDB/XQuery and Berkeley DB XML.

• Spatial Information Research
• /
• v.20 no.5
• /
• pp.99-109
• /
• 2012

Recently, due to the growth of social media and spread of smart-phone, the amount of data has considerably increased by full use of SNS (Social Network Service). According to it, the Big Data concept is come up and many researchers are seeking solutions to make the best use of big data. To maximize the creative value of the big data held by many companies, it is required to combine them with existing data. The physical and theoretical storage structures of data sources are so different that a system which can integrate and manage them is needed. In order to process big data, MapReduce is developed as a system which has advantages over processing data fast by distributed processing. However, it is difficult to construct and store a system for all key words. Due to the process of storage and search, it is to some extent difficult to do real-time processing. And it makes extra expenses to process complex event without structure of processing different data. In order to solve this problem, the existing Complex Event Processing System is supposed to be used. When it comes to complex event processing system, it gets data from different sources and combines them with each other to make it possible to do complex event processing that is useful for real-time processing specially in stream data. Nevertheless, unstructured data based on text of SNS and internet articles is managed as text type and there is a need to compare strings every time the query processing should be done. And it results in poor performance. Therefore, we try to make it possible to manage unstructured data and do query process fast in complex event processing system. And we extend the data complex function for giving theoretical schema of string. It is completed by changing the string key word into integer type with filtering which uses keyword set. In addition, by using the Complex Event Processing System and processing stream data at real-time of in-memory, we try to reduce the time of reading the query processing after it is stored in the disk.

• Annual Conference of KIPS
• /
• 2023.05a
• /
• pp.377-379
• /
• 2023

Concept drift detection on data stream is the major issue to maintain the performance of the machine learning model. Since the online stream is to be a function of time, the classical statistic methods are hard to apply. In particular case of seasonal time series, a novel window strategy with Fourier analysis however, gives a chance to adapt the classical methods on the series. We explore the KS-test for an adaptation of the periodic time series and show that this strategy handles a complicate time series as an ordinary tabular dataset. We verify that the detection with the strategy takes the second place in time delay and shows the best performance in false alarm rate and detection accuracy comparing to that of arbitrary window sizes.

• Applied Microscopy
• /
• v.46 no.1
• /
• pp.6-13
• /
• 2016

Fluorescence lifetime imaging microscopy (FLIM) has been considered an effective technique to investigate chemical properties of the specimens, especially of biological samples. Despite of this advantageous trait, researchers in this field have had difficulties applying FLIM to their systems because acquiring an image using FLIM consumes too much time. Although analog mean-delay (AMD) method was introduced to enhance the imaging speed of commonly used FLIM based on time-correlated single photon counting (TCSPC), a real-time image reconstruction using AMD method has not been implemented due to its data processing obstacles. In this paper, we introduce a real-time image restoration of AMD-FLIM through fast parallel data processing by using Threading Building Blocks (TBB; Intel) and octa-core processor (i7-5960x; Intel). Frame rate of 3.8 frames per second was achieved in $1,024{\times}1,024$ resolution with over 4 million lifetime determinations per second and measurement error within 10%. This image acquisition speed is 184 times faster than that of single-channel TCSPC and 9.2 times faster than that of 8-channel TCSPC (state-of-art photon counting rate of 80 million counts per second) with the same lifetime accuracy of 10% and the same pixel resolution.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2018.05a
• /
• pp.150-150
• /
• 2018

The Numerical Weather Prediction (NWP) models provide information for weather forecasts. The highly nonlinear and complex interactions in the atmosphere are simplified in meteorological models through approximations and parameterization. Therefore, the simplifications may lead to biases and errors in model results. Although the models have improved over time, the biased outputs of these models are still a matter of concern in meteorological and hydrological studies. Thus, bias removal is an essential step prior to using outputs of atmospheric models. The main idea of statistical bias correction methods is to develop a statistical relationship between modeled and observed variables over the same historical period. The Model Output Statistics (MOS) would be desirable to better match the real time forecast data with observation records. Statistical post-processing methods relate model outputs to the observed values at the sites of interest. In this study three methods are used to remove the possible biases of the real-time outputs of the Weather Research and Forecast (WRF) model in Imjin basin (North and South Korea). The post-processing techniques include the Linear Regression (LR), Linear Scaling (LS) and Power Scaling (PS) methods. The MOS techniques used in this study include three main steps: preprocessing of the historical data in training set, development of the equations, and application of the equations for the validation set. The expected results show the accuracy improvement of the real-time forecast data before and after bias correction. The comparison of the different methods will clarify the best method for the purpose of the forecast skill enhancement in a real-time case study.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2009.10a
• /
• pp.371-374
• /
• 2009

This Paper deal with a design, making a prtotype and test methods of Real-time towed Line Array Sensor Data processing board for fast data communication and long range transmission with SFM(Serial FPDP Module) through Optic-fiber channel. The LAS A,B,C group Data from towed line array sensor which is installed in FFX(Fast Frigate eXperimental) of Korean Navy is packed a previously agreed protocol and transmitted to the Signal processing unit. Consider the limited space of VME 6U size, LAS Data processing board is designed with MPC8265 PowerPC Controller of Freescale for main system control and Altera's CycloneIII FPGA for sensor data packing, self-test simulation data generation, S/W FIFO et cetera. LAS Data processing board have VxWorks, the RTOS(Real Time Operating System) that present many device drivers, peripheral control libraries on board for real-time data processing.

• Korean Journal of Fisheries and Aquatic Sciences
• /
• v.44 no.3
• /
• pp.308-317
• /
• 2011

A MBES (multibeam echo sounder) survey around Yokji Island, Korea, was conducted to find an effective method for removing error data. Two post-processing software programs, PDS2000 (RESON) and HIPS (CARIS), were used to remove the error data using an interactive editing method and the CUBE algorithm filter. The post-processing with the PDS2000 and HIPS programs, using the interactive editing method, took 120 and 168 hours, respectively, and there was little difference in the seafloor images. The processing time of the PDS2000 and HIPS programs using the CUBE algorithm filter was 36 and 60 hours, respectively. Nevertheless, there was little difference in the seafloor images because of differences in the factor parameters in each of the post-processing programs. Therefore, post-processing using CUBE filtering can save time in data processing and provide consistent results, excluding the subjective decisions of the operator. This method is more effective than other methods for rejecting erroneous multibeam echo sounder data.

• International journal of advanced smart convergence
• /
• v.10 no.2
• /
• pp.64-72
• /
• 2021

XML message brokers have a lot of importance because XML has become a practical standard for data exchange in many applications. Message brokers covered in this document store many users. This paper is a study of the processing of twig pattern queries in XML documents using branching node streams in XML message broker structures. This work is about query processing in XML documents, especially for query processing with XML twig patterns in the XML message broker structure and proposed a method to reduce query processing time when parsing documents with XML twig patterns by processing information. In this paper, the twig pattern query processing method of documents using the branching node stream removes the twigging value of the branch node that does not include the labeling value of the branch node stream when it receives a twig query from the client. In this paper, the leaf node discovery time can be reduced by reducing the navigation time of nodes in XML documents that are matched to leaf nodes in twig queries for client twig queries. Overall, the overall processing time to respond to queries is reduced, allowing for rapid question-answer processing.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.60 no.2
• /
• pp.245-251
• /
• 2011

Quantities of data in the DMS (Distribution management system) or SCADA (Supervisory control and data acquisition) system is enormously large as illustrated by the usage of term flooding of data. This enormous quantity of data is transmitted to the status data or event data of the on-site apparatus in real-time. In addition, if GIS (Geographic information system) and AMR (Automatic meter reading), etc are integrated, the quantity of data to be processed in real-time increases unimaginably. Increase in the quantity of data due to addition of system or increase in the on-site facilities cannot be handled through the currently used Single Thread format of data processing technology. However, if Multi Thread technology that utilizes LF-POOL (Leader Follower -POOL) is applied in processing large quantity of data, large quantity of data can be processed in short period of time and the load on the server can be minimized. In this Study, the actual materialization and functions of LF POOL technology are examined.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.8 no.4 s.23
• /
• pp.24-31
• /
• 2005

This paper presents an implementation of a real-time multi-sensor data fusion algorithm for Flight Test Computer. The sensor data consist of positional information of the target from a radar, a GPS receiver and an INS. The data fusion algorithm is designed by the 21st order distributed Kalman Filter which is based on the PVA model with sensor bias states. A fault detection and correction logics are included in the algorithm for bad measurements and sensor faults. The statistical parameters for the states are obtained from Monte Carlo simulations and covariance analysis using test tracking data. The designed filter is verified by using real data both in post processing and real-time processing.