• Title/Summary/Keyword: Stream Analytics

Search Result 8, Processing Time 0.028 seconds

Multi-dimensional Query Authentication for On-line Stream Analytics

  • Chen, Xiangrui;Kim, Gyoung-Bae;Bae, Hae-Young
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.4 no.2
    • /
    • pp.154-173
    • /
    • 2010
  • Database outsourcing is unavoidable in the near future. In the scenario of data stream outsourcing, the data owner continuously publishes the latest data and associated authentication information through a service provider. Clients may register queries to the service provider and verify the result's correctness, utilizing the additional authentication information. Research on On-line Stream Analytics (OLSA) is motivated by extending the data cube technology for higher multi-level abstraction on the low-level-abstracted data streams. Existing work on OLSA fails to consider the issue of database outsourcing, while previous work on stream authentication does not support OLSA. To close this gap and solve the problem of OLSA query authentication while outsourcing data streams, we propose MDAHRB and MDAHB, two multi-dimensional authentication approaches. They are based on the general data model for OLSA, the stream cube. First, we improve the data structure of the H-tree, which is used to store the stream cube. Then, we design and implement two authentication schemes based on the improved H-trees, the HRB- and HB-trees, in accordance with the main stream query authentication framework for database outsourcing. Along with a cost models analysis, consistent with state-of-the-art cost metrics, an experimental evaluation is performed on a real data set. It exhibits that both MDAHRB and MDAHB are feasible for authenticating OLSA queries, while MDAHRB is more scalable.

Fast Visualization Technique and Visual Analytics System for Real-time Analyzing Stream Data (실시간 스트림 데이터 분석을 위한 시각화 가속 기술 및 시각적 분석 시스템)

  • Jeong, Seongmin;Yeon, Hanbyul;Jeong, Daekyo;Yoo, Sangbong;Kim, Seokyeon;Jang, Yun
    • Journal of the Korea Computer Graphics Society
    • /
    • v.22 no.4
    • /
    • pp.21-30
    • /
    • 2016
  • Risk management system should be able to support a decision making within a short time to analyze stream data in real time. Many analytical systems consist of CPU computation and disk based database. However, it is more problematic when existing system analyzes stream data in real time. Stream data has various production periods from 1ms to 1 hour, 1day. One sensor generates small data but tens of thousands sensors generate huge amount of data. If hundreds of thousands sensors generate 1GB data per second, CPU based system cannot analyze the data in real time. For this reason, it requires fast processing speed and scalability for analyze stream data. In this paper, we present a fast visualization technique that consists of hybrid database and GPU computation. In order to evaluate our technique, we demonstrate a visual analytics system that analyzes pipeline leak using sensor and tweet data.

ADA: Advanced data analytics methods for abnormal frequent episodes in the baseline data of ISD

  • Biswajit Biswal;Andrew Duncan;Zaijing Sun
    • Nuclear Engineering and Technology
    • /
    • v.54 no.11
    • /
    • pp.3996-4004
    • /
    • 2022
  • The data collected by the In-Situ Decommissioning (ISD) sensors are time-specific, age-specific, and developmental stage-specific. Research has been done on the stream data collected by ISD testbed in the recent few years to seek both frequent episodes and abnormal frequent episodes. Frequent episodes in the data stream have confirmed the daily cycle of the sensor responses and established sequences of different types of sensors, which was verified by the experimental setup of the ISD Sensor Network Test Bed. However, the discovery of abnormal frequent episodes remained a challenge because these abnormal frequent episodes are very small signals and may be buried in the background noise of voltage and current changes. In this work, we proposed Advanced Data Analytics (ADA) methods that are applied to the baseline data to identify frequent episodes and extended our approach by adding more features extracted from the baseline data to discover abnormal frequent episodes, which may lead to the early indicators of ISD system failures. In the study, we have evaluated our approach using the baseline data, and the performance evaluation results show that our approach is able to discover frequent episodes as well as abnormal frequent episodes conveniently.

Scalable Big Data Pipeline for Video Stream Analytics Over Commodity Hardware

  • Ayub, Umer;Ahsan, Syed M.;Qureshi, Shavez M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.4
    • /
    • pp.1146-1165
    • /
    • 2022
  • A huge amount of data in the form of videos and images is being produced owning to advancements in sensor technology. Use of low performance commodity hardware coupled with resource heavy image processing and analyzing approaches to infer and extract actionable insights from this data poses a bottleneck for timely decision making. Current approach of GPU assisted and cloud-based architecture video analysis techniques give significant performance gain, but its usage is constrained by financial considerations and extremely complex architecture level details. In this paper we propose a data pipeline system that uses open-source tools such as Apache Spark, Kafka and OpenCV running over commodity hardware for video stream processing and image processing in a distributed environment. Experimental results show that our proposed approach eliminates the need of GPU based hardware and cloud computing infrastructure to achieve efficient video steam processing for face detection with increased throughput, scalability and better performance.

A Study on Adaptive Learning Model for Performance Improvement of Stream Analytics (실시간 데이터 분석의 성능개선을 위한 적응형 학습 모델 연구)

  • Ku, Jin-Hee
    • Journal of Convergence for Information Technology
    • /
    • v.8 no.1
    • /
    • pp.201-206
    • /
    • 2018
  • Recently, as technologies for realizing artificial intelligence have become more common, machine learning is widely used. Machine learning provides insight into collecting large amounts of data, batch processing, and taking final action, but the effects of the work are not immediately integrated into the learning process. In this paper proposed an adaptive learning model to improve the performance of real-time stream analysis as a big business issue. Adaptive learning generates the ensemble by adapting to the complexity of the data set, and the algorithm uses the data needed to determine the optimal data point to sample. In an experiment for six standard data sets, the adaptive learning model outperformed the simple machine learning model for classification at the learning time and accuracy. In particular, the support vector machine showed excellent performance at the end of all ensembles. Adaptive learning is expected to be applicable to a wide range of problems that need to be adaptively updated in the inference of changes in various parameters over time.

The Big Data Analytics Regarding the Cadastral Resurvey News Articles

  • Joo, Yong-Jin;Kim, Duck-Ho
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.32 no.6
    • /
    • pp.651-659
    • /
    • 2014
  • With the popularization of big data environment, big data have been highlighted as a key information strategy to establish national spatial data infrastructure for a scientific land policy and the extension of the creative economy. Especially interesting from our point of view is the cadastral information is a core national information source that forms the basis of spatial information that leads to people's daily life including the production and consumption of information related to real estate. The purpose of our paper is to suggest the scheme of big data analytics with respect to the articles of cadastral resurvey project in order to approach cadastral information in terms of spatial data integration. As specific research method, the TM (Text Mining) package from R was used to read various formats of news reports as texts, and nouns were extracted by using the KoNLP package. That is, we searched the main keywords regarding cadastral resurvey, performing extraction of compound noun and data mining analysis. And visualization of the results was presented. In addition, new reports related to cadastral resurvey between 2012 and 2014 were searched in newspapers, and nouns were extracted from the searched data for the data mining analysis of cadastral information. Furthermore, the approval rating, reliability, and improvement of rules were presented through correlation analyses among the extracted compound nouns. As a result of the correlation analysis among the most frequently used ones of the extracted nouns, five groups of data consisting of 133 keywords were generated. The most frequently appeared words were "cadastral resurvey," "civil complaint," "dispute," "cadastral survey," "lawsuit," "settlement," "mediation," "discrepant land," and "parcel." In Conclusions, the cadastral resurvey performed in some local governments has been proceeding smoothly as positive results. On the other hands, disputes from owner of land have been provoking a stream of complaints from parcel surveying for the cadastral resurvey. Through such keyword analysis, various public opinion and the types of civil complaints related to the cadastral resurvey project can be identified to prevent them through pre-emptive responses for direct call centre on the cadastral surveying, Electronic civil service and customer counseling, and high quality services about cadastral information can be provided. This study, therefore, provides a stepping stones for developing an account of big data analytics which is able to comprehensively examine and visualize a variety of news report and opinions in cadastral resurvey project promotion. Henceforth, this will contribute to establish the foundation for a framework of the information utilization, enabling scientific decision making with speediness and correctness.

Method for Preference Score Based on User Behavior (웹 사이트 이용 고객의 행동 정보를 기반으로 한 고객 선호지수 산출 방법)

  • Seo, Dong-Yal;Kim, Doo-Jin;Yun, Jeong-Ki;Kim, Jae-Hoon;Moon, Kang-Sik;Oh, Jae-Hoon
    • CRM연구
    • /
    • v.4 no.1
    • /
    • pp.55-68
    • /
    • 2011
  • Recently with the development of Web services by utilizing a variety of web content, the studies on user experience and personalization based on web usage has attracted much attention. Majority of personalized analysis are have been carried out based on existing data, primarily using the database and statistical models. These approaches are difficult to reflect in a timely mannerm, and are limited to reflect the true behavioral characteristics because the data itself was just a result of customers' behaviors. However, recent studies and commercial products on web analytics try to track and analyze all of the actions from landing to exit to provide personalized service. In this study, by analyzing the customer's click-stream behaviors, we define U-Score(Usage Score), P-Score (Preference Score), M-Score(Mania Score) to indicate variety of customer preferences. With the devised three indicators, we can identify the customer's preferences more precisely, provide in-depth customer reports and customer relationship management, and utilize personalized recommender services.

  • PDF

The Effect of Online Multiple Channel Marketing by Device Type (디바이스 유형을 고려한 온라인 멀티 채널 마케팅 효과)

  • Hajung Shin;Kihwan Nam
    • Information Systems Review
    • /
    • v.20 no.4
    • /
    • pp.59-78
    • /
    • 2018
  • With the advent of the various device types and marketing communication, customer's search and purchase behavior have become more complex and segmented. However, extant research on multichannel marketing effects of the purchase funnel has not reflected the specific features of device User Interface (UI) and User Experience (UX). In this study, we analyzed the marketing channel effects of multi-device shoppers using a unique click stream dataset from global online retailers. We examined device types that activate online shopping and compared the differences between marketing channels that promote visits. In addition, we estimated the direct and indirect effects on visits and purchase revenue through customer's accumulated experience and channel conversions. The findings indicate that the same customer selects a different marketing channel according to the device selection. These results can help retailers gain a better understanding of customers' decision-making process in multi-marketing channel environment and devise the optimal strategy taking into account various device types. Our empirical analyses yield business implications based on the significant results from global big data analytics and contribute academically meaningful theoretical framework using an economic model. We also provide strategic insights attributed to the practical value of an online marketing manager.