• Title/Summary/Keyword: Stream Data

Search Result 2,521, Processing Time 0.028 seconds

Comparative Analysis on the Application of Biotic Indices for Environmental Assessment of a Polluted Stream (Jinwi Stream) (오염하천(진위천)의 환경평가를 위한 생물지수간 적용성 비교분석)

  • Oh, Min Woo;Lee, Ok-Min;Song, Ho-Bok;Park, Sun Jin;Song, Mee Young;Kong, Dongsoo
    • Journal of Korean Society on Water Environment
    • /
    • v.27 no.6
    • /
    • pp.760-768
    • /
    • 2011
  • Jinwi Stream is considerably polluted. The urban development in a Jinwi Stream basin can make state of aquatic ecosystem worse. However, researches for aquatic ecosystems in Jinwi Stream are insufficient. In this study, biotic indices of periphytic diatoms (DAIpo and TDI), benthic macroinvertebrates (EPT, KSI and ESB) and fish (IBI) were compared with the annual water quality data. Benthic macroinvertebrates indices showed highly significant correlations with concentrations of organic materials and nutrients, while DAIpo, TDI and IBI showed low correlations with them. In particular, ESB can be considered as an useful indicator that reflects the degree of diversity and abundance of biotic community as well as water quality. In polluted and disturbed streams as Jinwi Stream, DAIpo, TDI and IBI appeared to be not available for evaluating and discriminating the water quality, although they have been known as good indices in general streams.

Ecosystem management system of Wangsuk stream region by geographical information systems (GIS를 이용한 왕숙천 유역의 생태계 관리 시스템)

  • 이웅재;원두희
    • Journal of environmental and Sanitary engineering
    • /
    • v.16 no.3
    • /
    • pp.54-60
    • /
    • 2001
  • The need and concern about ecosystem are growing rapidly. However, ecosystem management systems are still in the first stage since the data are handled locally and separately. It results in the waste of money and time. In this research, we designed and implemented ecosystem management system of stream region using geographical information system(GIS) that is able to be used to manage the natural resource efficiently. It is expected to be used as a useful tool for Improvement of environment and management of ecosystem as well as recovery of natural environment.

  • PDF

Preprocessing Method for Handling Multi-Way Join Continuous Queries over Data Streams (데이터 스트림에서 다중 조인 연속질의의 효과적인 처리를 위한 전처리 기법)

  • Seo, Ki-Yeon;Lee, Joo-Il;Lee, Won-Suk
    • Journal of Internet Computing and Services
    • /
    • v.13 no.3
    • /
    • pp.93-105
    • /
    • 2012
  • A data stream is a series of tuples which are generated in real-time, incessant, immense, and volatile manner. As new information technologies are actively emerging, stream processing methods are being needed to efficiently handle data streams. Especially, finding out an efficient evaluation for a multi-way join would make outstanding contributions toward improving the performance of a data stream management system because a join operation is one of the most resource-consuming operators for evaluating queries. In this paper, in order to evaluate efficiently a multi-way join continuous query, we propose a novel method to decrease the cost of a query by eliminating unsuccessful intermediate results. For this, we propose a matrix-based structure for monitoring data streams and estimate the number of final result tuples of the query and find out unsuccessful tuples by matrix multiplication operations. And then using these information, we process efficiently a multi-way join continuous query by filtering out the unsuccessful tuples in advance before actual evaluation of the query.

Load Balancing for Distributed Processing of Real-time Spatial Big Data Stream (실시간 공간 빅데이터 스트림 분산 처리를 위한 부하 균형화 방법)

  • Yoon, Susik;Lee, Jae-Gil
    • Journal of KIISE
    • /
    • v.44 no.11
    • /
    • pp.1209-1218
    • /
    • 2017
  • A variety of sensors is widely used these days, and it has become much easier to acquire spatial big data streams from various sources. Since spatial data streams have inherently skewed and dynamically changing distributions, the system must effectively distribute the load among workers. Previous studies to solve this load imbalance problem are not directly applicable to processing spatial data. In this research, we propose Adaptive Spatial Key Grouping (ASKG). The main idea of ASKG is, by utilizing the previous distribution of the data streams, to adaptively suggest a new grouping scheme that evenly distributes the future load among workers. We evaluate the validity of the proposed algorithm in various environments, by conducting an experiment with real datasets while varying the number of workers, input rate, and processing overhead. Compared to two other alternative algorithms, ASKG improves the system performance in terms of load imbalance, throughput, and latency.

Comparison of Topographical Parameter for DTED and Grid DEM from 1:50,000 Digital Map (DTED와 1:50,000 수치지형도에 의한 격자 DEM의 지형 매개변수 비교)

  • Kim, Yeon-Jun;Shin, Ke-Jong
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.5 no.3
    • /
    • pp.19-32
    • /
    • 2002
  • Topographic information is indispensable in the applications that require elevational data. These applications are exemplified by watershed partition, extraction of drainage networks, viewshed analysis, derivation of geomorphologic features, quantification of landslide-terrain, and identification of topographic settings susceptible to landsliding. Therefore, we study the accuracy of data on topographic parameters derived from digital elevation models(DEMs). This research wished to analyze the effect that data source and grid size get in topography parameter using gridded DEM. An analysis of topography parameter extract and compared drainage basin, watershed slope, stream network using DEM is constructed by digital map and DTED DEM. Especially, when extract stream network from gridded DEM, received much effects according to threshold value of flowaccumulation regardless of DEM grid size. Therefore, this study applied equal threshold value of flowaccumulation for two data sources, and compare and analyzed stream network.

  • PDF

Effective Streaming of XML Data for Wireless Broadcasting (무선 방송을 위한 효과적인 XML 스트리밍)

  • Park, Jun-Pyo;Park, Chang-Sup;Chung, Yon-Dohn
    • Journal of KIISE:Databases
    • /
    • v.36 no.1
    • /
    • pp.50-62
    • /
    • 2009
  • In wireless and mobile environments, data broadcasting is recognized as an effective way for data dissemination due to its benefits to bandwidth efficiency, energy-efficiency, and scalability. In this paper, we address the problem of delayed query processing raised by tree-based index structures in wireless broadcast environments, which increases the access time of the mobile clients. We propose a novel distributed index structure and a clustering strategy for streaming XML data which enable energy and latency-efficient broadcast of XML data. We first define the DIX node structure to implement a fully distributed index structure which contains tag name, attributes, and text content of an element as well as its corresponding indices. By exploiting the index information in the DIX node stream, a mobile client can access the wireless stream in a shorter latency. We also suggest a method of clustering DIX nodes in the stream, which can further enhance the performance of query processing over the stream in the mobile clients. Through extensive performance experiments, we demonstrate that our approach is effective for wireless broadcasting of XML data and outperforms the previous methods.

Squall: A Real-time Big Data Processing Framework based on TMO Model for Real-time Events and Micro-batch Processing (Squall: 실시간 이벤트와 마이크로-배치의 동시 처리 지원을 위한 TMO 모델 기반의 실시간 빅데이터 처리 프레임워크)

  • Son, Jae Gi;Kim, Jung Guk
    • Journal of KIISE
    • /
    • v.44 no.1
    • /
    • pp.84-94
    • /
    • 2017
  • Recently, the importance of velocity, one of the characteristics of big data (5V: Volume, Variety, Velocity, Veracity, and Value), has been emphasized in the data processing, which has led to several studies on the real-time stream processing, a technology for quick and accurate processing and analyses of big data. In this paper, we propose a Squall framework using Time-triggered Message-triggered Object (TMO) technology, a model that is widely used for processing real-time big data. Moreover, we provide a description of Squall framework and its operations under a single node. TMO is an object model that supports the non-regular real-time processing method for certain conditions as well as regular periodic processing for certain amount of time. A Squall framework can support the real-time event stream of big data and micro-batch processing with outstanding performances, as compared to Apache storm and Spark Streaming. However, additional development for processing real-time stream under multiple nodes that is common under most frameworks is needed. In conclusion, the advantages of a TMO model can overcome the drawbacks of Apache storm or Spark Streaming in the processing of real-time big data. The TMO model has potential as a useful model in real-time big data processing.

Real-Time IoT Big-data Processing for Stream Reasoning (스트림-리즈닝을 위한 실시간 사물인터넷 빅-데이터 처리)

  • Yun, Chang Ho;Park, Jong Won;Jung, Hae Sun;Lee, Yong Woo
    • Journal of Internet Computing and Services
    • /
    • v.18 no.3
    • /
    • pp.1-9
    • /
    • 2017
  • Smart Cities intelligently manage numerous infrastructures, including Smart-City IoT devices, and provide a variety of smart-city applications to citizen. In order to provide various information needed for smart-city applications, Smart Cities require a function to intelligently process large-scale streamed big data that are constantly generated from a large number of IoT devices. To provide smart services in Smart-City, the Smart-City Consortium uses stream reasoning. Our stream reasoning requires real-time processing of big data. However, there are limitations associated with real-time processing of large-scale streamed big data in Smart Cities. In this paper, we introduce one of our researches on cloud computing based real-time distributed-parallel-processing to be used in stream-reasoning of IoT big data in Smart Cities. The Smart-City Consortium introduced its previously developed smart-city middleware. In the research for this paper, we made cloud computing based real-time distributed-parallel-processing available in the cloud computing platform of the smart-city middleware developed in the previous research, so that we can perform real-time distributed-parallel-processing with them. This paper introduces a real-time distributed-parallel-processing method and system for stream reasoning with IoT big data transmitted from various sensors of Smart Cities and evaluate the performance of real-time distributed-parallel-processing of the system where the method is implemented.

TriSec: A Secure Data Framework for Wireless Sensor Networks Using Authenticated Encryption

  • Kumar, Pardeep;Cho, Sang-Il;Lee, Dea-Seok;Lee, Young-Dong;Lee, Hoon-Jae
    • Journal of information and communication convergence engineering
    • /
    • v.8 no.2
    • /
    • pp.129-135
    • /
    • 2010
  • Wireless sensor networks (WSNs) are an emerging technology and offers economically viable monitoring solution to many challenging applications. However, deploying new technology in hostile environment, without considering security in mind has often proved to be unreasonably unsecured. Apparently, security techniques face many critical challenges in WSNs like data security and secrecy due to its hostile deployment nature. In order to resolve security in WSNs, we propose a novel and efficient secure framework called TriSec: a secure data framework for wireless sensor networks to attain high level of security. TriSec provides data confidentiality, authentication and data integrity to sensor networks. TriSec supports node-to-node encryption using PingPong-128 stream cipher based-privacy. A new PingPong-MAC (PP-MAC) is incorporated with PingPong stream cipher to make TriSec framework more secure. PingPong-128 is fast keystream generation and it is very suitable for sensor network environment. We have implemented the proposed scheme on wireless sensor platform and our result shows their feasibility.

Current Status of Refractory Dissolved Organic Carbon in the Nakdong River Basin (낙동강유역 난분해성 용존 유기탄소 배출 현황 분석)

  • Lee, Jeonghoon;Kim, Jungsun;Lee, Jae Kwan;Kang, Limseok;Kim, Sangdan
    • Journal of Korean Society on Water Environment
    • /
    • v.28 no.4
    • /
    • pp.538-550
    • /
    • 2012
  • This study suggests a general methodology which is designed for assessing RDOC behavior at the catchment scale by coupling properly a series of steam flow and water quality simulation models and actual monitoring data set. The modified TANK model in which a river routing function is incorporated to the conventional one is applied to simulate the long-term daily stream flow data, and the simulated stream flow data is combined with the 7-parameter log-linear model coupled to the minimum variance unbiased estimator to simulate the long-term daily water quality (BOD, COD and TOC) loads. Finally, the regression analysis between the usually monitored water quality data (BOD, COD and TOC) and RDOC is combined with the simulated water quality data to manifest the spatio-temporal variability of RDOC flux behavior at the Korean TMDL catchment scale.