• Title/Summary/Keyword: Very large real-time data

Search Result 132, Processing Time 0.034 seconds

GPU-based Object Extraction for Real-time Analysis of Large-scale Radar Signal (대규모 레이더 신호 데이터의 실시간 분석을 위한 GPU 기반 객체 추출 기법)

  • Kang, Young-Min
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.8
    • /
    • pp.1297-1309
    • /
    • 2016
  • In this paper, an efficient connected component labeling (CCL) method was proposed. The proposed method is based on GPU parallelism. The CCL is very important in various applications where images are analysed. However, the label of each pixel is dependent on the connectivity of adjacent pixels so that it is not very easy to be parallelized. In this paper, a GPU-based parallel CCL techniques were proposed and applied to the analysis of radar signal. Since the radar signals contains complex and large data, the efficiency of the algorithm is crucial when realtime analysis is required. The experimental results show the proposed method is efficient enough to be successfully applied to this application.

Approximate Top-k Subgraph Matching Scheme Considering Data Reuse in Large Graph Stream Environments (대용량 그래프 스트림 환경에서 데이터 재사용을 고려한 근사 Top-k 서브 그래프 매칭 기법)

  • Choi, Do-Jin;Bok, Kyoung-Soo;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.20 no.8
    • /
    • pp.42-53
    • /
    • 2020
  • With the development of social network services, graph structures have been utilized to represent relationships among objects in various applications. Recently, a demand of subgraph matching in real-time graph streams has been increased. Therefore, an efficient approximate Top-k subgraph matching scheme for low latency in real-time graph streams is required. In this paper, we propose an approximate Top-k subgraph matching scheme considering data reuse in graph stream environments. The proposed scheme utilizes the distributed stream processing platform, called Storm to handle a large amount of stream data. We also utilize an existing data reuse scheme to decrease stream processing costs. We propose a distance based summary indexing technique to generate Top-k subgraph matching results. The proposed summary indexing technique costs very low since it only stores distances among vertices that are selected in advance. Finally, we provide k subgraph matching results to users by performing an approximate Top-k matching on the summary indexing. In order to show the superiority of the proposed scheme, we conduct various performance evaluations in diverse real world datasets.

Development of a Scheduling System for Mould and Die Manufacturing Factory Using Microsoft Project 98 (Microsoft Project 98을 이용한 금형공장의 일정계획 시스템 개발)

  • Ju, Sang-Yoon;Ok, Kyung-Jin
    • IE interfaces
    • /
    • v.13 no.2
    • /
    • pp.246-252
    • /
    • 2000
  • As moulds and dies are manufactured through complex processes under the make-to-order production environment, it is very difficult that the manufacturing activities as like observance of the due date, trace of the progress, etc are controlled with a real time. In this paper, a schedule-planning system using the commercial software Microsoft Project 98 is developed to control the procedures of mould and die manufacturing with real time. Once an initial schedule is planned from the BOM information in the intranet, it is rescheduled by data collected from machines on the shop floor. The system is suitable to medium- or small-sized manufacturing companies as well as large-sized ones, because it can be installed with a low cost.

  • PDF

Hardware-Accelerated Multipipe Parallel Rendering of Large Data Streams

  • Park, Sanghun;Park, Sangmin;Bajaj, Chandrajit;Ihm, Insung
    • Journal of the Korea Computer Graphics Society
    • /
    • v.7 no.2
    • /
    • pp.21-28
    • /
    • 2001
  • As a result of the recent explosive growth of scientific data, extremely large volume datasets have become increasingly commonplace. While several texture-based volume rendering algorithms have been proposed, most of them focused on volumes smaller than the hardware's available texture memory. This paper presents a new parallel volume rendering scheme for very large static and time-varying data on a multipipe system architecture. Our scheme subdivides large volumes dynamically into smaller bricks, and assigns them adaptively to graphics pipes to minimize the costs of texture swapping. With the new method, Phong shaded images can be easily created by computing the gradients on the fly and using the color matrix feature of OpenGL. We report experimental results on an SGI Onyx2 for the various large datasets.

  • PDF

T-START: Time, Status and Region Aware Taxi Mobility Model for Metropolis

  • Wang, Haiquan;Lei, Shuo;Wu, Binglin;Li, Yilin;Du, Bowen
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.7
    • /
    • pp.3018-3040
    • /
    • 2018
  • The mobility model is one of the most important factors that impacts the evaluation of any transportation vehicular networking protocols via simulations. However, to obtain a realistic mobility model in the dynamic urban environment is a very challenging task. Several studies extract mobility models from large-scale real data sets (mostly taxi GPS data) in recent years, but they do not consider the statuses of taxi, which is an important factor affected taxi's mobility. In this paper, we discover three simple observations related to the taxi statuses via mining of real taxi trajectories: (1) the behavior of taxi will be influenced by the statuses, (2) the macroscopic movement is related with different geographic features in corresponding status, and (3) the taxi load/drop events are varied with time period. Based on these three observations, a novel taxi mobility model (T-START) is proposed with respect to taxi statuses, geographic region and time period. The simulation results illustrate that proposed mobility model has a good approximation with reality in trajectory samples and distribution of nodes in four typical time periods.

Dynamic Load Management Method for Spatial Data Stream Processing on MapReduce Online Frameworks (맵리듀스 온라인 프레임워크에서 공간 데이터 스트림 처리를 위한 동적 부하 관리 기법)

  • Jeong, Weonil
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.8
    • /
    • pp.535-544
    • /
    • 2018
  • As the spread of mobile devices equipped with various sensors and high-quality wireless network communications functionsexpands, the amount of spatio-temporal data generated from mobile devices in various service fields is rapidly increasing. In conventional research into processing a large amount of real-time spatio-temporal streams, it is very difficult to apply a Hadoop-based spatial big data system, designed to be a batch processing platform, to a real-time service for spatio-temporal data streams. This paper extends the MapReduce online framework to support real-time query processing for continuous-input, spatio-temporal data streams, and proposes a load management method to distribute overloads for efficient query processing. The proposed scheme shows a dynamic load balancing method for the nodes based on the inflow rate and the load factor of the input data based on the space partition. Experiments show that it is possible to support efficient query processing by distributing the spatial data stream in the corresponding area to the shared resources when load management in a specific area is required.

Design of Extended Real-time Data Pipeline System Architecture (확장형 실시간 데이터 파이프라인 시스템 아키텍처 설계)

  • Shin, Hoseung;Kang, Sungwon;Lee, Jihyun
    • Journal of KIISE
    • /
    • v.42 no.8
    • /
    • pp.1010-1021
    • /
    • 2015
  • Big data systems are widely used to collect large-scale log data, so it is very important for these systems to operate with a high level of performance. However, the current Hadoop-based big data system architecture has a problem in that its performance is low as a result of redundant processing. This paper solves this problem by improving the design of the Hadoop system architecture. The proposed architecture uses the batch-based data collection of the existing architecture in combination with a single processing method. A high level of performance can be achieved by analyzing the collected data directly in memory to avoid redundant processing. The proposed architecture guarantees system expandability, which is an advantage of using the Hadoop architecture. This paper confirms that the proposed architecture is approximately 30% to 35% faster in analyzing and processing data than existing architectures and that it is also extendable.

COMPARISON OF GLOBAL SEA SURFACE TEMPERATURE PRODUCTS

  • Kubota, Masahisa.;Iwasaki, Shinzuke
    • Proceedings of the KSRS Conference
    • /
    • v.2
    • /
    • pp.993-996
    • /
    • 2006
  • NOAA operational bulk SST product (Reynolds et al, 2002) is very popular global SST data sets and is extensively used for various studies. However, the original time resolution is weekly and relatively large. On the other hand, there exist many new global SST data sets at present. In this study, we compare many global SST data sets including NOAA operational bulk SST product, CAOS OI SST product, Microwave Optimum Interpolation (MWOI) SST, Real Time Global (RTG) SST and JMA merged satellite and in situ Global Daily (MGD) SST.

  • PDF

EFFICIENT MANAGEMENT OF VERY LARGE MOVING OBJECTS DATABASE

  • Lee, Seong-Ho;Lee, Jae-Ho;An, Kyoung-Hwan;Park, Jong-Hyun
    • Proceedings of the KSRS Conference
    • /
    • v.2
    • /
    • pp.725-727
    • /
    • 2006
  • The development of GIS and Location-Based Services requires a high-level database that will be able to allow real-time access to moving objects for spatial and temporal operations. MODB.MM is able to meet these requirements quite adequately, providing operations with the abilities of acquiring, storing, and querying large-scale moving objects. It enables a dynamic and diverse query mechanism, including searches by region, trajectory, and temporal location of a large number of moving objects that may change their locations with time variation. Furthermore, MODB.MM is designed to allow for performance upon main memory and the system supports the migration on out-of-date data from main memory to disk. We define the particular query for truncation of moving objects data and design two migration methods so as to operate the main memory moving objects database system and file-based location storage system with.

  • PDF

A Study on Disaster Information Support using Big Data (빅 데이터를 이용한 재해 정보 지원에 관한 연구)

  • Shin, Bong-Hi;Jeon, Hye-Kyoung
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.8
    • /
    • pp.25-32
    • /
    • 2018
  • Recently, the size and type of disasters in Korea has been diversified. However, Korea has not been able to build various information support systems to predict these disasters.Many other organizations also provide relevant information. This information is mainly provided on the Web, but most of it is not real time information. In this study, we have paid attention to support information using big data to provide better quality real - time information together with information provided by institutions. Big data has a large amount of information with real-time property, and it can make customized service using it. Among them, SNS such as Twitter and Facebook can be used as a new information collection medium in case of disaster. However, it is very difficult to retrieve necessary information from too much information, and it is difficult to collect intuitive information. For this purpose, this study develops an information support system using Twitter. The system retrieves information using the Twitter hashtag. Also, information mapping is performed on the map so that intuitive information can be grasped. For system evaluation, information extraction, degree of mapping, and recommendation speed are evaluated.