• Title/Summary/Keyword: Very large real-time data

Search Result 130, Processing Time 0.032 seconds

An Efficient Storing Scheme of Real-time Large Data to improve Semiconductor Process Productivities (반도체 공정의 생산성 향상을 위한 실시간 대용량 데이터의 효율적인 저장 기법)

  • Chung, Weon-Il;Kim, Hwan-Koo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.10 no.11
    • /
    • pp.3207-3212
    • /
    • 2009
  • Automatic semiconductor manufacturing systems are demanded to improve the efficiency of the semiconductor production process. These systems include the functionalities such as the analysis and management schemes for very large real-time data in order to enhance the productivities. So, it requires the efficient storage management system to store very large real-time data. Traditional database management systems(e.g. Oracle, MY-SQL, MS-SQL) are based on disk. However, previous DBMS's have the limitation on the low storing performance. In this paper, we propose a compress-merge storing method of very large real-time data using insert transaction of a block unit. The proposed method shows better processing performances compare to conventional DBMS's. Also compress-merge method makes it possible that it can store large real-time data on low storage cost. Therefore, the proposed method can be applied to an efficient storage management system in the semiconductor production process.

A High-rate GPS Data Processing for Large-scale Structure Monitoring (대형구조물 모니터링을 위한 high-rate GPS 자료처리)

  • Bae, Tea-Suk
    • Proceedings of the Korean Society of Surveying, Geodesy, Photogrammetry, and Cartography Conference
    • /
    • 2010.04a
    • /
    • pp.181-182
    • /
    • 2010
  • For real-time displacement monitoring of large-scale structures, the high-rate (>1 Hz) GPS data processing is necessary, which is not possible even for the scientific GPS data processing softwares. Since the baseline is generally very short in this case, most of the atmospheric effects are removed, resulting in the unknowns of position and integer ambiguity. The number of unknowns in real-time kinematic GPS positioning makes the positioning impossible with usual approach, thus two-step approach is tested in this study.

  • PDF

A STUDY ON ENCODING/DECODING TECHNIQUE OF SENSOR DATA FOR A MOBILE MAPPING SYSTEM

  • Bae, Sang-Keun;Kim, Byung-Guk
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.705-708
    • /
    • 2005
  • Mobile Mapping Systems using the vehicle equipped the GPS, IMU, CCD Cameras is the effective system for the management of the road facilities, update of the digital map, and etc. They must provide users with the sensor data which is acquired by Mobile Mapping Systems in real-time so that users can process what they want by using the latest data. But it' s not an easy process because the amount of sensor data is very large, particularly image data to be transmitted. So it is necessary to reduce the amount of image data so that it is transmitted effectively. In this study, the effective method was suggested for the compression/decompression image data using the Wavelet Transformation and Huffman Coding. This technique will be possible to transmit of the geographic information effectively such as position data, attitude data, and image data acquired by Mobile Mapping Systems in the wireless internet environment when data is transmitted in real-time.

  • PDF

Estimation of Moving Information for Tracking of Moving Objects

  • Park, Jong-An;Kang, Sung-Kwan;Jeong, Sang-Hwa
    • Journal of Mechanical Science and Technology
    • /
    • v.15 no.3
    • /
    • pp.300-308
    • /
    • 2001
  • Tracking of moving objects within video streams is a complex and time-consuming process. Large number of moving objects increases the time for computation of tracking the moving objects. Because of large computations, there are real-time processing problems in tracking of moving objects. Also, the change of environment causes errors in estimation of tracking information. In this paper, we present a new method for tracking of moving objects using optical flow motion analysis. Optical flow represents an important family of visual information processing techniques in computer vision. Segmenting an optical flow field into coherent motion groups and estimating each underlying motion are very challenging tasks when the optical flow field is projected from a scene of several moving objects independently. The problem is further complicated if the optical flow data are noisy and partially incorrect. Optical flow estimation based on regulation method is an iterative method, which is very sensitive to the noisy data. So we used the Combinatorial Hough Transform (CHT) and Voting Accumulation for finding the optimal constraint lines. To decrease the operation time, we used logical operations. Optical flow vectors of moving objects are extracted, and the moving information of objects is computed from the extracted optical flow vectors. The simulation results on the noisy test images show that the proposed method finds better flow vectors and more correctly estimates the moving information of objects in the real time video streams.

  • PDF

A VLSI Architecture for the Real-Time 2-D Digital Signal Processing (실시간 2차원 디지털 신호처리를 위한 VLSI 구조)

  • 권희훈
    • Information and Communications Magazine
    • /
    • v.9 no.9
    • /
    • pp.72-85
    • /
    • 1992
  • The throughput requirement for many digital signal processing is such that multiple processing units are essential for real-time implementation. Advances in VLSI technology make it feasible to design and implement computer systems consisting of a large number of function units. The research on a very high throughput VLSI architecture for digital signal processing applications requires the development of an algorithm, decomposition scheme which can minimize data communication requirements as well as minimize computational complexity. The objectives of the research are to investigate computationally efficient algorithms for solution of the class of problems which can be modeled as DLSI systems or adaptive system, and develop VLSI architectures and associated multiprocessor systems which can be used to implement these algorithms in real-time. A new VLSI architecture for real-time 2-D digital signal processing applications is proposed in this research. This VLSI architecture extends the concept of having a single processing units in a chip. Because this VLSI architecture has the advantage that the complexity and the number of computations per input does not increase as the size of the input data in increased, it can process very large 2-D date in near real-time.

  • PDF

Boosting Algorithms for Large-Scale Data and Data Batch Stream (대용량 자료와 순차적 자료를 위한 부스팅 알고리즘)

  • Yoon, Young-Joo
    • The Korean Journal of Applied Statistics
    • /
    • v.23 no.1
    • /
    • pp.197-206
    • /
    • 2010
  • In this paper, we propose boosting algorithms when data are very large or coming in batches sequentially over time. In this situation, ordinary boosting algorithm may be inappropriate because it requires the availability of all of the training set at once. To apply to large scale data or data batch stream, we modify the AdaBoost and Arc-x4. These algorithms have good results for both large scale data and data batch stream with or without concept drift on simulated data and real data sets.

Real time Storage Manager to store very large datausing block transaction (블록 단위 트랜잭션을 이용한 대용량 데이터의 실시간 저장관리기)

  • Baek, Sung-Ha;Lee, Dong-Wook;Eo, Sang-Hun;Chung, Warn-Ill;Kim, Gyoung-Bae;Oh, Young-Hwan;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.10 no.2
    • /
    • pp.1-12
    • /
    • 2008
  • Automatic semiconductor manufacture system generating transaction from 50,000 to 500,000 per a second needs storage management system processing very large data at once. A lot of storage management systems are researched for storing very large data. Existing storage management system is typical DBMS on a disk. It is difficult that the DBMS on a disk processes the 500,000 number of insert transaction per a second. So, the DBMS on main memory appeared to use memory. But it is difficultthat very large data stores into the DBMS on a memory because of limited amount of memory. In this paper we propose storage management system using insert transaction of a block unit that can process insert transaction over 50,000 and store data on low storage cost. A transaction of a block unit can decrease cost for a log and index per each tuple as transforming a transaction of a tuple unit to a block unit. Besides, the proposed system come cost to decompress all block of data because the information of each field be loss. To solve the problems, the proposed system generates the index of each compressed block to prevent reducing speed for searching. The proposed system can store very large data generated in semiconductor system and reduce storage cost.

  • PDF

Spatial-Temporal Modelling of Road Traffic Data in Seoul City

  • Lee, Sang-Yeol;Ahn, Soo-Han;Park, Chang-Yi;Jeon, Jong-Woo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.13 no.2
    • /
    • pp.261-270
    • /
    • 2002
  • Recently, the demand of the Intelligent Transportation System(ITS) has been increased to a large extent, and a real-time traffic information service based on the internet system became very important. When ITS companies carry out real-time traffic services, they find some traffic data missing, and use the conventional method of reconstructing missing values by calculating average time trend. However, the method is found unsatisfactory, so that we develop a new method based the spatial and spatial-temporal models. A cross-validation technique shows that the spatial-temporal model outperforms the others.

  • PDF

A Diagnosing System Development for Insulated Stator of Large Generator (대형 발전기의 고정자 절연 진단용 시스템 개발)

  • Park, Seong-Wook
    • The Transactions of the Korean Institute of Electrical Engineers P
    • /
    • v.51 no.1
    • /
    • pp.52-54
    • /
    • 2002
  • This research presented in this paper focuses on a system development for checking stator's wedge of large generator in heavy industry. The system is composed of impactor, A/D converter and digital signal processing board. Also this paper proposes a method for detection of stator's insulating state using analysis algorithm. As developed impact system is acting, impact wave is generated real time and transferred data to microprocessor. Using communication program this signal move microprocessor to hard directory of note book with 1000 data/sec. The developed system and analysis program performed very well by real stator's wedge in large generator.

Large Flows Detection, Marking, and Mitigation based on sFlow Standard in SDN

  • Afaq, Muhammad;Rehman, Shafqat;Song, Wang-Cheol
    • Journal of Korea Multimedia Society
    • /
    • v.18 no.2
    • /
    • pp.189-198
    • /
    • 2015
  • Despite the fact that traffic engineering techniques have been comprehensively utilized in the past to enhance the performance of communication networks, the distinctive characteristics of Software Defined Networking (SDN) demand new traffic engineering techniques for better traffic control and management. Considering the behavior of traffic, large flows normally carry out transfers of large blocks of data and are naturally packet latency insensitive. However, small flows are often latency-sensitive. Without intelligent traffic engineering, these small flows may be blocked in the same queue behind megabytes of file transfer traffic. So it is very important to identify large flows for different applications. In the scope of this paper, we present an approach to detect large flows in real-time without even a short delay. After the detection of large flows, the next problem is how to control these large flows effectively and prevent network jam. In order to address this issue, we propose an approach in which when the controller is enabled, the large flow is mitigated the moment it hits the predefined threshold value in the control application. This real-time detection, marking, and controlling of large flows will assure an optimize usage of an overall network.