• Title/Summary/Keyword: Massive Information Processing Technology

Search Result 72, Processing Time 0.024 seconds

An Efficient Complex Event Processing Algorithm based on Multipattern Sharing for Massive Manufacturing Event Streams

  • Wang, Jianhua;Lan, Yubin;Lu, Shilei;Cheng, Lianglun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.3
    • /
    • pp.1385-1402
    • /
    • 2019
  • Quickly picking up some valuable information from massive manufacturing event stream usually faces with the problem of long detection time, high memory consumption and low detection efficiency due to its stream characteristics of large volume, high velocity, many variety and small value. Aiming to solve the problem above for the current complex event processing methods because of not sharing detection during the detecting process for massive manufacturing event streams, an efficient complex event processing method based on multipattern sharing is presented in this paper. The achievement of this paper lies that a multipattern sharing technology is successfully used to realize the quick detection of complex event for massive manufacturing event streams. Specially, in our scheme, we firstly use pattern sharing technology to merge all the same prefix, suffix, or subpattern that existed in single pattern complex event detection models into a multiple pattern complex event detection model, then we use the new detection model to realize the quick detection for complex events from massive manufacturing event streams, as a result, our scheme can effectively solve the problems above by reducing lots of redundant building, storing, searching and calculating operations with pattern sharing technology. At the end of this paper, we use some simulation experiments to prove that our proposed multiple pattern processing scheme outperforms some general processing methods in current as a whole.

Introduction to general purpose GPU computing (GPU를 이용한 범용 계산의 소개)

  • Yu, Donghyeon;Lim, Johan
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.5
    • /
    • pp.1043-1061
    • /
    • 2013
  • Recent advances in computer technology introduce massive data and their analysis becomes important. The high performance computing is one of the most essential part in analysis of massive data. In this paper, we review the general purpose of the graphics processing unit and its application to parallel computing, which has been of great interest in statistics communities.

Massive MIMO Channel Estimation Algorithm Based on Weighted Compressed Sensing

  • Lv, Zhiguo;Wang, Weijing
    • Journal of Information Processing Systems
    • /
    • v.17 no.6
    • /
    • pp.1083-1096
    • /
    • 2021
  • Compressed sensing-based matching pursuit algorithms can estimate the sparse channel of massive multiple input multiple-output systems with short pilot sequences. Although they have the advantages of low computational complexity and low pilot overhead, their accuracy remains insufficient. Simply multiplying the weight value and the estimated channel obtained in different iterations can only improve the accuracy of channel estimation under conditions of low signal-to-noise ratio (SNR), whereas it degrades accuracy under conditions of high SNR. To address this issue, an improved weighted matching pursuit algorithm is proposed, which obtains a suitable weight value uop by training the channel data. The step of the weight value increasing with successive iterations is calculated according to the sparsity of the channel and uop. Adjusting the weight value adaptively over the iterations can further improve the accuracy of estimation. The results of simulations conducted to evaluate the proposed algorithm show that it exhibits improved performance in terms of accuracy compared to previous methods under conditions of both high and low SNR.

ECPS: Efficient Cloud Processing Scheme for Massive Contents (클라우드 환경에서 대규모 콘텐츠를 위한 효율적인 자원처리 기법)

  • Na, Moon-Sung;Kim, Seung-Hoon;Lee, Jae-Dong
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.15 no.4
    • /
    • pp.17-27
    • /
    • 2010
  • Major IT vendors expect that cloud computing technology makes it possible to reduce the contents service cycle, speed up application deployment and skip the installation process, reducing operational costs, proactive management etc. However, cloud computing environment for massive content service solutions requires high-performance data processing to reduce the time of data processing and analysis. In this study, Efficient_Cloud_Processing_Scheme(ECPS) is proposed for allocation of resources for massive content services. For high-performance services, optimized resource allocation plan is presented using MapReduce programming techniques and association rules that is used to detect hidden patterns in data mining, based on levels of Hadoop platform(Infrastructure as a service). The proposed ECPS has brought more than 20% improvement in performance and speed compared to the traditional methods.

Parallelization of Raster GIS Operations Using PC Clusters (PC 클러스터를 이용한 래스터 GIS 연산의 병렬화)

  • 신윤호;박수홍
    • Spatial Information Research
    • /
    • v.11 no.3
    • /
    • pp.213-226
    • /
    • 2003
  • With the increasing demand of processing massive geographic data, conventional GISs based on the single processor architecture appear to be problematic. Especially, performing complex GIS operations on the massive geographic data is very time consuming and even impossible. This is due to the processor speed development does not keep up with the data volume to be processed. In the field of GIS, this PC clustering is one of the emerging technology for handling massive geographic data effectively. In this study, a MPI(Message Passing Interface)-based parallel processing approach was conducted to implement the existing raster GIS operations that typically requires massive geographic data sets in order to improve the processing capabilities and performance. Specially for this research, four types of raster CIS operations that Tomlin(1990) has introduced for systematic analysis of raster GIS operation. A data decomposition method was designed and implemented for selected raster GIS operations.

  • PDF

Development of Technical and Economic Evaluation Model for Seafloor Massive Sulfide Deposits (해저열수광상 기술.경제성평가 모델 개발)

  • Park, Se-Hun;Park, Seong-Wook;Kwon, Suk-Jae
    • Ocean and Polar Research
    • /
    • v.28 no.2
    • /
    • pp.187-199
    • /
    • 2006
  • The Kuroko-type seafloor massive sulfide deposits found in the western Pacific have been considered to have potentials for economic recovery of Au, Ag, Cu, Zn, and Pb. In this study, a preliminary model was developed for the technical and economic evaluation of them. The FRSC site on Lau Basin in the Tonga EEZ was selected as a target. In this study, no construction In for the metallurgical processing subsystem was accounted for. Instead, it was assumed to sell the Cu, Zn, and Pb concentrates to the existing sulfide customer smelter. The low total investment costs for the development make the venture very attractive. However, the result of the economic feasibility evaluation is still less attractive with the mean metal yield of the Kuroko on land. It is considered that commercial mining may be plausible if the richer metal yields are applied to the development. Quantitative information for metal yield is necessary for a more accurate evaluation. However, the important resource potential information regarding the amount of ore body, the inside structure, and the metal yields have not yet been clarified sufficiently. h addition, the flotation of ore body using seawater has not been tested yet. It is necessary to solve these problems through the experimental R&D and a survey.

Performance Enhancement of A Massive Scientific Data Visualization System on Virtual Reality Environment by Using Data Locality (Data Locality를 활용한 VR환경에서의 대용량 데이터 가시화 시스템의 성능 개선)

  • Lee, Se-Hoon;Kim, Min-Ah;Lee, Joong-Yeon;Hur, Young-Ju
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.11a
    • /
    • pp.284-287
    • /
    • 2012
  • GLOVE(GLObal Virtual reality visualization Environment for scientific simulation)는 컴퓨팅 자원의 성능 향상으로 데이터 양이 급속히 증가한 응용 과학과 전산 시뮬레이션 분야의 대용량 과학 데이터를 효율적으로 가시화하여 분석하기 위한 도구이다. GLOVE의 데이터 관리자인 GDM(GLOVE Data Manager)은 대용량 데이터의 분산 병렬 가시화를 위해 분산 공유 메모리를 제공하는 GA(Global Array)를 이용해 테라 바이트 단위의 데이터를 실시간으로 처리한다. 그러나 대용량 과학 데이터를 가시화 하는 과정에서 기존의 Data Locality를 고려하지 않은 데이터 접근 방식으로 인한 성능 저하를 확인했다. 본 논문은 기존 GLOVE에서 발견한 성능 저하 현상을 밝히고, 이에 대한 해결 방법을 제시한다.

Design and Implementation of MQTT Message Server for a massive connection processing in IoT Environment (IoT 환경에서 대량 접속처리를 위한 MQTT 메시지 서버 설계 및 구현)

  • Cha, Woosuk;Yoo, Eunkuk;Kim, Yeongjun;Kim, Jinsoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.10a
    • /
    • pp.936-938
    • /
    • 2018
  • 본 논문은 IoT 환경에서 MQTT 대량 브로커와 대량 클라이언트간 효율적인 접속을 지원하기 위해 대량 접속처리 기능을 제공하는 CA(Connectionb Agent)를 포함한 MQTT 메시지 서버를 설계, 구현하였다. MTQQ 프로토콜은 MQTT 브로커와 MQTT 클라이언트로 구성되며, 각 MQTT 클라이언트는 MQTT 브로커의 중재를 받아 Pub/Sub 방식으로 메시지를 상호 전송한다. 이를 위해 MQTT 프로토콜은 MQTT 브로커와 MQTT 클라이언트 간 접속기능만을 제공한다. 실험결과에서 MQTT 메시지 서버는 초당 평균 12,500 건의 클라이언트 접속요청을 처리하였고, 20만건의 접속요청 처리에 16초가 소요되었다.

Design and Implementation of Dynamic Recommendation Service in Big Data Environment

  • Kim, Ryong;Park, Kyung-Hye
    • Journal of Information Technology Applications and Management
    • /
    • v.26 no.5
    • /
    • pp.57-65
    • /
    • 2019
  • Recommendation Systems are information technologies that E-commerce merchants have adopted so that online shoppers can receive suggestions on items that might be interesting or complementing to their purchased items. These systems stipulate valuable assistance to the user's purchasing decisions, and provide quality of push service. Traditionally, Recommendation Systems have been designed using a centralized system, but information service is growing vast with a rapid and strong scalability. The next generation of information technology such as Cloud Computing and Big Data Environment has handled massive data and is able to support enormous processing power. Nevertheless, analytic technologies are lacking the different capabilities when processing big data. Accordingly, we are trying to design a conceptual service model with a proposed new algorithm and user adaptation on dynamic recommendation service for big data environment.

Widely-Linear Beamforming and RF Impairment Suppression in Massive Antenna Arrays

  • Hakkarainen, Aki;Werner, Janis;Dandekar, Kapil R.;Valkama, Mikko
    • Journal of Communications and Networks
    • /
    • v.15 no.4
    • /
    • pp.383-397
    • /
    • 2013
  • In this paper, the sensitivity of massive antenna arrays and digital beamforming to radio frequency (RF) chain in-phase quadrature-phase (I/Q) imbalance is studied and analyzed. The analysis shows that massive antenna arrays are increasingly sensitive to such RF chain imperfections, corrupting heavily the radiation pattern and beamforming capabilities. Motivated by this, novel RF-aware digital beamforming methods are then developed for automatically suppressing the unwanted effects of the RF I/Q imbalance without separate calibration loops in all individual receiver branches. More specifically, the paper covers closed-form analysis for signal processing properties as well as the associated radiation and beamforming properties of massive antenna arrays under both systematic and random RF I/Q imbalances. All analysis and derivations in this paper assume ideal signals to be circular. The well-known minimum variance distortionless response (MVDR) beamformer and a widely-linear (WL) extension of it, called WL-MVDR, are analyzed in detail from the RF imperfection perspective, in terms of interference attenuation and beamsteering. The optimum RF-aware WL-MVDR beamforming solution is formulated and shown to efficiently suppress the RF imperfections. Based on the obtained results, the developed solutions and in particular the RF-aware WL-MVDR method can provide efficient beamsteering and interference suppressing characteristics, despite of the imperfections in the RF circuits. This is seen critical especially in the massive antenna array context where the cost-efficiency of individual RF chains is emphasized.