• Title/Summary/Keyword: MapReduce

Search Result 849, Processing Time 0.026 seconds

Similar Sequence Matching based on SE Matching with MapReduce (맵리듀스를 이용한 SE Matching 기반 유사 검색 기법)

  • Kwon, Jeong-Min;Seo, Jun-Il;An, Jin-Woo;Lee, Jeong-Joon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.04a
    • /
    • pp.224-227
    • /
    • 2012
  • 시계열 데이터 검색은 금융, 생명정보 등 많은 분야에서 요구되는 주요한 기술로서 연구되어 왔다. 특히 기존에 제안된 스케일링(Scaling)과 쉬프팅(Shifting)을 이용한 검색인 SE Match는 유사 서브시퀀스를 효과적으로 찾아내는 방법으로 알려지고 있다. 본 논문에서는 이 방법에 맵리듀스를 적용하는 MRSE-검색 (MapReduce-based Searching with Shift-Eliminated)방법을 제안한다. 본 논문이 제안하는 방법으로 분산처리를 통하여 응답시간의 개선과 대용량의 시계열 데이터에서 효율적인 검색이 가능 할 것으로 사료된다.

Design an Indexing Structure System Based on Apache Hadoop in Wireless Sensor Network

  • Keo, Kongkea;Chung, Yeongjee
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2013.05a
    • /
    • pp.45-48
    • /
    • 2013
  • In this paper, we proposed an Indexing Structure System (ISS) based on Apache Hadoop in Wireless Sensor Network (WSN). Nowadays sensors data continuously keep growing that need to control. Data constantly update in order to provide the newest information to users. While data keep growing, data retrieving and storing are face some challenges. So by using the ISS, we can maximize processing quality and minimize data retrieving time. In order to design ISS, Indexing Types have to be defined depend on each sensor type. After identifying, each sensor goes through the Indexing Structure Processing (ISP) in order to be indexed. After ISP, indexed data are streaming and storing in Hadoop Distributed File System (HDFS) across a number of separate machines. Indexed data are split and run by MapReduce tasks. Data are sorted and grouped depend on sensor data object categories. Thus, while users send the requests, all the queries will be filter from sensor data object and managing the task by MapReduce processing framework.

A MapReduce-based Artificial Neural Network Churn Prediction for Music Streaming Service

  • Chen, Min
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.1
    • /
    • pp.55-60
    • /
    • 2022
  • Churn prediction is a critical long-term problem for many business like music, games, magazines etc. The churn probability can be used to study many aspects of a business including proactive customer marketing, sales prediction, and churn-sensitive pricing models. It is quite challenging to design machine learning model to predict the customer churn accurately due to the large volume of the time-series data and the temporal issues of the data. In this paper, a parallel artificial neural network is proposed to create a highly-accurate customer churn model on a large customer dataset. The proposed model has achieved significant improvement in the accuracy of churn prediction. The scalability and effectiveness of the proposed algorithm is also studied.

A Survey on Massive Data Processing Model in Cloud Computing (클라우드 컴퓨팅에서의 대용량 데이터 처리 모델에 관한 조사)

  • Jin, Ah-Yeon;Park, Young-Ho
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2011.11a
    • /
    • pp.145-146
    • /
    • 2011
  • 클라우드 컴퓨팅은 세계적인 시장조사기관인 가트너사의 10대전략기술에서 2년 연속 1위를 할 정도로 많은 각광을 받고 있다. 클라우드 컴퓨팅이란 인터넷 기술을 활용하여 가상화된 컴퓨팅 자원을 서비스로 제공하는 것으로, 사용자는 IT자원을 필요한 만큼 빌려서 사용하고 사용한 만큼 비용을 지불하는 컴퓨팅을 지칭한다. 이러한 클라우드 컴퓨팅 상에서 폭발적으로 증가하고 있는 데이터를 효율적으로 병렬 처리할 수 있는 방법에 대하여 많은 연구가 활발히 이루어지고 있다. 이러한 대용량 데이터 처리를 위한 대표적인 모델에는 MapReduce와 Dryad가 있으며, 서로간에 많은 공통점이 있지만 MapReduce는 범용 프로그래밍 언어를 기반으로 쉬운 병렬 프로그래밍을 가능하게 했다는 점에서 많이 사용되고 있으며 Dryad는 재사용이 쉽고 데이터 처리 흐름을 유연하게 작성할 수 있다는 점에서 장점을 가지고 있다.

Graph-based Wi-Fi Radio Map Construction and Update Method (그래프 기반 Wi-Fi 신호 지도 구축 및 갱신 기법)

  • Yu, Subin;Choi, Wonik
    • Journal of KIISE
    • /
    • v.44 no.6
    • /
    • pp.643-648
    • /
    • 2017
  • Among Wi-Fi based indoor positioning systems, fingerprinting localization is the most common technique with high precision. However, construction of the initial radio map and the update process require considerable labor and time effort. To address this problem, we propose an efficient method that constructs the initial radio map at each vertex based on a graph. In addition, we introduce a method to update the radio map automatically by mapping signal data acquired from users to the reference point created on each edge. Since the proposed method collects signal data manually only at the vertex of the graph to build the initial radio map and updates it automatically, our proposed method can dramatically reduce labor and time effort, which are the disadvantages of the conventional fingerprinting method. In our experimental study, we show validity of our radio map update method by comparing with the actual reference point data. We also show that our proposed method is able to construct the radio map with an accuracy of about 3.5m by automatically updating the radio map.

Successive MAP Detection with Soft Interference Cancellation for Iterative Receivers in Hierarchical M-ary QAM Systems (M-레벨 QAM 계층 변조 시스템에서 연 간섭 제거를 이용한 연속 MAP 판정 기법)

  • Kim, Jong-Kyung;Seo, Jong-Soo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.3C
    • /
    • pp.304-310
    • /
    • 2009
  • This paper proposes a successive MAP (maximum a posteriori probability) detection scheme with SoIC(soft interference cancellation) to reduce the receiver complexity of hierarchical M-ary QAM system. For the successive MAP detection, modulation symbols generated from the other data streams are treated as Gaussian noise or eliminated as the soft interference according to their priorities. The log-likelihood ratio of the a posteriori probability (LAPRP) of each bit is calculated by the MAP detector with an adjusted noise variance in order to take the elimination and Gaussian assumption effect into account. By separating the detection process into the successive steps, the detection complexity is reduced to increase linearly with the number of bits per hierarchical M-ary QAM symbol. Simulation results show that the proposed detection provides a small performance degradation as compared to the optimal MAP detection.

An Identification of Outlying Cells in Contingency Table via Correspondence Analysis Map

  • Hong, Chong Sun;Lee, Jong Cheol
    • Communications for Statistical Applications and Methods
    • /
    • v.8 no.1
    • /
    • pp.39-49
    • /
    • 2001
  • When an appropriate model is fitted to explain a certain categorical data, outlying cell detection plays very important role to reduce the lack of fit. There exist many statistical methods to identify outlying cells in contingency table. In this paper, correspondence analysis is applied to identify one or two outlying cells. When corresponding relationships between categories of the row and columns are explored, we find that outlying cells could be identified via the correspondence analysis map.

  • PDF

Higher Order Shimming for Ultra-fast Spiral-Scan Imaging at 3 Tesla MRI System (3 Tesla MRI 시스템에서 초고속 나선주사영상을 위한 고차 shimming)

  • Kim, P.K.;Lim, J.W.;Ahn, C.B.
    • Investigative Magnetic Resonance Imaging
    • /
    • v.11 no.2
    • /
    • pp.95-102
    • /
    • 2007
  • Purpose: To acquire high-resolution spiral-scan images at higher magnetic field, high homogeneous magnetic field is needed. Field inhomogeneity mapping and in-vivo shimming are important for rapid imaging such as spiral-scan imaging. The rapid scanning sequences are very susceptible to inhomogeneity. In this paper, we proposed a higher-order shimming method to obtain homogeneous magnetic field. Materials and Methods: To reduce measurement time for field inhomogeneity mapping, simultaneous axial/ sagittal, and coronal acquisitions are done using multi-slice based Fast Spin echo sequence. Acquired field inhomogeneity map is analyzed using the spherical harmonic functions, and shim currents are obtained by the multiplication of the pseudo-inverse of the field pattern with the inhomogeneity map. Results: Since the field inhomogeneity is increasing in proportion to the magnetic field, higher order shimming to reduce the inhomogeneity becomes more important in high field imaging. The shimming technique in which axial, sagittal, and coronal section inhomogeneity maps are obtained in one scan is developed, and the shimming method based on the analysis of spherical harmonics of the imhomogenity map is applied. The proposed technique is applicable to a localized shimming as well. High resolution spiral-scan imaging was successfully obtained with the proposed higher order shimming. Conclusion: Proposed pulse sequence for rapid measurement of inhomogeneity map and higher order shimming based on the inhomogeneity map work very well at 3 Tesla MRI system. With the proposed higher order shimming and localized higher order shimming techniques, high resolution spiral-scan images are successfully obtained at 3 T MRI system.

  • PDF