• Title/Summary/Keyword: distributed parallel processing

Search Result 257, Processing Time 0.03 seconds

Implementation of a Wi-Fi Based Cluster System using Raspberry Pi for Multidisciplinary Education

  • Koo, Geum-Seo;Sim, Gab-Sig
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.1
    • /
    • pp.1-7
    • /
    • 2019
  • In this paper, we implemented a Wi-Fi based cluster system using raspberry pi for multidisciplinary education. The cluster implementation on the desktop was more difficult to maintain the complexity, big size, high price, power consumption as the number of nodes increased. In this paper, we implemented a cluster using Raspberry Pi, which is developed for educational purposes, to reduce the cost of connecting nodes. In addition, the complexity of system construction is reduced by replacing the connection between each node with Wi-Fi. Also, the inconvenience of configuration due to node increase was reduced. It is expected that the implementation of the cluster will be a good alternative in the educational environment where distributed processing and parallel processing are performed in the embedded environment. Also, it is confirmed that it can be applied to the multidisciplinary education.

Fuzzy Inference of Large Volumes in Parallel Computing Environments (병렬컴퓨팅 환경에서의 대용량 퍼지 추론)

  • 김진일;이상구
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.10 no.4
    • /
    • pp.293-298
    • /
    • 2000
  • In fuzzy expert systems or database systems that have volumes of fuzzy data or large fuzzy rules, the inference time is much increased. Therefore, a high performance parallel fuzzy computing environment is needed. In this paper, we propose a parallel fuzzy inference mechanism in parallel computing environments. In this, fuzzy rules are distributed and executed simultaneously. The ONE_TO_ALL algorithm is used to broadcast the fuzzy input input vector to the all nodes. The results of the MIN/MAX operations are transferred to the output processor by the ALL_TO_ONE algorithm. By parallel processing of fuzzy or data, the parallel fuzzy inference algortihm extracts effective and achieves and achieves a good speed factor.

  • PDF

Real Time Distributed Parallel Processing to Visualize Noise Map with Big Sensor Data and GIS Data for Smart Cities (스마트시티의 빅 센서 데이터와 빅 GIS 데이터를 융합하여 실시간 온라인 소음지도로 시각화하기 위한 분산병렬처리 방법론)

  • Park, Jong-Won;Sim, Ye-Chan;Jung, Hae-Sun;Lee, Yong-Woo
    • Journal of Internet Computing and Services
    • /
    • v.19 no.4
    • /
    • pp.1-6
    • /
    • 2018
  • In smart cities, data from various kinds of sensors are collected and processed to provide smart services to the citizens. Noise information services with noise maps using the collected sensor data from various kinds of ubiquitous sensor networks is one of them. This paper presents a research result which generates three dimensional (3D) noise maps in real-time for smart cities. To make a noise map, we have to converge many informal data which include big image data of geographical Information and massive sensor data. Making such a 3D noise map in real-time requires the processing of the stream data from the ubiquitous sensor networks in real-time and the convergence operation in real-time. They are very challenging works. We developed our own methodology for real-time distributed and parallel processing for it and present it in this paper. Further, we developed our own real-time 3D noise map generation system, with the methodology. The system uses open source softwares for it. Here in this paper, we do introduce one of our systems which uses Apache Storm. We did performance evaluation using the developed system. Cloud computing was used for the performance evaluation experiments. It was confirmed that our system was working properly with good performance and the system can produce the 3D noise maps in real-time. The performance evaluation results are given in this paper, as well.

A Hadoop-based Multimedia Transcoding System for Processing Social Media in the PaaS Platform of SMCCSE

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku;Jeong, Changsung
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.6 no.11
    • /
    • pp.2827-2848
    • /
    • 2012
  • Previously, we described a social media cloud computing service environment (SMCCSE). This SMCCSE supports the development of social networking services (SNSs) that include audio, image, and video formats. A social media cloud computing PaaS platform, a core component in a SMCCSE, processes large amounts of social media in a parallel and distributed manner for supporting a reliable SNS. Here, we propose a Hadoop-based multimedia system for image and video transcoding processing, necessary functions of our PaaS platform. Our system consists of two modules, including an image transcoding module and a video transcoding module. We also design and implement the system by using a MapReduce framework running on a Hadoop Distributed File System (HDFS) and the media processing libraries Xuggler and JAI. In this way, our system exponentially reduces the encoding time for transcoding large amounts of image and video files into specific formats depending on user-requested options (such as resolution, bit rate, and frame rate). In order to evaluate system performance, we measure the total image and video transcoding time for image and video data sets, respectively, under various experimental conditions. In addition, we compare the video transcoding performance of our cloud-based approach with that of the traditional frame-level parallel processing-based approach. Based on experiments performed on a 28-node cluster, the proposed Hadoop-based multimedia transcoding system delivers excellent speed and quality.

A Synchronous/Asynchronous Hybrid Parallel Power Iteration for Large Eigenvalue Problems by the MPMD Methodology (MPMD 방식의 동기/비동기 병렬 혼합 멱승법에 의한 거대 고유치 문제의 해법)

  • Park, Pil-Seong
    • The KIPS Transactions:PartA
    • /
    • v.11A no.1
    • /
    • pp.67-74
    • /
    • 2004
  • Most of today's parallel numerical schemes use synchronous algorithms, where some processors that have finished their tasks earlier than others must wait at synchronization points for correct computation. Hence overall performance of the system is dependent upon the speed of the slowest processor. In this paper, we det·ise a synchronous/asynchronous hybrid algorithm to accelerate convergence of the solution for finding the dominant eigenpair of a large matrix, by reducing the idle times of faster processors using MPMD programming methodology.

Hybrid Channel Model in Parallel File System (병렬 파일 시스템에서의 하이브리드 채널 모델)

  • Lee, Yoon-Young;Hwangbo, Jun-Hyung;Seo, Dae-Wha
    • The KIPS Transactions:PartA
    • /
    • v.10A no.1
    • /
    • pp.25-34
    • /
    • 2003
  • Parallel file system solves I/O bottleneck to store a file distributedly and read it parallel exchanging messages among computers that is connected multiple computers with high speed networks. However, they do not consider the message characteristics and performances are decreased. Accordingly, the current study proposes the Hybrid Channel model (HCM) as a message-management method, whereby the messages of a parallel file system are classified by a message characteristic between control messages and file data blocks, and the communication channel is divided into a message channel and data channel. The message channel then transfers the control messages through TCP/IP with reliability, while the data channel that is implemented by Virtual Interface Architecture (VIA) transfers the file data blocks at high speed. In tests, the proposed parallel file system that is implemented by HCM exhibited a considerably improved performance.

Performance Analysis of Distributed Hadoop Systems (분산 하둡 시스템의 성능 비교 분석)

  • Bae, Byoung-Jin;Kim, Young-Joo;Kim, Young-Kuk
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2014.05a
    • /
    • pp.479-482
    • /
    • 2014
  • Nowadays open-source hadoop systems have been using widely to efficiently manage a fast-growing big data. Hadoop systems consist of distributed file processing system called HDFS (Hadoop Distributed File System) and distributed parallel processing system called MapReduce. The MapReduce reads and processes big data from HDFS and then processed results are written in HDFS again by the MapReduce. Such a processing method has different system structure respectively according to hadoop version. Therefore, this paper shows analysis results for performance of hadoop systems. For this, we devise a way which monitors hadoop systems and measure occurrence frequency of processes, threads, and variables generated in hadoop system itself using the devised way. So, by using the measured results as analysis indicator, we help the indicator predict inner performance of hadoop systems.

  • PDF

A Global Framework for Parallel and Distributed Application with Mobile Objects (이동 객체 기반 병렬 및 분산 응용 수행을 위한 전역 프레임워크)

  • Han, Youn-Hee;Park, Chan-Yeol;Hwang, Chong-Sun;Jeong, Young-Sik
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.6 no.6
    • /
    • pp.555-568
    • /
    • 2000
  • The World Wide Web has become the largest virtual system that is almost universal in scope. In recent research, it has become effective to utilize idle hosts existing in the World Wide Web for running applications that require a substantial amount of computation. This novel computing paradigm has been referred to as the advent of global computing. In this paper, we implement and propose a mobile object-based global computing framework called Tiger, whose primary goal is to present novel object-oriented programming libraries that support distribution, dispatching, migration of objects and concurrency among computational activities. The programming libraries provide programmers with access, location and migration transparency for distributed and mobile objects. Tiger's second goal is to provide a system supporting requisites for a global computing environment - scalability, resource and location management. The Tiger system and the programming libraries provided allow a programmer to easily develop an objectoriented parallel and distributed application using globally extended computing resources. We also present the improvement in performance gained by conducting the experiment with highly intensive computations such as parallel fractal image processing and genetic-neuro-fuzzy algorithms.

  • PDF

An Efficient Distributed Shared Memory System for Parallel GIS (병렬 GIS를 위한 효율적인 분산공유메모리 시스템)

  • Jeong, Sang-Hwa;Ryu, Gwang-Yeol;Go, Yun-Yeong;Gwak, Min-Seok
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.6
    • /
    • pp.700-707
    • /
    • 1999
  • 본 논문에서는 GIS 관련 연산을 실시간에 효율적으로 처리하기 위한 분산공유메모리 기반 병렬처리 시스템을 제안한다. 본 논문의 분산공유메모리 시스템은 메시지전달 방식의 분산메모리 MIMD 컴퓨터 상에 소프트웨어 기반 분산공유메모리 모듈을 탑재함으로써 구현되었다. 또한 GIS 연산의 기본이 되는 공간 객체를 공유의 기본 단위로 설정하고, GIS 데이타의 특성을 반영하여 읽기전용 공유데이타 타입을 추가하였으며, 네트워크 오버헤드를 줄이기 위하여 복수의 객체를 한번에 읽어오는 bulk access가 가능하도록 하였다. 본 시스템에서는 GIS 데이타의 효율적인 분배를 위하여 부하균등화 기법으로 guided self scheduling을 사용하였다. 실험결과 본 시스템은 네트워크 캐쉬의 효율적인 활용을 통하여 소프트웨어 기반 분산메모리 시스템의 오버헤드에도 불구하고 MPI 기반 메시지전달 방식에 비하여 향상된 성능을 얻을 수 있었다.Abstract In this paper, we propose a distributed shared memory(DSM) based parallel processing system to process GIS related computations efficiently in real time. The system is based on a software DSM module implemented on top of a distributed MIMD computer. In the DSM system, spatial object, which is a fundamental structure to represent GIS data, is used as a basic unit for sharing, and a read-only shared data type is added to reflect the characteristics of GIS data. In addition, a bulk access to multiple shared data is made possible to reduce the network overhead. A guided self scheduling method is devised for efficient load balancing in distributing GIS data to parallel processors. The experimental results show that the DSM system performs better than an MPI based message-passing system through the efficient utilization of network cache in spite of the system's software overhead.

Development of High Performance Massively Parallel Processing Simulator for Semiconductor Etching Process (건식 식각 공정을 위한 초고속 병렬 연산 시뮬레이터 개발)

  • Lee, Jae-Hee;Kwon, Oh-Seob;Ban, Yong-Chan;Won, Tae-Young
    • Journal of the Korean Institute of Telematics and Electronics D
    • /
    • v.36D no.10
    • /
    • pp.37-44
    • /
    • 1999
  • This paper report the implementation results of Monte Carlo numerical calculation for ion distributions in plasma dry etching chamber and of the surface evolution simulator using cell removal method for topographical evolution of the surface exposed to etching ion. The energy and angular distributions of ion across the plasma sheath were calculated by MC(Monte Carlo) algorithm. High performance MPP(Massively Parallel Processing) algorithm developed in this paper enables efficient parallel and distributed simulation with an efficiency of more than 95% and speedup of 16 with 16 processors. Parallelization of surface evolution simulator based on cell removal method reduces simulation time dramatically to 15 minutes and increases capability of simulation required enormous memory size of 600Mb.

  • PDF