• Title/Summary/Keyword: distributed programming

Search Result 288, Processing Time 0.028 seconds

An Efficient Implementation of Mobile Raspberry Pi Hadoop Clusters for Robust and Augmented Computing Performance

  • Srinivasan, Kathiravan;Chang, Chuan-Yu;Huang, Chao-Hsi;Chang, Min-Hao;Sharma, Anant;Ankur, Avinash
    • Journal of Information Processing Systems
    • /
    • v.14 no.4
    • /
    • pp.989-1009
    • /
    • 2018
  • Rapid advances in science and technology with exponential development of smart mobile devices, workstations, supercomputers, smart gadgets and network servers has been witnessed over the past few years. The sudden increase in the Internet population and manifold growth in internet speeds has occasioned the generation of an enormous amount of data, now termed 'big data'. Given this scenario, storage of data on local servers or a personal computer is an issue, which can be resolved by utilizing cloud computing. At present, there are several cloud computing service providers available to resolve the big data issues. This paper establishes a framework that builds Hadoop clusters on the new single-board computer (SBC) Mobile Raspberry Pi. Moreover, these clusters offer facilities for storage as well as computing. Besides the fact that the regular data centers require large amounts of energy for operation, they also need cooling equipment and occupy prime real estate. However, this energy consumption scenario and the physical space constraints can be solved by employing a Mobile Raspberry Pi with Hadoop clusters that provides a cost-effective, low-power, high-speed solution along with micro-data center support for big data. Hadoop provides the required modules for the distributed processing of big data by deploying map-reduce programming approaches. In this work, the performance of SBC clusters and a single computer were compared. It can be observed from the experimental data that the SBC clusters exemplify superior performance to a single computer, by around 20%. Furthermore, the cluster processing speed for large volumes of data can be enhanced by escalating the number of SBC nodes. Data storage is accomplished by using a Hadoop Distributed File System (HDFS), which offers more flexibility and greater scalability than a single computer system.

Scalable Ontology Reasoning Using GPU Cluster Approach (GPU 클러스터 기반 대용량 온톨로지 추론)

  • Hong, JinYung;Jeon, MyungJoong;Park, YoungTack
    • Journal of KIISE
    • /
    • v.43 no.1
    • /
    • pp.61-70
    • /
    • 2016
  • In recent years, there has been a need for techniques for large-scale ontology inference in order to infer new knowledge from existing knowledge at a high speed, and for a diversity of semantic services. With the recent advances in distributed computing, developments of ontology inference engines have mostly been studied based on Hadoop or Spark frameworks on large clusters. Parallel programming techniques using GPGPU, which utilizes many cores when compared with CPU, is also used for ontology inference. In this paper, by combining the advantages of both techniques, we propose a new method for reasoning large RDFS ontology data using a Spark in-memory framework and inferencing distributed data at a high speed using GPGPU. Using GPGPU, ontology reasoning over high-capacity data can be performed as a low cost with higher efficiency over conventional inference methods. In addition, we show that GPGPU can reduce the data workload on each node through the Spark cluster. In order to evaluate our approach, we used LUBM ranging from 10 to 120. Our experimental results showed that our proposed reasoning engine performs 7 times faster than a conventional approach which uses a Spark in-memory inference engine.

ABox Realization Reasoning in Distributed In-Memory System (분산 메모리 환경에서의 ABox 실체화 추론)

  • Lee, Wan-Gon;Park, Young-Tack
    • Journal of KIISE
    • /
    • v.42 no.7
    • /
    • pp.852-859
    • /
    • 2015
  • As the amount of knowledge information significantly increases, a lot of progress has been made in the studies focusing on how to reason large scale ontology effectively at the level of RDFS or OWL. These reasoning methods are divided into TBox classifications and ABox realizations. A TBox classification mainly deals with integrity and dependencies in schema, whereas an ABox realization mainly handles a variety of issues in instances. Therefore, the ABox realization is very important in practical applications. In this paper, we propose a realization method for analyzing the constraint of the specified class, so that the reasoning system automatically infers the classes to which instances belong. Unlike conventional methods that take advantage of the object oriented language based distributed file system, we propose a large scale ontology reasoning method using spark, which is a functional programming-based in-memory system. To verify the effectiveness of the proposed method, we used instances created from the Wine ontology by W3C(120 to 600 million triples). The proposed system processed the largest 600 million triples and generated 951 million triples in 51 minutes (696 K triple / sec) in our largest experiment.

A Study on IoT information Generation Tool for User Defined Web Services (사용자 정의 웹 서비스를 위한 IoT 정보 자동생성 도구에 관한 연구)

  • Sim, Sungho
    • Journal of Digital Convergence
    • /
    • v.16 no.11
    • /
    • pp.329-334
    • /
    • 2018
  • Web services are standardized software technologies that enable interoperability of operating systems and programming languages through networks and related standards. Web services are distributed computing services that provide and discover services making it possible to access various services. Since the search method of web service considers only the functional aspect, it has a limitation on user-oriented search when selecting a service. In order to solve these problems, this study proposes an automatic IoT information generation tool, and provides IoT extension information when searching a web service, thereby improving the problem so that a suitable service can be selected for a user. Automatic IoT extension information generation tool proposed in this study collects and stores various information generated in the process of sensing, networking, and information processing by collaborating autonomously in a distributed environment of user, object, and service. The proposed method supports the service search suitable for the user by providing the information generated by the user as extended information when searching the web service. The proposed method can be applied to the 4th industry sector to provide a customized service that meets various environment requirements.

Molecular Docking System using Parallel GPU (병렬 GPU를 이용한 분자 도킹 시스템)

  • Park, Sung-Jun
    • The Journal of the Korea Contents Association
    • /
    • v.8 no.12
    • /
    • pp.441-448
    • /
    • 2008
  • The molecular docking system needs a large amount of computation and requires super-computing power. Since the experiment requires a large amount of time, the experiment is conducted in the distributed environment or in the grid environment. Recently, researches on using parallel GPU of far higher performance than that of CPU in scientific computing have been very actively conducted. CUDA is an open technique by which a parallel GPU programming is made possible. This study proposes the molecular docking system using CUDA. It also proposes algorithm that parallels energy-minimizing-computation. To verify such experiments, this study conducted a comparative analysis on the time required for experimenting molecular docking in general CPU and the time and performance of the parallel GPU-based molecular docking which is proposed in this study.

Profiler Design for Evaluating Performance of WebCL Applications (WebCL 기반 애플리케이션의 성능 평가를 위한 프로파일러 설계 및 구현)

  • Kim, Cheolwon;Cho, Hyeonjoong
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.4 no.8
    • /
    • pp.239-244
    • /
    • 2015
  • WebCL was proposed for high complex computing in Javascript. Since WebCL-based applications are distributed and executed on an unspecified number of general clients, it is important to profile their performances on different clients. Several profilers have been introduced to support various programming languages but WebCL profiler has not been developed yet. In this paper, we present a WebCL profiler to evaluate WebCL-based applications and monitor the status of GPU on which they run. This profiler helps developers know the execution time of applications, memory read/write time, GPU statues such as its power consumption, temperature, and clock speed.

Exploitation of IP-based Intelligent Networked Measuring and Control Device and System

  • Liu, Gui-Xiong;Luo, Yi;Fang, Xiao-Dong
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2003.10a
    • /
    • pp.1235-1239
    • /
    • 2003
  • On the base of network frame and protocol system of Ethernet the networked sensing technology based on Ethernet is studied and the design principles of industrial Ethernet measurement of control system is put forward, and the general structure model is built in the paper. An eight-bit economical MCU scheme is proposed, and a general scheme of distributed intelligent networked measuring and control equipment based on TCP/IP is designed too. A compact TCP/IP protocol stack are successfully implemented in eight-bit MCU. With C51 program language, method of modularized programming is applied in soft design. The problem of in-system modifying measuring and control strategy of its system is solved successfully by assigning memory dynamically and saving parameter with EEPROM, and it makes the intelligent networked measurement and control system can explain and analyses control strategy from PC. Experiment result shows that, the research of intelligent networked measurement and control equipment and system base on TCP/IP is successful, with flexible network, convenient usage, and good commonality.

  • PDF

Pilot Symbol Assisted Weighted Data Fusion Scheme for Uplink Base-Station Cooperation System

  • Zhang, Zhe;Yang, Jing;Zhang, Jiankang;Mu, Xiaomin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.2
    • /
    • pp.528-544
    • /
    • 2015
  • Base Station Cooperation (BSC) has been a promising technique for combating the Inter-Cell Interference (ICI) by exchanging information through a high-speed optical fiber back-haul to increase the diversity gain. In this paper, we propose a novel pilot symbol assisted data fusion scheme for distributed Uplink BSC (UBSC) based on Differential Evolution (DE) algorithm. Furthermore, the proposed scheme exploits the pre-defined pilot symbols as the sample of transmitted symbols to constitute a sub-optimal Weight Calculation (WC) model. To circumvent the non-linear programming problem of the proposed sub-optimal model, DE algorithm is employed for searching the proper fusion weights. Compared with the existing equal weights based soft combining scheme, the proposed scheme can adaptively adjust the fusion weights according to the accuracy of cooperative information, which remains the relatively low computational complexity and back-haul traffic. Performance analysis and simulation results show that, the proposed scheme can significantly improve the system performance with the pilot settings of the existing standards.

A Joint Resource Allocation Scheme for Relay Enhanced Multi-cell Orthogonal Frequency Division Multiple Networks

  • Fu, Yaru;Zhu, Qi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.7 no.2
    • /
    • pp.288-307
    • /
    • 2013
  • This paper formulates resource allocation for decode-and-forward (DF) relay assisted multi-cell orthogonal frequency division multiple (OFDM) networks as an optimization problem taking into account of inter-cell interference and users fairness. To maximize the transmit rate of system we propose a joint interference coordination, subcarrier and power allocation algorithm. To reduce the complexity, this semi-distributed algorithm divides the primal optimization into three sub-optimization problems, which transforms the mixed binary nonlinear programming problem (BNLP) into standard convex optimization problems. The first layer optimization problem is used to get the optimal subcarrier distribution index. The second is to solve the problem that how to allocate power optimally in a certain subcarrier distribution order. Based on the concept of equivalent channel gain (ECG) we transform the max-min function into standard closed expression. Subsequently, with the aid of dual decomposition, water-filling theorem and iterative power allocation algorithm the optimal solution of the original problem can be got with acceptable complexity. The third sub-problem considers dynamic co-channel interference caused by adjacent cells and redistributes resources to achieve the goal of maximizing system throughput. Finally, simulation results are provided to corroborate the proposed algorithm.

Application Methodology of XML Test Assertion for BioAPI Standard Conformance Tests in Distributed Environment (분산 환경에서의 생체인증 API 표준 적합성을 위한 XML Test Assertion 적용 방안)

  • Son, Min-Woo;Kim, Yong-Chai;Shin, Dong-Il;Shin, Dong-Kyoo;Sin, Yong-Nyeo;Kim, Jae-Seong
    • 한국IT서비스학회:학술대회논문집
    • /
    • 2007.11a
    • /
    • pp.562-567
    • /
    • 2007
  • 분산 환경에서 신분 확인을 위한 생체인증기기가 이용되는 경우 그 기기가 생체인증 표준인 BioAPI를 준용하여 제대로 구현된 것인가에 대한 적합성 시험이 필요하게 된다. 이러한 적합성 시험은 분산 환경의 사용자 및 서비스 제공자에게 표준 규격을 준용한 제품이라는 신뢰성을 주게 된다. 기존에 제공되는 있는 BioAPI(Biometric Application Programming Interface) v2.0 기반의 BSP(Biometric Service Provider)는 오프라인 상에서 BioAPI기반의 제품의 준용 여부만을 평가하기 때문에 분산 환경에서 여러 사람이 동시에 준용 여부를 평가 받기 힘들며 이에 따른 동시 서비스 제공도 불가능하다. 본 논문에서는 BioAPI v2.0 기반의 제품들이 분산 환경에서 제공되는 9개 모델의 표준화된 환경으로 구분하고, 원활한 적합성 시험을 위하여 XML기반의 Test Assertion을 설계하여 생체인증 API 표준 적합성을 시험하였다. XML Test Assertion을 이용한 생체인증 적합성 시험을 위한 메시지 플로우를 밝혀 그 타당성을 입증하였다.

  • PDF