• Title/Summary/Keyword: distributed memory system

Search Result 212, Processing Time 0.026 seconds

SPARQL Query Processing System over Scalable Triple Data using SparkSQL Framework (SparQLing : SparkSQL 기반 대용량 트리플 데이터를 위한 SPARQL 질의 시스템 구축)

  • Jeon, MyungJoong;Hong, JinYoung;Park, YoungTack
    • Journal of KIISE
    • /
    • v.43 no.4
    • /
    • pp.450-459
    • /
    • 2016
  • Every year, RDFS data tends further toward scalability; hence, the manner of SPARQL processing needs to be changed for fast query. The query processing method of SPARQL has been studied using a scalable distributed processing framework. Current studies indicate that the query engine based on the scalable distributed processing framework i.e., Hadoop(MapReduce) is not suitable for real-time processing because of the repetitive tasks; in addition, it is difficult to construct a query engine based on an In-memory Distributed Query engine, because distributed structure on the low-level is required to be considered. In this paper, we proposed a method to construct a query engine for improving the speed of the query process with the mass triple data. The query engine processes the query of SPARQL using the SparkSQL, which is an In-memory based, distributed query processing framework. SparkSQL is a high-level distributed query engine that facilitates existing SQL statement. In order to process the SPARQL query, after generating the Algebra Tree using Jena, the Algebra Tree is required to be translated to Spark Algebra Tree for application in the Spark system, and construction of the system that generated the SparkSQL query. Furthermore, we proposed the design of triple property table based on DataFrame for more efficient query processing in the Spark system. Finally, we verified the validity through comparative evaluation with the query engine, which is the existing distributed processing framework.

Performance Analysis of Bus Arbitration Schemes for Multiple-bus Multiprocessor System (다중버스 다중프로세서 시스템을 위한 버스 중재 방식의 성능 분석)

  • 김종현
    • Journal of the Korea Society for Simulation
    • /
    • v.2 no.1
    • /
    • pp.13-22
    • /
    • 1993
  • In a multiple-bus multiprocessor system in which processors and memory modulus are interconnected through system buses, time delay due to bus contention degrades system performance. In order to reduce such a problem , and optimal bus arbitration scheme and its hardware are neccessary. In this study, performaces of four arbitration schemes are analyzed and compared : fixed-priority, equal-priority, rotating-priority and round-robin priority schemes. For the study, the software simulator of a multiple-bus multiprocessor system is developed by using SLAM II. Simulation results show that, when memory sccesses are evenly distributed to all memory modulus, round-robin priority scheme provides the best performance. But when a hot spot exists, the use of the fixed priority scheme results in the shortest access time.

  • PDF

Intelligent Control by Immune Network Algorithm Based Auto-Weight Function Tuning

  • Kim, Dong-Hwa
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2002.10a
    • /
    • pp.120.2-120
    • /
    • 2002
  • In this paper auto-tuning scheme of weight function in the neural networks has been suggested by immune algorithm for nonlinear process. A number of structures of the neural networks are considered as learning methods for control system. A general view is provided that they are the special cases of either the membership functions or the modification of network structure in the neural networks. On the other hand, since the immune network system possesses a self organizing and distributed memory, it is thus adaptive to its external environment and allows a PDP (parallel distributed processing) network to complete patterns against the environmental situation. Also. It can provi..

  • PDF

SWAT: A Study on the Efficient Integration of SWRL and ATMS based on a Distributed In-Memory System (SWAT: 분산 인-메모리 시스템 기반 SWRL과 ATMS의 효율적 결합 연구)

  • Jeon, Myung-Joong;Lee, Wan-Gon;Jagvaral, Batselem;Park, Hyun-Kyu;Park, Young-Tack
    • Journal of KIISE
    • /
    • v.45 no.2
    • /
    • pp.113-125
    • /
    • 2018
  • Recently, with the advent of the Big Data era, we have gained the capability of acquiring vast amounts of knowledge from various fields. The collected knowledge is expressed by well-formed formula and in particular, OWL, a standard language of ontology, is a typical form of well-formed formula. The symbolic reasoning is actively being studied using large amounts of ontology data for extracting intrinsic information. However, most studies of this reasoning support the restricted rule expression based on Description Logic and they have limited applicability to the real world. Moreover, knowledge management for inaccurate information is required, since knowledge inferred from the wrong information will also generate more incorrect information based on the dependencies between the inference rules. Therefore, this paper suggests that the SWAT, knowledge management system should be combined with the SWRL (Semantic Web Rule Language) reasoning based on ATMS (Assumption-based Truth Maintenance System). Moreover, this system was constructed by combining with SWRL reasoning and ATMS for managing large ontology data based on the distributed In-memory framework. Based on this, the ATMS monitoring system allows users to easily detect and correct wrong knowledge. We used the LUBM (Lehigh University Benchmark) dataset for evaluating the suggested method which is managing the knowledge through the retraction of the wrong SWRL inference data on large data.

A MAC System Design for High-speed UWB SoC (고속 UWB SoC의 MAC 시스템 설계)

  • Kim, Do-Hoon;Wee, Jeong-Wook;Lee, Chung-Yong
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.48 no.4
    • /
    • pp.1-5
    • /
    • 2011
  • We present the implementation of MAC system for MBOA UWB SoC. The implemented MBOA MAC algorithm is not master control mechanism, but distributed network mechanism. Therefore, mesh network can be easily constructed because MAC consists of distributed network and administrates network. The ARM926EJ with cache is adopted for high performnace and AMBA bus is applied for system design and reuse. In addition, the system operating clock management algorithm is implemented for low power consumption. The dedicated DMA for MAC is designed between the system memory buffer and MAC hardware, and the dedicated DMA for USB 2.0 is also implemented between system memory buffer and host for high data transaction.

Performance Enhancement and Evaluation of Distributed File System for Cloud (클라우드 분산 파일 시스템 성능 개선 및 평가)

  • Lee, Jong Hyuk
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.7 no.11
    • /
    • pp.275-280
    • /
    • 2018
  • The choice of a suitable distributed file system is required for loading large data and high-speed processing through subsequent applications in a cloud environment. In this paper, we propose a write performance improvement method based on GlusterFS and evaluate the performance of MapRFS, CephFS and GlusterFS among existing distributed file systems in cloud environment. The write performance improvement method proposed in this paper enhances the response time by changing the synchronization level used by the synchronous replication method from disk to memory. Experimental results show that the distributed file system to which the proposed method is applied is superior to other distributed file systems in the case of sequential write, random write and random read.

Development of Real time Air Quality Prediction System

  • Oh, Jai-Ho;Kim, Tae-Kook;Park, Hung-Mok;Kim, Young-Tae
    • Proceedings of the Korean Environmental Sciences Society Conference
    • /
    • 2003.11a
    • /
    • pp.73-78
    • /
    • 2003
  • In this research, we implement Realtime Air Diffusion Prediction System which is a parallel Fortran model running on distributed-memory parallel computers. The system is designed for air diffusion simulations with four-dimensional data assimilation. For regional air quality forecasting a series of dynamic downscaling technique is adopted using the NCAR/Penn. State MM5 model which is an atmospheric model. The realtime initial data have been provided daily from the KMA (Korean Meteorological Administration) global spectral model output. It takes huge resources of computation to get 24 hour air quality forecast with this four step dynamic downscaling (27km, 9km, 3km, and lkm). Parallel implementation of the realtime system is imperative to achieve increased throughput since the realtime system have to be performed which correct timing behavior and the sequential code requires a large amount of CPU time for typical simulations. The parallel system uses MPI (Message Passing Interface), a standard library to support high-level routines for message passing. We validate the parallel model by comparing it with the sequential model. For realtime running, we implement a cluster computer which is a distributed-memory parallel computer that links high-performance PCs with high-speed interconnection networks. We use 32 2-CPU nodes and a Myrinet network for the cluster. Since cluster computers more cost effective than conventional distributed parallel computers, we can build a dedicated realtime computer. The system also includes web based Gill (Graphic User Interface) for convenient system management and performance monitoring so that end-users can restart the system easily when the system faults. Performance of the parallel model is analyzed by comparing its execution time with the sequential model, and by calculating communication overhead and load imbalance, which are common problems in parallel processing. Performance analysis is carried out on our cluster which has 32 2-CPU nodes.

  • PDF

Distributed In-Memory based Large Scale RDFS Reasoning and Query Processing Engine for the Population of Temporal/Spatial Information of Media Ontology (미디어 온톨로지의 시공간 정보 확장을 위한 분산 인메모리 기반의 대용량 RDFS 추론 및 질의 처리 엔진)

  • Lee, Wan-Gon;Lee, Nam-Gee;Jeon, MyungJoong;Park, Young-Tack
    • Journal of KIISE
    • /
    • v.43 no.9
    • /
    • pp.963-973
    • /
    • 2016
  • Providing a semantic knowledge system using media ontologies requires not only conventional axiom reasoning but also knowledge extension based on various types of reasoning. In particular, spatio-temporal information can be used in a variety of artificial intelligence applications and the importance of spatio-temporal reasoning and expression is continuously increasing. In this paper, we append the LOD data related to the public address system to large-scale media ontologies in order to utilize spatial inference in reasoning. We propose an RDFS/Spatial inference system by utilizing distributed memory-based framework for reasoning about large-scale ontologies annotated with spatial information. In addition, we describe a distributed spatio-temporal SPARQL parallel query processing method designed for large scale ontology data annotated with spatio-temporal information. In order to evaluate the performance of our system, we conducted experiments using LUBM and BSBM data sets for ontology reasoning and query processing benchmark.

Distributed Moving Objects Management System for a Smart Black Box

  • Lee, Hyunbyung;Song, Seokil
    • International Journal of Contents
    • /
    • v.14 no.1
    • /
    • pp.28-33
    • /
    • 2018
  • In this paper, we design and implement a distributed, moving objects management system for processing locations and sensor data from smart black boxes. The proposed system is designed and implemented based on Apache Kafka, Apache Spark & Spark Streaming, Hbase, HDFS. Apache Kafka is used to collect the data from smart black boxes and queries from users. Received location data from smart black boxes and queries from users becomes input of Apache Spark Streaming. Apache Spark Streaming preprocesses the input data for indexing. Recent location data and indexes are stored in-memory managed by Apache Spark. Old data and indexes are flushed into HBase later. We perform experiments to show the throughput of the index manager. Finally, we describe the implementation detail in Scala function level.

병렬분산 환경에서의 DEVS형식론의 시뮬레이션

  • Seong, Yeong-Rak;Jung, Sung-Hun;Kon, Tag-Gon;Park, Kyu-Ho-
    • Proceedings of the Korea Society for Simulation Conference
    • /
    • 1992.10a
    • /
    • pp.5-5
    • /
    • 1992
  • The DEVS(discrete event system specification) formalism describes a discrete event system in a hierarchical, modular form. DEVSIM++ is C++ based general purpose DEVS abstract simulator which can simulate systems to be modeled by the DEVS formalism in a sequential environment. We implement P-DEVSIM++ which is a parallel version of DEVSIM++. In P-DEVSIM++, the external and internal event of models can be processed in parallel. To process in parallel, we introduce a hierarchical distributed simulation technique and some optimistic distributed simulation techniques. But in our algorithm, the rollback of a model is localized itself in contrast to the Time Warp approach. To evaluate its performance, we simulate a single bus multiprocessor architecture system with an external common memory. Simulation result shows that significant speedup is made possible with our algorithm in a parallel environment.

  • PDF