• Title/Summary/Keyword: distributed memory environment

Search Result 80, Processing Time 0.023 seconds

Study of In-Memory based Hybrid Big Data Processing Scheme for Improve the Big Data Processing Rate (빅데이터 처리율 향상을 위한 인-메모리 기반 하이브리드 빅데이터 처리 기법 연구)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.12 no.2
    • /
    • pp.127-134
    • /
    • 2019
  • With the advancement of IT technology, the amount of data generated has been growing exponentially every year. As an alternative to this, research on distributed systems and in-memory based big data processing schemes has been actively underway. The processing power of traditional big data processing schemes enables big data to be processed as fast as the number of nodes and memory capacity increases. However, the increase in the number of nodes inevitably raises the frequency of failures in a big data infrastructure environment, and infrastructure management points and infrastructure operating costs also increase accordingly. In addition, the increase in memory capacity raises infrastructure costs for a node configuration. Therefore, this paper proposes an in-memory-based hybrid big data processing scheme for improve the big data processing rate. The proposed scheme reduces the number of nodes compared to traditional big data processing schemes based on distributed systems by adding a combiner step to a distributed system processing scheme and applying an in-memory based processing technology at that step. It decreases the big data processing time by approximately 22%. In the future, realistic performance evaluation in a big data infrastructure environment consisting of more nodes will be required for practical verification of the proposed scheme.

A Development of Distributed Parallel Processing algorithm for Power Flow analysis (전력 조류 계산의 분산 병렬처리기법에 관한 연구)

  • Lee, Chun-Mo;Lee, Hae-Ki
    • Proceedings of the KIEE Conference
    • /
    • 2001.07e
    • /
    • pp.134-140
    • /
    • 2001
  • Parallel processing has the potential to be cost effectively used on computationally intense power system problems. But this technology is not still available is not only parallel computer but also parallel processing scheme. Testing these algorithms to ensure accuracy, and evaluation of their performance is also an issue. Although a significant amount of parallel algorithms of power system problem have been developed in last decade, actual testing on processor architectures lies in the beginning stages. This paper presents the parallel processing algorithm to supply the base being able to treat power flow by newton's method by the distributed memory type parallel computer. This method is to assign and to compute teared blocks of sparse matrix at each parallel processors. The testing to insure accuracy of developed method have been done on serial computer by trying to simulate a parallel environment.

  • PDF

A Study on a Conceptual Model for Distributed Problem Solving (분산 문제 해결을 위한 개념적 모델에 관한 연구)

  • Kim, Eun-Gyeong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.3 no.1
    • /
    • pp.107-117
    • /
    • 1996
  • This paper proposes a conceptual model for distributed problem solving(DPS), where cooperation and communication among agents are based on virtual shared memory (VSM) In this model, all agents in a DPS system view all the memory in a distributed computer system as a single shared memory in which data, tasks, and results can in a distributed computer system as a single shared memory, in which data, tasks, and results can / be accessed by any of the agents. Several agents cooperatively solve a complex problem, as agents write data, intermediate results of execution, and tasks requested to other agents to VSM and other agents read or execute them. Also, in order to develop DPS systems based on the proposed conceptual model, this paper designs a DPS system development environment(DPS-VSM)using Network Linda. Thi spaper shows utility ofthe proposed model by presenting an example of simulation a VSM-based DPS system, and analyzes the model by comparing the features of DPS- VSM with some other DAI programming shells.

  • PDF

Human Indicator and Information Display using Space Human Interface in Networked Intelligent Space

  • Jin Tae-Seok;Niitsuma Mihoko;Hashimoto Hideki
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.15 no.5
    • /
    • pp.632-638
    • /
    • 2005
  • This paper describes a new data-handing, based on a Spatial Human Interface as human indicator, to the Spatial-Knowledge-Tags (SKT) in the spatial memory the Spatial Human Interface (SHI) is a new system that enables us to facilitate human activity in a working environment. The SHI stores human activity data as knowledge and activity history of human into the Spatial Memory in a working environment as three-dimensional space where one acts, and loads them with the Spatial-Knowledge-Tags(SKT) by supporting the enhancement of human activity. To realize this, the purpose of SHI is to construct new relationship among human and distributed networks computers and sensors that is based on intuitive and simultaneous interactions. In this paper, the specified functions of SKT and the realization method of SKT are explained. The utility of SKT is demonstrated in designing a robot motion control.

Intelligent Control by Immune Network Algorithm Based Auto-Weight Function Tuning

  • Kim, Dong-Hwa
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2002.10a
    • /
    • pp.120.2-120
    • /
    • 2002
  • In this paper auto-tuning scheme of weight function in the neural networks has been suggested by immune algorithm for nonlinear process. A number of structures of the neural networks are considered as learning methods for control system. A general view is provided that they are the special cases of either the membership functions or the modification of network structure in the neural networks. On the other hand, since the immune network system possesses a self organizing and distributed memory, it is thus adaptive to its external environment and allows a PDP (parallel distributed processing) network to complete patterns against the environmental situation. Also. It can provi..

  • PDF

병렬분산 환경에서의 DEVS형식론의 시뮬레이션

  • Seong, Yeong-Rak;Jung, Sung-Hun;Kon, Tag-Gon;Park, Kyu-Ho-
    • Proceedings of the Korea Society for Simulation Conference
    • /
    • 1992.10a
    • /
    • pp.5-5
    • /
    • 1992
  • The DEVS(discrete event system specification) formalism describes a discrete event system in a hierarchical, modular form. DEVSIM++ is C++ based general purpose DEVS abstract simulator which can simulate systems to be modeled by the DEVS formalism in a sequential environment. We implement P-DEVSIM++ which is a parallel version of DEVSIM++. In P-DEVSIM++, the external and internal event of models can be processed in parallel. To process in parallel, we introduce a hierarchical distributed simulation technique and some optimistic distributed simulation techniques. But in our algorithm, the rollback of a model is localized itself in contrast to the Time Warp approach. To evaluate its performance, we simulate a single bus multiprocessor architecture system with an external common memory. Simulation result shows that significant speedup is made possible with our algorithm in a parallel environment.

  • PDF

Call admission control for ATM networks using a sparse distributed memory (ATM 망에서 축약 분산 기억 장치를 사용한 호 수락 제어)

  • 권희용;송승준;최재우;황희영
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • v.35S no.3
    • /
    • pp.1-8
    • /
    • 1998
  • In this paper, we propose a Neural Call Admission Control (CAC) method using a Sparse Distributed Memory(SDM). CAC is a key technology of TM network traffic control. It should be adaptable to the rapid and various changes of the ATM network environment. conventional approach to the ATM CAC requires network analysis in all cases. So, the optimal implementation is said to be very difficult. Therefore, neural approach have recently been employed. However, it does not mett the adaptability requirements. because it requires additional learning data tables and learning phase during CAC operation. We have proposed a neural network CAC method based on SDM that is more actural than conventioal approach to apply it to CAC. We compared it with previous neural network CAC method. It provides CAC with good adaptability to manage changes. Experimenatal results show that it has rapid adaptability and stability without additional learning table or learning phase.

  • PDF

Performance Enhancement and Evaluation of Distributed File System for Cloud (클라우드 분산 파일 시스템 성능 개선 및 평가)

  • Lee, Jong Hyuk
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.7 no.11
    • /
    • pp.275-280
    • /
    • 2018
  • The choice of a suitable distributed file system is required for loading large data and high-speed processing through subsequent applications in a cloud environment. In this paper, we propose a write performance improvement method based on GlusterFS and evaluate the performance of MapRFS, CephFS and GlusterFS among existing distributed file systems in cloud environment. The write performance improvement method proposed in this paper enhances the response time by changing the synchronization level used by the synchronous replication method from disk to memory. Experimental results show that the distributed file system to which the proposed method is applied is superior to other distributed file systems in the case of sequential write, random write and random read.

Location Based Routing Service In Distributed Web Environment

  • Kim, Do-Hyun;Jang, Byung-Tae
    • Proceedings of the KSRS Conference
    • /
    • 2003.11a
    • /
    • pp.340-342
    • /
    • 2003
  • Location based services based on positions of moving objects are expanding the business area gradually. The location is included all estimate position of the future as well as the position of the present and the past. Location based routing service is active business application in which the position information of moving objects is applied efficiently. This service includes the trajectory of past positions, the real-time tracing of present position of special moving objects, and the shortest and optimized paths combined with map information. In this paper, we describes the location based routing services is extend in distributed web GIS environment. Web GIS service systems provide the various GIS services of analyzing and displaying the spatial data with friendly user - interface. That is, we propose the efficient architecture and technologies for servicing the location based routing services in distributed web GIS environment. The position of moving objects is acquired by GPS (Global Positioning System) and converted the coordinate of real world by map matching with geometric information. We suppose the swapping method between main memory and storages to access the quite a number of moving objects. And, the result of location based routing services is wrapped the web-styled data format. We design the schema based on the GML. We design these services as components were developed in object-oriented computing environment, and provide the interoperability, language-independent, easy developing environment as well as re - usability.

  • PDF

A Dynamic Routing Algorithm Adaptive to Traffic for Multistage Bus Networks in Distributed Shared Memory Environment (분산 공유메모리 환경의 다단계 버스망에서 트래픽에 적응하는 동적 라우팅 알고리즘)

  • Hong, Kang-Woon;Jeon, Chang-Ho
    • The KIPS Transactions:PartA
    • /
    • v.9A no.4
    • /
    • pp.547-554
    • /
    • 2002
  • This paper proposes an efficient dynamic routing algorithm for Multistage Bus Networks(MBN's) in distributed shared memory environment. Our algorithm utilizes extra paths available on MBN and determines routing paths adaptively according to switch traffic in order to distribute traffic among switches. Precisely, a packet is transmitted to the next switch on an extra path having a lighter traffic. As a consequence the proposed algorithm reduces the mean response time and the average number of waiting tasks. The results of simulations, carried out with varying numbers of processors and varying switch sizes, show that the proposed algorithm improves the mean response time by 9% and the average number of waiting tasks by 21.6%, compared to the existing routing algorithms which do not consider extra paths on MBN.