• 제목/요약/키워드: distributed storage systems

검색결과 207건 처리시간 0.028초

분산 데이터베이스 시스템에서의 최적 파일 이동 정책 (Optimal File Migration Policies in Distributed Database Systems)

  • 이기태;김재련
    • 산업경영시스템학회지
    • /
    • 제18권33호
    • /
    • pp.1-10
    • /
    • 1995
  • The allocation of files is essential to the efficiency and effectiveness of a distributed system that must meet geographically dispersed data processing demands. In this paper, we address an optimization model that generates optimal file migration policies in distributed database systems. The proposed model is a more generalized model that includes system's capacity constraints - computing sites' storage capacity and communication networks' capacity - which have not taken into consideration in previous researches. Using this model, we can establish initial file allocation, file reallocation and file migration polices that minimize a system operating cost under system's capacity constraints at an initial system design or reorganization point The proposed model not only can be adopted by small-sized systems but also provides a foundation for effective and simple heuristics for adaptive file migration in large systems.

  • PDF

Intermedia Synchronization Protocol for Continuous Media Using MPEG-4 in Mobile Distributed Systems

  • Dominguez, Eduardo Lopez;Hernandez, Saul Eduardo Pomares;Gil, Pilar Gomez;Calleja, Jorge De La;Benitez, Antonio;Marin-Hernandez, Antonio
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제6권6호
    • /
    • pp.1689-1706
    • /
    • 2012
  • The preservation of temporal dependencies among a group of processes that exchange continuous media at runtime is a key issue for emerging mobile distributed systems (MDS), such as monitoring of biosignals and interactive multiuser games. Although several works are oriented to satisfy temporal dependencies, most of them are not suitable for MDSs. In general, an MDS is characterized by the absence of global references (e.g. shared memory and wall clock), host mobility, limited processing and storage capabilities in mobile hosts, and limited bandwidth on wireless communication channels. This paper proposes an asymmetric synchronization protocol to be used at runtime in an MDS without using a common reference. One main aspect of our synchronization protocol is that it translates temporal constraints to causal dependencies of the continuous media data as seen by the mobile hosts. We simulate the protocol by considering a cellular network environment and by using MPEG-4 encoders. The simulation results show that our protocol is effective in reducing the synchronization error. In addition, the protocol is efficient in terms of processing and storage costs at the mobile devices, as well as in the overhead attached per message across the wired and wireless channels.

하둡 환경에 적합한 데이터 저장 및 복원 기법에 관한 연구 (A Study on Data Storage and Recovery in Hadoop Environment)

  • 김수현;이임영
    • 정보처리학회논문지:컴퓨터 및 통신 시스템
    • /
    • 제2권12호
    • /
    • pp.569-576
    • /
    • 2013
  • 최근 많은 관심을 받고 있는 클라우드 컴퓨팅이 해결해야할 가장 큰 문제는 바로 보안이다. 일반적인 클라우드 컴퓨팅 환경에서는 사용자의 데이터 보호를 위해 수많은 분산서버를 이용하여 데이터를 저장한다. 하지만 분산서버에 저장된 데이터를 암호화 과정을 거치지 않고 그대로 저장하게 된다면, 마스터 서버에 저장된 분산파일 위치를 추적하여 그대로 유출이 되는 문제가 발생할 수 있다. 이러한 문제를 방지하기 위해 비밀키를 이용하여 분산된 데이터를 암호화해야할 필요성이 존재한다. 그러나 대용량 데이터의 경우 수십, 수백 개의 조각으로 나누어지게 되는데 분산서버마다 각각의 비밀키를 이용하게 된다면, 관리의 어려움이 존재할 뿐 아니라 분산 서버에 대한 정당한 인증, 암복호화 과정을 수없이 거치게 되어 막대한 오버헤드가 발생하게 된다. 따라서 본 논문에서는 이와 같은 문제점을 해결하기 위해 Hadoop 환경에 적합한 XOR 및 RAID기반의 효율적인 분산 저장 및 복구 기법을 제안하였다.

RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment

  • Muhammad Faseeh Qureshi, Nawab;Shin, Dong Ryeol
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제10권9호
    • /
    • pp.4063-4086
    • /
    • 2016
  • Cloud computing is a robust technology, which facilitate to resolve many parallel distributed computing issues in the modern Big Data environment. Hadoop is an ecosystem, which process large data-sets in distributed computing environment. The HDFS is a filesystem of Hadoop, which process data blocks to the cluster nodes. The data block placement has become a bottleneck to overall performance in a Hadoop cluster. The current placement policy assumes that, all Datanodes have equal computing capacity to process data blocks. This computing capacity includes availability of same storage media and same processing performances of a node. As a result, Hadoop cluster performance gets effected with unbalanced workloads, inefficient storage-tier, network traffic congestion and HDFS integrity issues. This paper proposes a storage-tier-aware Robust Data Placement (RDP) scheme, which systematically resolves unbalanced workloads, reduces network congestion to an optimal state, utilizes storage-tier in a useful manner and minimizes the HDFS integrity issues. The experimental results show that the proposed approach reduced unbalanced workload issue to 72%. Moreover, the presented approach resolve storage-tier compatibility problem to 81% by predicting storage for block jobs and improved overall data block placement by 78% through pre-calculated computing capacity allocations and execution of map files over respective Namenode and Datanodes.

Scalable Blockchain Storage Model Based on DHT and IPFS

  • Chen, Lu;Zhang, Xin;Sun, Zhixin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제16권7호
    • /
    • pp.2286-2304
    • /
    • 2022
  • Blockchain is a distributed ledger that combines technologies such as cryptography, consensus mechanism, peer-to-peer transmission, and time stamping. The rapid development of blockchain has attracted attention from all walks of life, but storage scalability issues have hindered the application of blockchain. In this paper, a scalable blockchain storage model based on Distributed Hash Table (DHT) and the InterPlanetary File System (IPFS) was proposed. This paper introduces the current research status of the scalable blockchain storage model, as well as the basic principles of DHT and the InterPlanetary File System. The model construction and workflow are explained in detail. At the same time, the DHT network construction mechanism, block heat identification mechanism, new node initialization mechanism, and block data read and write mechanism in the model are described in detail. Experimental results show that this model can reduce the storage burden of nodes, and at the same time, the blockchain network can accommodate more local blocks under the same block height.

Modeling of a storage subsystem in multimedia information system

  • Lim, Cheol-Su
    • 한국통신학회논문지
    • /
    • 제22권11호
    • /
    • pp.2521-2530
    • /
    • 1997
  • In this paepr, we present a video-on-demand (VOD) system design model that address and integrates a number of inter-related issues. Then with analysis and performance evaluation, we investigate various aspects of disk and buffer managements in the given model. Based on the analysis results, we suggest that a distributed buffering scheme with intermediate buffers may te useful to transform bursty disk accesses into a continuous stream for for glitch-free performance of VOD systems. Also, through simulation, we illustrate that massive multimedia information storage design techniques such as prefetching, clustered striping, and real-time disk scheduling integrated with the distributed buffering mechanism may enhance end-to-end real-time performance of VOD systems under wide-area networks.

  • PDF

Agents for Autonomous Distributed Secret Sharing Storage System

  • Hayashi, Daisuke;Miyamoto, Toshiyuki;Doi, Shinji;Kumagai, Sadatoshi
    • 대한전자공학회:학술대회논문집
    • /
    • 대한전자공학회 2002년도 ITC-CSCC -1
    • /
    • pp.482-485
    • /
    • 2002
  • For mission-critical and safe-critical operations of medical information, financial, or administrative systems, a reliable and robust storage system is indispensable. The main purpose of our research is to develop a high-confidential, reliable, and survivable storage system.

  • PDF

완전다분할그래프 기반 이진 부분접속복구 부호 (Binary Locally Repairable Codes from Complete Multipartite Graphs)

  • 김정현;남미영;송홍엽
    • 한국통신학회논문지
    • /
    • 제40권9호
    • /
    • pp.1734-1740
    • /
    • 2015
  • 본 논문은 분산 저장 시스템에서 기존의 부분접속수를 일반화한 개념인 결합 부분접속수를 소개하고, 결합 부분접속수($r_1$=2, $r_2$=3 or 4)를 만족하는 부호 설계 방식을 제안한다. 결합 부분접속수란 다양한 수의 노드 손실을 복구하기 위해 필요한 노드 수 집합을 의미한다. 제안된 방식은 완전다분할그래프를 사용하여 부호 설계를 단순화한다. 또한 제안된 방식으로 임의의 양의 정수 t에 대해 (2,t)-가용도를 갖는 이진 부분접속복구 부호를 설계할 수 있다. 즉, 1개 노드 손실 시 t개의 서로소인 복구 집합으로부터 각각 복구가 가능하며, 이때 각 복구 집합의 크기는 최대 2이다. 이러한 성질은 핫 데이터의 병렬처리를 가능하게 하므로 분산 저장 시스템에서 중요한 의미를 갖는다.

전력시스템 관리 및 Vehicle to Grid 전력시장 개발을 위한 가상발전소의 활용방안 (Review of Virtual Power Plant Applications for Power System Management and Vehicle-to-Grid Market Development)

  • 진태환;박혜리;정모;신기열
    • 전기학회논문지
    • /
    • 제65권12호
    • /
    • pp.2251-2261
    • /
    • 2016
  • The use of renewable energy sources and energy storage systems is increasing due to new policies in the energy industries. However, the increase in distributed generation hinders the reliability of power systems. In order to stabilize power systems, a virtual power plant has been proposed as a novel power grid management system. The virtual power plant plays includes different distributed energy resources and energy storage systems. We define a core virtual power plant technology related to demand response and ancillary service for the cases of Korea, America, and Europe. We also suggest applications of the proposed virtual power plant to the vehicle-to-grid market for restructuring national power industries in Korea.

A COMPARATIVE STUDY ON BLOCKCHAIN DATA MANAGEMENT SYSTEMS: BIGCHAINDB VS FALCONDB

  • Abrar Alotaibi;Sarah Alissa;Salahadin Mohammed
    • International Journal of Computer Science & Network Security
    • /
    • 제23권5호
    • /
    • pp.128-134
    • /
    • 2023
  • The widespread usage of blockchain technology in cryptocurrencies has led to the adoption of the blockchain concept in data storage management systems for secure and effective data storage and management. Several innovative studies have proposed solutions that integrate blockchain with distributed databases. In this article, we review current blockchain databases, then focus on two well-known blockchain databases-BigchainDB and FalconDB-to illustrate their architecture and design aspects in more detail. BigchainDB is a distributed database that integrates blockchain properties to enhance immutability and decentralization as well as a high transaction rate, low latency, and accurate queries. Its architecture consists of three layers: the transaction layer, consensus layer, and data model layer. FalconDB, on the other hand, is a shared database that allows multiple clients to collaborate on the database securely and efficiently, even if they have limited resources. It has two layers: the authentication layer and the consensus layer, which are used with client requests and results. Finally, a comparison is made between the two blockchain databases, revealing that they share some characteristics such as immutability, low latency, permission, horizontal scalability, decentralization, and the same consensus protocol. However, they vary in terms of database type, concurrency mechanism, replication model, cost, and the usage of smart contracts.