• 제목/요약/키워드: Distributed Storage

검색결과 697건 처리시간 0.031초

분산 저장 블록체인 시스템을 위한 효율적인 결함 내성 향상 기법 (Fault Tolerance Enhancement for Distributed Storage Blockchain Systems)

  • Kim, Junghyun
    • 한국정보통신학회논문지
    • /
    • 제24권11호
    • /
    • pp.1558-1561
    • /
    • 2020
  • In this paper, we propose a blockchain scheme to enhance fault tolerance in distributed storage blockchain systems. Traditional blockchain systems suffer from ever-increasing storage cost. To overcome this problem, distributed storage blockchain techniques have been proposed. Distributed storage blockchain schemes effectively reduce the storage cost, but there are still limitations in reducing recovery cost and fault tolerance. The proposed approach recovers multiple errors within a group by utilizing locally repairable codes with availability. This improves the fault tolerance of the blockchain systems. Simulation results show that the proposed scheme enhances the fault tolerance while minimizing storage cost and recovery cost compared to other state-of-art schemes.

Adaptive-and-Resolvable Fractional Repetition Codes Based on Hypergraph

  • Tiantian Wang;Jing Wang;Haipeng Wang;Jie Meng;Chunlei Yu;Shuxia Wang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제17권4호
    • /
    • pp.1182-1199
    • /
    • 2023
  • Fractional repetition (FR) codes can achieve exact uncoded repair for multiple failed nodes, with lower computational complexity and bandwidth overhead, and effectively improve repair performance in distributed storage systems (DSS). The actual distributed storage system is dynamic, that is, the parameters such as node storage overhead and number of storage nodes will change randomly and dynamically. Considering that traditional FR codes cannot be flexibly applied to dynamic distributed storage systems, a new construction scheme of adaptive-and-resolvable FR codes based on hypergraph coloring is proposed in this paper. Specifically, the linear uniform regular hypergraph can be constructed based on the heuristic algorithm of hypergraph coloring proposed in this paper. Then edges and vertices in hypergraph correspond to nodes and coded packets of FR codes respectively, further, FR codes is constructed. According to hypergraph coloring, the FR codes can achieve rapid repair for multiple failed nodes. Further, FR codes based on hypergraph coloring can be generalized to heterogeneous distributed storage systems. Compared with Reed-Solomon (RS) codes, simple regenerating codes (SRC) and locally repairable codes (LRC), adaptive-and-resolvable FR codes have significant advantages over repair locality, repair bandwidth overhead, computational complexity and time overhead during repairing failed nodes.

Prototype Design of Mass Distributed Storage System based on PC using Ceph for SMB

  • Cha, ByungRae;Kim, Yongil
    • 스마트미디어저널
    • /
    • 제4권3호
    • /
    • pp.62-67
    • /
    • 2015
  • The trend keywords in ICT sector will be Big Data, Internet of Things, and Cloud Computing. The rear end to support those techniques requires a large-capacity storage technology of low-cost. Therefore, we proposed the prototype of low-cost and mass distributed storage system based on PC using open-source Ceph FS for SMB.

Network Coding for Energy-Efficient Distributed Storage System in Wireless Sensor Networks

  • Wang, Lei;Yang, Yuwang;Zhao, Wei;Lu, Wei
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제7권9호
    • /
    • pp.2134-2153
    • /
    • 2013
  • A network-coding-based scheme is proposed to improve the energy efficiency of distributed storage systems in WSNs (Wireless Sensor Networks). We mainly focus on two problems: firstly, consideration is given to effective distributed storage technology; secondly, we address how to effectively repair the data in failed storage nodes. For the first problem, we propose a method to obtain a sparse generator matrix to construct network codes, and this sparse generator matrix is proven to be the sparsest. Benefiting from this matrix, the energy consumption required to implement distributed storage is reduced. For the second problem, we designed a network-coding-based iterative repair method, which adequately utilizes the idea of re-encoding at intermediate nodes from network coding theory. Benefiting from the re-encoding, the energy consumption required by data repair is significantly reduced. Moreover, we provide an explicit lower bound of field size required by this scheme, which implies that it can work over a small field and the required computation overhead is very low. The simulation result verifies that the proposed scheme not only reduces the total energy consumption required to implement distributed storage system in WSNs, but also balances energy consumption of the networks.

Efficient Distributed Storage for Space Information Network Based on Fountain Codes and Probabilistic Broadcasting

  • Kong, Bo;Zhang, Gengxin;Zhang, Wei;Dong, Feihong
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제10권6호
    • /
    • pp.2606-2626
    • /
    • 2016
  • This article investigates the distributed data storage problem in the space information network (SIN) using distributed fountain codes. Since space nodes in the SIN are resource-limited, in order to reduce energy consumption while improving the storage reliability, an efficient distributed storage based on fountain codes and probabilistic broadcasting (DSFPB) strategy is proposed. In the proposed strategy, source packets are disseminated among the entire network according to probabilistic broadcasting (PBcast), and the final degree distribution is close to the desired robust soliton distribution (RSD), this is benefited from the appropriate packets encoding procedure of the proposed strategy. As presented by the analysis and simulations, the total cost of data dissemination is greatly reduced compared with existing representative strategies, while improving the decoding performance.

분산형 저류시설의 실시간 네트워크 제어기술 적용시 고려 사항 (Methods for an application of real-time network control on distributed storage facilities)

  • 백현욱;류재나;오재일;김태형
    • 상하수도학회지
    • /
    • 제27권6호
    • /
    • pp.711-721
    • /
    • 2013
  • Optimal operation of a combined sewer network with distributed storage facilities aims to use the whole retention capacity of all reservoirs efficiently before overflows take place somewhere in the considered network system. An efficient real-time network control (RTNC) strategy has been emerging as an attractive approach for reducing substantially the overflows from a sewer network compared to the conventional fixed or manually adjusted gate setting method, but the related concrete framework for RTC development has not been throughly introduced so far. The main goal of this study is to give a detailed description of the RTNC systems via reviewing several guidelines published abroad, and finally to suggest methods for the proper application of RTNC on distributed storage facilities. Especially, this study is focused on emphasizing the importance of hierarchical structure of RTNC system that consists of three control layers (management, global control and local control). Further, with regard to the global control layer which is responsible for the central overall network control, the wide-ranging details of two components (adaption and optimization layers) are also presented. This study can provide the valuable basis for the RTNC implementation in the particular sewer network with distributed multiple storage facilities.

인터넷 기반의 그룹 오피스 시스템 구현 (Implementation of Group Office System Based on Internet)

  • 김경록;송석일;곽윤식
    • 한국정보통신학회논문지
    • /
    • 제9권3호
    • /
    • pp.540-546
    • /
    • 2005
  • 본 논문에서는 기존의 웹 하드디스크보다 쉽게 기능을 확장할 수 있고, 웹에서의 일반 어플리케이션을 보다 좀 더 간편하게 사용자의 요구에 맞도록 커스터마이징이 가능한 웹 스토리지 기반의 그룹오피스 시스템을 설계하고 구현하였다. 제안하는 웹스토리지는 다수의 스토리지 서버가 모여서 구성되는 분산 시스템이다 이 웹스토리지는 사용자의 위치에 따라 서비스해 주는 서버를 달리하는 방법을 제공하여 보다 빠른 스토리지 서비스가 가능하다.

HTSC and FH HTSC: XOR-based Codes to Reduce Access Latency in Distributed Storage Systems

  • Shuai, Qiqi;Li, Victor O.K.
    • Journal of Communications and Networks
    • /
    • 제17권6호
    • /
    • pp.582-591
    • /
    • 2015
  • A massive distributed storage system is the foundation for big data operations. Access latency performance is a key metric in distributed storage systems since it greatly impacts user experience while existing codes mainly focus on improving performance such as storage overhead and repair cost. By generating parity nodes from parity nodes, in this paper we design new XOR-based erasure codes hierarchical tree structure code (HTSC) and high failure tolerant HTSC (FH HTSC) to reduce access latency in distributed storage systems. By comparing with other popular and representative codes, we show that, under the same repair cost, HTSC and FH HTSC codes can reduce access latency while maintaining favorable performance in other metrics. In particular, under the same repair cost, FH HTSC can achieve lower access latency, higher or equal failure tolerance and lower computation cost compared with the representative codes while enjoying similar storage overhead. Accordingly, FH HTSC is a superior choice for applications requiring low access latency and outstanding failure tolerance capability at the same time.

토러스 연결망 기반의 대용량 멀티미디어용 분산 스토리지 시스템 (Torus Network Based Distributed Storage System for Massive Multimedia Contents)

  • 김재열;김동오;김홍연;김영균;서대화
    • 한국멀티미디어학회논문지
    • /
    • 제19권8호
    • /
    • pp.1487-1497
    • /
    • 2016
  • Explosively growing service of digital multimedia data increases the need for highly scalable low-cost storage. This paper proposes the new storage architecture based on torus network which does not need network switch and erasure coding for efficient storage usage for high scalability and efficient disk utilization. The proposed model has to compensate for the disadvantage of long network latency and network processing overhead of torus network. The proposed storage model was compared to two most popular distributed file system, GlusterFS and Ceph distributed file systems through a prototype implementation. The performance of prototype system shows outstanding results than erasure coding policy of two file systems and mostly even better results than replication policy of them.

클라우딩 기반에서 클라이언트와 서버간 협상을 위한 자가 조직 저장매체의 DDMPF(Distributed Data Management Protocol using FAT) 설계 (A DDMPF(Distributed Data Management Protocol using FAT) Design of Self-organized Storage for Negotiation among a Client and Servers based on Clouding)

  • 이병관;정은희;양승해
    • 한국멀티미디어학회논문지
    • /
    • 제15권8호
    • /
    • pp.1048-1058
    • /
    • 2012
  • 본 논문은 클라우딩 환경에서 클라이언트와 저장 서버, 검증 서버로 구성하여 자가 조직 저장 매체의 데이터 손실을 방지하고, 보안을 유지하기 위한 DDMPF( Distributed Data Management Protocol using FAT)을 제안한다. DDMPF는 클라우드 컴퓨팅 환경에서 자가 조직 저장 서버를 구축하고, 데이터를 분할하여 저장 서버에 분산 저장함으로써 기존의 클라우딩 저장 매체의 중앙 집중화 문제와 저장 서버 문제로 인한 데이터 손실 문제를 해결하였고, 파일할당테이블을 이용해 분산 저장된 데이터 관리의 효율성도 향상시켰다. 그리고 DDMPF는 저장 서버의 데이터 무결성을 검증 서버가 검증함으로써 데이터의 신뢰성을 향상시키고, 클라이언트의 비밀키와 EC-DH 알고리즘을 이용하여 생성된 시스템 마스터 키로 이중 암호화하여 전송함으로써 보안을 강화시켰다. 또한, 자가 조직 저장 매체를 구성할 때, 검증서버의 개수를 제한하고, 검증요청메시지에 대한 TS(Time Stamp)을 설정함으로써 플러딩 공격 탐지하였고, 검증을 요청할 때마다 새롭게 생성된 nonce 값을 이용하여 재전송 공격을 탐지하도록 하였다.