• 제목/요약/키워드: storage overhead

검색결과 181건 처리시간 0.032초

Adaptive-and-Resolvable Fractional Repetition Codes Based on Hypergraph

  • Tiantian Wang;Jing Wang;Haipeng Wang;Jie Meng;Chunlei Yu;Shuxia Wang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제17권4호
    • /
    • pp.1182-1199
    • /
    • 2023
  • Fractional repetition (FR) codes can achieve exact uncoded repair for multiple failed nodes, with lower computational complexity and bandwidth overhead, and effectively improve repair performance in distributed storage systems (DSS). The actual distributed storage system is dynamic, that is, the parameters such as node storage overhead and number of storage nodes will change randomly and dynamically. Considering that traditional FR codes cannot be flexibly applied to dynamic distributed storage systems, a new construction scheme of adaptive-and-resolvable FR codes based on hypergraph coloring is proposed in this paper. Specifically, the linear uniform regular hypergraph can be constructed based on the heuristic algorithm of hypergraph coloring proposed in this paper. Then edges and vertices in hypergraph correspond to nodes and coded packets of FR codes respectively, further, FR codes is constructed. According to hypergraph coloring, the FR codes can achieve rapid repair for multiple failed nodes. Further, FR codes based on hypergraph coloring can be generalized to heterogeneous distributed storage systems. Compared with Reed-Solomon (RS) codes, simple regenerating codes (SRC) and locally repairable codes (LRC), adaptive-and-resolvable FR codes have significant advantages over repair locality, repair bandwidth overhead, computational complexity and time overhead during repairing failed nodes.

HTSC and FH HTSC: XOR-based Codes to Reduce Access Latency in Distributed Storage Systems

  • Shuai, Qiqi;Li, Victor O.K.
    • Journal of Communications and Networks
    • /
    • 제17권6호
    • /
    • pp.582-591
    • /
    • 2015
  • A massive distributed storage system is the foundation for big data operations. Access latency performance is a key metric in distributed storage systems since it greatly impacts user experience while existing codes mainly focus on improving performance such as storage overhead and repair cost. By generating parity nodes from parity nodes, in this paper we design new XOR-based erasure codes hierarchical tree structure code (HTSC) and high failure tolerant HTSC (FH HTSC) to reduce access latency in distributed storage systems. By comparing with other popular and representative codes, we show that, under the same repair cost, HTSC and FH HTSC codes can reduce access latency while maintaining favorable performance in other metrics. In particular, under the same repair cost, FH HTSC can achieve lower access latency, higher or equal failure tolerance and lower computation cost compared with the representative codes while enjoying similar storage overhead. Accordingly, FH HTSC is a superior choice for applications requiring low access latency and outstanding failure tolerance capability at the same time.

A Data-Consistency Scheme for the Distributed-Cache Storage of the Memcached System

  • Liao, Jianwei;Peng, Xiaoning
    • Journal of Computing Science and Engineering
    • /
    • 제11권3호
    • /
    • pp.92-99
    • /
    • 2017
  • Memcached, commonly used to speed up the data access in big-data and Internet-web applications, is a system software of the distributed-cache mechanism. But it is subject to the severe challenge of the loss of recently uncommitted updates in the case where the Memcached servers crash due to some reason. Although the replica scheme and the disk-log-based replay mechanism have been proposed to overcome this problem, they generate either the overhead of the replica synchronization or the persistent-storage overhead that is caused by flushing related logs. This paper proposes a scheme of backing up the write requests (i.e., set and add) on the Memcached client side, to reduce the overhead resulting from the making of disk-log records or performing the replica consistency. If the Memcached server fails, a timestamp-based recovery mechanism is then introduced to replay the write requests (buffered by relevant clients), for regaining the lost-data updates on the rebooted Memcached server, thereby meeting the data-consistency requirement. More importantly, compared with the mechanism of logging the write requests to the persistent storage of the master server and the server-replication scheme, the newly proposed approach of backing up the logs on the client side can greatly decrease the time overhead by up to 116.8% when processing the write workloads.

메모리 파일시스템에서 메모리 매핑을 이용한 파일 입출력의 오버헤드 분석 (Analyzing the Overhead of the Memory Mapped File I/O for In-Memory File Systems)

  • 최정식;한환수
    • 정보과학회 컴퓨팅의 실제 논문지
    • /
    • 제22권10호
    • /
    • pp.497-503
    • /
    • 2016
  • 비휘발성 메모리 같은 차세대 저장장치의 등장으로 저장장치 지연시간은 거의 사라질 것이다. 예전에는 저장장치 지연시간이 가장 큰 문제였기 때문에 소프트웨어의 효율성은 중요한 문제가 아니었다. 하지만 이제는 소프트웨어 오버헤드가 해결해야 할 문제점으로 나타나고 있다. 소프트웨어 오버헤드를 최소화하기 위해 많은 연구자들은 메모리 매핑을 이용한 파일 입출력 기법을 제안하고 있다. 메모리 맵 파일 입출력 기법을 사용하면 기존 운영체제의 복잡한 파일 입출력 스택을 피할 수 있을 뿐 아니라 빈번한 사용자/커널 모드 변환도 최소화할 수 있다. 또한 다수의 메모리 복사 오버헤드도 최소화 할 수 있다. 하지만 메모리 맵 파일 입출력 기법에도 해결해야 할 문제점이 존재한다. 메모리 맵 파일 입출력 메커니즘도 느린 블록 디바이스를 효율적으로 관리하기 위해 설계된 기존 운영체제의 일부이기 때문이다. 본 논문에서는 메모리 맵 파일 입출력의 오버헤드 문제점을 설명하고 실험을 통해 그 문제점을 확인한다.

초저지연 저장장치를 위한 적응형 폴링 선택 기법 (An Adaptive Polling Selection Technique for Ultra-Low Latency Storage Systems)

  • 천명준;김윤아;김지홍
    • 대한임베디드공학회논문지
    • /
    • 제14권2호
    • /
    • pp.63-69
    • /
    • 2019
  • Recently, ultra-low latency flash storage devices such as Z-SSD and Optane SSD were introduced with the significant technological improvement in the storage devices which provide much faster response time than today's other NVMe SSDs. With such ultra-low latency, $10{\mu}s$, storage devices the cost of context switch could be an overhead during interrupt-driven I/O completion process. As an interrupt-driven I/O completion process could bring an interrupt handling overhead, polling or hybrid-polling for the I/O completion is known to perform better. In this paper, we analyze tail latency problem in a polling process caused by process scheduling in data center environment where multiple applications run simultaneously under one system and we introduce our adaptive polling selection technique which dynamically selects efficient processing method between two techniques according to the system's conditions.

Analysis Performance Characterics of Dynamic Signature File Methods

  • Yoo, Jae-Soo;Choi, Kil-Seong;Kim, Myoung-Ho
    • Journal of Electrical Engineering and information Science
    • /
    • 제2권4호
    • /
    • pp.37-45
    • /
    • 1997
  • With rapid increase of information requirements from various application areas, there has been much research on dynamic information storage structures that effectively support insertions, deletions and updates. In this appear we evaluate the performance of he existing dynamic signature file methods such as the S-tree, Quick Filter and HS file and provide guidelines for the most effective usage to a given operational environment. We derive analytic performance evaluation models of the storage structures based on retrieval time, storage overhead and insertion time. We also perform extensive experiments with various data distributions such as uniform, normal and exponential distributions. The relationships among various performance parameters are throughly investigated. We show through performance comparison based on analytic models and experiments that regardless of data distribution, the HS file significantly improves performance in both the retreival time and the storage overhead over S-tree and Quick Filter.

  • PDF

$PRR_gp$ 기반 결함허용 연속 매체 저장시스템의 설계와 성능평가 (Design And Performance Evaluation of Fault-Tolerant Continuous Media Storage System Based on $PRR_gp$)

  • 오유영;김성수
    • 한국정보처리학회논문지
    • /
    • 제7권4호
    • /
    • pp.1290-1298
    • /
    • 2000
  • Multimedia Systems such as VOD(Video On Demand) and MOD (Multimedia On Demand) need to support continuous media operations which are randomly called by concurrent users and require that stored media be accessed in real-tim. To satisfy such a requirements, disk arrays consisting of multiple disks are generally used as storage systems. Under the real-time environments to provide users with accessing continuous media in the parallel and concurrent manner, storage systems should be able to deal with user requests independently. In this paper, we present a new fault-tolerant continuous media storage system called PADA(PRR\ulcorner bAsed Disk Array), which is based on a PRR\ulcorner (Prime Round Robin with Grouped Parties) disk placement scheme with enhanced reliability nd load-balancing. We have compared and evaluated the storage space overhead for fault-tolerance, the reliability of diks array systems, the degree of disk load0-balancing, the demanded buffer space, the maximum number of users being capable of supporting and the fault recovery overhead for PADA, RAID 5 and Declustered storage systems. According to the results, PADA is the best among them in that PADA satisfies load-balancing more effectively and servces more user in case of arbitrary-rate retrievals.

  • PDF

Secure and Efficient Conjunctive Keyword Search Scheme without Secure Channel

  • Wang, Jianhua;Zhao, Zhiyuan;Sun, Lei;Zhu, Zhiqiang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제13권5호
    • /
    • pp.2718-2731
    • /
    • 2019
  • Conjunctive keyword search encryption is an important technique for protecting sensitive data that is outsourced to cloud servers. However, the process of searching outsourced data may facilitate the leakage of sensitive data. Thus, an efficient data search approach with high security is critical. To solve this problem, an efficient conjunctive keyword search scheme based on ciphertext-policy attribute-based encryption is proposed for cloud storage environment. This paper proposes an efficient mechanism for removing the secure channel and resisting off-line keyword-guessing attacks. The storage overhead and the computational complexity are regardless of the number of keywords. This scheme is proved adaptively secure based on the decisional bilinear Diffie-Hellman assumption in the standard model. Finally, the results of theoretical analysis and experimental simulation show that the proposed scheme has advantages in security, storage overhead and efficiency, and it is more suitable for practical applications.

순환최소자승법을 이용한 직류도시철도 변전소의 가선전압변동 모델링 (Modelling Voltage Variation at DC Railway Traction Substation using Recursive Least Square Estimation)

  • 배창한
    • 전력전자학회논문지
    • /
    • 제20권6호
    • /
    • pp.534-539
    • /
    • 2015
  • The DC overhead line voltage of an electric railway substation swings depending on the accelerating and regenerative-braking energy of trains, and it deteriorates the energy quality of the electric facility in the DC railway substation and restricts the powering and braking performance of subway trains. Recently, an energy storage system or a regenerative inverter has been introduced into railway traction substations to diminish both the variance of the overhead line voltage and the peak power consumption. In this study, the variance of the overhead line voltage in a DC railway substation is modelled by RC parallel circuits in each feeder, and the RC parameters are estimated using the recursive least mean square (RLMS) scheme. The forgetting factor values for the RLMS are selected using simulated annealing optimization, and the modelling scheme of the overhead line voltage variation is evaluated through raw data measured in a downtown railway substation.

3GPP 네트워크에서 효율적인 인증 데이터 관리를 위한 개선된 AKA 프로토콜 (Improved AKA Protocol for Efficient Management of Authentication Data in 3GPP Network)

  • 김두환;정수환
    • 정보보호학회논문지
    • /
    • 제19권2호
    • /
    • pp.93-103
    • /
    • 2009
  • 본 논문에서는 3GPP 네트워크에서 USIM 기반의 사용자 인증 기법을 제안한다. 제안 기법은 기존의 3GPP 네트워크 접속을 위한 인증 방식에서 발생 가능한 Sequence Number 동기 문제, 인증 데이터 Overhead 문제, 네트워크 간 시그널링 Overhead 문제 등을 개선한다. 제안 기법은 기존의 USIM 기반의 AKA 인증 프로토콜을 기본 모델로 사용하고 단말과 SN이 공유한 SK와 Time Stamp를 통해서 AKA 인증 절차를 수행하도록 한다. 이렇게 함으로써 인증 벡터의 Sequence Number의 동기 여부를 확인할 필요 없이 Time Stamp 값으로 인증 벡터의 맵핑을 수행하여 Sequence Number 동기 문제를 해결할 수 있다. 뿐만 아니라 하나의 인증 벡터만을 관리하여 사용하기 때문에 SN에서의 인증 데이터 Overhead문제를 해결하고, SN과 HN 사이의 시그널링 Overhead 문제를 개선할 수 있다.